# AI-Empowered Multiple Access for 6G: A Survey of Spectrum Sensing, Protocol
Designs, and Optimizations

Xuelin Cao, Bo Yang, Kaining Wang, Xinghua Li, Zhiwen Yu, ,
Chau Yuen, , Yan Zhang, , Zhu Han

###### Abstract

With the rapidly increasing number of bandwidth-intensive terminals capable of
intelligent computing and communication, such as smart devices equipped with
shallow neural network models, the complexity of multiple access for these
intelligent terminals is increasing due to the dynamic network environment and
ubiquitous connectivity in 6G systems. Traditional multiple access (MA) design
and optimization methods are gradually losing ground to artificial
intelligence (AI) techniques that have proven their superiority in handling
complexity. AI-empowered MA and its optimization strategies aimed at achieving
high Quality-of-Service (QoS) are attracting more attention, especially in the
area of latency-sensitive applications in 6G systems. In this work, we aim to:
1) present the development and comparative evaluation of AI-enabled MA; 2)
provide a timely survey focusing on spectrum sensing, protocol design, and
optimization for AI-empowered MA; and 3) explore the potential use cases of
AI-empowered MA in the typical application scenarios within 6G systems.
Specifically, we first present a unified framework of AI-empowered MA for 6G
systems by incorporating various promising machine learning techniques in
spectrum sensing, resource allocation, MA protocol design, and optimization.
We then introduce AI-empowered MA spectrum sensing related to spectrum sharing
and spectrum interference management. Next, we discuss the AI-empowered MA
protocol designs and implementation methods by reviewing and comparing the
state-of-the-art, and we further explore the optimization algorithms related
to dynamic resource management, parameter adjustment, and access scheme
switching. Finally, we discuss the current challenges, point out open issues,
and outline potential future research directions in this field.

###### Index Terms:

AI-empowered MA, spectrum sensing, and protocol design and optimization.

TABLE I: List of abbreviations ABS | Almost Blank Sub-Frame | MEC | Multi-Access Edge Computing
---|---|---|---
ACL | Actor-Critic Learning | MABL | Multi-Armed Bandit Learning
AIFS | Arbitration Inter-Frame Space | MA | Multiple Access
AI | Artificial Intelligence | MIMO | Multiple-Input Multiple-Output
ANN | Attention-Based Neural Network | MISO | Multiple-Input Single-Output
AR | Augmented Reality | NNs | Neural Networks
BDMA | Beam-division multiple access | NGMA | Next-Generation Multiple Access
BLER | Block Error Rate | NOMA | Non-Orthogonal Multiple Access
BnB | Branch-and-Bound | OFDMA | Orthogonal Frequency Division Multiple Access
CDMA | Code Division Multiple Access | OMA | Orthogonal Multiple Access
CNN | Convolutional Neural Networks | OA-FMTL | Over-The-Air Federated Multi-Task Learning
CRN | Cognitive Radio Networks | PDMA | Pattern Division Multiple Access
CSAT | Carrier Sense Adaptive Transmission | PHY | Physical
CSMA | Carrier Sense Multiple Access | PCA | Principal Component Analysis
CSMA/CA | CSMA with Collision Avoidance | QoS | Quality of Service
CW | Contention Window | RA | Random Access
DCF | Distributed Coordination Function | RFR | Random Forest Regressor
DIFS | DCF Inter-Frame space | RS | Rate Splitting
DCM | Duty Cycle Management | RSMA | Rate Splitting Multiple Access
DT | Decision Tree | RIS | Reconfigurable Intelligent Surface
DBN | Deep Belief Networks | RNN | Recurrent Neural Networks
DDPG | Deep Deterministic Policy Gradient | RL | Reinforcement Learning
DNN | Deep Neural Networks | RB | Resource Block
DRL | Deep Reinforcement Learning | 6G | Sixth Generation
EH | Energy Harvesting | SC | Superposition Coding
EM | Expectation-Maximization | SCF | Spectral Correlation Function
FL | Federated Learning | SCMA | Sparse-Code Multiple Access
1G | First Generation | SDMA | Space-Division Multiple Access
FDD | Frequency Division Duplexing | SDN | Software Defined Networking
FDMA | Frequency Division Multiple Access | SIC | Successive Interference Cancellation
GSM | Global System for Mobile | SIFS | Short Inter-Frame Spacing
HB | Holographic Beamforming | SIM | Spectrum Interference Mitigation
IM | Index Modulation | SL | Supervised Learning
I/Q | In-Phase/Quadrature | SVM | Support Vector Machine
ISAC | Integration of Sensing and Communication | TCMA | Trellis-Coded Multiple Access
IoT | Internet of Things | TDD | Time Division Duplexing
KNN | K Nearest Neighbors | TDMA | Time Division Multiple Access
LAA | License Assisted Access | UAV | Unmanned Aerial Vehicle
LTE | Long Term Evolution | URA | Unsourced Random Access
LTE-U | LTE in Unlicensed Spectrum | USL | Unsupervised Learning
LSTM | Long-Short Term Memory | uRLLC | ultra-Reliable Low-Latency Communication
LEO | Low Earth Orbit | V2X | Vehicle-to-Everything
LDS | Low-Density Signature | VR | Virtual Reality
MAC | Medium Access Control | WANET | Wireless Ad Hoc Networks
ML | Machine Learning | WFL | Wireless Federated Learning
MR | Mixed Reality | WPT | Wireless Power Transfer
MINLP | Mixed-Integer Nonlinear Programming | WSN | Wireless Sensor Networks

##  I Introduction

With emerging scenarios in the sixth-generation (6G) systems, such as
immersive communication, massive communication, hyper-reliable low-latency
communication, ubiquitous connectivity, integrated sensing and communication
(ISAC), and integrated AI and communication, 6G communication systems are
expected to support extremely high data rates (up to 1 Tbps), massive dense
connectivity of billions of devices (up to $10^{7}$ devices per km2), a wide
range of coverage ($100\%$ land area and 3D coverage), and high mobility
(500-1,000 km/h) with a wide variety of applications and services [1],
attracting enormous attention from academia and industry. However, the
multiple access (MA) techniques developed in previous generations are not
scalable to meet the following unprecedented requirements, such as ubiquitous
connectivity, intelligent communication, ultra-high reliability, low latency,
integration of network communication and computing, etc. due to limited
spectrum resources, high hardware cost, high power consumption, and high
complexity. In addition, existing MA schemes face significant challenges, such
as ultra-high spectral/energy efficiency to provide ubiquitous connectivity in
resource-constrained IoT networks, enabling massive device connectivity with
low overhead, providing fairness, achieving strict timeliness for autonomous
vehicles and remote surgery services, and enabling compatibility and
scalability for 6G and beyond. With the emergence of 6G technologies, resource
optimization, interference management, channel estimation, signal processing,
and security issues also appeared to be more challenging in MA. These
challenges, interacted with the high demands of 6G, are prompting a rethink
and redesign of next generation multiple access (NGMA) [2, 3] techniques for a
green and intelligent network.

With billions of smart devices seamlessly connecting to the Internet and a
rapidly growing number of bandwidth-hungry mobile broadband devices, it is
envisioned that MA and its optimization methods for achieving better Quality-
of-Service (QoS), especially for latency-sensitive applications, are
attracting more attention. For example, emerging novel applications have posed
unprecedented demands on bandwidth, power, latency, and connectivity in
wireless networks, including but not limited to the super-smart society
(referred to as autonomous driving, smart cities, and smart healthcare),
augmented reality (referred to as XR, including augmented reality (AR), mixed
reality (MR), and virtual reality (VR)), and automation and manufacturing
(Industry 4.0). However, traditional MA design and optimization approaches are
losing ground due to a number of drawbacks. For example, low spectral
efficiency in the typical orthogonal multiple access (OMA) schemes, spatial
division multiple access (SDMA) schemes are not feasible in densely populated
urban areas or indoor environments due to the complexity of multi-antenna
processing techniques, non-orthogonal multiple access (NOMA) schemes sacrifice
their hardware complexity and system overhead to improve overall spectral
efficiency, DCF-based random MA schemes are impossible to avoid collisions,
etc. Artificial Intelligence (AI) solutions are able to learn the inference
information online and make optimal decisions in near real-time. In addition,
the intelligence characteristic of 6G has triggered AI applications in MA, and
AI-empowered MA designs will exploit the full potential of radio signals to
address these requirements and challenges, thereby having been widely
investigated by academic and industrial communities.

Figure 1: Roadmap of multiple access.

###  I-A Roadmap of Multiple Access

####  I-A1 MA in Cellular Networks

Over the last few decades, the development of MA schemes has been seen as an
important factor in the evolution of cellular networks from the first-
generation (1G) to 6G. As shown in Fig. 1, the MA schemes used in wireless
networks, including frequency division multiple access (FDMA) in 1G systems;
Time division multiple access (TDMA) [4], which is widely used in the 2G
global system for mobile communications (GSM); Code division multiple access
(CDMA)[5], which supports multiple users with user-specific spreading
sequences within the same resource block (RB), was originally proposed by
Qualcomm and is widely used in 3G; and orthogonal frequency division multiple
access (OFDMA)[6], which is implemented in 4G networks by dividing the
frequency and time RB into narrow subcarriers and slots. A defining feature of
all MA schemes is their orthogonality, which ensures that wireless network
resources, including frequency, time, and code resources, are theoretically
allocated to different users. It allows multiple users to access the network
while avoiding multi-user interference. In general, FDMA, TDMA, CDMA, and
OFDMA can be regarded as OMA schemes, which typically provide access service
to only one user within the same RB at one time. In 4G and 5G, the
introduction of multiple-input multiple-output (MIMO) technology creates a
spatial dimension, facilitating beam-division multiple access (BDMA)[7] and
SDMA[8]. BDMA and SDMA are capable of serving multiple users on the same RB by
utilizing spatial resources and multi-antenna processing.

TABLE II: Comparison of different MA category MA category |  Schemes |  Scheduling domain |  Advantages |  Disadvantages
---|---|---|---|---
OMA |  FDMA |  Frequency |  $\star$ Simple transceiver design |  $\star$ Less number of users are supported
|  TDMA |  Time |  $\star$ Avoidance of multi-user interference |  $\star$ Low spectral efficiency
|  CDMA |  Code |  $\star$ Easy implementation |  $\star$ Inefficient use of spectrum
|  OFDMA |  Time and frequency |  $\star$ Low cost |  $\star$ High signalling overhead
NOMA |  CD-NOMA |  Code |  $\star$ More number of users are supported |  $\star$ Complexity transceiver design
|  PD-NOMA |  Power |  $\star$ High spectral efficiency |  $\star$ SIC error propagation
RA |  CSMA |  Null |  $\star$ Highly dynamic and adaptive |  $\star$ High collision and interference

However, the orthogonality characteristics of the aforementioned OMA schemes
limit the number of access users. To allow more users to access the network
and significantly improve the overall spectral efficiency, NOMA schemes have
been proposed [9, 10, 11]. These schemes allow multiple users to share the
same slot and frequency RB in the different domains, thus achieving high
spectral efficiency at the cost of additional hardware complexity. On this
basis, NOMA schemes can be divided into two types: code-domain non-orthogonal
multiple access (CD-NOMA) and power-domain non-orthogonal multiple access (PD-
NOMA). CD-NOMA separates multiple users via non-orthogonal codes, while PD-
NOMA separates multiple users in the power domain. Typical CD-NOMA schemes
include trellis-coded multiple access (TCMA) [12], low-density signature (LDS)
sequence-based CDMA [13], interleave-division multiple access (IDMA) [14],
pattern-division multiple access (PDMA) [15], and sparse-code multiple access
(SCMA). Unlike CD-NOMA schemes, PD-NOMA schemes use superposition coding (SC)
techniques at the transmitter and successive interference cancellation (SIC)
at the receiver to optimize resource allocation.

To further improve the performance of NOMA, rate-splitting multiple access
(RSMA) has recently been proposed with the concept of rate splitting (RS).
Specifically, RSMA splits user messages into shared and private content based
on RS, then enables the ability to partially decode the interference and
partially treat the interference as noise, which is different from the extreme
interference management strategies used in SDMA and NOMA. RSMA can dynamically
switch between SDMA and NOMA by adjusting power and streams, thereby ensuring
effective performance under varying levels of interference [16].

####  I-A2 MA in Wi-Fi Networks

In contrast, MA schemes in Wi-Fi networks mainly focus on contention-based
random access (RA) schemes. The earliest RA schemes, such as ALOHA and slotted
ALOHA, limit the channel efficiency to $18\%$ and $36\%$, respectively, [17,
18]. Carrier sense multiple access (CSMA)-based schemes have been proposed to
reduce the collisions suffered from ALOHA-based schemes [19]. However, CSMA
cannot eliminate collisions due to the hidden and exposed terminal problems.
The hidden terminal problem can be solved by using busy tones [20] or
collision avoidance [21]. The most classical RA scheme refers to IEEE 802.11,
based on CSMA with collision avoidance (CSMA/CA) [22], where the distributed
coordination function (DCF) is a mechanism to avoid collisions when accessing
the channel, its performance has been widely studied [23, 24, 25]. The
performance of IEEE 802.11 DCF can be influenced by adjusting the contention
window (CW) values. In particular, higher CW values result in fewer collisions
but increase the number of backoff slots, and lower CW values result in faster
backoff but increase the probability of collisions. Therefore, as device
density increases, CW values should be set high to avoid collisions, which in
turn reduces spectral efficiency. As the number of access devices decreases,
the CW values should be set lower to improve spectral efficiency, but this
also results in more collisions and long latency. Although IEEE 802.11 DCF
performance improvement analysis has been studied, the main problems with RA-
based schemes remain, especially for massive connectivity scenarios. In
addition, these conventional RA schemes are not well suited for 6G scenarios
as the signalling overhead, hardware cost, and complexity will limit their
applicability. Table II compares the different MA in terms of their scheduling
domain and features.

Figure 2: Roadmap of AI-empowered multiple access.

###  I-B Evolution of AI for Multiple Access

Machine learning (ML) was first introduced as a popular technique in the late
1950s and has undergone rapid development. It has been successfully applied to
wireless networks and is likely to become a potential technique for
significantly improving the performance of next-generation networks [26, 27,
28]. In particular, ML techniques have been used to solve wireless resource
management problems, achieving optimal performance in a near-real-time manner,
especially in applications such as power control, beamforming, multi-access
edge computing (MEC), and reconfigurable intelligent surfaces (RIS).
Consequently, the optimization of wireless networks using ML techniques has
become a hot topic that has attracted much attention in academia and industry.
When considering ML techniques for network architecture, offline and online ML
are two fundamental mechanisms; the latter is embedded in network algorithms
or protocols, while the former is performed on a local computing unit or can
be supported by remote computing units.

Recently, many studies have been conceived on ML paradigms in wireless
scenarios, such as HetNets, cognitive radio networks (CRNs), Internet of
Things (IoT), wireless ad hoc networks (WANETs), wireless sensor networks
(WSNs), etc. In addition, ML techniques have been reviewed in the past decades
to address compelling problems of wireless networks [28, 29, 30, 31, 32, 33].
Specifically, Wang et al. [28] addresses the implementation of supervised
learning (SL), unsupervised learning (USL), and reinforcement learning (RL) in
wireless scenarios. Xie et al. [29] focus on addressing the problems of cyber
intrusion detection using ML and data mining. Pacheco et al. [30] apply ML
algorithms to software-defined networking (SDN) for classification/regression,
routing optimization, resource management, etc. Several papers explore the
progress of ML techniques in the physical (PHY) layer, medium access control
(MAC) layer, network layer, and application layer [34, 31, 28, 35, 36, 37, 38,
39, 40, 41, 33, 42, 2, 43, 44, 7, 45, 46, 47, 32, 48, 3, 49, 50, 51]. More
details are shown in Table III, Compared to these existing surveys and reviews
that mainly focus on resource management at the MAC layer and its ML-based
solutions, in this paper we mainly focus on the survey of AI-empowered MA in
terms of spectrum sensing, protocol design, optimization, applications, and
open issues.

TABLE III: Summary of existing survey on AI-empowered MA Type |  Ref. |  Network |  Main Scope |  Contributions |  Year
---|---|---|---|---|---
Existing
|  [34] |  5G |  OMA |  $\star$ Explores the deep learning techniques in the radio access. |  2019
[31] |  5G |  TDMA, OFDMA |  $\star$ Surveys the applications of ML in resource management at the MAC layer, networking and mobility management at the network layer, and localization in the application layers. |  2019
[52] |  6G |  NOMA |  $\star$ Surveys the AI applications in RIS-assisted NOMA systems. |  2019
Survey |  [28] |  WNs |  OFDM, NOMA |  $\star$ Reviews ML-enabled wireless networks by elaborating on various learning models. |  2020
[35] |  6G |  RSMA |  $\star$ Presents the first comprehensive tutorial on RSMA by detailing its architecture, taxonomy, applications, and complexities. |  2020
[36] |  Vehicular |  TDD, OFDMA, NOMA |  $\star$ Explores various ML techniques applied to the radio access, networking, and security parts of the 6G vehicular network. |  2020
[37] |  B5G or 6G |  OFDMA, NOMA |  $\star$ Investigates AI-enabled HAPS in terms of design, topology management, handoff, and resource allocation. |  2021
[38] |  5G HetNets |  OFDMA, NOMA |  $\star$ Investigates the learning-based resource allocation (RA) structures, models, and algorithms. |  2021
[33] |  Wi-Fi |  RA |  $\star$ Surveys IEEE 802.11 performance improvement with machine learning. |  2022
[39] |  5G HetNets |  OFDMA, NOMA |  $\star$ Surveys the AI-enabled radio resource management methods for improving spectrum utilization, load balancing, and network energy efficiency. |  2022
[40] |  B5G or 6G |  PD-NOMA |  $\star$ Focuses on optimizing and improving the power domain NOMA with ML. |  2022
[41] |  5G |  Radio Access |  $\star$ Surveys the main energy efficiency enabling technologies that 3GPP NR provides massive MIMO, lean carrier design and sleep modes with ML. |  2022
[33] |  Wi-Fi |  RA |  $\star$ Investigates on improving IEEE 802.11 performance with ML. |  2022
[42] |  5G,6G |  OMA, RA |  $\star$ Surveys the applications of ML in resource management at the MAC layer, networking and mobility management in the network layer, and localization in the application layer. |  2022
[2] |  6G |  NOMA |  $\star$ Explores advanced ML techniques for facilitating the design of NOMA communications. |  2022
[43] |  6G |  NOMA |  $\star$ Explores the spectrum policies and technologies in the above 100 GHz spectrum. |  2023
[44] |  Ad hoc |  RA |  $\star$ Surveys RL-based MAC protocols and their related solutions. |  2023
[7] |  5G |  BDMA |  $\star$ Investigates the design of multiplexing methods and beamwidth optimization for UAV communications. |  2019
[53] |  6G |  MAC |  $\star$ Elaborates how to employ the AI techniques to efficiently and effectively optimize the network performance. |  2020
Magazine |  [45] |  5G |  NOMA |  $\star$ Reviews AI-enhanced spectrum sensing for improving the spectral efficiency of multiple access. |  2020
[46] |  MTC |  NOMA |  $\star$ Studies NOMA transceiver design with deep learning. |  2021
[47] |  5G |  RA |  $\star$ Investigates a framework for estimating access parameters using a DNN model to avoid preamble collision. |  2021
[54] |  6G |  NOMA |  $\star$ Reviews AI-driven UAV-NOMA-MEC systems. |  2022
[48] |  Satellites |  NOMA, RA |  $\star$ Summarizes the routing technologies from single-layer and multilayer satellite constellations. |  2022
[3] |  6G |  NOMA |  $\star$ Applies ML in NOMA and its SIC for massive connectivity. |  2022
[49] |  6G |  Unsourced RA |  $\star$ Proposes URA and takes federated learning (FL) for privacy protection of URA. |  2023
[50] |  5G and 6G |  NOMA |  $\star$ Focuses on applying AI technologies in NOMA to achieve automated, adaptive and highly efficient multiple access. |  2023
[51] |  B5G |  NOMA |  $\star$ Discusses the potential of ML approaches for RIS-aided NOMA. |  2023
[55] |  6G |  MD-MA |  $\star$ Investigates intelligent multi-dimensional multiple access. |  2023
Ours
Survey |  This article |  6G |  OMA, NOMA, RA |  $\star$ Surveys the AI-empowered spectrum sensing, protocol design, optimization, and applications at the MAC layer. |  null

Specifically, the roadmap of ML techniques at the MAC layer is shown in Fig.
2. In the late 1990s, learning-automata-based TDMA was proposed for stations
to select efficiently in each time slot under bursty and correlated traffic
[56, 57, 58]. Around the year 2000, many ML techniques were proposed for CDMA,
such as linear RL was used in CDMA for adaptive energy estimation [59], Tabu
learning was used to solve optimization problems of CDMA [60], and self-
learning was used to improve the performance of CDMA [61]. From 2004 to 2009,
Q-learning was introduced for radio resource management in multimedia WCDMA
[62], and fuzzy Q-learning was proposed for admission control in WCDMA [63,
64, 65]. In 2010, distributive stochastic learning was proposed for OFDMA for
power and sub-band allocation optimization [66], and after this, ML has been
used in OFDMA for interference management and resource management [67, 68, 69,
70, 71, 72, 73, 74]. Furthermore, RL was used for intercell interference
management and resource allocation [67, 68, 69, 71], utility-based learning
and attachment learning were used for bandwidth allocation [72, 73, 74]. In
2015, null space learning was proposed in SDMA for channel interference
management [75]. Meanwhile, ML has also been applied in MIMO-OFDM for resource
allocation and optimization [76, 77] in 2016. Subsequently, DL, RL, and SL
approaches have been extensively studied in MA protocols, and especially ML
techniques have been explosively studied with the appearance of NOMA and SCMA
[78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92]. In particular,
SIC plays an important role on the receiver side of NOMA. The application of
AI techniques to reduce the complexity and improve the accuracy of SIC in NOMA
needs to be considered for 6G and beyond [50, 93, 87].

In contrast to the above work, the authors of [33] investigate the use of ML
for layer-specific operation in IEEE 802.11 networks. The new amendments to
IEEE 802.11, such as the IEEE 802.11 n/ac/ax/amendments, introduce augmented
functionalities (e.g., channel bonding, massive transmissions, short guard
interval, and advanced modulations) to improve network operation and user
experience. Unlike previous versions, IEEE 802.11be introduces new advanced MA
techniques such as OFDMA, multi-resource (MU)-MIMO, multi-AP coordination, and
multi-link operation (MLO) to improve spectral efficiency. Compared to IEEE
802.11be, IEEE 802.11bn addresses the latency-sensitive traffic by extending
enhanced distributed channel access (EDCA), and also enables multi-AP
coordination transmission through coordinated beamforming (CBF) or joint
transmission and reception (JTR) [94, 95, 96]. However, these augmented
functionalities have a limited impact on network performance due to the
dynamic nature of Wi-Fi environments and user mobility. By applying ML
algorithms to these augmented functionalities, it is possible to gain
knowledge, generalize, and learn from experience to design intelligent
communication systems. Existing ML solutions for the IEEE 802.11 MAC layer
mainly focus on optimizing the parameter design of the DCF. For example, the
dynamic selection of CW values based on SL and RL methods can compensate for
collisions and idle periods. In addition, self-learning has been used to
reduce energy consumption [97, 98], support self-configuration [99], and
select the MA scheme according to the environmental conditions and application
requirements [100]. In summary, SL [101, 102], RL [103, 104, 105, 106], deep
reinforcement learning (DRL) [107, 108, 109], and federated learning (FL)
[110, 109], models are applied to the IEEE 802. 11 standards [104, 109] and
its amendments, most notably in 802.11ac [101], 802.11e [103, 111], 802.11n
[102], and 802.11ax [107, 108].

Overall, AI technologies are driving MA techniques in terms of resource
allocation, optimization, signal processing, sensing, security, dynamic access
control, etc. These aspects are driving the latest versions of standards,
technical reports, and technical reports and industry white papers. For
example, 3GPP Release 17$\&$18 define various advances such as beamforming,
MIMO enhancements, mobility enhancements, network power savings, intelligence
in MA, IEEE 802.11be and IEEE 802.11bn standards advocate for AI applications
to enhance spectral efficiency, enable dynamic access control, facilitate
predictive resource allocation, optimize intelligent beamforming, and provide
security. Some reports and whitepapers released by IMT-2030, HuaweiTech, or
Ericsson propose advanced MA coupled with AI techniques for space-air-ground
integrated networks (SAGIN), ISAC, RIS, etc. With the boom of AI technologies,
some works on datasets for signal processing, sensing, optimization, and
security of MA have been analyzed to improve communication reliability,
throughput and latency [112, 113, 114, 115, 116, 117]. These indicate that the
quality of data sets is critical to ensuring the effectiveness and reliability
of 6G systems.

Figure 3: Structure of the article.

###  I-C Contributions and Organization

In this paper, we start with an overview of AI-empowered MA, including the new
requirements and key techniques that meet the 6G features, and then propose a
unified framework. We continue in the context of spectrum sensing, protocol
design, and optimization techniques for AI-empowered MA. Finally, we point out
the open issues of AI-empowered MA techniques and discuss the potential
applications of AI-empowered MA. Our contributions to this survey are detailed
below.

  1. 1.

We survey the recent progress and the state of the art in AI-empowered MA
techniques. In particular, we first discuss the implementation issues and
challenges and then contrast various popular AI-empowered MA approaches to
show the recent progress in the interplay between MA and AI techniques in
wireless networks. Meanwhile, the basic principles of AI-empowered MA are
defined, and the key characteristics are summarized accordingly. By studying
the interplay between applied MA techniques and advanced AI techniques in 6G
systems, the emerging new requirements, potential techniques, and a unified
framework for AI-empowered MA are also illustrated.

  2. 2.

We elaborate on the spectrum sensing of AI-empowered MA techniques, including
spectrum sharing and interference management, which belong to the fundamental
problems for improving the performance of AI-empowered MA. We then summarize
the AI-empowered MA protocols in terms of the frame, scheme, and
implementation designs. Furthermore, we identify the protocol optimization of
AI-empowered MA, where efficient AI algorithms are designed for resource
management, parameter adjustment, and access switching. The potential problems
and solutions based on the current research are also discussed, respectively.

  3. 3.

We discuss the open issues and research related to the emerging AI-empowered
MA techniques, with particular emphasis on spectrum and interference
management, diverse requirements, security and privacy, compatibility,
complexity, lower power, and adaptability. In addition, we look forward to
applications of AI-empowered MA in typical 6G networks, such as SAGIN,
wireless power transfer (WPT), Holographic Beamforming (HBF), Terahertz (THz),
ISAC, RIS, MEC, etc.

The remainder of this article is organized as follows (see Fig. 3). Section II
introduces AI-empowered MA for 6G systems in terms of its emerging features,
new requirements, key techniques, and a unified framework model. Section III
presents the possible spectrum sensing, sharing, and interference management
techniques for AI-empowered MA that influence spectral efficiency and energy
efficiency. Section IV presents the feasible AI-empowered MA protocols
including the frame, the scheme, and the implementation designs, and then
Section V explores the optimization of AI-empowered MA protocols in terms of
the resource management, the parameter adjustment, and the access switching.
Furthermore, Section VI explores the open issues and the research areas of AI-
empowered MA that accelerate the evolution of 6G networks. Finally, Section
VII concludes this article with an outlook. Table I gives the list of the
abbreviations used here for readability.

##  II AI-Empowered MA

AI techniques enable devices to proactively respond to the wireless
environment by learning the network states (e.g., channel state information,
traffic rates, link, interference, etc.) and link states (e.g., signal-to-
noise ratio (SNR), acknowledgment/negative acknowledgment (ACK/NACK), etc.) to
achieve a pre-defined goal in a wireless system. As a result, AI techniques
have become one of the most important tools and have been widely applied in
wireless networks, especially for optimizing and improving the operation of
wireless networks. In addition, it is estimated that 5G will reach its
capacity limit by 2030, and 6G will emerge to accommodate user services beyond
5G’s capabilities, leading to a fully digital and intelligent connectivity
era. In this context, AI techniques are expected to play a crucial role in
improving the MA performance of 6G services, such as extremely high data
rates, massive connectivity, very low latency, high spectral/energy
efficiency, high reliability, etc. In this section, the 6G features are
aligned with MA in Section II-A, the new challenges of MA are shown in Section
II-B, the key techniques enabled on MA are presented in Section II-C, and the
framework modules of MA are introduced in Section II-D.

###  II-A Meet 6G Features

According to the 6G trends in IMT-2030 and future network forecasts, the AI-
empowered MA technologies align with the 6G features as shown in Fig. 4, which
are detailed as follows.

####  II-A1 Native AI

In contrast to 4G and 5G communication systems with partial or limited AI,
intelligence is expected to be a fundamental characteristic of 6G networks, as
AI techniques have already penetrated wireless networks and will be ubiquitous
across all domains (core, access, edge, device). The wireless evolution from
“connected things” to “connected intelligence” in 6G will have a profound
impact on the evolution of AI and proliferate novel applications. In
particular, a new paradigm of AI-empowered MA will simplify and improve radio
resource management, adaptability, and efficiency of 6G networks.

####  II-A2 ISAC

The emergence of ISAC is boosting new services and applications such as
automated driving, navigation, etc. In particular, AI-driven ISAC [118, 119,
120] in 6G networks will simultaneously improve communication and sensing
aspects through full interaction of both, which will help converge the
physical, biological, and cyber worlds, making real-time digital twins a
reality. To achieve this vision and lay a solid foundation for the
intelligence of everything in the future, AI-empowered MA systems are required
to satisfy this emerging trend.

####  II-A3 Ubiquitous Connectivity

With the explosion of low-orbit satellite and UAV technologies in recent
years, the integration of ground, air, and space networks in 6G systems will
support ubiquitous super-3D connectivity and communications in the vertical
dimension. Unlike current terrestrial networks, 3D networks will be able to
cover remote areas and provide seamless global access. To make this possible,
an efficient AI-empowered MA system must be developed to enable 6G services
for users wherever they are.

####  II-A4 Multi-band

Unlike the limited radio frequency (RF) bands used in 5G, 6G will exploit the
rich band resources, such as sub-6 GHz, mmWave bands (30–300 GHz), terahertz
band (0.1–10 THz), and even the optical frequency band. It is expected to
result in high data rates, with a peak data rate of 1,000 Gbps, and a
significant increase in total band capacity by at least 11.11 times.
Furthermore, the use of multi-band and massive MIMO technologies is expected
to improve spectral efficiency, and the move to higher frequency bands will be
a key factor in the investigation of AI-empowered MA for 6G networks.

####  II-A5 Sustainability

It is estimated that by 2030, research on 6G networks will be in a new era
where sustainability will drive its development. This means that 6G wireless
networks will not exist in isolation, but should be linked to the triple
bottom line of economic sustainability, social sustainability, and
environmental sustainability. To realize this sustainable development, a
reliable and scalable AI-empowered MA will play an important role in 6G
networks.

####  II-A6 Security

Human-centric 6G network services will require security, confidentiality, and
privacy features. The PHY layer security challenges in 5G (e.g.,
decentralization, transparency, data interoperability, and network privacy
vulnerabilities) can be solved by the traditional encryption algorithms,
thereby providing secret access and transmission services. However, due to the
extensive application of Big Data and AI technologies in wireless networks,
the current privacy and security methods in 5G networks will become insecure
in 6G networks. Based on this, AI-driven MA security can be ensured by jointly
adopting AI security techniques (e.g., adversarial training, federated
learning, differential privacy, etc.) and physical security technologies
(e.g., encryption, authentication, detection, regular updates, etc.) [121,
122, 123, 124]. Thus, the novel regulation and process of privacy and security
for the design of AI-empowered MA has to be considered with respect to AI
technologies and MA systems that are still in their early stages.

(a)

(b)

Figure 4: AI-empowered MA meets 6G features and new requirements. (a) Meet 6G
features. (b) New challenges.

###  II-B New Challenges of MA

To adapt to the new features and emerging technologies of 6G, AI-empowered MA
systems face a number of new challenges.

####  II-B1 Spectral Efficiency

The increase in the number of connected devices and traffic poses a challenge
to improve spectral efficiency, especially the effective use of higher
frequency bands (e.g., mmWave and THz) to facilitate the development of 6G
communications targeting at Tbps data rate [125]. However, in the traditional
RA, OMA, and NOMA schemes, the channel access collisions, empty slots, and
control overheads can result in low spectral efficiency, thereby degrading the
data rate and throughput of the whole system. Meanwhile, the shared spectrum
for each device is limited, and it is difficult to accommodate access requests
from millions of devices in the network. Therefore, achieving high spectral
efficiency becomes a major challenge in the development of MA for 6G networks.

####  II-B2 Energy Efficiency

The 6G system requires ten times the energy efficiency compared to 5G. Against
this background, the interconnection and collaboration of various intelligent
entities that have limited or, in some cases, no access to power, the AI-
empowered MA paradigm needs to satisfy the green and sustainable development
requirements of 6G networks. Consequently, a major challenge for the
development of AI-enabled MA is that it can support low-power communications
such as on-demand transmission, sleep or wake schedules, and power control to
improve energy efficiency. In addition, AI-empowered MA systems are also
exploring advanced battery technologies (e.g., energy harvesting (EH), power
transfer, etc.) and AI methods to support low-power communications and further
eliminate energy waste in 6G networks.

####  II-B3 Ultra-Low Latency and Reliability

Ultra-low latency and reliable services in 6G systems are important for
mission- and time-critical applications (e.g., military, disaster management,
etc.). However, with traditional contention-free MAC schemes (e.g., OMA and
NOMA), users have to wait for effective resource allocation to grant access to
their transmissions, resulting in high latency. Meanwhile, contention-based RA
schemes also suffer from latency and reliability issues due to channel access
collisions and control overhead. Ultra-low latency and reliability are
therefore becoming key challenges in developing AI-empowered MA for 6G
networks, especially for ultra-high speed with low latency communication
(uHSLLC) services in 6G is a key challenge.

####  II-B4 Massive Connectivity

The 6G system is expected to connect a massive number of devices (up to 10
million/$km^{2}$) and provide Gbps coverage everywhere with 3D network
coverage (i.e., a global 3D network overlay in ground, air, and space
networks), thereby achieving ubiquitous connectivity for exponentially growing
devices. The International Telecommunication Union (ITU) expects the number of
mobile subscriptions to reach 17.1 billion and the total mobile data traffic
to exceed 5TB per month by 2030. Against this background, AI-empowered MA
needs to be developed to enable massive, dense, and ubiquitous connectivity
for 3D network coverage, which is a major challenge in 6G networks.

####  II-B5 Compatibility and Scalability

In the context of complex communications in 6G systems, the critical
considerations for an AI-empowered MA are compatibility and scalability, which
define the ability to cope and perform well with an increasing or expanding
workload or scope. In addition, the wireless network environment can be
dynamic due to the mobility of the device (e.g., the device joins or leaves
the network). Therefore, the challenge is to make the AI-empowered MA protocol
scalable and adaptable to the varying wireless network environments without
requiring complex computation or additional control overhead while maintaining
fairness and access efficiency.

###  II-C Key Techniques for MA

Key technologies such as AI, THz, RIS, Ultra massive MIMO, WPT, HBF, IM, ISAC,
and MEC are expected to solve existing MA issues.

####  II-C1 AI

AI is a fundamental technology in the 6G system [26, 27, 28] because of its
ability to solve complex problems without explicit programming. Its successful
applications in 6G, such as spectrum sensing, resource management, and MAC
layer optimization, have attracted much attention from the industry and
research community. In particular, AI technologies have been widely used to
solve various problems arising in MAC [33, 26], such as power allocation,
spectrum management, beamformer design, switching control, user association,
clustering, scheduling, security, and so on. Compared to the very limited
application of AI from 2G to 5G, the full integration of AI and MA in 6G will
greatly improve access efficiency with low complexity and low power
consumption, especially for ubiquitous intelligent networks.

####  II-C2 THz

In the 6G system, the near exhaustion of the RF band is pushing the boundaries
of the frequency band to the THz band [126] to accelerate the deployment of
uMUB, uHSLLC, and uHDD services [127] and enhance the potential of 6G. Unlike
the mmWave band, the shorter wavelength THz band can provide hundreds of beams
by integrating a large number of antennas [128, 129, 130]. THz combined with
antenna techniques is possible to increase spectral efficiency and support
very high data rates, thereby improving the throughput performance of MA.

####  II-C3 RIS

RIS is becoming a promising technology that is capable of reconfiguring the
wireless propagation environment by exploiting the unique properties of
metamaterials-based integrated large arrays of low-cost antennas [131, 132,
133, 134]. By doing so, it is possible to increase the coverage area of ultra-
small cells, and it also has immense potential to improve the energy
efficiency and safety of MA by jointly optimizing RIS size, deployment, phase
shift, and transmit power. Furthermore, RIS can also be integrated with
existing MAC technology, such as OMA, NOMA, and RA schemes, in order to
further improve spectral efficiency while consuming less power [135, 136, 137,
138, 139, 140].

####  II-C4 Ultra-Massive MIMO

Ultra-massive MIMO technology will be critical in 6G systems due to its high
spectral and energy efficiency, higher data rates, and higher frequencies
[141]. The successful application of ultra-massive MIMO in 6G will trigger the
development of uHSLLC, mMTC, and uHDD services. Unlike the conventional MIMO
technologies in 5G systems, the ultra-massive MIMO technologies are expected
to be integrated with the emerging MA schemes to achieve massive connectivity
and high spectral and energy efficiency in the upcoming 6G networks.

####  II-C5 WPT

WPT offers a new way of powering electric-powered devices, which will be one
of the most innovative technologies in 6G communications, breaking the current
technical bottlenecks of batteries [142]. Using WPT technologies in wireless
battery charging systems, the electric-powered devices receive wireless power
from an electromagnetic field in the air and charge their batteries
wirelessly, thus achieving energy migration. In addition, the integration of
WPT technologies and MA techniques can support the massive connectivity of
power-limited devices, which will be widely applied in some specific regions,
such as smart industries and smart cities.

####  II-C6 HBF

BF [143, 144] is a new dynamic beamforming technique that differs from
conventional phased array or MIMO systems using a software-defined antenna
(SDA). Unlike phased arrays, beamforming is achieved using holograms rather
than discrete phase shifters, resulting in symmetrical transmit and receive
characteristics for HBF. In 6G, HBF shows its advantages in achieving
efficient and flexible transmission and reception of signals in multi-antenna
communication devices. In this case, HBF technologies are beneficial for MA
systems, including spectrum and energy efficiency, access convergence, and
security areas.

####  II-C7 Index Modulation (IM)

IM is a promising 6G technology for solving existing problems at the MAC
layer, such as improving spectral and energy efficiency and reducing
complexity. This is because IM uses index patterns as an additional dimension
to modulate information bits for data transmission, where the pattern can be
the activation state of time slots, subcarriers, transmit or receive antennas,
spreading codes, power levels, etc.[145]. MA combined with IM technologies
(such as IMMA)[146] allows each device to transmit additional index bits over
partial resources, thereby improving spectral and energy efficiency.

####  II-C8 ISAC

ISAC is a promising technology for improving spectral efficiency, network
reliability, and reducing hardware costs [147]. It will enable 6G systems to
continuously sense the dynamically changing wireless environment and establish
connections between different devices, thereby supporting autonomous wireless
networks. Through ISAC, the full interaction between sensing and communication
(i.e., cross-layer, cross-module, and cross-node information sharing) will
significantly improve spectral efficiency and mitigate interference. This will
enable the mutual benefits from sensing-aided communication and communication-
aided sensing, while reducing overall cost and power consumption.

####  II-C9 MEC

Due to the limited storage and computing capabilities of devices, large
amounts of raw data cannot be efficiently processed by themselves, so MEC
technology is proposed to allow devices to offload their tasks to the edge or
cloud, thereby accelerating data processing by providing intensive computing
capabilities [148, 149]. On the one hand, highly efficient channel access and
data transmission are essential for the implementation of MEC technologies. On
the other hand, MA combined with the MEC system can enhance high throughput
and timeliness, especially for the upcoming ubiquitous services in 6G
scenarios.

###  II-D Framework of MA

To provide further clarification on AI-empowered MA, we present a general
framework where the AI approaches and three models with sensing, design, and
optimization are detailed.

####  II-D1 General Framework

MA protocols are designed to adapt to the dynamic wireless environment and
satisfy various requirements. However, the limited channel resources may
restrict the performance of MA protocols when massive users have to access the
channel, so the desirable AI-empowered MAC schemes that improve the
spectral/energy efficiency while maintaining a low time cost and overhead have
to be investigated. On this basis, it is imperative that AI-empowered MA can
be flexible, scalable, reliable, and highly efficient as the applications and
scenarios change. Fig. 5 shows a general framework of AI-empowered MA schemes,
including spectrum sensing, protocol design, and performance optimization,
where each module interacts with each other, and all of them can be supported
by AI technologies. In particular, accurate spectrum sensing techniques are a
prerequisite for MA, which will benefit the design of highly efficient MA
protocols by improving spectrum utilization and mitigating spectrum
interference. Effective protocol designs are the main body of MA, ensuring
effective access and successful transmission. Advanced optimization algorithms
are an important part of MA, which can optimize and improve the performance of
the designed MA protocols. In addition, AI technologies can be used separately
in each part or adopted jointly in three parts to enhance the performance of
each part and improve the overall performance of MA for 6G.

Figure 5: AI-empowered MA framework.

####  II-D2 Three Modules

The implementation of AI-empowered MA protocols refers to three models, i.e.,
sensing, design, and optimization, the function of each model is shown in Fig.
6.

  1. 1.

Sensing Module: It is a fundamental component for the establishment of AI-
empowered MA, especially for its protocol design and optimization. Spectrum
sensing is the task of obtaining awareness of spectrum usage to avoid
collisions, which not only includes the spectrum usage characteristics across
multiple dimensions such as frequency, time, space, code, and beam, but also
determines who is occupying the spectrum. Therefore, accurate spectrum sensing
can enable the MAC to achieve high spectral efficiency and energy efficiency.
In addition, integrating spectrum sensing with AI approaches can achieve
better channel coordination and spectrum sharing mechanisms. The sensing model
supported by AI technologies is shown in Fig. 6, where network status
information (e.g., signal, interference, channel utilization, link, etc.) is
used to achieve accurate spectrum sensing and sharing information with low
complexity.

  2. 2.

Design Module: Protocol design, including frame structure and channel access
control, is the most important component for the establishment of AI-enabled
MA. It is mainly responsible for coordinating channel access among massive
devices in the shared medium using various methods (e.g., centralized,
distributed, and hybrid), which affects the performance of the communication
system (e.g., network throughput, energy consumption, latency, etc.). In
addition, integrating the MA protocol design with AI approaches could further
address the diverse service requirements and unique traffic characteristics of
devices. The design model supported by AI technologies is shown in Fig. 6,
where network status information (e.g., channel utilization, collisions,
throughput, etc.) and link state information are used to support protocol
design and frame aggregation, thereby improving channel efficiency with low-
cost hardware.

  3. 3.

Optimization Module: MA protocol optimization refers to resource management
(e.g., user association, resource allocation, collision elimination, etc.),
parameter adjustment (e.g., CW/backoff setting, frame size setting,
transmission rate setting, etc.), and access switching between different MA
schemes (e.g., OMA, NOMA, RA, etc.) is essential for obtaining high
throughput, low power consumption, and low latency. Additionally, protocol
optimization is required to adapt to highly dynamic wireless environments and
improve the overall network performance. As shown in Fig. 6, network status
information (e.g., throughput, collisions, delay, power, load, etc.) and link
state information are used to support resource management, parameter
adjustment, access switching, and collision elimination, thereby optimizing
the performance of MA protocols.

####  II-D3 AI Approaches

We have thoroughly reviewed AI technologies and algorithms for MA systems in
UAV, V2X, MEC, and RIS scenarios[54, 150, 151, 152, 153, 154], which are shown
in Table IV. Each approach is characterized as follows.

TABLE IV: The AI approaches used in AI-empowered MA schemes ML-based Approaches |  Suitable Conditions |  Applications |  Ref.
---|---|---|---
SL-based MA |  Solution is unique, a good classic algorithm |  Spectrum sensing, interference elimination, traffic estimation, channel selection, user association |  [155, 156, 157, 158, 159, 104, 102, 160, 161, 162, 163]
USL-based MA |  Solution is non-unique, automatic algorithm design |  Channel state detection, clustering, frame aggregation, interference cancellation, symbol detection, active user detection, MIMO, RIS configuration, offloading |  [164, 165, 166, 167, 168, 169, 170]
RL-based MA |  Long-term planning, the action affects the states |  Resource management, parameter adjustment, interference management, access scheme selection, sensing policy |  [171, 69, 67, 172, 173, 174, 175, 176, 177, 178, 179]
DL-based MA |  Data-driven, is capable of remembering |  Resource allocation, power control, traffic control, user scheduling, interference identification, spectrum access |  [180, 181, 182, 183, 184, 185, 186, 187]
FL-based MA |  Proving security and low computing cost |  Security, resource allocation, spectrum sharing, offloading, interference management |  [188, 189, 190, 191, 192, 193, 194]
Figure 6: Three modules of AI-empowered multiple-access.

  1. 1.

Supervised Learning: SL algorithms are trained on a certain amount of labeled
data set. Given a labeled data set, SL infers a function that maps the input
data to the output label. According to the continuity of the output, SL
algorithms are extensively used in regression, classification, and prediction,
and aim to solve MAC problems in wireless networks, such as spectrum sensing,
power control, scheduling, user association, and security issues in UAV, V2X,
MEC, and RIS scenarios. The typical SL algorithms mainly refer to support
vector machine (SVM) [195], K nearest neighbors (KNN) [196], decision tree
(DT)[197], neural networks (NNs) [198], and Bayesian statistics [199].

  2. 2.

Unsupervised Learning: USL algorithms try to explore the hidden features of
the unlabeled data. Compared to SL, there is no standard accuracy evaluation
for the output due to the lack of a labeled data set. USL algorithms aim to
classify the data set into different groups by investigating the similarity
between them. Generally, USL algorithms are widely used at the MAC layer for
solving its clustering and frame aggregation issues in UAV, V2X, MEC, and RIS
scenarios. The typical USL algorithms mainly refer to K-Means Clustering
[200], expectation-maximization (EM) [201], principal component analysis (PCA)
[202], and independent component analysis [203].

  3. 3.

Reinforcement Learning: RL algorithms allow the agent to learn by interacting
with its environment [204]. Specifically, the agent will take the appropriate
actions to optimize a long-term objective based on its trial and reward. In
contrast to the aforementioned two ML algorithms, RL algorithms are conceived
for decision-making relying on their own experience. As expected, RL
algorithms have been widely used at the MAC layer for solving mobility
management, resource management, and parameter adjustment issues in UAV, V2X,
MEC, and RIS scenarios. The most well-known RL algorithms are Q-learning
[205], multi-armed bandit learning (MABL) [206], actor-critic learning (ACL)
[207], joint utility and strategy estimation based learning (JUSEBL) [208],
and DRL [209].

  4. 4.

Deep learning: DL algorithms rely on a multi-layer network for feature
extraction and transformation, inspired by the neural network. Specifically,
each layer takes the output of the previous layer as input and then infers its
output. In principle, some DL algorithms, such as deep neural networks (DNN),
deep belief networks (DBN), recurrent neural networks (RNN) with LSTM units
[210], and convolutional neural networks (CNN), have been successfully used at
the MAC layer to solve resource allocation and optimization problems in UAV,
V2X, MEC, and RIS scenarios, thereby improving spectral efficiency and energy
efficiency.

  5. 5.

Federated Learning: FL algorithms focus on collaborative training between
multiple devices, enabling each device to build a general, robust ML model
without sharing raw data, and thus becoming the standard for compliance with a
range of new regulations on the handling and storage of private data [211,
212]. It is therefore expected to provide reliability, security, and privacy
for MA systems while reducing computational complexity issues in UAV, V2X,
MEC, and RIS scenarios.

##  III Spectrum Sensing for AI-Empowered MA

As mentioned above, the demand for high spectral and energy efficiency at the
MAC layer motivates researchers to explore spectrum sensing techniques that
support highly efficient channel access. Therefore, spectrum sensing
techniques are crucial to improve the performance of MA. Recent advances in MA
show that spectral and energy efficiency have been significantly improved by
applying AI techniques (e.g., DL, RL, SL, etc. ) in spectrum sensing, spectrum
sharing, and interference management. Additionally, AI technologies can
potentially be combined with THz or RIS for MA spectrum sensing and dynamic
spectrum management in 6G. In this section, AI-enabled spectrum sensing is
presented in Section III-A, AI-enabled spectrum sharing is shown in Section
III-B, AI-enabled spectrum interference management is depicted in Section
III-C, and their discussions are given in Section III-D.

Figure 7: AI-empowered spectrum sensing.

###  III-A Spectrum Sensing

Accurate spectrum sensing aims to help devices share the spectrum and access
the channel while avoiding multi-user interference. It is considered to be a
prerequisite for dynamic channel access and data transmission, and it also
poses a challenge for designing high-performance MA protocols, since accurate
spectrum sensing depends on the spectrum sensing period. For example, a long
spectrum sensing period could reduce the transmission time and consume more
energy, thus reducing the spectral efficiency. Therefore, how to realize
accurate and effective spectrum sensing has attracted much effort from
researchers. In this case, recent studies focusing on AI-enhanced solutions
for spectrum sensing and dynamic spectrum management have been widely
investigated in cognitive radio communication, which aims to increase spectral
efficiency. In addition, THz and RIS combined with AI technologies have also
been identified as candidates for spectrum sensing and dynamic spectrum
management. A typical AI method applied to spectrum sensing is shown in Fig.
7, where the AP senses the CSI and decides the frequency occupation of each
device using AI technologies.

Recent improvements in spectrum sensing using various AI techniques for
dynamic spectrum management are presented [213, 214, 215, 216, 217, 45, 218,
219, 220, 221, 222, 223, 224]. In particular, SL and USL techniques (e.g.,
SVMs, the K-means clustering algorithm, and the Gaussian mixture model) are
investigated for cooperative spectrum sensing [213, 225]. Various AI
techniques (e.g., SL, USL, QL, DL, etc.) are applied in spectrum sensing and
dynamic spectrum management of 6G communications to improve spectral
efficiency [218]. The deep learning techniques are further applied in CRN for
spectrum monitoring, modulation detection, and assignment[217, 214, 215].
K-nearest neighbor learning based reliable spectrum sensing for CNN is
investigated to improve spectral efficiency[220]. The multi-agent DRL
techniques in resource allocation for D2D communication are presented to
improve spectral efficiency where global historical states, actions, and
policies are shared without signal interaction[219]. Q-learning-based
approaches are used in cognitive networks to enable secondary users to perform
effective spectrum management individually and intelligently, thereby
addressing interference coordination between secondary users and interference
suppression for primary users [216]. A long-short term memory (LSTM)-based
approach is proposed to extract the energy correlation features and improve
the detection performance [226]. A DL method is developed for different types
of signal detection based on in-phase/quadrature (I/Q) samples as well [227],
thereby achieving spectrum management. In addition, Saim et al. [126]
investigate spectrum management techniques for THz communications, as the new
spectrum can be used, it is challenging to implement efficient spectrum
management, new radio access, and intelligent automation in THz communications
using AI techniques. Yang et al. [223, 224] propose intelligent spectrum
learning for spectrum utilization and combine it with RIS techniques to
significantly improve the spectral efficiency of 6G networks.

Different from AI-assisted spectrum sensing for OMA, Shi et al. [45]
investigated learning-based cooperative spectrum sensing for NOMA, which has
great potential to improve spectral efficiency and support massive
connectivity. An SVM-based SL algorithm is introduced to achieve cooperative
spectrum sensing with less prior information requirement, higher sensing
accuracy, and lower computational complexity. Compared to traditional OMA
mechanisms, SVM-based cooperative spectrum sensing in NOMA allows more than
one user to occupy the same time-frequency resource, thus achieving much
higher spectral efficiency. However, cooperative spectrum sensing will face
more challenges as the number of simultaneous transmissions increases, and the
classifiers cannot correct the existing classification error, which affects
the accuracy of cooperative spectrum sensing.

###  III-B Spectrum Sharing

Effective spectrum sharing relies on accurate spectrum sensing, which has been
extensively investigated as mentioned above. With the development of AI
techniques in spectrum sensing, significant efforts have been made to develop
intelligent spectrum sharing solutions, which enable AI-enhanced spectrum
sharing to be a promising solution due to the reduced need for prior
information requirements and higher sensing accuracy. Intelligent spectrum
sharing, which improves spectral efficiency, is required to satisfy the
following rules: i) interference is reduced or mitigated, or ii) sharing is
enabled in time, frequency/band, or spatial domain. Existing research on
intelligent spectrum sharing focuses on three main areas: spectrum sharing
between secondary and primary users in CR networks, spectrum sharing across
licensed and unlicensed bands for coexistence networks (such as Wi-Fi and
cellular coexistence), and spectrum sharing between active and passive users
[43].

Figure 8: AI-empowered spectrum sharing.

For example, the intelligent spectrum sharing solutions for CR networks based
on CNN, K-means, Gaussian mixture model, and KNN have been presented [213,
225, 216]. Baesd on a multi-task learning approach and data sets for resource
optimization [112], a great performance improvement has been observed there
[113]. Fig. 8 shows a framework for AI-assisted spectrum sharing, where the DL
model is trained and tested offline with the historical radio frequency (RF)
traces, and the real-time RF traces are examined to determine whether the
concept drift has occurred, if not, they are fed to the well-trained model to
perform online predictions by signal detection and identification, otherwise
they are trained online or fine-tuned and then fed to the well-trained model.
Although DL-based spectrum sharing solutions can achieve a better
classification performance for ultra-high dimensional models, their
computational cost cannot be ignored. Li et al. [228] provided an SVM-based
spectrum sharing solution, which utilizes the user grouping method to reduce
the cooperation overhead and improve the detection performance. Several works
use RL-based techniques to control the duty cycle management (DCM) function
that is a part of the carrier sense adaptive transmission (CSAT) algorithm or
implement the almost blank sub-frame (ABS) allocation mechanism, thereby
improving the spectrum sharing efficiency for coexistence networks. For
instance, Q-learning is used for adjusting the allocation of LTE subframes in
the CSAT duty cycles [229], and it is also used for carrier selection of LTE-
LAA systems coexisting with Wi-Fi systems [230]. Besides, the RL-based
spectrum sharing between LTE-LAA and Wi-Fi systems is explored to improve
coexistence fairness [231]. Similarly, the spectrum sharing between active and
passive users is explored to alleviate spectrum congestion [232].

Figure 9: Spectrum interference classification.

###  III-C Spectrum Interference

Spectrum interference or spectrum overlap has a strong impact on sensing
performance, and thus spectrum interference mitigation (SIM) becomes a
necessary step in sensing processing pipelines to decide how the spectrum
should be used. In the context of spectrum sharing, SIM places the burden of
interference management on the users, which must be combined for a balanced
approach to spectrum sharing.

In general, spectrum interference is divided into three types as shown in Fig.
9, i.e., transient bursts generated by impulse-like waveforms and narrowband
or wideband. Spectrum interference can be caused by sources transmitting in
the band of interest or by out-of-band emissions, which can lead to the
following problems[43].

  * •

Introduces identifiable distortions in the gain or frequency response of the
received waveform.

  * •

Obscures the received signal with a strong and identifiable emission
throughout the observed spectrum.

  * •

Introduces more subtle alterations in the received signals (e.g., when the
interference power is close to the noise threshold) that are more difficult to
detect and are therefore filtered out.

Figure 10: AI-empowered spectrum interference management. TABLE V: Summary of works on spectrum sensing for AI-empowered MA Ref. |  Focuses |  ML category |  ML approach |  Applications |  Improvements |  Scenarios
---|---|---|---|---|---|---
[213] |  Spectrum sharing |  DL |  CNN |  For cooperative spectrum sensing |  Delay |  CRN
[214] |  Spectrum sensing |  DL |  DNN |  For signal detection without prior knowledge |  Spectrum sensing efficiency |  CRN
[215] |  Spectrum sensing |  DL |  CNN |  For spectrum monitoring without complex multi-stage ML |  Sensing accuracy |  IoT
[218] |  Spectrum sensing |  RL |  DNN |  For dynamic spectrum management |  Spectral efficiency |  Intelligent IoT
[216] |  Spectrum sensing |  RL |  DQN |  For dynamic spectrum access without accurate CSI |  Achievable data rate and PU protections |  CRN
[226] |  Spectrum sensing |  DL |  CNN, LSTM |  For spectrum sensing detection |  Detection probability |  CRN
[223] |  Spectrum sensing |  SL |  DNN |  For exploiting the inherent characteristics of the radio frequency spectrum and help the RIS controller make a decision |  Energy efficiency |  RIS, 6G
[228] |  Spectrum sensing |  SL |  SVM |  For cooperative spectrum sensing with user grouping |  Sensing efficiency, security |  CRN
[221] |  Spectrum sharing |  SL |  DNN |  For spectrum detection in the unlicensed band |  Spectrum utilization |  IoT, Wi-Fi
[229] |  Spectrum sharing |  RL |  Q-learning |  For spectrum sharing between LTE and Wi-Fi in the unlicensed spectrum |  Spectral efficiency |  LTE-U, Wi-Fi
[230] |  Spectrum sharing |  RL |  Double Q-learning |  For spectrum sharing between LTE-LAA and Wi-Fi in the unlicensed spectrum |  Spectral efficiency |  LTE-LAA, Wi-Fi
[231] |  Spectrum sharing |  RL |  MABL |  For spectrum sharing between LTE-LAA and Wi-Fi with cooperative learning |  Access fairness |  LTE-LAA, Wi-Fi
[233] |  Spectrum interference |  DL |  ANN |  For spectral interference correction |  Prediction error |  WN
[234] |  Spectrum interference |  DL |  CNN |  For automatic modulation classification with two-level data augmentation |  Classification accuracy |  CRN
[235] |  Spectrum interference |  DL |  DNN |  For replacing the interference cancellation blocks of SIC to facilitate high-integrity detection |  SIC performance |  WNs
[224] |  Spectrum interference |  DL |  CNN |  For help the RIS controllers infering the interference signals from the incident signals |  SINR |  RIS
[222] |  Spectrum interference |  SL |  CNN |  For sensing the spectrum to avoid access collisions dynamically |  Access efficiency |  Wi-Fi
[236] |  Spectrum interference |  RL |  LSTM |  For dynamically avoiding mutual interference among automotive radars |  Success rate |  V2X
[237] |  Spectrum interference |  DRL |  CNN |  For cooperative spectrum sensing to decrease the signalling |  Spectrum utilization |  CRN
[238] |  Spectrum interference |  RL |  MABL |  For predicting the idle probability of each channel and selecting idle channels |  Spectrum access probability |  Industry IOT

SIM solutions involve time, frequency, or spatial filtering, which are based
on specific characteristics of the sensed signals. In the time and frequency
domains, spectrum interference can be detected and mitigated if it is confined
to specific times or sub-bands. Recently, comprehensive SIM technologies based
on DL (e.g.,[215, 239, 240, 234, 241, 233, 242, 226, 235]) and RL (e.g.,[236,
238, 237]) have been investigated to promote spectral efficiency, using DL
techniques for spectrum monitoring and signal representation[215, 240, 233],
the modulation classification [240, 241, 234], the interference identification
[242, 239], and SIC [235]. In particular, Kulin et al. [215] explore the end-
to-end learning-based CNN model for spectrum monitoring and signal
representations (i.e., temporal IQ data, amplitude/phase, and frequency domain
representations that affect the sensing accuracy.). Similarly, Tekbiyik et al.
[239] propose a CNN-based spectral correlation function (SCF) to achieve
better SIM in cellular bands without a priori information. Li et al. [233]
investigate ANN-based DL for SIM, to provide more accurate determinations.
Rajendran et al. [240] employ LSTM to learn good representations of temporal
IQ data, amplitude/phase, and frequency domain representations, aiming to
perform automatic modulation classification and achieve SIM. Shi et al. [241]
present a DL-based modulation classification for dynamic channel access and
SIM in CR networks, thereby increasing the throughput of secondary users and
the success probability of primary users. Zheng et al. [234] use radio signal
augmentation in DL to promote modulation classification and SIM. Zhang et al.
[242] train samples from a 10 MHz band in the 2.4 GHz ISM band and analyze the
classification accuracy considering CNN, ResNet, CLDNN, and LSTM, and further
reduce the training time by band, SNR, and sample selection while maintaining
the classification accuracy. Van et al. [235] introduce a DL-based SIC
detector to replace the interference cancellation blocks and detect the
superimposed symbols for the downlink transmissions of non-orthogonal systems.

Fig. 10 shows a typical intelligent SIM method to avoid spectrum interference
and improve medium access efficiency [221, 222]. A DL algorithm is used to
achieve SIM based on the classification probability, i.e., by extracting the
I/Q sequences and inputting the I/Q data to the neural network-based
predictor, the AP predicts the number of devices from the overlapped signal
according to the prediction probability and classifies these devices, so as to
avoid spectrum interference. Based on the proposed intelligent spectrum
learning, Yang et al. [223, 224] further explore the interference mitigation
of RIS-assisted communications, where the trained DL model helps RISs to infer
the interfering signals directly from the incident signals, and then
configures RISs to avoid interference.

Different from DL-based SIM techniques, RL-based SIM techniques are adaptive
to the dynamic environment. For example, Liu et al. [236] use RL to make a
decision in a dynamic environment, to achieve frequency sub-band selection
based on the present and past observations. To maximize the spectrum
utilization for dynamic CR networks, Sarikhani et al. [237] propose a DRL-
based cooperative spectrum sensing method to reduce secondary user signalling
and perform SIM. Considering the cognitive industrial system, Liu et al. [238]
also use DL with Bayesian fusion to dynamically find idle channels while
avoiding spectrum interference. Furthermore, by setting double thresholds and
weighted energy detection, a better SIM can be achieved in comparison with
cooperative spectrum sensing and energy detection.

###  III-D Discussions

The developed intelligent spectrum learning technologies for MA protocols have
become a popular research area that has attracted much attention from academia
and industry. However, AI-empowered spectrum sensing, spectrum sharing, and
SIM still face significant challenges.

  1. 1.

Security and Privacy: While most of the presented AI-empowered spectrum
sensing, spectrum sharing, and SIM for MA functions are centralized, the
distributed operation approaches are more suitable for random traffic request
scenarios. Moreover, there exist potential security and privacy risks
associated with centralized operation, such as sensitive user data being
captured at the central controller. Therefore, decentralized and distributed
intelligent spectrum learning is encouraged to provide security and privacy,
especially the information interaction and sharing are required for multiple
agents.

  2. 2.

Heterogeneity: Intelligent spectrum sharing may face problems of heterogeneity
in the underlying technologies, such as different MAC and PHY operations,
which are usually managed by separate operators and require either common
solutions or coordination between different standardization and technology
development forums. In this case, fair sharing issues are therefore still an
open challenge.

  3. 3.

Scalability: Intelligent spectrum learning mostly focuses on the lower
frequencies (i.e., below 24 GHz) and active users, so the approaches involved
are difficult to apply directly to the ultra-high frequencies (i.e., above 100
GHz) or passive users. Moreover, the combination with other technologies such
as THz, RISs, and Massive MIMO complicates the implementation of intelligent
spectrum sensing, sharing, and SIM.

  4. 4.

Power Cost: The cooperative sensing of multiple devices, the combination of
different technologies, and the dynamically changing wireless environment
result in particularly high power consumption. Therefore, the energy
efficiency of intelligent spectrum learning is also a critical challenge.

Figure 11: The general frame structure of AI-empowered MA. Figure 12: Three
types of AI-empowered MA: centralized, distributed, and hybrid.

##  IV Protocol Design for AI-Empowered MA

With the efficiency and accuracy of AI-empowered spectrum sensing, the
performance of MA protocols can be further enhanced by AI techniques in each
of the three dimensions: collision reduction, throughput improvement, and
adaptivity enhancement. Moreover, AI-enabled structure-based designs can
significantly enhance the above benefits without introducing additional
reliability losses. In addition, AI technologies are expected to be combined
with HBF, RIS, or MIMO in MA for 6G ubiquitous intelligent services. In this
section, the related designs at the MAC layer include the following three
structure-based aspects: frame, protocol, and implementation designs are
discussed and summarized. Specifically, the AI-enabled MA frame design is
presented in Section IV-A, the AI-enabled MA scheme design is shown in Section
IV-B, the AI-enabled MA implementation is depicted in Section IV-C, and their
discussions are given in Section IV-D.

###  IV-A Frame Design

The frame structure is important in the protocol design of AI-empowered MA,
and the ability to achieve high MA efficiency is intrinsically related to the
designed frame structure. For example, the frame structure design aims to
ensure ultra-reliability, fast processing, and wide bandwidth is recommended
in [243], where a highly flexible frame structure supports resource
multiplexing of scheduled users. Pedersen et al. [244] propose a flexible
frame structure that integrates frequency division duplexing (FDD) and time
division duplexing (TDD). The frame structure allows users to multiplex with
dynamically adjusted frame sizes, i.e., the frame size can be dynamically
adjusted for each user’s scheduling. Pedersen et al. [245] further explore
their flexible frame structure with a dynamically changing frame size. Jiang
et al. [246] propose a general frame structure that includes general frames,
large frames, and superframes, where the MAC operation is repeated with either
a fixed format or a random interval. Liu et al. [247] design a superframe
structure, where the slot selection algorithm is based on the QL model.

A typical frame structure of various MA protocols is shown in Fig. 11, which
is based on frame division, where each frame is an operation cycle, i.e., a
repeated time epoch with either a fixed or a random interval, it is
responsible for the confirmation of the MA scheme (e.g., TDMA, CDMA, FDMA,
NOMA, RA, etc.) and access transmission, where the MA scheme for accessing
channels in an operation cycle can be confirmed using AI techniques, and the
then successive access transmission with specific access units depends mainly
on the confirmed MA scheme, which can also be modified using AI techniques.
The access unit used to share the medium is determined not only by the MA
scheme but also by the multiplexing solutions in the PHY layer. Each MA scheme
in a frame has different running sequences, exchanged information, capacity,
and the number of available access units, which vary according to the protocol
design and the function of different MA. Following this general frame
structure, the protocol and implementation designs of various AI-empowered MA
for their performance enhancement are presented below.

TABLE VI: AI-empowered centralized protocols: OMA  Ref. |  Protocols |  Mechanisms |  Applications |  Improvements
---|---|---|---|---
[56] |  TDMA |  RL |  For operating efficiently under bursty and correlated traffic |  Access efficiency
[57] |  TDMA |  RL |  For the selection of stations in one slot |  Real time and spectral efficiency
[58] |  TDMA |  RL |  For the selection of transmission in each slot |  Delay and network performance
[69] |  TDMA |  RL |  For adapting MAC parameters with the incoming traffic, buffer, and channel conditions |  Energy efficiency and latency
[171] |  TDMA |  RL |  For optimizing resource utilization |  Spectral/energy efficiency
[248] |  TDMA |  DRL |  For data transmission and task calculation |  Delay, data rate, and energy consumption
[249] |  Grant-free MA |  DRL |  For mitigating potential pilot sequence collisions |  Throughput
[72] |  OFDMA |  RL |  For multi-channel allocation strategy |  Throughput
[67] |  OFDMA |  RL |  For frequency resource allocation |  Spectral efficiency and inter-cell interference suppression
[74] |  OFDMA |  RL |  For solving the uncertainty of Nash equilibrium |  Layer interference and spectral efficiency
[66] |  OFDMA |  DL |  For determining control actions |  Delay and energy consumption
[180] |  OFDMA |  DL |  For channel estimation-equalization and carrier frequency offset (CFO) estimation |  Spectral efficiency
[181] |  OFDMA |  DL |  For the two tasks-user scheduling and beamforming |  Delay and total rate
[174] |  OFDMA |  DRL |  For resource optimization |  Overall rate performance
[175] |  OFDMA |  DRL |  For beamforming coordination, power and carrier allocation |  Sum data rate
[176] |  OFDMA |  DRL |  For allocating time slots, subcarriers, and modulation formats dynamically |  Delay and energy efficiency
[191] |  FL-MAC |  FL |  For MAC privacy and security |  Convergence speed and model accuracy
[190] |  GBMA |  FL |  For solving the distributed access of MAC |  Delay and access efficiency
[188] |  OA-FMTL MAC |  FL |  For solving inter-task interference |  Computational complexity

###  IV-B Scheme Design

AI-empowered MAC protocol designs for wireless environments have received
considerable attention from academics, consisting of centralized, distributed,
and hybrid modes, as shown in Fig. 12.

####  IV-B1 Centralized Schemes

AI-empowered centralized MAC protocols allow an intelligent central controller
to coordinate multiple user access and eliminate collisions using ML
techniques (such as DL, RL, etc.). Typical centralized-based AI-enabled MAC
protocols include OMA (e.g., FDMA, TDMA, CDMA, and OFDMA) and NOMA schemes. In
these schemes, multiple users are scheduled by an intelligent central
controller to access the channel over a fixed number of resources (e.g.,
times, frequencies, codes, and power domains). Although these protocols can
avoid collisions by pre-allocating resources, they are not very efficient at
low loads and lack flexibility under dynamic network conditions. In
particular, TDMA-type protocols have strict time synchronization requirements
and result in additional bandwidth and power consumption. In addition, access
with a large delay is a concern for delay-sensitive applications. CDMA-type
protocols require tight power control to overcome the near-far problem at the
receiver due to multiple access interference. Power control imposes
computational and hardware requirements that increase the overall system cost.
FDMA-type protocols require additional circuitry to communicate and switch
between different radio channels at a high cost, and the strict linearity
requirement on the medium limits their practicality.

A large number of works applying ML techniques to the design of centralized-
based OMA protocols have been studied [248, 191, 58, 190, 57, 56, 171, 69,
188, 72, 174, 175, 74, 66, 176, 180, 181, 67]. To improve spectral efficiency
and reduce delay, a learning automata algorithm is applied to TDMA for station
or transmission selection in each slot [56, 57, 58]. To improve resource
utilization, a collaborative multi-agent RL framework for TDMA in wireless
sensor networks is proposed [69], and an RL is used for node scheduling in
TDMA [171], DRL is used in IoT networks to support grant-free MA systems, thus
mitigating potential pilot sequence collisions without requiring additional
information exchange [249], and DRL is also combined with TDMA to integrate
communication and computing [248]. Some works focus on improving OFDMA
performance using different learning methods [72, 74, 66, 174, 175, 176, 180,
181, 67]. For example, attachment learning for multi-channel allocation is
used to improve throughput [72], utility-based learning for solving Nash
equilibrium uncertainty is used to reduce layer interference [74], stochastic
learning for determining control actions is used to reduce delay and energy
consumption [66], DL for channel estimation and user scheduling to improve
spectral efficiency [180, 181], and RL or DRL for resource allocation and
beamforming coordination is used to improve sum rate [174, 175, 176, 67]. To
provide privacy and security for MAC, FL combined with MAC protocol design is
investigated, such as FL-MAC [191], GMBA [190], and [188]. Table VI compares
the aforementioned AI-empowered OMA protocols in terms of their ML mechanisms,
applications, and improvements.

TABLE VII: The comparison of various NOMA, where L, M, and H refer to low, medium, and high, respectively; Sep. and Int. refer to separated and integrated respectively; SC and MC refer to single-carrier and multiple-carrier respectively; SIC, IMD, and MPA refer to successive interference cancellation, iterative multiuser detection, and message passing algorithm, respectively. Paradigm |  Flexibility |  Complexity |  Benefits | Carrier | Modulation | Decoding
---|---|---|---|---|---|---
|  |  SC |  MC |  Sep. |  Int. |  SIC |  IMD |  MPA
NOMA |  PD-NOMA |  H |  L |  Spectral efficiency |  ✓ |  |  ✓ |  |  ✓ |  |
IDMA |  M |  L |  Diversity |  ✓ |  |  ✓ |  |  |  ✓ |
LDS-CDMA |  H |  M |  Without CSI |  ✓ |  |  ✓ |  |  |  |  ✓
LDS-OFDM |  L |  M |  Wideband signals |  |  ✓ |  |  ✓ |  |  |  ✓
SCMA |  L |  H |  Diversity |  |  ✓ |  |  ✓ |  |  |  ✓
PDMA |  M |  H |  Diversity |  |  ✓ |  ✓ |  |  |  |  ✓
RSMA |  H |  L |  Spectral efficiency |  |  ✓ |  |  ✓ |  ✓ |  |
TABLE VIII: AI-empowered NOMA protocols NOMA category |  Ref. |  Protocols |  Mechanisms |  Applications |  Improvements
---|---|---|---|---|---
|  [182] |  MIMO-NOMA |  DL |  For analyzing the channel state information (CSI) and transmission detection |  Detection efficiency, energy consumption
[183] |  MIMO-NOMA-OMA |  DL |  For optimizing multi-dimensional resource allocation |  Delay, data rate
[179] |  PDMA |  DRL |  For solving optimization problem |  Energy consumption
[186] |  HMAS |  DL |  For signal detection in a high overloaded multiuser system |  Symbol error rate
[189] |  AGMA |  Gradient learning |  For solving distributed access problem |  Convergence rate
PD-NOMA |  [250] |  Real-time NOMA |  DRL |  For scheduling multiple transmissions |  Scalability, delay, energy consumption, throughput
[251] |  Multi-carrier NOMA |  DL |  For transmit power optimizing |  Energy consumption, data rate
[252] |  Downlink NOMA |  DRL |  For power allocation |  Energy consumption, data rate
[253] |  NOMA-BF |  DRL |  For investigating RIS phase shifts design and optimization |  Energy consumption, sum rate
[254] |  NOMA-CRS |  Lightweight ML |  For the sharing-power allocation |  Bit error probability, reliability, energy consumption
[255] |  NOMA-VLC |  DL |  For signal demodulator |  Scalability, data rate
[256] |  Downlink NOMA |  Anderson-Darling ML |  For detecting the modulation order of interference signals in downlink NOMA system |  Detection accuracy
[257] |  Downlink NOMA |  DRL |  For solving resource allocation in multi-antenna downlink NOMA systems |  Spectral efficiency, sum rate, energy consumption
[258] |  Uplink NOMA |  DRL |  For resource allocation in uplink NOMA-URLLC system |  Energy consumption, delay
[259] |  MISO NOMA |  DRL |  For channel estimation |  Sum-rate, estimation accuracy, reliability
[260] |  FTN-NOMA |  DL |  For sliding-window detection |  Detection accuracy, spectral efficiency, delay
[261] |  Cooperative NOMA |  DL |  For classifying channels of imperfect CSI and improving the accuracy of CSI |  Spectral efficiency, CSI accuracy
[262] |  NOMA-SWIPT |  DRL |  For efficient data compression in an uplink NOMA communication system |  Energy consumption,compression efficiency
[263] |  Random NOMA |  DRL |  For allocating RA slots |  RA slots allocation efficiency
[193] |  RSMA |  FL |  For uplink transmission of edge devices |  Delay and energy consumption
[184] |  RSMA |  DL |  For calculating the precoder vector and obtaining the common rate variable |  Computational complexity and energy consumption
[185] |  RSMA |  DL |  For improving the robustness against CSI defects |  Spectral efficiency and overhead
[178] |  RSMA |  DRL |  For power allocation |  Energy consumption
[192] |  RSMA |  FL |  For solving a multi optimization problem |  Delay, reliability
[172] |  RSMA |  RL |  For user clustering of fog radio access networks |  Average sum-rate
[264] |  SDMA |  DL |  For mobile stations clustering and scheduling |  Computational complexity
[265] |  SDMA |  RL |  For beam tracking and switching in multiple UAVs systems |  Transmission efficiency
[266] |  SCMA |  DL |  For designing end-to-end autoencoder |  Bit error rate, computational complexity
[267] |  SCMA |  DL |  For making the decoding strategy |  Bit error rate, computational complexity
[268] |  SCMA |  DL |  For multiuser SCMA detection |  Computing complexity, delay
CD-NOMA |  [269] |  SCMA |  DRL |  For adaptively constructing codebook |  Throughput, reliability, delay
[270] |  SCMA |  DL |  For designing encoder and decoder |  Decoding delay, decoding efficiency
[271] |  SCMA |  DL |  For incorporating prior knowledge into SCMA detection |  Delay, data rate
[272] |  SCMA |  DL |  For jointly designing the downlink SCMA encoder and decoder |  Error rate, computational complexity
[273] |  SCMA |  DRL |  For selecting the subframe and codebook |  Throughput
[274] |  SCMA |  DL |  For cross-physical-and-application-layer codebook allocation |  Delay, scalability, data rate
[275] |  C-NOMA |  DL |  For cross-physical-and-application-layer codebook allocation |  Data rate
[276] |  CD-NOMA |  DL |  For jointly optimizing resource mapping and codebook |  Bit error rate

Additionally, with the development of various NOMA technologies (as shown in
Table VII), extensive research has been conducted on AI-empowered NOMA to
further improve NOMA performance, in particular to reduce the complexity and
increase the accuracy of SIC at NOMA receivers. ln conclusion, NOMA
performance can be improved by applying AI technologies in to channel
estimation, interference mitigation, detection or modulation, resource
optimization, and signal processing [50, 93, 277, 87, 278, 279]. For example,
most AI solutions for various PD-NOMA designs include DL (e.g., [182, 183,
184, 185, 186, 251, 255, 260, 261]), DRL (e.g., [178, 179, 250, 252, 253, 257,
258, 259, 262, 263]), and FL (e.g., [193, 192]). In particular, DL is used in
MIMO NOMA for CSI [182] and resource allocation [183], in RSMA for the
precoder vector [184] and robustness [185], in HMAS for signal detection
[186], in multi-carrier NOMA for power optimizing [255], in NOMA-VLC for
signal demodulation [255], in FTN-NOMA for sliding window detection [260], and
in cooperative NOMA for channel classification [261]. DRL is used in RSMA and
PDMA for resource allocation and optimization [178, 179], in up/downlink NOMA
system for transmission scheduling [250], resource allocation [252, 257, 258],
beamforming [253], channel estimation [259], and data compression [262], and
in random NOMA for slot allocation[263]. FL is used in RSMA for optimizing
[192] and edge transmission [193]. On the other hand, some works aim at
implementing DL (e.g., [266, 267, 268, 270, 271, 272, 274, 275, 276]) and DRL
(e.g., [269, 273]) approaches on CD-NOMA. For example, DL is used in SCMA for
encoder and decoder [266, 267, 270, 272], detection [268, 271], and codebook
allocation [274], and also in C-NOMA or CD-NOMA for codebook allocation [275,
276]. DRL is used in SCMA for codebook construction and selection [269, 273].
ML algorithms are used to address MA optimization in SAGIN [278, 279]. Table
VIII summarizes the typical AI-empowered NOMA in terms of their ML mechanisms,
applications, and improvements.

####  IV-B2 Distributed Schemes

In contrast to the centralized schemes, AI-empowered distributed MA protocols
allow each intelligent user to compete for multiple accesses and avoid
collisions. In this case, scheduling by the BS or AP is not required, and each
intelligent user accesses the channel using DL or RL approaches in CSMA-based
schemes. Typical RA protocols are the IEEE 802.11 channel access mechanism
known as carrier sense (i.e., DCF) and some derivative schemes based on
CSMA/CA. Such protocols are slotted, and the busy/idle state of channels is
determined by carrier sensing; they perform well in a small network and can
eliminate collisions to some extent while satisfying users’ RA requirements.
However, as user density increases, their delay and throughput degrade
rapidly, especially as the load approaches saturation. They also fail to scale
as the network expands, wasting additional energy may occur due to collisions,
idle listening, and information exchange overheads.

Different from the centralized MAC protocols, there are many AI solutions for
the IEEE 802.11 MAC layer to adaptively design the internal parameters of the
core Wi-Fi functions. Typically, these AI solutions for distributed MAC
protocols include CW value selection [107, 111, 105, 101, 102, 108], backoff
value selection [103, 109, 104, 106], time slot selection [280, 281, 282],
frame size selection [159, 160, 161, 162, 283], transmission rate selection
[162, 284], and link parameter configuration [285, 110, 163, 286]. In
particular, RL-based and DRL-based approaches are generally adopted to
conceive distributed MA protocols and set channel access parameters, and SL-
based approaches are used to predict CSI. In the proposed RL and SL models,
loss functions and rewards are addressed in terms of reduced collisions [107,
105], the increased difference between successful and collided frames [103],
improved channel utilization [101], increased successful channel access
attempts [102, 109], throughput [108], network utility [285], and a
combination of improved throughput, reduced energy, and reduced number of
collisions [104]. We provide a summary of the most important ones in Table IX,
while an illustrative example of using RL to optimize MAC protocol design is
given in Fig. 13, where an agent learns a model based on the current MAC
building blocks and network configuration to evaluate how the environment
affects the MA protocol design from their interaction. The agent then uses the
learned model to adaptively design an MA protocol that includes the selection
of CW/backoff values, time slots, frame sizes, and transmission rates.

Figure 13: MA protocol design with RL. TABLE IX: Distributed AI-empowered MA protocols  Ref. |  Protocols |  Mechanisms |  Applications |  Improvements
---|---|---|---|---
[107] |  IEEE 802.11ax |  DRL |  For CW selection |  Throughput, delay, and fairness
[105] |  IEEE 802.11 standard |  RL |  For minimum CW parameter selection |  Network-level utility
[103] |  IEEE 802.11e |  RL |  For backoff value selection |  Flexibility and overhead
[101] |  IEEE 802.11 standard |  SL |  For the minimum CW value selection |  Fairness
[102] |  IEEE 802.11n |  SL |  For CW value selection |  Average throughput, delay, and fairness
[109] |  IEEE 802.11 standard |  DRL |  For back-off value selection |  Throughput and fairness
[108] |  IEEE 802.11ax |  DRL |  For CW values selection |  Efficiency and computational cost
[285] |  IEEE 802.11 standard |  DRL |  For interference patterns and configurations |  Delay
[104] |  IEEE 802.11 standard |  RL |  For back-off value selection |  Throughput and convergence speed
[110] |  IEEE 802.11ax |  FL |  For channel access parameter selection |  Throughput
[111] |  IEEE 802.11 standard |  SL |  For AIFS and CW selection |  Throughput and QoS
[159] |  IEEE 802.11 standard |  SL |  For frame-size selection |  Effectiveness and reaction speed
[160] |  IEEE 802.11 standard |  SL |  For frame size selection |  Goodput
[161] |  IEEE 802.11 standard |  SL |  For frame size selection |  Aggregated network throughput
[162] |  IEEE 802.11 standard |  SL |  For frame size and transmission rate selection |  Throughput
[163] |  IEEE 802.11ac |  DL |  For solving the backhaul |  High rate and low delay
[280] |  IEEE 802.11 standard |  RL |  For backoff value selection |  Collisions and network performance
[106] |  IEEE 802.11 standard |  RL |  For backoff value selection |  QoS
[283] |  IEEE 802.11ac |  RL |  For frame size selection |  Energy efficiency
[284] |  IEEE 802.11ac |  DRL |  For experience-driven rate adaptation |  Overhead
[286] |  IEEE 802.11ac |  SL |  For link configuration parameter selection |  Link layer performance
[282] |  IEEE 802.11 standard |  RL |  For time slot selection |  Delay, robust, and efficiency
[281] |  IEEE 802.11be |  RL |  For time slot selection |  Interference and sum throughputs
[33] |  IEEE 802.11bn |  ML |  Resource allocation |  Throughput, delay

####  IV-B3 Hybrid Schemes

AI-empowered distributed MA protocols easily adapt to dynamic networks and are
more suitable for small and low load networks. AI-empowered centralized MA
protocols eliminate collisions and achieve high channel utilization at higher
loads. AI-empowered hybrid MAC protocols combine and balance aspects of both,
where users can be scheduled to access the channel by an intelligent central
controller, and the intelligent users can also access the channel via the
contention scheme. Switching between RA and scheduled access effectively
reduces collisions of RA protocols at high loads and improves channel
utilization of scheduled access at low loads. However, collisions during the
random access, which relies on slot/code/frequency/power resource reservation
of hybrid protocols, become the bottleneck preventing the network from
achieving high utilization. In addition, dynamic switching usually results in
wasted slots, frequencies, and additional overhead.

In general, existing hybrid MA protocols combine CSMA with TDMA, FDMA, and
CDMA and can achieve dynamic switching between two modes, these protocols
behave as the distributed access (e.g., CSMA) at low contention levels and
switch to the centralized access (e.g., TDMA, FDMA, or CDMA) at high
contention levels [287, 288, 289], or they allocate slots and codes according
to CSMA-based bandwidth requests[290]. To overcome the above bottleneck and
further enhance the benefits of both, various AI techniques are used to
achieve the switching between the distributed mode and the centralized mode,
thus satisfying access flexibility while maintaining high access efficiency.
For example, Qiao et al. [291] propose an intelligent MAC switching scheme to
support large and dense networks with low computational complexity, where the
contention MAC protocol combined with contention-free MAC protocol is
presented, the dynamic switching of both based on ML techniques can adapt to
the changing network state. Yang et al. [292] design a scalable intelligent
MAC protocol that combines the contention access and the scheduling access to
improve the system throughput, where the time length of the contention access
in each superframe is selected based on the learned network states. Yang et
al. [221] further optimize the proposed scalable intelligent MAC protocol for
coexisting networks including Wi-Fi devices and IoT devices. Cao et al. [293]
investigate the coexisting networks consisting of Wi-Fi devices and
backscatter devices, and propose a hybrid MAC protocol that combines the
contention access with the reservation access using DRL. In addition, Cao et
al. [137] develop an AI-assisted hybrid MAC protocol for RIS-based multi-user
communication systems, where AI techniques are used to jointly design
centralized access and distributed access in a superframe. Gomes et al. [294]
use RL techniques to switch the MAC protocol between the contention scheme and
the reservation scheme based on the current network demand. In addition,
several hybrid MAC protocols using RL techniques are proposed to achieve the
scheme selection between OMA and NOMA to adapt to the different requirements
[186, 295].

###  IV-C Implementation Design

Referring to these discussed intelligent MA scheme designs [137], a typical
implementation design of AI-empowered centralized, distributed, and hybrid MA
framework is shown in Fig. 14.

In the proposed AI-empowered centralized MA framework, the center controller
(e.g., BS or AP) tightly coordinates the multiple access of users. Explicitly,
the center controller schedules and allocates resources for users via AI
models, as detailed in the left part of Fig. 14, each subframe is divided into
three periods: the pilot period, the computation period, and the scheduled
transmission period. The pilot period and the scheduled transmission period
can be further divided into $K$ pilot slots and $J$ data slots, respectively.
The users transmit their data in the $J$ data slots on the $N$ non-overlapping
subchannels. Based on this frame structure, the AI-empowered centralized MA
protocol is designed with full consideration of channel access, computation,
and data transmission. The AI-empowered centralized MA protocol combines with
TDMA and FDMA, where each user follows the time division scheme in each
subchannel. In particular, after synchronization, each user initiates pilot
transmission to the central controller in dedicated pilot slots. During the
computation period, the central controller first estimates the CSI, followed
by time, frequency, and power resource allocation. The central controller then
configures the MA parameters and schedules the access of users requiring their
data transmissions. Given the excessive complexity of high-dimensional, full-
search-based centralized MA protocols, a DL-based computational model is
trained offline at the central controller and can be used to find a near-
optimal solution with reduced complexity. The input to the trained DL model
can be the number of users, the number of subframes, the number of channels,
and the channel information. Online inference, which shifts the complexity to
offline training, is performed at the central controller to determine the MA
parameters and the resource allocation strategy. More explicitly, these
related learning tasks share the same input parameters, so that learning
multiple related tasks together can improve the prediction accuracy and
generalization ability compared to learning them separately.

Figure 14: Implementation design.

In contrast to the centralized scheme, in the proposed AI-empowered MA
framework, each user configures multiple access and calculates the required
resources based on the dynamic network environment. In this case, no
assistance from the central controller is required. The users must negotiate
with the central controller for channel access and perform the subsequent data
transmission, as shown in the right part of Fig. 14. In particular, channel
sensing and computation are carried out at the user side via RL to determine
the MAC parameters and access the channel. The AI-empowered distributed MA
protocol combines CSMA and FDMA, where each user follows the IEEE 802.11 DCF
scheme in each channel. Specifically, a competing user senses the state of
each subchannel. Once a channel is sensed to be idle, the user contends for
access to the channel. Waiting for a DCF inter-frame space (DIFS) and backoff,
the user sends a request to the central controller. If the channel is
available to the user, after a short inter-frame spacing (SIFS) the central
controller sends its feedback to the user. After a SIFS, the user then
transmits the data to the central controller. Here, an RL-based calculation
model can be used by each user to set MA parameters (e.g., CW value,
backoff/DIFS value, frame size). The RL model includes the current MA
parameter setting and the current network configuration. Model actions include
updating the CW value, updating the backoff/DIFS value, and updating the frame
size. The reward function is determined by the user’s throughput requirement.
If the action taken by the user increases the throughput, the user will
receive a positive reward. Correspondingly, if the action taken by the user
reduces the throughput, the user will receive a negative reward.

TABLE X: The comparison of AI-empowered centralized, distributed, and hybrid MA schemes. Ref. | MA schemes | AI techniques | Applications
---|---|---|---
OMA | NOMA | RA | Switch | DL | RL | SL | FL | RIS | MIMO | THz | IoT | AdHoc | SAGIN
| [70] | ✓ |  |  |  |  | ✓ |  |  |  |  |  |  |  | ✓
[71] | ✓ |  |  |  |  | ✓ |  |  |  |  |  | ✓ |  |
[76] | ✓ |  |  |  |  | ✓ |  |  |  | ✓ |  |  |  |
Centralized | [180] | ✓ |  |  |  | ✓ |  |  |  |  |  |  | ✓ |  |
[181] | ✓ |  |  |  | ✓ |  |  |  |  |  | ✓ |  |  |
[82] |  | ✓ |  |  | ✓ |  |  |  |  |  |  | ✓ |  |
[84] |  | ✓ |  |  | ✓ |  |  |  |  | ✓ |  |  |  |
[85] |  | ✓ |  |  |  | ✓ |  |  |  |  |  | ✓ |  |
[87] |  | ✓ |  |  | ✓ | ✓ |  |  | ✓ |  |  |  |  |
[88] |  | ✓ |  |  |  |  |  | ✓ | ✓ |  |  | ✓ |  |
[90] |  | ✓ |  |  | ✓ |  |  |  | ✓ |  | ✓ |  |  |
[178] |  | ✓ |  |  | ✓ |  |  |  |  |  |  |  |  | ✓
[105] |  |  | ✓ |  |  | ✓ |  |  |  |  |  |  | ✓ |
[103] |  |  | ✓ |  |  | ✓ |  |  |  |  |  |  | ✓ |
Distributed | [101] |  |  | ✓ |  |  |  | ✓ |  |  |  |  |  | ✓ |
[102] |  |  | ✓ |  |  |  | ✓ |  |  |  |  |  | ✓ |
[109] |  |  | ✓ |  |  | ✓ |  | ✓ |  |  |  |  | ✓ |
[108] |  |  | ✓ |  |  | ✓ |  |  |  |  |  |  | ✓ |
[110] |  |  | ✓ |  |  | ✓ |  | ✓ |  |  |  |  | ✓ |
[221] |  |  | ✓ |  |  |  | ✓ |  |  |  |  | ✓ |  |
[137] | ✓ |  | ✓ | ✓ | ✓ | ✓ |  |  | ✓ |  |  | ✓ |  | ✓
Hybrid | [186] | ✓ | ✓ |  | ✓ | ✓ |  |  |  |  | ✓ |  |  |  |
[291] | ✓ |  | ✓ | ✓ |  |  | ✓ |  |  | ✓ |  |  | ✓ |
[292] | ✓ |  | ✓ | ✓ | ✓ |  |  |  |  |  |  | ✓ | ✓ |
[293] | ✓ |  | ✓ | ✓ |  | ✓ |  |  |  |  |  | ✓ | ✓ |
[294] | ✓ |  | ✓ | ✓ |  | ✓ |  |  |  |  |  | ✓ | ✓ |
[295] | ✓ | ✓ |  | ✓ |  | ✓ |  |  |  | ✓ |  | ✓ |  |

The AI-empowered hybrid MAC designs that alternately implement the centralized
and distributed modes in a single frame by controlling the weight of each are
detailed in Fig. 14, which includes the following three cases.

Case 1: As shown in Fig. 14, the scheduled and contention transmissions are
combined after the pilot transmissions and the computation, while allowing
users to switch between them to meet different QoS requirements. Each frame is
divided into four parts, namely the pilot period, the computation period, the
scheduled transmission period, and the contention transmission period.
According to the proposed design of the AI-empowered distributed MA schemes,
the scheduled users transmit their data to the central controller in the
scheduled transmission period. Then, based on the AI-empowered distributed MA
scheme design, the unscheduled users (i.e., the unserved users who have sent
their pilots and the new requesting users) transmit their data to the
receivers during the contention transmission period. By dynamically switching
between two transmission modes, this scheme is able to maintain the target
rate. Note that the AI-empowered hybrid MA scheme switches to the AI-empowered
centralized MAC scheme when the weight of w2 is zero, or switches to the AI-
empowered distributed MA scheme when the weight of w1 is zero.

Case 2: The contention requests and the scheduled transmissions are combined
into a single frame. Each frame consists of the contention request period, the
computation period, and the scheduled transmission period. The user sends a
request to the central controller when a subchannel becomes available during
the contention request period, and the central controller responds with an
acknowledgment. Based on the received requests, the central controller sends
the scheduling information to the users during the computation period. The
scheduled users then transmit their data to the central controller within the
scheduled transmission period.

Case 3: Similar to Case 2, the contention requests and the reserved data
transmissions are combined into a single frame, and the computation is done at
the user rather than at the central controller. When a subchannel becomes
idle, the user occupies the channel, then calculates the required resources
based on the RL model and sends a request to the central controller to reserve
the channel resources for future data transmissions. The central controller
sends feedback to the user once a request is registered. When the reserved
transmission period arrives, the user transmits the data to the receiver in
the reserved slots.

The comparison of the existing AI-empowered centralized, distributed, and
hybrid MA protocols in terms of designs, AI approaches, and applications is
summarized in Table X.

###  IV-D Discussions

The designed AI-empowered MA protocols have been extensively studied in many
works. However, the frame, protocol, and implementation designs combined with
AI techniques still face some challenges.

  1. 1.

The MA frame design allows nodes to access the channel using the same or a
different MA scheme, which may lead to priority or fairness issues. Therefore,
AI solutions for frame design (i.e., superframe or subframe) considering
priority or fairness becomes a challenge. In addition, the changing networks
and topologies can affect the frame design and lead to adaptive problems, and
dynamic frame designs also introduce additional overhead and power
consumption.

  2. 2.

Different MA protocols are designed to address different issues. For example,
although the AI-empowered centralized MA protocol avoids collisions through
scheduling, it does not fully address the complexity and overhead issues. On
the other hand, the advanced AI algorithms are important for the collision
avoidance of the distributed MA protocol, especially for the high load
requirements. Furthermore, the cooperation of different MA schemes brings new
complex problems. For example, adaptive switching between the two for
different network scenarios is required to take full account of the current
and historical network states, and how to use AI techniques to select the
optimal MAC scheme and reasonably set parameters for the selected MA scheme
are still open problems. In addition, the cross-layer AI-empowered MA protocol
design involving the PHY layer and network layer optimization is a challenge
for system performance improvement.

  3. 3.

The MA implementation designs mainly focus on a single network and ignore the
integration of different networks (e.g., SAGIN), so the flexible and
compatible MA implementation design for different networks is still a
challenge. For example, the characteristics of different networks, such as an
unstable channel, rapidly changing channel states, low power, and long
propagation delay need to be fully considered. Thus, the use of AI techniques
to learn these characteristics to reduce collisions and improve MA performance
remains to be solved. In addition, the structure-based designs require
significant changes to the system framework, and then face challenges in
hardware implementation and system compatibility.

Figure 15: Resource management for AI-empowered MA protocol optimization.

##  V Optimization for AI-Empowered MA

Compared to typical optimization algorithms, AI-based approaches to MA
protocol optimization are particularly efficient in reducing computational
complexity and improving resource utilization, especially when dealing with
complicated time-varying traffic or even unknown environments. Furthermore, AI
technologies combined with MEC or WPT are considered to ensure timeliness and
energy reduction for 6G systems. Therefore, the AI-enabled MA resource
management is presented in Section V-A, the AI-enabled MA parameter adjustment
is shown in Section V-B, the AI-enabled MA access switching is depicted in
Section V-C, and their discussions are given in Section V-D.

###  V-A Resource Management

Resource management is an important aspect that significantly affects MAC
performance and aims to achieve high spectrum and energy efficiency while
suppressing inter/intra-cell interference. Resource management at the MAC
layer is required to dynamically schedule users, allocate RBs and control
transmit power levels. Traditional resource management can be performed using
mathematical optimization theory, including convex optimization methods (e.g.,
interior-point method, Newton’s method, Lagrangian duality method, etc.) and
non-convex optimization methods (e.g., branch-and-bound (BnB), game theory,
successive convex approximation, etc.). However, the mathematical optimization
methods generally spend a lot of time and energy solving optimization problems
in time-varying and unpredictable network environments, and the results
obtained may be suboptimal. As shown in Fig. 15, AI algorithms are being
explored in resource management at the MAC layer for user association,
resource allocation, and power control, thereby reducing computational
complexity and achieving approximate real-time solutions.

####  V-A1 User Association

At the MAC layer, appropriate association schemes between users and the
central controller (i.e., user scheduling by the central controller) are
required for the designed MAC protocols. Considering the information related
to SINR, load balancing, QoS requirements, and dynamic channel environment,
some existing AI solutions are used to find the optimal user association
scheme. For example, Pervez et al. [296] propose a distributed user cell
association scheme. It uses a fuzzy Q-learning algorithm to learn the optimal
bias values that guide users to associate with preferred cells. Li et al.
[297] develop an online RL approach to solve the vehicle-BS association
problem in vehicular networks. Initial RL and history-based RL are defined. In
the initial RL phase, the BS decides the associated vehicles based on the
reward defined to minimize the data rate deviation of the vehicles. In the
history-based RL phase, the association schemes obtained in the initial RL
phase enable the BSs to achieve load balancing as the environment changes
dynamically. Challita et al. [298] propose a DRL method to solve the UAV-cell
association problem, thereby reducing the latency and interference of UAVs.
Mauricio et al. [264] use the K-means algorithm to divide mobile stations into
spatially compatible clusters, thereby supporting multiple spatial streams per
cluster. Raharya et al. [299] propose a parallel pursuit learning algorithm to
solve a joint optimization problem combining pilot assignment and multi-BS
association. Unlike pursuit learning approaches that use only a binary reward,
the proposed parallel pursuit learning algorithm allows a continuous reward to
provide an accurate estimate of spectral efficiency. Furthermore, by
introducing a heuristic solution in the pursuit-learning, the UE is associated
with a BS with high network spectral efficiency while maintaining low
complexity. Liu et al. [300] develop an SL-based user association approach to
support multi-connectivity in mmWave networks, where a graphical model is used
to represent the mmWave user-BS association, and this association can be
trained in a supervised manner with appropriate features obtained from both
geographic location and topological information. To satisfy the required
highly autonomous and decentralized nature, Dinh et al. [301] propose a
distributed user-AP association method to maximize the long-term total rate
under QoS constraints and AP load constraints, where the DRL algorithm is used
to help APs make an association decision based on their local network state
knowledge.

####  V-A2 Resource Allocation

Spectral efficiency and energy efficiency are major concerns in 6G systems and
therefore it is desirable to achieve highly efficient RB (i.e., time,
frequency, code, and power) allocation for the designed MAC protocols to
achieve these objectives. In this case, different AI techniques for RB
allocation are presented to optimize the designed MAC protocols. In
particular, the typical AI-based RB allocation for protocol optimization can
be divided into four categories: DL-based resource allocation [302, 302, 181,
139, 140, 303, 304, 305, 84, 306, 184, 185], RL-based RB allocation [183, 171,
177, 176, 175, 81, 307, 86, 179, 85, 258], FL-based RB allocation [89, 308,
309, 188], and other SL/USL-based RB allocation [310, 311], which are detailed
below.

DL techniques have been widely investigated in the protocol optimization of
various MAC schemes due to their lower computational complexity and signalling
overhead. For example, Yang et al. [302, 302] design a multi-task learning-
based feedforward neural network model for the OMA/NOMA-based MEC system. The
authors first use multi-task learning to address the joint optimization of
offloading strategy and computational resource allocation, and also verify the
effectiveness of the multi-task learning method in solving mixed-integer
nonlinear programming (MINLP). To reduce computational complexity and mitigate
multi-user interference, Jiang et al. [181] also use a multi-task learning
model to achieve optimal multi-user hybrid beamforming in mm-wave massive MIMO
OFDMA systems. Cao et al. [139] propose a RIS-aided OMA protocol, and use a
DNN-based multi-task learning model to predict the optimal transmission
strategy of the proposed protocol in real-time, Cao et al. [140] further
investigate the RIS resource optimization of the RIS-aided MAC protocol
relying on various AI techniques. In addition, most DL work has investigated
resource optimization in NOMA systems. Lei et al. [303] adopt the DL model to
tackle the NOMA resource allocation problems with high computational
efficiency. Yang et al. [304] use a DNN-based method for NOMA systems to
achieve real-time resource optimization. Zhang et al. [305] use the DL model
to deal with user association, subchannel, and power allocation in NOMA
systems, thereby achieving higher energy efficiency and lower complexity.
Huang et al. [84] propose a DNN algorithm for MIMO-NOMA networks to address
the dynamic power allocation problem. Ali et al. [306] develop a DNN method
for joint uplink-downlink NOMA optimization, where the DNN model is used to
solve joint power loading problems at source and relay nodes, thereby
maximizing the sum rate. As a further step, Camana et al. [184] apply the DNN
model in RSMA to provide the optimal precoder vector with lower computational
complexity. To address dynamic mobility issues, Gao et al. [312] propose a DL-
based algorithm for beam management and optimization in mmWave NOMA
communications. To achieve accurate high-dimensional CSI and reduce signalling
overhead in RIS-aided THz massive MIMO systems, Wu et al. [185] propose a
transformer-based data-driven RIS reflecting network to realize passive
precoding at the RIS, and further design a model-driven deep unfolding active
precoding network for RSMA digital active precoding at the BS.

To address resource allocation problems in a dynamic or uncertain environment,
RL models are used to allocate RB efficiently from their historical
experience. Considering a time-division-duplex (TDD)-based multi-dimensional
intelligent multiple access, Liu et al. [183] use a deterministic policy
gradient algorithm, a model-free DRL technique, to optimize the multi-
dimensional radio resource allocation in real-time with faster convergence
speed. Sah et al. [171] develop an RL algorithm to provide an optimal
scheduling policy in TDMA systems, and use Q-learning to find the optimal cost
to improve throughput and reduce latency. To improve spectral utilization, Li
et al. [177] use a DRL algorithm to learn the best access strategy in FDMA and
NOMA systems. Zhu et al. [176] propose a DRL algorithm to support efficient
bandwidth allocation in OFDMA passive optical networks, where slots,
subcarriers, and modulation formats are dynamically allocated using the DRL
model. Wang et al. [175] design a multi-agent deep Q-network algorithm for
beamforming coordination and resource allocation in multi-cell MISO-OFDMA
systems, where the proposed algorithm enables the new agents to predict the
resource allocation based on their experience and the learning results of the
pre-trained agents. Besides the above conventional OMA schemes, extensive RL
approaches have been studied for resource allocation in NOMA systems. To fully
exploit the advantages of NOMA techniques, He et al. [81] propose an
attention-based neural network (ANN)-based DRL algorithm to optimally allocate
power and channel resources in the multi-carrier NOMA system. Considering the
continuity of resource allocation, Shi et al. [307] propose a deep
deterministic policy gradient (DDPG) algorithm for multi-dimensional resource
allocation in NOMA communication. To reduce energy consumption and computation
delay, Li et al. [86] use the mean-field DDPG algorithm to achieve power and
computation resource allocation in NOMA-based MEC systems. Li et al. [179]
also propose a DQN-based constrained Markov decision process to optimize the
resource allocation for PDMA scheme. To increase the throughput of NOMA-IoT
networks, Ahsan et al. [85] develop DRL and SARSA learning algorithms to
explore the fair resource allocation in NOMA while balancing RB and network
traffic. Ahsan et al. [258] further consider reliable resource sharing in
NOMA-URLLC, and propose a DRL approach to provide long-term resource
allocation based on instantaneous reward feedback.

To improve the efficiency and effectiveness of RL in RIS-aided NOMA systems,
Zhong et al. [89] adopt a FL-aided DRL algorithm to optimize RIS design and
power allocation policy, where FL is used to assist multiple agents to
exchange experience, and these cooperated agents can obtain a reward gain
compared to the independent agents. To ensure fair resource sharing among
wireless stations, Poposka et al. [308] use an FL model to transmit local
model parameters to the base station and achieve fair resource allocation with
NOMA. Here, only a single wireless station is allowed to transmit at maximum
power, while the other wireless stations transmit at lower power. Bouzinis et
al. [309] introduce and optimize a compute-then-transmit NOMA for wireless
federated learning (WFL) networks, where the users simultaneously transmit the
locally trained parameters to the server, and the server jointly optimizes
computation and communication resources to reduce the overall delay during a
WFL communication round. Zhong et al. [188] study over-the-air federated
multi-task learning (OA-FMTL) for the MIMO-MAC optimization to reduce the
impact of inter-task interference on the FL performance. To maximize the sum
rate, Gao et al. [310] propose a K-means algorithm to optimize the RIS phase
shift matrix and power allocation coefficient vector in RIS-aided MISO NOMA
systems. To improve the energy efficiency in THz-NOMA-MIMO systems, Zhang et
al. [311] use an enhanced K-means ML algorithm for solving user clustering and
power allocation problems.

####  V-A3 Power Control

Optimal power control and management are beneficial to reduce interference and
increase system throughput when optimizing the AI-empowered MA protocol.
Existing AI methods for power control and management in different MA protocols
are detailed below.

To reduce primary user interference and avoid signalling overhead in CNN,
Galindo et al. [313] introduce a decentralized Q-learning algorithm to learn
an optimal power control policy and manage the aggregated interference. Bennis
et al. [314] formulate the power control and carrier selection as a non-
cooperative game among femto BSs, and propose an RL algorithm to help FBSs to
reach a Logit equilibrium. To mitigate inter-cell interference and maximize
the system performance, Simsek et al. [315] propose a two-stage RL formulation
for learning the optimal cell range expansion bias, power control schemes, and
frequency band allocation. Asheralieva et al. [316] develop a fully autonomous
multi-agent Q-learning to obtain the best channel and power control strategies
in heterogeneous cellular networks, achieving fast convergence and near-
optimal performance. Xiao et al. [78] propose a Q-learning-based power
allocation algorithm for the downlink NOMA transmission without knowing the
jamming and radio channel parameters, and a Dyna architecture based on real
anti-jamming transmission experience is used to accelerate the convergence of
the Q-learning-based power allocation and improve NOMA efficiency. Chu et al.
[317] apply an actor-critic DQN-based RL algorithm in an EH-MA system for
orthogonal channel access and continuous power control, while an LSTM-based
algorithm is used for battery state learning. Bouzinis et al. [309] propose an
online stochastic learning algorithm for delay-optimal power control and sub-
band allocation in the OFDMA uplink system. To deal with Q-learning model
overestimation issues, Fayaz et al. [91] propose a multi-agent DQN-based
grant-free NOMA algorithm to optimize transmit power levels for open-loop
power control. To ensure UAV flight safety and satisfy the sum rate
requirement, Zhao et al. [318] propose a sample-efficient DRL algorithm for
power control in RIS-NOMA-aided UAV systems. To balance system performance and
computational complexity, Zhang et al. [319] propose a USL algorithm for power
control strategies in cell-free massive MIMO systems. It can be learned from
[313, 314, 315, 316, 78] that distributed Q–earning-based power control
strategies perform better in channel access in CRNs and heterogeneous
networks. According to [317, 309], the performance of OMA can be improved by
applying RL algorithms in power control. In addition, using DRL algorithms to
achieve real-time power allocation in NOMA systems is also a potential way to
reduce the computational complexity and improve NOMA performance [91, 318,
319].

Figure 16: Parameter adjustment for AI-empowered MA protocol optimization.

###  V-B Parameter Adjustment

Employing AI solutions for IEEE 802.11, MA protocols can be optimized by
dynamically adjusting their access parameters, including CW/backoff, frame
size, and transmission rate setting, as shown in Fig. 16, to reduce MA
collisions and improve MA performance.

####  V-B1 CW and Back off Setting

According to [107, 111, 105, 101, 102, 108, 103, 109, 104, 106], applying RL-
based and SL-based methods, the CW/backoff values of MA protocols can be
optimized to improve throughput by reducing both collisions and idle periods.
For example, the existing RL [107, 111, 105, 108, 103, 109, 104, 106] and SL
[101, 102] solutions for IEEE 802.11 standards mainly focus on improving
throughput by adjusting the CW/backoff value of the basic 802.11 MAC protocol,
where a large CW or a small CW value may reduce throughput due to the
increasing idle slots or increasing collision probability, and how to utilize
different AI techniques to set an optimal CW value has been widely studied.
For example, Zhu et al. [103] implement an adaptation-based programming
algorithm to halve or leave unchanged the CW size depending on the
transmission state. Amuru et al. [104] propose a post-decision state-based
learning to use the previous knowledge of the CW setting to increase the
convergence speed. Based on the channel observation-based scaled backoff
protocol, Ali et al. [107] optimize the CW value according to the learned
channel collision probabilities. Edalat et al. [102] achieve a balance between
small CW and large CW using fixed-share SL. Abyaneh et al. [101] adopt SL to
balance the minimum CW size to achieve fairness. To ensure access priority,
Ali et al. [106] implement QL to infer the network density and set the CW
value. Zhang et al. [109] combine FL and DRL models to set the CW value to
achieve fairness. Kumar et al. [105] propose DQL to train minimum CW selection
to achieve per-user fairness. Wydmanski et al. [108] use DRL to learn the
optimal CW value setting to maintain a stable throughput as the number of
users increases. For distributed channel access (EDCA) in the 802.11e
amendment, arbitration inter-frame space (AIFS) and transmission opportunity
(TXOP) are included as the extended MAC parameters, and the joint optimization
of AIFS and CW is discussed to balance delay and throughput. Coronado et al.
[111] select the optimized combination of CW and AIFS via RL. The AIFS and CW
values are first set relying on decision tree algorithms, and then the optimal
combination for AIFS and CW is predicted. In EDCA, the optimal CW setting is
adaptive to the priority of the traffic.

####  V-B2 Frame Size Setting

The frame size setting has a direct impact on the efficiency of MA protocols
in terms of useful data transmitted and overhead. In general, SL-based and RL-
based approaches have been used to dynamically optimize the frame size for
IEEE 802.11 networks to improve throughput. For example, some works have
developed slot selection methods using RL models. In particular, Lee et al.
[280] select available time slots using RL. Kihira et al. [281] perform the
best time slot-based transmission policies via multi-agent cooperative RL, and
Guo et al. [282] propose a multi-agent RL algorithm to select time slots for
the optimal channel access behaviour. Several learning mechanisms are
presented to dynamically select the frame size, including SL [159, 160, 161,
162] and RL [283]. Lin et al. [159] jointly select both the frame size and the
CW value to address channel collisions, where an ANN model is trained with
frame size-throughput patterns to indicate the optimal frame size and the CW
value. Karmakar et al. [283] design the frame size selection using the RL
model to achieve the energy-throughput trade-off. Coronado et al. [160]
implement a random forest regressor (RFR) model to configure the frame size
and the modulation and coding scheme settings. Coronado et al. [161] further
develop an RFR-based frame size selection method for software-defined WLANs to
maximize the goodput of each user. This solution defines a set of suitable
aggregation levels based on an online learning algorithm and selects the