Google's another unique variant is the Text-to-Text Transfer Transformer (T5) model, which approaches all NLP task as a text-to-text problem, where both the input and output are sequences of text. Being pretrained on a large corpus of text data using a denoising autoencoder objective, T5 allows for its application on a wide range of tasks with minimal task-specific modifications.

7 Microsoft's Turing NLP <cit.>:
The initial version of Turing-NLG is a LLM trained on a diverse and extremely large corpus of data with 17 billion parameters. Similar to OpenAI's and Google's LLMs, Turing-NLG also utilizes the above GMT architecture and is pretrained in a generative manner to generate human-like text and has demonstrated remarkable performance in tasks like writing essays, summaries, and answering questions. Turing-NLG has been taken to another level by Microsoft and NVIDIA's Megatron-Turing Natural Language Generation (NLG), which is trained using DeepSpeed and Megatron, advanced tools designed to streamline the training of exceptionally large neural networks, over 530 billion of parameters, pushing the boundaries of what is possible in LLM and NLP.

Schematic illustration of GAMs.

Comparison of deep generative autoregressive models.

Model     Strengths     Weaknesses     Applications
FVSBN     c]@l@Simple architecture, easy to train     c]@l@Limited capacity, not scalable     General-purpose
NADE     c]@l@Efficient training,
good generalization     c]@l@More complex than FVSBN,  less efficient than MADE     General-purpose
MADE     c]@l@Efficient training, good performance     c]@l@Autoencoder-based,  less interpretable     General-purpose
PixelRNN/CNN     c]@l@High-quality images,  captures spatial relationships     c]@l@High computational cost,  sequential generation     Image generation
WaveNet     c]@l@High-fidelity audio, long-range dependencies     c]@l@High computational cost,  slow generation     Audio generation

§.§ GENERATIVE AUTOREGRESSIVE MODELS (GAMs)

GAMs have their roots in traditional autoregressive models, which were originally developed for time series analysis <cit.>. They are designed to generate data by learning the conditional probability distribution of a sequence of data points. These models leverage the autoregressive property, where each data point in the sequence is assumed to be dependent on its previous data points.  To be more specific, consider a sequence of data points represented by the vector $\mathbf{x} = [x_1, x_2, \ldots, x_N]$, as shown in Fig. <ref>. The GAMs aim to factorize the joint probability distribution of the sequence as a product of conditional probability distributions $p(\mathbf{x}) = \prod_{n=1}^{N} p(x_n \mid x_{<n})$, where data point $x_n, \: \forall n,$ is conditioned on  all its preceding data points in the sequence, $x_{<n}$. By modeling these dependencies, GAMs can generate realistic and diverse new data points by sampling from these conditional probability distributions sequentially, leading to the development of powerful GMs capable of modeling complex data distributions. Training a generative GAMs typically involves using maximum likelihood estimation (MLE), which is given by

\begin{equation}
\nonumber \mathcal{L}(\theta) = \sum_{n=1}^{N} \log p(x_n \mid \theta),
\end{equation}

where $\theta$ represents the model parameters. During the generation/sampling process, GAMs generate new samples sequentially by sampling from the learned conditional probability distributions.

By employing DL techniques, deep GAMs have pushed the boundaries of GenAI thanks to the following key characteristics: 1) capable of capturing complex dependencies between variables in the data distribution, resulting in more accurate and realistic samples; 2) scalable to model large datasets and high-dimensional data samples, 3) flexible to be adapted to different data types and domains by tailoring DNNs to suit the specific characteristics of the data; and 4) able to capture long-range dependencies in data, which was previously challenging for traditional autoregressive models, through incorporation of techniques such as attention mechanisms and dilated convolutions. The most common deep GAMs are explained below:

12 Fully Visible Sigmoid Belief Networks (FVSBNs)  are a type of GAM that uses a feedforward neural network (FNN) to model the conditional probability distribution of each variable given its preceding variables <cit.>. The network consists of a single hidden layer with sigmoid activation functions and is trained using maximum likelihood estimation.

12 Neural Autoregressive Distribution Estimator (NADE) extends FVSBNs by employing a DNN to model the conditional probability distributions <cit.>. NADE shares parameters across different dimensions of the input, resulting in more efficient training and better generalization.

12 Masked Autoencoder for Distribution Estimation (MADE) is a deep GAM that uses an AE architecture <cit.>. MADE applies masking to the weights of the AE to enforce the autoregressive property, ensuring that each output dimension only depends on the preceding input dimensions. This masking strategy leads to more efficient training and better performance compared to NADE.

12 PixelRNN <cit.> and PixelCNN <cit.> are deep GAMs specifically designed for generating images. PixelRNN uses a RNN architecture, while PixelCNN employs a CNN. Both models are capable of capturing long-range dependencies and complex spatial relationships in images, leading to high-quality image generation.

12 WaveNet <cit.> is a deep GAM designed for generating raw audio waveforms. It employs dilated convolutions to capture long-range dependencies in the audio data efficiently. WaveNet has been successful in generating high-fidelity, realistic-sounding speech and music.

Schematic illustration of RBMs.

§.§ RESTRICTED BOLTZMANN MACHINES (RBMs)

RBMs are a type of GM that can learn probability distributions over binary-valued input data.
In 2002, Geoffrey Hinton introduced the RBM, which is a simplified version of the Boltzmann Machine that removes intra-layer connections, making the model more computationally tractable and suitable for training with gradient-based methods like contrastive divergence <cit.>. As shown in Fig. <ref>, RBMs consist of a visible layer and a hidden layer, with connections only between the visible and hidden nodes. This structure is known as a symmetric bipartite graph, and the "restricted" in the name arises from the absence of intra-layer connections.

RBMs model the joint probability distribution between the visible units $\mathbf{v}$ and hidden units $\mathbf{h}$, $P(\mathbf{v}, \mathbf{h})$ in Fig. <ref>.(1), where $E(\mathbf{v}, \mathbf{h})$ is the energy function as given in Fig. <ref>.(2), $\mathbf{W}$ is the weight matrix, and $\mathbf{b}$/$\mathbf{c}$ is the bias offset vector for the visible/hidden units.
The RBM training process is typically performed using an unsupervised learning algorithm called Contrastive Divergence (CD) which follows below steps:

* Initialization: Random initialization of the weight matrix and bias offset vectors.

* Positive phase (forward propagation): Given an input data sample, compute the hidden layer activations using the current weights and biases, i.e., $P(h_j = 1 \mid \mathbf{v})$ as given in Fig. <ref>.(3).

* Negative phase (backward propagation): Reconstruct the visible layer activations by sampling from the distribution using the current weights and biases, i.e. $P(v_i = 1 \mid \mathbf{h})$ as given in Fig. <ref>.(4).

* Update: Update the weights and biases based on the difference between the outer products of the input data and the hidden activations from the positive phase, and the outer products of the reconstructed visible layer activations and the hidden activations from the negative phase, i.e., Fig. <ref>.(6)-Fig. <ref>.(8).

* Repeat: Perform steps 2-4 for a fixed number of iterations or until convergence, using different input data samples for each iteration.

Once an RBM has been trained, it can sample the hidden layer activations from the prior distribution over the hidden layer, then use the sampled hidden activations to generate visible layer activations (input data) through the learned conditional probability distributions. This generative process modeling the joint probability distribution between the visible and hidden layers allows RBMs to produce new samples that share similar characteristics with the input data they were trained on.

Even though RBMs were originally designed for binary data, this limitation has been overcome with the development of variations of RBMs that can handle different types of data, such as real-valued, categorical, or even multi-modal data. One such variation is the Gaussian-Bernoulli RBM, which is designed to handle continuous real-valued data <cit.>. There are other types of RBMs as well, such as the softmax RBM <cit.>, which can handle categorical data, or continuous-valued RBM to incorporate additional structure or sparsity constraints (e.g., sparse RBMs) <cit.>. These variants have further expanded the applicability of RBMs to a broader range of ML problems such as feature learning, dimensionality reduction, and collaborative filtering <cit.>.

Schematic illustration of DBNs.

§.§ DEEP BELIEF NETWORKS (DBNs)

Before the introduction of DBNs, training DNNs was challenging due to the vanishing gradient problem and the difficulty of optimizing the objective functions. Hinton et. al. proposed a new method for training deep architectures by stacking multiple RBMs on top of one another to form a DBN <cit.>. As shown in Fig. <ref>, a DBN with $L$ hidden layers can learn the joint probability distribution

\begin{equation}
\nonumber p(\mathbf{v}, \mathbf{h}_1, \mathbf{h}_2, \dots, \mathbf{h}_L) = p(\mathbf{v}|\mathbf{h}_1) \prod_{l=1}^{L-1} p(\mathbf{h}_l|\mathbf{h}_{l+1}),
\end{equation}

where each term $p(\mathbf{h}_{l}|\mathbf{h}_{l+1})$ corresponds to an RBM, representing the conditional probability distribution between adjacent layers of hidden variables. In this manner, DBNs can efficiently learn  hierarchical representations from data by a greedy layer-wise unsupervised pre-training method to initialize the weights of the network, followed by a supervised fine-tuning step using backpropagation.

In the unsupervised pre-training phase, the goal is to initialize the weights of the network by learning a hierarchical representation of the input data. The DBN is trained layer by layer, where each layer is an individual RBM. Train the first layer RBM using the input data. The input data is treated as visible units, and the RBM learns to reconstruct it through the hidden units by leveraging CD algoritm as explained in the previous section. The hidden unit activations of the first layer RBM are used as input (data) for the second layer RBM to train the second layer RBM, which is repeated for all layers in the DBN. After completing the unsupervised pre-training, the DBN has been initialized with weights that can capture a hierarchical representation of the input data.

In the supervised fine-tuning phase, the aim is to adjust the weights of the DBN using labeled data to perform a specific task, such as classification or regression.
A supervised output layer (e.g., softmax) uses the labeled data to perform backpropagation to update the weights of the entire network, including the pre-trained layers and the output layer. This step fine-tunes the weights to optimize the DBN for the specific SL task.
The backpropagation process is repeated until a predefined stopping criterion is met, such as reaching a maximum number of epochs or achieving a desired level of performance on the validation set.

The success of DBNs sparked a resurgence of interest in neural networks and DL, paving the way for the development of more advanced architectures and training techniques, such as CNNs, RNNs, and AEs. As a GM, DBNs have been applied to various tasks, including image generation, feature learning, dimensionality reduction, anomaly detection, collaborative filtering and recommendation, sequence modeling, and natural language processing.

High-level comparison of GMs

Criterion     GANs     VAEs     GAMs     RBMs     DBNs     FGMs     DGMs
Sample Quality            High       Medium     High       Medium     Medium     High       High
Training Stability        Low        High       Medium     High       High       High       Medium
Inference Speed           High       High       Medium     High       High       High       Medium
Scalability               High       High       High       Medium     Medium     High       High
Expressiveness            High       High       High       Medium     Medium     High       High
Disentanglement          Medium     High       Medium     Low        Low        Medium     Medium
Ease of Training          Medium     High       Medium     High       Medium     High       Medium
Evaluation Metrics        Medium     High       High       High       High       High       High
Interpretability          Low        Medium     Medium     Medium     Medium     Medium     Medium
Cond. Generation    High       High       High       Medium     Medium     High       High

GMs along with Potential Complementary Roles

Comp. Role         Generative Model Examples
Data Augmentation     DCGAN, CycleGAN, VAEs, RBMs, PixelCNN/RNN, FGMs
Data Imputation       VAEs, RBMs, DBNs, PixelCNN/RNN, Normalizing FGMs
Disentanglement       $\beta$-VAEs, InfoGANs, FactorVAEs, FGMs
Regularization        VAEs (as Bayesian prior), RBMs (as pre-training)
Dim. Reduction        VAEs, RBMs, DBNs, FGMs
Feature Learning     BiGAN, InfoGAN, VAEs, RBMs, DBNs, PixelCNN/RNN, FGMs
Transfer Learning      StarGAN, CycleGAN, conditional VAEs, PixelCNN/RNN
Semi-SL       SGAN, CatGAN, VAEs, PixelCNN/RNN
Multi-Task Learning       MT-GAN, MT-VAEs, PixelCNN/RNN
Balancing Exp.        VAEs, GANs , PixelCNN/RNN (all with model-based RL)
Imitation Learning        GAIL, VAEs, PixelCNN/RNN (all with behavioral cloning)

§.§ KEY GM PERFORMANCE METRICS

It is non-trivial to provide a quantitative or qualitative comparison among the wide variety of GMs  presented above because performance may vary depending on the specific GM model variant, architecture, and dataset. Instead, we provide a set of key performance metrics that should be taken into account while comparing GMs:

14 Sample quality: How realistic and diverse are the generated samples? High-quality samples should be visually or statistically indistinguishable from real data.

14 Training Stability: How stable is the training process? Some GMs may suffer from instability issues, making it difficult to converge to an optimal solution.

* Model Size and Computational Efficiency:How large is the model in terms of its parameters, architecture, and memory footprint? The size of a model often correlates with its computational demands. Training large models can be resource-intensive and may require specialized hardware, like powerful GPUs or TPUs. Furthermore, larger models may result in slower inference times, which is not ideal for real-time applications.

* Memory Consumption:Larger models demand more memory, both during training and inference. This can be a challenge when deploying models in resource-constrained environments like mobile devices or edge devices. When evaluating GMs, it's beneficial to consider the trade-offs between model size and performance.

14 Inference Speed: How quickly can new samples be generated? Faster inference speeds are desirable for real-time or large-scale applications.

14 Scalability: How well does the model handle large datasets and high-dimensional data? GMs should be able to scale efficiently to model complex data distributions.

14 Expressiveness: How flexible is the model in capturing a wide range of data distributions? A more expressive model can adapt to different types of data and learn more complex relationships.

14 Disentanglement: How well does the model separate the underlying factors of variation in the data? Disentangled representations can improve interpretability and facilitate downstream tasks.

14 Ease of Training: How easy is it to train the model, considering factors like hyperparameter tuning and the availability of training resources?

14 Interpretability: How interpretable are the model's learned representations and the generative process itself?

14 Conditional generation: How well can the model generate samples conditioned on specific attributes, such as labels or latent variables?

Following from the discussion in Section <ref>, we provide a comprehensive list of potential GMs relevant to complementary roles across various ML techniques in Table <ref>.  Please also note that this table is not exhaustive, and there might be other GMs and their corresponding roles that are not listed here. Additionally, many GMs can be used in combination or can be extended to address various complementary roles in different ML techniques.

_vfiz0mefo/.code =  0 bp   0 bp     0   2

_xpyvgylry/.code =  0 bp   0 bp     0   2

_gz1rj77vr/.code =  0 bp   0 bp     0   2

_xts2ltbes/.code =  0 bp   0 bp     0   2

_d4hybhio8/.code =  0 bp   0 bp     0   2

_vmnvuuhp4/.code =  0 bp   0 bp     0   2

_2xed3gyq2/.code =  0 bp   0 bp     0   2

_swt2keckk/.code =  0 bp   0 bp     0   2
_8f3u4406y/.code = 0 bp   0 bp     0   2
_m1ai9fexg 150bp color(0bp)=(transparent!0);
_2mkyfkzud[shading=_m1ai9fexg,_8f3u4406y] (0,0) rectangle (50bp,50bp);

_kimojzrk3/.code =  0 bp   0 bp     0   2
_2zxuulbyz/.code = 0 bp   0 bp     0   2
_khzraokfv 150bp color(0bp)=(transparent!0);
_e55g982lk[shading=_khzraokfv,_2zxuulbyz] (0,0) rectangle (50bp,50bp);

_h3wk6oik4/.code =  0 bp   0 bp     0   2
_7q3y9e8q6/.code = 0 bp   0 bp     0   2
_3i9r57vio 150bp color(0bp)=(transparent!0);
_k840d7flb[shading=_3i9r57vio,_7q3y9e8q6] (0,0) rectangle (50bp,50bp);

_wieactbgr/.code =  0 bp   0 bp     0   2
_y67ygwi7q/.code = 0 bp   0 bp     0   2
_586kfdz28 150bp color(0bp)=(transparent!0);
_bdojxtnkf[shading=_586kfdz28,_y67ygwi7q] (0,0) rectangle (50bp,50bp);

_4h0zmvse5/.code =  0 bp   0 bp     0   2
_23ajntxdt/.code = 0 bp   0 bp     0   2
_ld11cvh1z 150bp color(0bp)=(transparent!0);
_vlossobzd[shading=_ld11cvh1z,_23ajntxdt] (0,0) rectangle (50bp,50bp);

_057zpgee6/.code =  0 bp   0 bp     0   2
_1wh1mdsq8/.code = 0 bp   0 bp     0   2
_mked6mizr 150bp color(0bp)=(transparent!0);
_4hpqgofic[shading=_mked6mizr,_1wh1mdsq8] (0,0) rectangle (50bp,50bp);

_0xko7aabv/.code =  0 bp   0 bp     0   2
_5piu3nlwo/.code = 0 bp   0 bp     0   2
_8s0hzldy4 150bp color(0bp)=(transparent!0);
_5ds34bwem[shading=_8s0hzldy4,_5piu3nlwo] (0,0) rectangle (50bp,50bp);

_75730233j/.code =  0 bp   0 bp     0   2
_jnfil0sdm/.code = 0 bp   0 bp     0   2
_f84mr59n9 150bp color(0bp)=(transparent!0);
_al4uakod3[shading=_f84mr59n9,_jnfil0sdm] (0,0) rectangle (50bp,50bp);

_4x279iqgz/.code =  0 bp   0 bp     0   2
_nflmq15f9/.code = 0 bp   0 bp     0   2
_6am1s5krr 150bp color(0bp)=(transparent!0);
_nqmhziysv[shading=_6am1s5krr,_nflmq15f9] (0,0) rectangle (50bp,50bp);

_mv5dmomcv/.code =  0 bp   0 bp     0   2
_uy7uxpo9n/.code = 0 bp   0 bp     0   2
_lhw1mw4rq 150bp color(0bp)=(transparent!0);
_jea9u4jv2[shading=_lhw1mw4rq,_uy7uxpo9n] (0,0) rectangle (50bp,50bp);

_z861abxpz/.code =  0 bp   0 bp     0   2
_541dcraao/.code = 0 bp   0 bp     0   2
_t3inxx61v 150bp color(0bp)=(transparent!0);
_i0tq6mjhx[shading=_t3inxx61v,_541dcraao] (0,0) rectangle (50bp,50bp);

_k7ijmsswm/.code =  0 bp   0 bp     0   2
_4xyjh95bn/.code = 0 bp   0 bp     0   2
_un4ogwhc8 150bp color(0bp)=(transparent!0);
_x0iczo0ya[shading=_un4ogwhc8,_4xyjh95bn] (0,0) rectangle (50bp,50bp);

_x4mlaiu23/.code =  0 bp   0 bp     0   2
_1afnqa3is/.code = 0 bp   0 bp     0   2
_lqc47jxdl 150bp color(0bp)=(transparent!0);
_em756vv6o[shading=_lqc47jxdl,_1afnqa3is] (0,0) rectangle (50bp,50bp);

_ffp6t1fmq/.code =  0 bp   0 bp     0   2
_nv39b6vt5/.code = 0 bp   0 bp     0   2
_5y6iop943 150bp color(0bp)=(transparent!0);
_59ndhnrif[shading=_5y6iop943,_nv39b6vt5] (0,0) rectangle (50bp,50bp);

_5z8a74eco/.code =  0 bp   0 bp     0   2
_9lygtm5ee/.code = 0 bp   0 bp     0   2
_32b7qz2gi 150bp color(0bp)=(transparent!0);
_470s4s3hd[shading=_32b7qz2gi,_9lygtm5ee] (0,0) rectangle (50bp,50bp);

_r2xlgk5ot/.code =  0 bp   0 bp     0   2
_je6j095o0/.code = 0 bp   0 bp     0   2
_weokwb0ar 150bp color(0bp)=(transparent!0);
_1v0vd7455[shading=_weokwb0ar,_je6j095o0] (0,0) rectangle (50bp,50bp);

_c4lqhvi4z/.code =  0 bp   0 bp     0   2
_8uj2p3w3c/.code = 0 bp   0 bp     0   2
_vdryavvzt 150bp color(0bp)=(transparent!0);
_8x4pstjj4[shading=_vdryavvzt,_8uj2p3w3c] (0,0) rectangle (50bp,50bp);

_ggey00k42/.code =  0 bp   0 bp     0   2
_onz138457/.code = 0 bp   0 bp     0   2
_olm1zqe9v 150bp color(0bp)=(transparent!0);
_uv5hyivgb[shading=_olm1zqe9v,_onz138457] (0,0) rectangle (50bp,50bp);

_8skof4iyr/.code =  0 bp   0 bp     0   2
_o3i2g9t0x/.code = 0 bp   0 bp     0   2
_yq45pwrhn 150bp color(0bp)=(transparent!0);
_xm9lxw43x[shading=_yq45pwrhn,_o3i2g9t0x] (0,0) rectangle (50bp,50bp);

_9rn0guqyl/.code =  0 bp   0 bp     0   2
_kr7v108cg/.code = 0 bp   0 bp     0   2
_0fhsk7jab 150bp color(0bp)=(transparent!0);
_jpj136xyn[shading=_0fhsk7jab,_kr7v108cg] (0,0) rectangle (50bp,50bp);

_r4yy5obmy/.code =  0 bp   0 bp     0   2
_vgzhg6h51/.code = 0 bp   0 bp     0   2
_qsobbzezu 150bp color(0bp)=(transparent!0);
_aa3cxad8r[shading=_qsobbzezu,_vgzhg6h51] (0,0) rectangle (50bp,50bp);
every picture/.style=line width=0.75pt

[shading=_5hckyvrcp,_vfiz0mefo] (-6,3.11) .. controls (-6,-0.81) and (-2.81,-4) .. (1.11,-4) – (448.89,-4) .. controls (452.81,-4) and (456,-0.81) .. (456,3.11) – (456,496.89) .. controls (456,500.81) and (452.81,504) .. (448.89,504) – (1.11,504) .. controls (-2.81,504) and (-6,500.81) .. (-6,496.89) – cycle ;
[color=rgb, 255:red, 155; green, 155; blue, 155   ,draw opacity=1 ] (-6,3.11) .. controls (-6,-0.81) and (-2.81,-4) .. (1.11,-4) – (448.89,-4) .. controls (452.81,-4) and (456,-0.81) .. (456,3.11) – (456,496.89) .. controls (456,500.81) and (452.81,504) .. (448.89,504) – (1.11,504) .. controls (-2.81,504) and (-6,500.81) .. (-6,496.89) – cycle ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][fill=rgb, 255:red, 128; green, 128; blue, 128   ,fill opacity=1 ] (305,455.33) .. controls (305,452.39) and (307.39,450) .. (310.33,450) – (439.67,450) .. controls (442.61,450) and (445,452.39) .. (445,455.33) – (445,489.67) .. controls (445,492.61) and (442.61,495) .. (439.67,495) – (310.33,495) .. controls (307.39,495) and (305,492.61) .. (305,489.67) – cycle ;

[shading=_xgjloiwkp,_xpyvgylry] (2,311.12) .. controls (2,306.09) and (6.09,302) .. (11.12,302) – (139.88,302) .. controls (144.91,302) and (149,306.09) .. (149,311.12) – (149,488.88) .. controls (149,493.91) and (144.91,498) .. (139.88,498) – (11.12,498) .. controls (6.09,498) and (2,493.91) .. (2,488.88) – cycle ;
[color=rgb, 255:red, 155; green, 155; blue, 155   ,draw opacity=1 ] (2,311.12) .. controls (2,306.09) and (6.09,302) .. (11.12,302) – (139.88,302) .. controls (144.91,302) and (149,306.09) .. (149,311.12) – (149,488.88) .. controls (149,493.91) and (144.91,498) .. (139.88,498) – (11.12,498) .. controls (6.09,498) and (2,493.91) .. (2,488.88) – cycle ;

[shading=_ap0b5cnvn,_gz1rj77vr] (151.88,311.12) .. controls (151.88,306.09) and (155.96,302) .. (161,302) – (289.75,302) .. controls (294.79,302) and (298.88,306.09) .. (298.88,311.12) – (298.88,488.88) .. controls (298.88,493.91) and (294.79,498) .. (289.75,498) – (161,498) .. controls (155.96,498) and (151.88,493.91) .. (151.88,488.88) – cycle ;
[color=rgb, 255:red, 155; green, 155; blue, 155   ,draw opacity=1 ] (151.88,311.12) .. controls (151.88,306.09) and (155.96,302) .. (161,302) – (289.75,302) .. controls (294.79,302) and (298.88,306.09) .. (298.88,311.12) – (298.88,488.88) .. controls (298.88,493.91) and (294.79,498) .. (289.75,498) – (161,498) .. controls (155.96,498) and (151.88,493.91) .. (151.88,488.88) – cycle ;

[shading=_0fko8kjzn,_xts2ltbes] (301.75,261.12) .. controls (301.75,256.09) and (305.84,252) .. (310.87,252) – (439.63,252) .. controls (444.66,252) and (448.75,256.09) .. (448.75,261.12) – (448.75,438.88) .. controls (448.75,443.91) and (444.66,448) .. (439.63,448) – (310.87,448) .. controls (305.84,448) and (301.75,443.91) .. (301.75,438.88) – cycle ;
[color=rgb, 255:red, 155; green, 155; blue, 155   ,draw opacity=1 ] (301.75,261.12) .. controls (301.75,256.09) and (305.84,252) .. (310.87,252) – (439.63,252) .. controls (444.66,252) and (448.75,256.09) .. (448.75,261.12) – (448.75,438.88) .. controls (448.75,443.91) and (444.66,448) .. (439.63,448) – (310.87,448) .. controls (305.84,448) and (301.75,443.91) .. (301.75,438.88) – cycle ;

[shading=_e9ef29gqk,_d4hybhio8] (301.75,11.12) .. controls (301.75,6.09) and (305.84,2) .. (310.87,2) – (439.63,2) .. controls (444.66,2) and (448.75,6.09) .. (448.75,11.12) – (448.75,238.88) .. controls (448.75,243.91) and (444.66,248) .. (439.63,248) – (310.87,248) .. controls (305.84,248) and (301.75,243.91) .. (301.75,238.88) – cycle ;
[color=rgb, 255:red, 155; green, 155; blue, 155   ,draw opacity=1 ] (301.75,11.12) .. controls (301.75,6.09) and (305.84,2) .. (310.87,2) – (439.63,2) .. controls (444.66,2) and (448.75,6.09) .. (448.75,11.12) – (448.75,238.88) .. controls (448.75,243.91) and (444.66,248) .. (439.63,248) – (310.87,248) .. controls (305.84,248) and (301.75,243.91) .. (301.75,238.88) – cycle ;

[shading=_hthwro570,_vmnvuuhp4] (151,111.12) .. controls (151,106.09) and (155.09,102) .. (160.12,102) – (288.88,102) .. controls (293.91,102) and (298,106.09) .. (298,111.12) – (298,288.88) .. controls (298,293.91) and (293.91,298) .. (288.88,298) – (160.12,298) .. controls (155.09,298) and (151,293.91) .. (151,288.88) – cycle ;
[color=rgb, 255:red, 155; green, 155; blue, 155   ,draw opacity=1 ] (151,111.12) .. controls (151,106.09) and (155.09,102) .. (160.12,102) – (288.88,102) .. controls (293.91,102) and (298,106.09) .. (298,111.12) – (298,288.88) .. controls (298,293.91) and (293.91,298) .. (288.88,298) – (160.12,298) .. controls (155.09,298) and (151,293.91) .. (151,288.88) – cycle ;

[shading=_dnft9jtv9,_2xed3gyq2] (1,11.12) .. controls (1,6.09) and (5.09,2) .. (10.12,2) – (138.88,2) .. controls (143.91,2) and (148,6.09) .. (148,11.12) – (148,288.88) .. controls (148,293.91) and (143.91,298) .. (138.88,298) – (10.12,298) .. controls (5.09,298) and (1,293.91) .. (1,288.88) – cycle ;
[color=rgb, 255:red, 155; green, 155; blue, 155   ,draw opacity=1 ] (1,11.12) .. controls (1,6.09) and (5.09,2) .. (10.12,2) – (138.88,2) .. controls (143.91,2) and (148,6.09) .. (148,11.12) – (148,288.88) .. controls (148,293.91) and (143.91,298) .. (138.88,298) – (10.12,298) .. controls (5.09,298) and (1,293.91) .. (1,288.88) – cycle ;

[fill=rgb, 255:red, 126; green, 211; blue, 33   ,fill opacity=1 ][line width=0.75]  (155,27) .. controls (155,17.61) and (162.61,10) .. (172,10) – (277,10) .. controls (286.39,10) and (294,17.61) .. (294,27) – (294,78) .. controls (294,87.39) and (286.39,95) .. (277,95) – (172,95) .. controls (162.61,95) and (155,87.39) .. (155,78) – cycle ;

[fill=rgb, 255:red, 126; green, 211; blue, 33   ,fill opacity=1 ] (5,13) .. controls (5,8.58) and (8.58,5) .. (13,5) – (137,5) .. controls (141.42,5) and (145,8.58) .. (145,13) – (145,37) .. controls (145,41.42) and (141.42,45) .. (137,45) – (13,45) .. controls (8.58,45) and (5,41.42) .. (5,37) – cycle ;

[fill=rgb, 255:red, 126; green, 211; blue, 33   ,fill opacity=1 ] (155,113) .. controls (155,108.58) and (158.58,105) .. (163,105) – (287,105) .. controls (291.42,105) and (295,108.58) .. (295,113) – (295,137) .. controls (295,141.42) and (291.42,145) .. (287,145) – (163,145) .. controls (158.58,145) and (155,141.42) .. (155,137) – cycle ;

[shading=_gz3ajexd7,_swt2keckk,path fading= _2mkyfkzud ,fading transform=xshift=2] (5,63) .. controls (5,58.58) and (8.58,55) .. (13,55) – (137,55) .. controls (141.42,55) and (145,58.58) .. (145,63) – (145,87) .. controls (145,91.42) and (141.42,95) .. (137,95) – (13,95) .. controls (8.58,95) and (5,91.42) .. (5,87) – cycle ;
[color=rgb, 255:red, 128; green, 128; blue, 128   ,draw opacity=1 ][line width=1.5]  (5,63) .. controls (5,58.58) and (8.58,55) .. (13,55) – (137,55) .. controls (141.42,55) and (145,58.58) .. (145,63) – (145,87) .. controls (145,91.42) and (141.42,95) .. (137,95) – (13,95) .. controls (8.58,95) and (5,91.42) .. (5,87) – cycle ;

[shading=_sh306xn8x,_kimojzrk3,path fading= _e55g982lk ,fading transform=xshift=2] (5,213) .. controls (5,208.58) and (8.58,205) .. (13,205) – (137,205) .. controls (141.42,205) and (145,208.58) .. (145,213) – (145,237) .. controls (145,241.42) and (141.42,245) .. (137,245) – (13,245) .. controls (8.58,245) and (5,241.42) .. (5,237) – cycle ;
[color=rgb, 255:red, 128; green, 128; blue, 128   ,draw opacity=1 ][line width=1.5]  (5,213) .. controls (5,208.58) and (8.58,205) .. (13,205) – (137,205) .. controls (141.42,205) and (145,208.58) .. (145,213) – (145,237) .. controls (145,241.42) and (141.42,245) .. (137,245) – (13,245) .. controls (8.58,245) and (5,241.42) .. (5,237) – cycle ;

[shading=_2pphjdg7g,_h3wk6oik4,path fading= _k840d7flb ,fading transform=xshift=2] (5,113) .. controls (5,108.58) and (8.58,105) .. (13,105) – (137,105) .. controls (141.42,105) and (145,108.58) .. (145,113) – (145,137) .. controls (145,141.42) and (141.42,145) .. (137,145) – (13,145) .. controls (8.58,145) and (5,141.42) .. (5,137) – cycle ;
[color=rgb, 255:red, 128; green, 128; blue, 128   ,draw opacity=1 ][line width=1.5]  (5,113) .. controls (5,108.58) and (8.58,105) .. (13,105) – (137,105) .. controls (141.42,105) and (145,108.58) .. (145,113) – (145,137) .. controls (145,141.42) and (141.42,145) .. (137,145) – (13,145) .. controls (8.58,145) and (5,141.42) .. (5,137) – cycle ;

[shading=_gzexlu90l,_wieactbgr,path fading= _bdojxtnkf ,fading transform=xshift=2] (5,163) .. controls (5,158.58) and (8.58,155) .. (13,155) – (137,155) .. controls (141.42,155) and (145,158.58) .. (145,163) – (145,187) .. controls (145,191.42) and (141.42,195) .. (137,195) – (13,195) .. controls (8.58,195) and (5,191.42) .. (5,187) – cycle ;
[color=rgb, 255:red, 128; green, 128; blue, 128   ,draw opacity=1 ][line width=1.5]  (5,163) .. controls (5,158.58) and (8.58,155) .. (13,155) – (137,155) .. controls (141.42,155) and (145,158.58) .. (145,163) – (145,187) .. controls (145,191.42) and (141.42,195) .. (137,195) – (13,195) .. controls (8.58,195) and (5,191.42) .. (5,187) – cycle ;

[shading=_xkrd3fhpp,_4h0zmvse5,path fading= _vlossobzd ,fading transform=xshift=2] (5,263) .. controls (5,258.58) and (8.58,255) .. (13,255) – (137,255) .. controls (141.42,255) and (145,258.58) .. (145,263) – (145,287) .. controls (145,291.42) and (141.42,295) .. (137,295) – (13,295) .. controls (8.58,295) and (5,291.42) .. (5,287) – cycle ;
[color=rgb, 255:red, 128; green, 128; blue, 128   ,draw opacity=1 ][dash pattern=on 5.63pt off 4.5pt][line width=1.5]  (5,263) .. controls (5,258.58) and (8.58,255) .. (13,255) – (137,255) .. controls (141.42,255) and (145,258.58) .. (145,263) – (145,287) .. controls (145,291.42) and (141.42,295) .. (137,295) – (13,295) .. controls (8.58,295) and (5,291.42) .. (5,287) – cycle ;

[shading=_qadrkgqxh,_057zpgee6,path fading= _4hpqgofic ,fading transform=xshift=2] (155,163) .. controls (155,158.58) and (158.58,155) .. (163,155) – (287,155) .. controls (291.42,155) and (295,158.58) .. (295,163) – (295,187) .. controls (295,191.42) and (291.42,195) .. (287,195) – (163,195) .. controls (158.58,195) and (155,191.42) .. (155,187) – cycle ;
[color=rgb, 255:red, 128; green, 128; blue, 128   ,draw opacity=1 ][line width=1.5]  (155,163) .. controls (155,158.58) and (158.58,155) .. (163,155) – (287,155) .. controls (291.42,155) and (295,158.58) .. (295,163) – (295,187) .. controls (295,191.42) and (291.42,195) .. (287,195) – (163,195) .. controls (158.58,195) and (155,191.42) .. (155,187) – cycle ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (75,245) – (75,255) ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (75,195) – (75,205) ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (75,145) – (75,155) ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (75,95) – (75,105) ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (75,45) – (75,55) ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (225,145) – (225,155) ;

[shading=_cbs8xz0ut,_0xko7aabv,path fading= _5ds34bwem ,fading transform=xshift=2] (155,213) .. controls (155,208.58) and (158.58,205) .. (163,205) – (287,205) .. controls (291.42,205) and (295,208.58) .. (295,213) – (295,237) .. controls (295,241.42) and (291.42,245) .. (287,245) – (163,245) .. controls (158.58,245) and (155,241.42) .. (155,237) – cycle ;
[color=rgb, 255:red, 128; green, 128; blue, 128   ,draw opacity=1 ][line width=1.5]  (155,213) .. controls (155,208.58) and (158.58,205) .. (163,205) – (287,205) .. controls (291.42,205) and (295,208.58) .. (295,213) – (295,237) .. controls (295,241.42) and (291.42,245) .. (287,245) – (163,245) .. controls (158.58,245) and (155,241.42) .. (155,237) – cycle ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (230,195) – (230,205) ;

[shading=_mcco1spgi,_75730233j,path fading= _al4uakod3 ,fading transform=xshift=2] (155,263) .. controls (155,258.58) and (158.58,255) .. (163,255) – (287,255) .. controls (291.42,255) and (295,258.58) .. (295,263) – (295,287) .. controls (295,291.42) and (291.42,295) .. (287,295) – (163,295) .. controls (158.58,295) and (155,291.42) .. (155,287) – cycle ;
[color=rgb, 255:red, 128; green, 128; blue, 128   ,draw opacity=1 ][line width=1.5]  (155,263) .. controls (155,258.58) and (158.58,255) .. (163,255) – (287,255) .. controls (291.42,255) and (295,258.58) .. (295,263) – (295,287) .. controls (295,291.42) and (291.42,295) .. (287,295) – (163,295) .. controls (158.58,295) and (155,291.42) .. (155,287) – cycle ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (230,245) – (230,255) ;

[fill=rgb, 255:red, 126; green, 211; blue, 33   ,fill opacity=1 ] (5.25,313) .. controls (5.25,308.58) and (8.83,305) .. (13.25,305) – (137.25,305) .. controls (141.67,305) and (145.25,308.58) .. (145.25,313) – (145.25,337) .. controls (145.25,341.42) and (141.67,345) .. (137.25,345) – (13.25,345) .. controls (8.83,345) and (5.25,341.42) .. (5.25,337) – cycle ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (75.25,345) – (75.25,355) ;

[shading=_khfr3f8kl,_4x279iqgz,path fading= _nqmhziysv ,fading transform=xshift=2] (5,363) .. controls (5,358.58) and (8.58,355) .. (13,355) – (137,355) .. controls (141.42,355) and (145,358.58) .. (145,363) – (145,387) .. controls (145,391.42) and (141.42,395) .. (137,395) – (13,395) .. controls (8.58,395) and (5,391.42) .. (5,387) – cycle ;
[color=rgb, 255:red, 128; green, 128; blue, 128   ,draw opacity=1 ][line width=1.5]  (5,363) .. controls (5,358.58) and (8.58,355) .. (13,355) – (137,355) .. controls (141.42,355) and (145,358.58) .. (145,363) – (145,387) .. controls (145,391.42) and (141.42,395) .. (137,395) – (13,395) .. controls (8.58,395) and (5,391.42) .. (5,387) – cycle ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (75.5,395) – (75.5,405) ;

[shading=_s61tmr484,_mv5dmomcv,path fading= _jea9u4jv2 ,fading transform=xshift=2] (5.25,413) .. controls (5.25,408.58) and (8.83,405) .. (13.25,405) – (137.25,405) .. controls (141.67,405) and (145.25,408.58) .. (145.25,413) – (145.25,437) .. controls (145.25,441.42) and (141.67,445) .. (137.25,445) – (13.25,445) .. controls (8.83,445) and (5.25,441.42) .. (5.25,437) – cycle ;
[color=rgb, 255:red, 128; green, 128; blue, 128   ,draw opacity=1 ][dash pattern=on 5.63pt off 4.5pt][line width=1.5]  (5.25,413) .. controls (5.25,408.58) and (8.83,405) .. (13.25,405) – (137.25,405) .. controls (141.67,405) and (145.25,408.58) .. (145.25,413) – (145.25,437) .. controls (145.25,441.42) and (141.67,445) .. (137.25,445) – (13.25,445) .. controls (8.83,445) and (5.25,441.42) .. (5.25,437) – cycle ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (75.5,445) – (75.5,455) ;

[shading=_on28dqyzf,_z861abxpz,path fading= _i0tq6mjhx ,fading transform=xshift=2] (5.25,463) .. controls (5.25,458.58) and (8.83,455) .. (13.25,455) – (137.25,455) .. controls (141.67,455) and (145.25,458.58) .. (145.25,463) – (145.25,487) .. controls (145.25,491.42) and (141.67,495) .. (137.25,495) – (13.25,495) .. controls (8.83,495) and (5.25,491.42) .. (5.25,487) – cycle ;
[color=rgb, 255:red, 128; green, 128; blue, 128   ,draw opacity=1 ][dash pattern=on 5.63pt off 4.5pt][line width=1.5]  (5.25,463) .. controls (5.25,458.58) and (8.83,455) .. (13.25,455) – (137.25,455) .. controls (141.67,455) and (145.25,458.58) .. (145.25,463) – (145.25,487) .. controls (145.25,491.42) and (141.67,495) .. (137.25,495) – (13.25,495) .. controls (8.83,495) and (5.25,491.42) .. (5.25,487) – cycle ;

[fill=rgb, 255:red, 126; green, 211; blue, 33   ,fill opacity=1 ] (155.25,313) .. controls (155.25,308.58) and (158.83,305) .. (163.25,305) – (287.25,305) .. controls (291.67,305) and (295.25,308.58) .. (295.25,313) – (295.25,337) .. controls (295.25,341.42) and (291.67,345) .. (287.25,345) – (163.25,345) .. controls (158.83,345) and (155.25,341.42) .. (155.25,337) – cycle ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (225.25,345) – (225.25,355) ;

[shading=_jgl44a42w,_k7ijmsswm,path fading= _x0iczo0ya ,fading transform=xshift=2] (155,363) .. controls (155,358.58) and (158.58,355) .. (163,355) – (287,355) .. controls (291.42,355) and (295,358.58) .. (295,363) – (295,387) .. controls (295,391.42) and (291.42,395) .. (287,395) – (163,395) .. controls (158.58,395) and (155,391.42) .. (155,387) – cycle ;
[color=rgb, 255:red, 128; green, 128; blue, 128   ,draw opacity=1 ][line width=1.5]  (155,363) .. controls (155,358.58) and (158.58,355) .. (163,355) – (287,355) .. controls (291.42,355) and (295,358.58) .. (295,363) – (295,387) .. controls (295,391.42) and (291.42,395) .. (287,395) – (163,395) .. controls (158.58,395) and (155,391.42) .. (155,387) – cycle ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (225.5,395) – (225.5,405) ;

[shading=_jtxppdhq0,_x4mlaiu23,path fading= _em756vv6o ,fading transform=xshift=2] (155.25,413) .. controls (155.25,408.58) and (158.83,405) .. (163.25,405) – (287.25,405) .. controls (291.67,405) and (295.25,408.58) .. (295.25,413) – (295.25,437) .. controls (295.25,441.42) and (291.67,445) .. (287.25,445) – (163.25,445) .. controls (158.83,445) and (155.25,441.42) .. (155.25,437) – cycle ;
[color=rgb, 255:red, 128; green, 128; blue, 128   ,draw opacity=1 ][line width=1.5]  (155.25,413) .. controls (155.25,408.58) and (158.83,405) .. (163.25,405) – (287.25,405) .. controls (291.67,405) and (295.25,408.58) .. (295.25,413) – (295.25,437) .. controls (295.25,441.42) and (291.67,445) .. (287.25,445) – (163.25,445) .. controls (158.83,445) and (155.25,441.42) .. (155.25,437) – cycle ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (225.5,445) – (225.5,455) ;

[shading=_yzvibl5jw,_ffp6t1fmq,path fading= _59ndhnrif ,fading transform=xshift=2] (155.25,463) .. controls (155.25,458.58) and (158.83,455) .. (163.25,455) – (287.25,455) .. controls (291.67,455) and (295.25,458.58) .. (295.25,463) – (295.25,487) .. controls (295.25,491.42) and (291.67,495) .. (287.25,495) – (163.25,495) .. controls (158.83,495) and (155.25,491.42) .. (155.25,487) – cycle ;
[color=rgb, 255:red, 128; green, 128; blue, 128   ,draw opacity=1 ][line width=1.5]  (155.25,463) .. controls (155.25,458.58) and (158.83,455) .. (163.25,455) – (287.25,455) .. controls (291.67,455) and (295.25,458.58) .. (295.25,463) – (295.25,487) .. controls (295.25,491.42) and (291.67,495) .. (287.25,495) – (163.25,495) .. controls (158.83,495) and (155.25,491.42) .. (155.25,487) – cycle ;

[fill=rgb, 255:red, 126; green, 211; blue, 33   ,fill opacity=1 ] (305.25,13) .. controls (305.25,8.58) and (308.83,5) .. (313.25,5) – (437.25,5) .. controls (441.67,5) and (445.25,8.58) .. (445.25,13) – (445.25,37) .. controls (445.25,41.42) and (441.67,45) .. (437.25,45) – (313.25,45) .. controls (308.83,45) and (305.25,41.42) .. (305.25,37) – cycle ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (370.5,45) – (370.5,55) ;

[shading=_td0eh7oah,_5z8a74eco,path fading= _470s4s3hd ,fading transform=xshift=2] (305,63) .. controls (305,58.58) and (308.58,55) .. (313,55) – (437,55) .. controls (441.42,55) and (445,58.58) .. (445,63) – (445,87) .. controls (445,91.42) and (441.42,95) .. (437,95) – (313,95) .. controls (308.58,95) and (305,91.42) .. (305,87) – cycle ;
[color=rgb, 255:red, 128; green, 128; blue, 128   ,draw opacity=1 ][line width=1.5]  (305,63) .. controls (305,58.58) and (308.58,55) .. (313,55) – (437,55) .. controls (441.42,55) and (445,58.58) .. (445,63) – (445,87) .. controls (445,91.42) and (441.42,95) .. (437,95) – (313,95) .. controls (308.58,95) and (305,91.42) .. (305,87) – cycle ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (370.75,95) – (370.75,105) ;

[shading=_85qxr01nz,_r2xlgk5ot,path fading= _1v0vd7455 ,fading transform=xshift=2] (305.25,113) .. controls (305.25,108.58) and (308.83,105) .. (313.25,105) – (437.25,105) .. controls (441.67,105) and (445.25,108.58) .. (445.25,113) – (445.25,137) .. controls (445.25,141.42) and (441.67,145) .. (437.25,145) – (313.25,145) .. controls (308.83,145) and (305.25,141.42) .. (305.25,137) – cycle ;
[color=rgb, 255:red, 128; green, 128; blue, 128   ,draw opacity=1 ][line width=1.5]  (305.25,113) .. controls (305.25,108.58) and (308.83,105) .. (313.25,105) – (437.25,105) .. controls (441.67,105) and (445.25,108.58) .. (445.25,113) – (445.25,137) .. controls (445.25,141.42) and (441.67,145) .. (437.25,145) – (313.25,145) .. controls (308.83,145) and (305.25,141.42) .. (305.25,137) – cycle ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (370.75,145) – (370.75,155) ;

[shading=_nco07y4lq,_c4lqhvi4z,path fading= _8x4pstjj4 ,fading transform=xshift=2] (305.25,163) .. controls (305.25,158.58) and (308.83,155) .. (313.25,155) – (437.25,155) .. controls (441.67,155) and (445.25,158.58) .. (445.25,163) – (445.25,187) .. controls (445.25,191.42) and (441.67,195) .. (437.25,195) – (313.25,195) .. controls (308.83,195) and (305.25,191.42) .. (305.25,187) – cycle ;
[color=rgb, 255:red, 128; green, 128; blue, 128   ,draw opacity=1 ][line width=1.5]  (305.25,163) .. controls (305.25,158.58) and (308.83,155) .. (313.25,155) – (437.25,155) .. controls (441.67,155) and (445.25,158.58) .. (445.25,163) – (445.25,187) .. controls (445.25,191.42) and (441.67,195) .. (437.25,195) – (313.25,195) .. controls (308.83,195) and (305.25,191.42) .. (305.25,187) – cycle ;

[shading=_dsno97ytu,_ggey00k42,path fading= _uv5hyivgb ,fading transform=xshift=2] (305.25,213) .. controls (305.25,208.58) and (308.83,205) .. (313.25,205) – (437.25,205) .. controls (441.67,205) and (445.25,208.58) .. (445.25,213) – (445.25,237) .. controls (445.25,241.42) and (441.67,245) .. (437.25,245) – (313.25,245) .. controls (308.83,245) and (305.25,241.42) .. (305.25,237) – cycle ;
[color=rgb, 255:red, 128; green, 128; blue, 128   ,draw opacity=1 ][line width=1.5]  (305.25,213) .. controls (305.25,208.58) and (308.83,205) .. (313.25,205) – (437.25,205) .. controls (441.67,205) and (445.25,208.58) .. (445.25,213) – (445.25,237) .. controls (445.25,241.42) and (441.67,245) .. (437.25,245) – (313.25,245) .. controls (308.83,245) and (305.25,241.42) .. (305.25,237) – cycle ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (370.25,195) – (370.25,205) ;

[fill=rgb, 255:red, 126; green, 211; blue, 33   ,fill opacity=1 ] (305.25,263) .. controls (305.25,258.58) and (308.83,255) .. (313.25,255) – (437.25,255) .. controls (441.67,255) and (445.25,258.58) .. (445.25,263) – (445.25,287) .. controls (445.25,291.42) and (441.67,295) .. (437.25,295) – (313.25,295) .. controls (308.83,295) and (305.25,291.42) .. (305.25,287) – cycle ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (370.5,295) – (370.5,305) ;

[shading=_kz4fp1oe7,_8skof4iyr,path fading= _xm9lxw43x ,fading transform=xshift=2] (305,313) .. controls (305,308.58) and (308.58,305) .. (313,305) – (437,305) .. controls (441.42,305) and (445,308.58) .. (445,313) – (445,337) .. controls (445,341.42) and (441.42,345) .. (437,345) – (313,345) .. controls (308.58,345) and (305,341.42) .. (305,337) – cycle ;
[color=rgb, 255:red, 128; green, 128; blue, 128   ,draw opacity=1 ][line width=1.5]  (305,313) .. controls (305,308.58) and (308.58,305) .. (313,305) – (437,305) .. controls (441.42,305) and (445,308.58) .. (445,313) – (445,337) .. controls (445,341.42) and (441.42,345) .. (437,345) – (313,345) .. controls (308.58,345) and (305,341.42) .. (305,337) – cycle ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (370.75,345) – (370.75,355) ;

[shading=_ebh0l9yux,_9rn0guqyl,path fading= _jpj136xyn ,fading transform=xshift=2] (305.25,363) .. controls (305.25,358.58) and (308.83,355) .. (313.25,355) – (437.25,355) .. controls (441.67,355) and (445.25,358.58) .. (445.25,363) – (445.25,387) .. controls (445.25,391.42) and (441.67,395) .. (437.25,395) – (313.25,395) .. controls (308.83,395) and (305.25,391.42) .. (305.25,387) – cycle ;
[color=rgb, 255:red, 128; green, 128; blue, 128   ,draw opacity=1 ][line width=1.5]  (305.25,363) .. controls (305.25,358.58) and (308.83,355) .. (313.25,355) – (437.25,355) .. controls (441.67,355) and (445.25,358.58) .. (445.25,363) – (445.25,387) .. controls (445.25,391.42) and (441.67,395) .. (437.25,395) – (313.25,395) .. controls (308.83,395) and (305.25,391.42) .. (305.25,387) – cycle ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (370.75,395) – (370.75,405) ;

[shading=_y2f6jx85k,_r4yy5obmy,path fading= _aa3cxad8r ,fading transform=xshift=2] (305.25,413) .. controls (305.25,408.58) and (308.83,405) .. (313.25,405) – (437.25,405) .. controls (441.67,405) and (445.25,408.58) .. (445.25,413) – (445.25,437) .. controls (445.25,441.42) and (441.67,445) .. (437.25,445) – (313.25,445) .. controls (308.83,445) and (305.25,441.42) .. (305.25,437) – cycle ;
[color=rgb, 255:red, 128; green, 128; blue, 128   ,draw opacity=1 ][dash pattern=on 5.63pt off 4.5pt][line width=1.5]  (305.25,413) .. controls (305.25,408.58) and (308.83,405) .. (313.25,405) – (437.25,405) .. controls (441.67,405) and (445.25,408.58) .. (445.25,413) – (445.25,437) .. controls (445.25,441.42) and (441.67,445) .. (437.25,445) – (313.25,445) .. controls (308.83,445) and (305.25,441.42) .. (305.25,437) – cycle ;

[color=rgb, 255:red, 255; green, 255; blue, 255   ,draw opacity=1 ][line width=1.5]  (310,459) .. controls (310,456.79) and (311.79,455) .. (314,455) – (371,455) .. controls (373.21,455) and (375,456.79) .. (375,459) – (375,471) .. controls (375,473.21) and (373.21,475) .. (371,475) – (314,475) .. controls (311.79,475) and (310,473.21) .. (310,471) – cycle ;

[color=rgb, 255:red, 255; green, 255; blue, 255   ,draw opacity=1 ][dash pattern=on 5.63pt off 4.5pt][line width=1.5]  (380,460) .. controls (380,457.79) and (381.79,456) .. (384,456) – (436,456) .. controls (438.21,456) and (440,457.79) .. (440,460) – (440,472) .. controls (440,474.21) and (438.21,476) .. (436,476) – (384,476) .. controls (381.79,476) and (380,474.21) .. (380,472) – cycle ;

(75,25) node  [font=] [align=left] [lt]100pt

A. Channel Estimation & Equalization

(225,125) node  [font=] [align=left] [lt]95.2pt

B. Modulation

(75,75) node  [font=] [align=left] [lt]95.2pt

mmWave/THz Channel Estimation <cit.>

(75.25,125) node  [font=] [align=left] [lt]94.86pt

Model-Free Stochastic Channel Approx. <cit.>

(75.25,175) node  [font=] [align=left] [lt]94.86pt

Autonomous Channel Modeling <cit.>

(75.25,225) node  [font=] [align=left] [lt]94.86pt

Score-Based MIMO Channel Estimation <cit.>

(80,275) node  [font=] [align=left] [lt]88.4pt

Blind Channel Equalization <cit.>

(225.25,175) node  [font=] [align=left] [lt]94.86pt

Data Augment. for Improved Perform. <cit.>

(225.25,225) node  [font=] [align=left] [lt]94.86pt

Real vs. Synthetic
RF Data <cit.>

(225.25,275) node  [font=] [align=left] [lt]94.86pt

Radio and Signal Classification <cit.>

(75.25,325) node  [font=] [align=left] [lt]95.2pt

C. Joint Source
Channel Coding

(75.25,375) node  [font=] [align=left] [lt]94.86pt

Perceptual Quality Optimization <cit.>

(75.25,425) node  [font=] [align=left] [lt]95.2pt

Robust and Adaptable JSCC <cit.>

(76.38,475) node  [font=] [align=left] [lt]93.67pt

Lightweight and Reliable Comm. <cit.>

(225.25,325) node  [font=] [align=left] [lt]95.2pt

D. Spectrum Sensing and Recovery

(222.88,375) node  [font=] [align=left] [lt]91.63pt

Data Augment. & Domain Adapt. <cit.>

(225.25,425) node  [font=] [align=left] [lt]95.2pt

Generative Adversarial Radios <cit.>

(225.25,475) node  [font=] [align=left] [lt]94.86pt

Recovering Missing Spectral Info. <cit.>

(375.25,25) node  [font=] [align=left] [lt]95.2pt

E. Signal Detection and Decoding

(375,75) node  [font=] [align=left] [lt]91.63pt

Interference Recognition <cit.>

(375.13,125) node  [font=] [align=left] [lt]95.03pt

Wireless Signal Enhancement <cit.>

(375.38,175) node  [font=] [align=left] [lt]95.03pt

Burst-Error Characterization <cit.>

(375.25,225) node  [font=] [align=left] [lt]95.03pt

MIMO Detection <cit.>

(375.25,275) node  [font=] [align=left] [lt]95.2pt

F. End-to-End System Design

(375.38,325) node  [font=] [align=left] [lt]95.2pt

Channel-Agnostic E2E Systems <cit.>

(375.13,375) node  [font=] [align=left] [lt]95.03pt

Synthesizing Modulation and Coding <cit.>

(375.44,425) node  [font=] [align=left] [lt]95.12pt

VAE-based E2E
System Design <cit.>

(224.5,52.5) node  [font=] [align=left] [lt]95.2pt

Sec. <ref> - PHY
Layer Design

(342.77,464.5) node  [font=] [align=left] [lt]43.84pt

(410,465) node  [font=] [align=left] [lt]40.8pt

(372.5,484) node  [font=] [align=left] [lt]30.51pt

GenAI applications in PHY Layer Design.

§ GENAI FOR PHYSICAL LAYER DESIGN

GMs have shown significant promise in addressing various challenges associated with the physical layer aspects of wireless networks. By learning the underlying structure and patterns in data, GM can contribute to improving the efficiency and reliability of wireless communication in several ways, which are the main focus of this section.

§.§ CHANNEL MODELING, ESTIMATION, AND EQUALIZATION

§.§.§ CHANNEL MODELING AND ESTIMATION

GMs provide a powerful means for learning complex channel characteristics in wireless communication systems, including those with mathematically intractable or unknown channel models. They are capable of capturing non-linear distortions and time-varying channel properties, thereby enabling accurate channel estimation and modeling. In the remainder, we will cover state-of-the-art GM-based channel modeling and estimation methods studied in the literature:

§.§.§ mmWave and THz Channel Estimation:

Due to distinct signal propagation characteristics and harsh channel impediments at high carrier frequencies, mmWave and THz communications present unique challenges in channel estimation and modeling, especially with large number of antenna elements. GMs have been successfully applied in this domain to improve the physical layer aspects of wireless networks. Li et al. <cit.> proposed a novel solution for highly-mobile mmWave systems, leveraging DL tools to efficiently learn and predict mmWave channel covariance matrices by treating them as 2D images. A cGAN is leveraged to learn the implicit mapping function between the received training signals, which act as RF signatures of the environment, and a sparse representation of the large-dimensional channel covariance matrix. In a separate study, Zhang et al. <cit.> developed a data-driven air-to-ground channel estimation framework for mmWave communications for UAV networks using a cGAN. To broaden the applicability of the trained channel model, a cooperative framework based on a distributed cGAN architecture is proposed, allowing UAVs to collaboratively learn the mmWave channel distribution in a fully-distributed manner. Meanwhile, Balevi et al. <cit.> proposed a GAN-based channel estimation method to accurately estimate frequency selective mmWave and THz channels, even with reduced number of pilots at low SNR conditions. GANs are trained to produce channel samples from the true but unknown channel distribution, which is then used as a prior to estimate the current channel, without need for retraining even if the number of clusters and rays change considerably.

§.§.§ Model-Free Stochastic Channel Approximations:

Recent works on training GMs to approximate wireless channel responses has focused on accurately reflecting the PDFs of stochastic channel behaviors <cit.>. Timothy et al. <cit.> introduced the use of variational GANs to provide appropriate architecture and loss functions that accurately capture these stochastic behaviors. Their model-free approach enables the optimization of communication system performance for specific over-the-air channel effects or scenarios exhibiting complex combinations of channel effects. Similarly, Smith et al. <cit.> demonstrated the versatility of GAN architecture in handling complex communication environments with non-linear amplifier distortion, pulse shape filtering, ISI, frequency-dependent group delay, multipath, and non-Gaussian statistics.

§.§.§ Autonomous Channel Modeling:

Yang et al. <cit.> proposed a novel GAN framework for autonomous wireless channel modeling, eliminating the need for complex theoretical analysis or data processing typically required in traditional methods like ray-tracing and geometry-based stochastic channel models. The GAN is trained with raw measurement data to approximate the distribution of a typical AWGN channel, showcasing its effectiveness and potential to improve the physical layer of wireless networks.

§.§.§ MIMO Channel Estimation with Score-Based GMs:

Arvinte et al. <cit.> presented a novel approach for MIMO channel estimation using score-based GMs, which estimate the gradient of the log-prior of channels in high-dimensional space by leveraging the model for channel estimation through posterior sampling. The score-based model is trained on channel realizations from the clustered delay line (CDL)-D model and demonstrates competitive in- and out-of-distribution performance compared to GAN and compressed sensing methods. In tests on CDL-D and CDL-C channels, the proposed approach achieves significant gains in channel estimation error and end-to-end coded performance, highlighting the robustness and effectiveness of the method for MIMO channel estimation in wireless networks.

§.§.§ BLIND CHANNEL EQUALIZATION

Channel equalization is essential to compensate for distortions introduced by communication channels. As the received signals are typically distorted versions of the transmitted ones, channel equalization aims to mitigate these distortions to enhance the quality of the received signals. Recent advancements in channel equalization involve the use of VAEs to develop blind equalization techniques, as outlined below:

In <cit.>, the authors propose an innovative approach for blind channel equalization and estimation using the vector quantized (VQ) VAE framework. This VQ-VAE-based blind channel equalizer addresses the limitations of previous VAE-based equalizers by broadening their applicability to nonlinear systems employing high-order modulation formats. Simulation results reveal that the proposed method achieves performance comparable to a data-aided equalizer using the minimum-mean-squared-error criterion, while surpassing blind constant modulus algorithm (CMA) and VAE-based channel equalizers.

In <cit.>, the authors introduce a novel maximum likelihood estimation approach for blind channel equalization using VAEs. The proposed VAE-based blind channel equalizer demonstrates significant improvements in the error rate of reconstructed symbols compared to the CMA. This new equalization method enables lower latency acquisition of unknown channel responses and extends the application of DL techniques to blind channel equalization for noisy ISI channels with unknown impulse responses.

In <cit.>, the authors explore the potential of adaptive blind equalizers based on variational inference for carrier recovery in optical communications. The proposed generalized VAE equalizer accommodates higher-order modulation formats, probabilistic constellation shaping (PCS), receiver oversampling, and dual-polarization transmission. The VAE equalizer provides reliable channel estimation and outperforms the state-of-the-art CMA for PCS in both fixed and time-varying channels. This research underscores the potential of GMs to enhance the physical layer design, particularly in coherent optical communication systems.

§.§ MODULATION RECOGNITION

GMs, particularly GANs, have demonstrated remarkable potential in recognizing and classifying modulation schemes used in wireless communication systems. By learning the unique features and patterns of different modulation schemes, these models accurately identify the modulation type of an incoming signal, paving the way for adaptive modulation techniques and promoting efficient spectrum utilization.

3 Data Augmentation and Improved Performance: In <cit.> and <cit.>, the authors propose data augmentation methods using cGAN and auxiliary classifier GANs (ACGANs), respectively, to tackle the challenge of limited high-quality labeled data in DL-based automatic modulation classification (AMC) for wireless communications. Both approaches generate high-quality, labeled wireless modulation data from a small set of real data while preserving the high-level features of the original dataset, improving the performance of CNN-based AMC systems. These cGAN-based and ACGAN-based data augmentation methods yield significant improvements in modulation classification accuracy, demonstrating the potential of GMs in enhancing the physical layer design.
3 Real vs. Synthetic RF Data: In <cit.>, the authors examine the impact of data types and data augmentation on the performance of RF-ML systems, particularly in the context of AMC. The authors explore three main questions: the effectiveness of synthetically trained systems in real-world scenarios, the role of augmentation in the RF-ML domain, and how knowledge of signal degradations caused by the transmission channel contributes to the system performance. The paper reveals the value of augmentation compared to extended data capture and whether understanding the distributions of degradation sources affects the network's ability to achieve peak performance.

3 Radio and Signal Classifiers: In <cit.> and <cit.>, the authors present Radio Classify GAN (RC-GANs) and Signal Classifier GANs (SC-GANs), respectively. These are scalable, end-to-end frameworks that introduce GANs into the radio ML domain for modulation recognition. RCGANs are trained on a synthetic RF dataset and learn features through self-optimization during an extensive data-driven training process. On the other hand, SC-GANs improve GANs by effectively avoiding nonconvergence and mode collapse problems caused by the complexity of RF signals. Both RC-GANs and SC-GAN demonstrate improvements in classification accuracy compared to traditional DL methods and exhibit robustness against noise. By employing GANs for modulation recognition, these methods offer a promising approach to improve the physical layer of wireless networks by enabling AMC without relying on manual experience.

§.§ JOINT SOURCE-CHANNEL CODING (JSCC)
GMs have been effectively utilized in JSCC schemes, where source and channel codes are designed together to achieve efficient and robust communication. By learning the statistical properties of the source data and the characteristics of the channel, these models facilitate the development of optimized joint source-channel codes, leading to improved end-to-end system performance. In the remainder, we outline recent developments in this area, focusing on the application of GMs to various domains and their ability to address the challenges faced in modern communication systems.

3 Image Transmission & Perceptual Quality Optimization: Perceptual quality has become an essential aspect of modern communication systems. In <cit.>, the authors propose the use StyleGAN for two novel JSCC schemes, InverseJSCC and GenerativeJSCC, for enhancing the perceptual quality of wireless image transmission in extreme physical conditions. These schemes leverage deep GMs to optimize a weighted sum of mean squared error and learn perceptual image patch similarity losses. Moreover, <cit.> investigates the performance of VAEs compared to standard AEs for the transmission of image data over Gaussian point-to-point wireless channels, demonstrating that VAEs are more robust to channel degradation than AEs. Additionally, the authors employ the structural similarity, a perception-based metric, to optimize neural networks for improved human perceptual quality of the reconstructed images at the receiver.

3 Robust and Adaptable JSCC with VAEs: JSCC schemes must maintain robust performance under varying channel conditions to be effective in real-world communication systems. In a series of studies, <cit.>, <cit.>, and <cit.>, the authors use VAE-based approaches that exhibit robustness to channel condition variations, offering more practical solutions for wireless networks. These approaches demonstrate adaptability in handling different types of data sources and communication scenarios, such as Gaussian sources over multiple AWGN channels, distributed Gaussian sources over a multiple access AWGN channel, and analog independent additive noise channels.

3 Lightweight Communication with VQ-VAEs: In <cit.>, the authors investigate the use of VQ-VAEs for point-to-point wireless communication. They modify the VQ-VAE's training process to design a JSCC that is robust against noisy wireless channels, aiming to make the physical/link layer lighter while preserving system reliability. This approach offers an alternative to conventional separated source-channel coding schemes and demonstrates its potential for lighter physical/link layers in IoT networks.

§.§ SPECTRUM SENSING AND RECOVERY

GMs have been effectively employed in spectrum sensing techniques, which are crucial for efficient spectrum management and dynamic spectrum access in cognitive radio networks. By learning the characteristics of signal occupancy and interference patterns in the spectrum, these models can accurately detect the presence or absence of primary users, enabling secondary users to access the spectrum without causing harmful interference.

3 Data Augmentation and Domain Adaptation: In <cit.>, the authors address the challenges of training data augmentation and domain adaptation in cognitive radio applications by proposing the spectrum augmentation/adaptation with GANs (SAGA) approach. This novel method leverages a deep GAN to generate additional synthetic training data and adapt the training data to spectrum dynamics. The results demonstrate that the SAGA approach significantly increases classifier accuracy, which is sustained with domain adaptation as spectrum conditions change. By augmenting and adapting training data on the fly, SAGA enables cognitive radios to use ML effectively and seamlessly transition between different spectrum conditions.

3 Generative Adversarial Radio Spectrum Networks: In <cit.>, the authors explore the use of GANs for generative replay to improve the physical layer of wireless networks. This approach allows for the creation of signals with similar structure and properties to arbitrary signals, without being exact replicas. The study extends the use of GANs to full-band spectral generation, validating the feasibility of the approach, refining the algorithm, and quantifying its capabilities. The authors aim to push the boundaries of GANs in the radio domain by examining their ability to generate large portions of radio signals, accurately produce samples from random generators, and simulate large radio environments using a naive learning approach.

3 Recovering Missing Spectral Information: In <cit.>, the authors propose GANs called SARGAN to recover missing spectral information in ultra-wideband radar systems. The proposed SARGAN learns the relationship between original and missing-frequency-band signals by observing numerous possible training pairs. The initial results indicate that the recovered signals achieve more than an 18 dB gain in the SNR, even when up to 90% of the operating spectrum is missing, without requiring prior knowledge of the missing frequency band locations. This approach offers advantages over traditional spectral recovery techniques, as it shifts the computational complexity to the training phase and does not require information on the missing band locations.

§.§ ENHANCED DETECTION AND DECODING

GMs have been employed in the development of advanced signal detection and decoding algorithms, enabling efficient signal detection in complex scenarios where traditional methods may fail. By learning the underlying relationships between transmitted and received signals, these models can adapt to varying channel conditions, reducing the need for explicit channel estimation and minimizing the impact of channel estimation errors on the performance of signal detection algorithms.

3 Interference Recognition with Quadruple GAN: In <cit.>, the authors propose an improved Quadruple GAN (QGAN) approachh for wireless interference signal recognition, which is crucial for mitigating cross-technology interference in wireless local area networks (WLANs). The improved QGAN enhances the performance of the original QGAN by substituting it with an auxiliary classifier GAN (ACGAN) and optimizing loss functions for generative, representation, and classification sub-networks. Furthermore, a lightweight model based on knowledge distillation is introduced to reduce memory consumption and computational complexity during the inference phase.

3 Wireless Signal Enhancement: <cit.> presents a wireless Signal Enhancement Generative Adversarial Network (WSE-GAN) for adaptively learning signal characteristics and enhancing wireless communication signals in time-varying systems. The WSE-GAN incorporates the raw time-domain signal as a condition to achieve state-of-the-art enhancement effects while preserving symbol information. The proposed network demonstrates robust learning ability for dynamic channel effects and excellent adversarial capabilities for signal jitter and skews. By focusing on original signals, WSE-GAN maintains exceptional enhancement performance even when the statistical characteristics of channel effects differ between training and testing stages.

3 Burst-Error Characterization: In <cit.>, the authors present deterministic-process-based GMs (DPBGMs) for simulating wireless channels with hard and soft decision outputs, which are crucial for designing and evaluating wireless communication protocols and error-control schemes. The proposed DPBGMs are applied to uncoded enhanced general packet radio service systems with typical urban and rural area channels, demonstrating an excellent approximation to the burst-error statistics of the target hard and soft error sequences. This research showcases the use of GMs to accurately and efficiently simulate digital wireless channels, enabling improved design and performance evaluation of wireless communication systems at the physical layer.

3 Semi-Supervised MIMO Detection with CycleGAN: <cit.> proposes a novel semi-supervised deep MIMO detection approach using CycleGAN for wireless communication systems without prior knowledge of underlying channel distributions. The proposed CycleGAN detector consists of a bidirectional loop of two modified LS-GAN, where the forward LS-GAN models the transmission process, and the backward LS-GAN detects the received signals. By optimizing the cycle-consistency of transmitted and received signals, the method can be trained online and semi-supervisedly using both pilots and received payload data, effectively reducing overhead. The CycleGAN detector does not require explicit channel estimation and performs well under unknown channel effects. The proposed method outperforms existing semi-blind DL detection techniques and conventional linear detectors in terms of BER and achievable rate, especially in scenarios with nonlinear power amplifier distortion.

§.§ END-TO-END (E2E) SYSTEM DESIGN

GMs offer promising advancements in E2E system design for wireless communications, optimizing the entire communication system from transmitter to receiver. By learning relationships between system parameters and performance metrics, these models can predict optimal configurations and settings for various components, resulting in improved network efficiency and reliability.

3 Channel-Agnostic E2E Systems: In <cit.>, the authors propose a channel-agnostic E2E wireless communication system using DNNs for all signal-related functionalities, including encoding, decoding, modulation, and equalization. They address the challenge of not having prior CSI by employing a cGAN to represent the channel effects. Through iterative training, the E2E loss can be optimized in a supervised manner, and the proposed method demonstrates effectiveness on AWGN and Rayleigh fading channels. This approach showcases the potential of building data-driven communication systems that do not rely on prior information about the channel.

3 Learning-Based Modulation and Coding Schemes: <cit.> presents a novel learning-based method for synthesizing new physical layer modulation and coding schemes for communication systems without requiring an analytic model of channel impairments. By adopting an adversarial approach, the authors jointly learn a solution to channel response approximation and information encoding applicable over a wide range of channel environments. The proposed adversarial system offers the potential to learn radio communication schemes directly from the sampled response of real physical hardware devices and channels, rather than relying on simplified models that may not fully capture real-world complexities. This method can enable the development of more efficient and adaptive modulation and demodulation functions that optimize system performance for global metrics or loss functions.

3 E2E Systems with VAEs: In <cit.>, the authors propose a novel E2E wireless communication system using VAEs to reconstruct transmitted symbols without sending data bits from the transmitter. The key innovation lies in representing the symbol as an image hot vector containing features such as spikes, closed squared frame, pixel index location, and pixel grayscale colors. These features are inferred by latent random variables (LRVs), which are transmitted through the physical wireless channel instead of the original bits or hot vectors. The proposed VAE architecture can reconstruct the transmitted symbol from the received LRVs, and the results demonstrate that using a VAE with a simple classifier provides a better SER compared to both autoencoder baseline and classical Hamming code with hard decision decoding, particularly at high $E_b/N_0$.

§.§ SUMMARY, INSIGHTS, AND FUTURE DIRECTIONS

The utilization of GMs in the physical layer design of wireless communication systems has shown remarkable promise. This paper discussed the key applications of GMs in various aspects of the physical layer, ranging from channel estimation to blind channel equalization, modulation recognition, and JSCC. In the rest, we provide key insights and prospective research directions:

72  GMs can accurately model and estimate complex channel characteristics, even in high-frequency mmWave and THz communications; offering state-of-the-art data-driven channel estimation techniques that can outperform traditional approaches, especially in highly dynamic environments. Future work could focus on optimizing these models for real-time deployment and studying their robustness under different environmental conditions. Additionally, investigating the performance trade-offs between model complexity and channel estimation accuracy can be valuable.

72GANs have shown to be capable of autonomously identifying various modulation schemes, thus enabling adaptive modulation techniques and promoting efficient spectrum utilization. Data augmentation methods improve classification accuracy, and studies have also confirmed the robustness of synthetically trained models. As adaptive modulation becomes more essential for dynamic spectrum management, real-time GM-based techniques will likely be a hot topic for research. Thus, the domain of modulation recognition can benefit from future GM research that focuses on real-time adaptive mechanisms and lower-latency operations.

72GMs have also been used effectively in JSCC schemes to improve the efficiency and reliability of end-to-end system performance. Novel approaches have been proposed to optimize perceptual quality in image transmission, demonstrating robustness against varying channel conditions. There is room for further investigation into optimization techniques that can handle different types of data transmission, such as video and text, in addition to images. Research can also look into the applicability of GMs in multi-user scenarios and their potential in optimizing system-level performance.

72In spectrum sensing and recovery, GANs have played a special role in addressing key challenges such as data augmentation, domain adaptation, and spectral recovery. The ability to generate synthetic data and adapt to spectrum dynamics provides a robust framework for dynamic spectrum management. Furthermore, the incorporation of GANs in spectral information recovery illustrates a significant advantage over traditional methods by providing a data-driven approach that moves the computational complexity to the training phase. Future research directions could focus on real-world applications, especially in CRNs with varying degrees of user density and spectrum usage. Exploring how GMs react to real-world radio frequency interference can also provide valuable insights.

72In the area of signal detection and decoding, GMs have enabled advanced algorithms for efficient signal recognition, especially in complex scenarios where traditional methods may struggle. They offer adaptive capabilities in varying channel conditions, mitigating the impact of channel estimation errors. Applications such as interference signal recognition and MIMO detection have particularly benefited from these advancements, displaying improved performance metrics. Potential areas for future investigation include GMs' adaptability to more complex interference conditions and their ability to learn and adapt without requiring an explicit training phase. The integration of GMs with other ML techniques (e.g., RL) could provide more adaptive solutions for varying channel conditions.

72GM-based E2E system design has been another milestone, representing a shift from component-wise optimization to holistic system design. Learning-based modulation, channel-agnostic systems, and innovative applications of VAEs exemplify the extensive applicability of GMs in this domain. E2E designs offer a data-driven approach to wireless communication systems, optimizing the overall performance and ensuring robustness against a variety of channel conditions. Future E2E system design research can concentrate on the development of GM-based frameworks that are capable of self-optimization in response to changes in network traffic and conditions. The incorporation of federated learning could be explored for edge-based E2E systems that require low latency and high reliability.

Overall, GMs have proven to be invaluable tools for tackling the challenges associated with the physical layer in wireless communication systems. Their ability to learn complex data distributions and generalize to new, unseen data sets them apart as powerful methods for advancing the state-of-the-art in physical layer design. In addition to domain specific directions presented above, future work could potentially explore applicability of GM in solving other pressing issues such as energy efficiency, low-latency communication, and integration with other layers of the communication protocol stack.

_iiz5x9p6b/.code =  0 bp   0 bp     0   2

_eca2vwzdi/.code =  0 bp   0 bp     0   2

_tk65delzy/.code =  0 bp   0 bp     0   2

_8s7ewvhaz/.code =  0 bp   0 bp     0   2

_hy3oxxp4x/.code =  0 bp   0 bp     0   2

_l9yqtp388/.code =  0 bp   0 bp     0   2

_uq6zqrc9k/.code =  0 bp   0 bp     0   2

_mbs2w2sbk/.code =  0 bp   0 bp     0   2

_wga9rngdx/.code =  0 bp   0 bp     0   2
_zf7xqah70/.code = 0 bp   0 bp     0   2
_t02z2lk5m 150bp color(0bp)=(transparent!90);
_a1d1neraa[shading=_t02z2lk5m,_zf7xqah70] (0,0) rectangle (50bp,50bp);

_ymrfz33i4/.code =  0 bp   0 bp     0   2

_y8n7zceo6/.code =  0 bp   0 bp     0   2
_fooqjy8v6/.code = 0 bp   0 bp     0   2
_s357ii8ym 150bp color(0bp)=(transparent!90);
_r8u8d2016[shading=_s357ii8ym,_fooqjy8v6] (0,0) rectangle (50bp,50bp);

_xwy7sr4tp/.code =  0 bp   0 bp     0   2

_hkqr9d32w/.code =  0 bp   0 bp     0   2
_2xmeamaeb/.code = 0 bp   0 bp     0   2
_7qexzahub 150bp color(0bp)=(transparent!90);
_fxzuillkw[shading=_7qexzahub,_2xmeamaeb] (0,0) rectangle (50bp,50bp);

_gf44mdya0/.code =  0 bp   0 bp     0   2
_fvfobsfuq/.code = 0 bp   0 bp     0   2
_6drid2ldp 150bp color(0bp)=(transparent!90);
_n6bi3rgve[shading=_6drid2ldp,_fvfobsfuq] (0,0) rectangle (50bp,50bp);

_075kpe1g9/.code =  0 bp   0 bp     0   2

_nuwrrpl33/.code =  0 bp   0 bp     0   2
_bsf1nmgam/.code = 0 bp   0 bp     0   2
_1wjw3a5jm 150bp color(0bp)=(transparent!90);
_amqmzivw7[shading=_1wjw3a5jm,_bsf1nmgam] (0,0) rectangle (50bp,50bp);

_3sm61z911/.code =  0 bp   0 bp     0   2
_5twxtjui0/.code = 0 bp   0 bp     0   2
_se0bcokgk 150bp color(0bp)=(transparent!90);
_2b08kntbq[shading=_se0bcokgk,_5twxtjui0] (0,0) rectangle (50bp,50bp);

_zxnng3t9p/.code =  0 bp   0 bp     0   2

_brnspl88w/.code =  0 bp   0 bp     0   2
_sockez8gz/.code = 0 bp   0 bp     0   2
_5tm7yzq5d 150bp color(0bp)=(transparent!90);
_cxthpmt4m[shading=_5tm7yzq5d,_sockez8gz] (0,0) rectangle (50bp,50bp);

_86pwd4rpv/.code =  0 bp   0 bp     0   2
_tl94xvdwr/.code = 0 bp   0 bp     0   2
_6w8u7xetc 150bp color(0bp)=(transparent!90);
_r1hnpvm92[shading=_6w8u7xetc,_tl94xvdwr] (0,0) rectangle (50bp,50bp);

_0voh1wtf1/.code =  0 bp   0 bp     0   2

_qvzx3csp3/.code =  0 bp   0 bp     0   2
_2xwnyi0dp/.code = 0 bp   0 bp     0   2
_tzsisykgz 150bp color(0bp)=(transparent!90);
_deo07dtt0[shading=_tzsisykgz,_2xwnyi0dp] (0,0) rectangle (50bp,50bp);

_oia713dxx/.code =  0 bp   0 bp     0   2
_ztrs82313/.code = 0 bp   0 bp     0   2
_cx6hygifa 150bp color(0bp)=(transparent!90);
_9i6rr27oq[shading=_cx6hygifa,_ztrs82313] (0,0) rectangle (50bp,50bp);

_ofic6u11z/.code =  0 bp   0 bp     0   2

_8dtgbzzql/.code =  0 bp   0 bp     0   2
every picture/.style=line width=0.75pt

[shading=_fzo5yqg7e,_iiz5x9p6b] (5,13.66) .. controls (5,8.88) and (8.88,5) .. (13.66,5) – (476.34,5) .. controls (481.12,5) and (485,8.88) .. (485,13.66) – (485,361.19) .. controls (485,365.97) and (481.12,369.85) .. (476.34,369.85) – (13.66,369.85) .. controls (8.88,369.85) and (5,365.97) .. (5,361.19) – cycle ;
[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ] (5,13.66) .. controls (5,8.88) and (8.88,5) .. (13.66,5) – (476.34,5) .. controls (481.12,5) and (485,8.88) .. (485,13.66) – (485,361.19) .. controls (485,365.97) and (481.12,369.85) .. (476.34,369.85) – (13.66,369.85) .. controls (8.88,369.85) and (5,365.97) .. (5,361.19) – cycle ;

[shading=_trs86552h,_eca2vwzdi] (140,303.87) .. controls (140,298.97) and (143.97,295) .. (148.87,295) – (316.13,295) .. controls (321.03,295) and (325,298.97) .. (325,303.87) – (325,356.13) .. controls (325,361.03) and (321.03,365) .. (316.13,365) – (148.87,365) .. controls (143.97,365) and (140,361.03) .. (140,356.13) – cycle ;
[color=rgb, 255:red, 155; green, 155; blue, 155   ,draw opacity=1 ] (140,303.87) .. controls (140,298.97) and (143.97,295) .. (148.87,295) – (316.13,295) .. controls (321.03,295) and (325,298.97) .. (325,303.87) – (325,356.13) .. controls (325,361.03) and (321.03,365) .. (316.13,365) – (148.87,365) .. controls (143.97,365) and (140,361.03) .. (140,356.13) – cycle ;

[shading=_uivunvpqa,_tk65delzy] (164.29,132.08) .. controls (164.29,128.17) and (167.45,125) .. (171.36,125) – (317.92,125) .. controls (321.83,125) and (325,128.17) .. (325,132.08) – (325,282.92) .. controls (325,286.83) and (321.83,290) .. (317.92,290) – (171.36,290) .. controls (167.45,290) and (164.29,286.83) .. (164.29,282.92) – cycle ;
[color=rgb, 255:red, 155; green, 155; blue, 155   ,draw opacity=1 ] (164.29,132.08) .. controls (164.29,128.17) and (167.45,125) .. (171.36,125) – (317.92,125) .. controls (321.83,125) and (325,128.17) .. (325,132.08) – (325,282.92) .. controls (325,286.83) and (321.83,290) .. (317.92,290) – (171.36,290) .. controls (167.45,290) and (164.29,286.83) .. (164.29,282.92) – cycle ;

[shading=_ufhlxs0bx,_8s7ewvhaz] (330,16.6) .. controls (330,12.96) and (332.96,10) .. (336.6,10) – (473.4,10) .. controls (477.04,10) and (480,12.96) .. (480,16.6) – (480,183.4) .. controls (480,187.04) and (477.04,190) .. (473.4,190) – (336.6,190) .. controls (332.96,190) and (330,187.04) .. (330,183.4) – cycle ;
[color=rgb, 255:red, 155; green, 155; blue, 155   ,draw opacity=1 ] (330,16.6) .. controls (330,12.96) and (332.96,10) .. (336.6,10) – (473.4,10) .. controls (477.04,10) and (480,12.96) .. (480,16.6) – (480,183.4) .. controls (480,187.04) and (477.04,190) .. (473.4,190) – (336.6,190) .. controls (332.96,190) and (330,187.04) .. (330,183.4) – cycle ;

[shading=_s0lcoq7cz,_hy3oxxp4x] (330,206.6) .. controls (330,202.96) and (332.96,200) .. (336.6,200) – (473.4,200) .. controls (477.04,200) and (480,202.96) .. (480,206.6) – (480,358.4) .. controls (480,362.04) and (477.04,365) .. (473.4,365) – (336.6,365) .. controls (332.96,365) and (330,362.04) .. (330,358.4) – cycle ;
[color=rgb, 255:red, 155; green, 155; blue, 155   ,draw opacity=1 ] (330,206.6) .. controls (330,202.96) and (332.96,200) .. (336.6,200) – (473.4,200) .. controls (477.04,200) and (480,202.96) .. (480,206.6) – (480,358.4) .. controls (480,362.04) and (477.04,365) .. (473.4,365) – (336.6,365) .. controls (332.96,365) and (330,362.04) .. (330,358.4) – cycle ;

[shading=_x8yywsngi,_l9yqtp388] (10,196.6) .. controls (10,192.96) and (12.96,190) .. (16.6,190) – (153.4,190) .. controls (157.04,190) and (160,192.96) .. (160,196.6) – (160,358.4) .. controls (160,362.04) and (157.04,365) .. (153.4,365) – (16.6,365) .. controls (12.96,365) and (10,362.04) .. (10,358.4) – cycle ;
[color=rgb, 255:red, 155; green, 155; blue, 155   ,draw opacity=1 ] (10,196.6) .. controls (10,192.96) and (12.96,190) .. (16.6,190) – (153.4,190) .. controls (157.04,190) and (160,192.96) .. (160,196.6) – (160,358.4) .. controls (160,362.04) and (157.04,365) .. (153.4,365) – (16.6,365) .. controls (12.96,365) and (10,362.04) .. (10,358.4) – cycle ;

[shading=_zoajrduz9,_uq6zqrc9k] (10,16.6) .. controls (10,12.96) and (12.96,10) .. (16.6,10) – (153.4,10) .. controls (157.04,10) and (160,12.96) .. (160,16.6) – (160,178.4) .. controls (160,182.04) and (157.04,185) .. (153.4,185) – (16.6,185) .. controls (12.96,185) and (10,182.04) .. (10,178.4) – cycle ;
[color=rgb, 255:red, 155; green, 155; blue, 155   ,draw opacity=1 ] (10,16.6) .. controls (10,12.96) and (12.96,10) .. (16.6,10) – (153.4,10) .. controls (157.04,10) and (160,12.96) .. (160,16.6) – (160,178.4) .. controls (160,182.04) and (157.04,185) .. (153.4,185) – (16.6,185) .. controls (12.96,185) and (10,182.04) .. (10,178.4) – cycle ;

[shading=_g0mwnny0o,_mbs2w2sbk] (165,22.33) .. controls (165,18.28) and (168.28,15) .. (172.33,15) – (317.67,15) .. controls (321.72,15) and (325,18.28) .. (325,22.33) – (325,112.67) .. controls (325,116.72) and (321.72,120) .. (317.67,120) – (172.33,120) .. controls (168.28,120) and (165,116.72) .. (165,112.67) – cycle ;
[line width=0.75]  (165,22.33) .. controls (165,18.28) and (168.28,15) .. (172.33,15) – (317.67,15) .. controls (321.72,15) and (325,18.28) .. (325,22.33) – (325,112.67) .. controls (325,116.72) and (321.72,120) .. (317.67,120) – (172.33,120) .. controls (168.28,120) and (165,116.72) .. (165,112.67) – cycle ;

[shading=_ghoxs4u76,_wga9rngdx,path fading= _a1d1neraa ,fading transform=xshift=2] (15,72) .. controls (15,68.13) and (18.13,65) .. (22,65) – (148,65) .. controls (151.87,65) and (155,68.13) .. (155,72) – (155,111) .. controls (155,114.87) and (151.87,118) .. (148,118) – (22,118) .. controls (18.13,118) and (15,114.87) .. (15,111) – cycle ;
[color=rgb, 255:red, 128; green, 128; blue, 128   ,draw opacity=1 ][line width=1.5]  (15,72) .. controls (15,68.13) and (18.13,65) .. (22,65) – (148,65) .. controls (151.87,65) and (155,68.13) .. (155,72) – (155,111) .. controls (155,114.87) and (151.87,118) .. (148,118) – (22,118) .. controls (18.13,118) and (15,114.87) .. (15,111) – cycle ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (86,55) – (86,65) ;

[shading=_e2qdpj2an,_ymrfz33i4] (16,20.71) .. controls (16,17.56) and (18.56,15) .. (21.71,15) – (149.29,15) .. controls (152.44,15) and (155,17.56) .. (155,20.71) – (155,49.29) .. controls (155,52.44) and (152.44,55) .. (149.29,55) – (21.71,55) .. controls (18.56,55) and (16,52.44) .. (16,49.29) – cycle ;
[line width=0.75]  (16,20.71) .. controls (16,17.56) and (18.56,15) .. (21.71,15) – (149.29,15) .. controls (152.44,15) and (155,17.56) .. (155,20.71) – (155,49.29) .. controls (155,52.44) and (152.44,55) .. (149.29,55) – (21.71,55) .. controls (18.56,55) and (16,52.44) .. (16,49.29) – cycle ;

[shading=_qj2lwuh4v,_y8n7zceo6,path fading= _r8u8d2016 ,fading transform=xshift=2] (15,134.33) .. controls (15,130.84) and (17.84,128) .. (21.33,128) – (148.67,128) .. controls (152.16,128) and (155,130.84) .. (155,134.33) – (155,173.67) .. controls (155,177.16) and (152.16,180) .. (148.67,180) – (21.33,180) .. controls (17.84,180) and (15,177.16) .. (15,173.67) – cycle ;
[color=rgb, 255:red, 128; green, 128; blue, 128   ,draw opacity=1 ][line width=1.5]  (15,134.33) .. controls (15,130.84) and (17.84,128) .. (21.33,128) – (148.67,128) .. controls (152.16,128) and (155,130.84) .. (155,134.33) – (155,173.67) .. controls (155,177.16) and (152.16,180) .. (148.67,180) – (21.33,180) .. controls (17.84,180) and (15,177.16) .. (15,173.67) – cycle ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (85,118) – (85,128) ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (85,235) – (85,245) ;

[shading=_ml3qez9sj,_xwy7sr4tp] (16,200.71) .. controls (16,197.56) and (18.56,195) .. (21.71,195) – (149.29,195) .. controls (152.44,195) and (155,197.56) .. (155,200.71) – (155,229.29) .. controls (155,232.44) and (152.44,235) .. (149.29,235) – (21.71,235) .. controls (18.56,235) and (16,232.44) .. (16,229.29) – cycle ;
[line width=0.75]  (16,200.71) .. controls (16,197.56) and (18.56,195) .. (21.71,195) – (149.29,195) .. controls (152.44,195) and (155,197.56) .. (155,200.71) – (155,229.29) .. controls (155,232.44) and (152.44,235) .. (149.29,235) – (21.71,235) .. controls (18.56,235) and (16,232.44) .. (16,229.29) – cycle ;

[shading=_djk7fqqkl,_hkqr9d32w,path fading= _fxzuillkw ,fading transform=xshift=2] (15,250.15) .. controls (15,247.31) and (17.31,245) .. (20.15,245) – (149.85,245) .. controls (152.69,245) and (155,247.31) .. (155,250.15) – (155,289.85) .. controls (155,292.69) and (152.69,295) .. (149.85,295) – (20.15,295) .. controls (17.31,295) and (15,292.69) .. (15,289.85) – cycle ;
[color=rgb, 255:red, 128; green, 128; blue, 128   ,draw opacity=1 ][line width=1.5]  (15,250.15) .. controls (15,247.31) and (17.31,245) .. (20.15,245) – (149.85,245) .. controls (152.69,245) and (155,247.31) .. (155,250.15) – (155,289.85) .. controls (155,292.69) and (152.69,295) .. (149.85,295) – (20.15,295) .. controls (17.31,295) and (15,292.69) .. (15,289.85) – cycle ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (85,295) – (85,305) ;

[shading=_4a1a3ij7d,_gf44mdya0,path fading= _n6bi3rgve ,fading transform=xshift=2] (15,311.33) .. controls (15,307.84) and (17.84,305) .. (21.33,305) – (148.67,305) .. controls (152.16,305) and (155,307.84) .. (155,311.33) – (155,353.67) .. controls (155,357.16) and (152.16,360) .. (148.67,360) – (21.33,360) .. controls (17.84,360) and (15,357.16) .. (15,353.67) – cycle ;
[color=rgb, 255:red, 128; green, 128; blue, 128   ,draw opacity=1 ][line width=1.5]  (15,311.33) .. controls (15,307.84) and (17.84,305) .. (21.33,305) – (148.67,305) .. controls (152.16,305) and (155,307.84) .. (155,311.33) – (155,353.67) .. controls (155,357.16) and (152.16,360) .. (148.67,360) – (21.33,360) .. controls (17.84,360) and (15,357.16) .. (15,353.67) – cycle ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (405,245) – (405,255) ;

[shading=_dxi5y7yar,_075kpe1g9] (336,210.71) .. controls (336,207.56) and (338.56,205) .. (341.71,205) – (469.29,205) .. controls (472.44,205) and (475,207.56) .. (475,210.71) – (475,239.29) .. controls (475,242.44) and (472.44,245) .. (469.29,245) – (341.71,245) .. controls (338.56,245) and (336,242.44) .. (336,239.29) – cycle ;
[line width=0.75]  (336,210.71) .. controls (336,207.56) and (338.56,205) .. (341.71,205) – (469.29,205) .. controls (472.44,205) and (475,207.56) .. (475,210.71) – (475,239.29) .. controls (475,242.44) and (472.44,245) .. (469.29,245) – (341.71,245) .. controls (338.56,245) and (336,242.44) .. (336,239.29) – cycle ;

[shading=_bycj24oqn,_nuwrrpl33,path fading= _amqmzivw7 ,fading transform=xshift=2] (335,259.67) .. controls (335,257.09) and (337.09,255) .. (339.67,255) – (470.33,255) .. controls (472.91,255) and (475,257.09) .. (475,259.67) – (475,290.33) .. controls (475,292.91) and (472.91,295) .. (470.33,295) – (339.67,295) .. controls (337.09,295) and (335,292.91) .. (335,290.33) – cycle ;
[color=rgb, 255:red, 128; green, 128; blue, 128   ,draw opacity=1 ][line width=1.5]  (335,259.67) .. controls (335,257.09) and (337.09,255) .. (339.67,255) – (470.33,255) .. controls (472.91,255) and (475,257.09) .. (475,259.67) – (475,290.33) .. controls (475,292.91) and (472.91,295) .. (470.33,295) – (339.67,295) .. controls (337.09,295) and (335,292.91) .. (335,290.33) – cycle ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (405,295) – (405,305) ;

[shading=_q2h20tgtl,_3sm61z911,path fading= _2b08kntbq ,fading transform=xshift=2] (335,310.33) .. controls (335,307.39) and (337.39,305) .. (340.33,305) – (469.67,305) .. controls (472.61,305) and (475,307.39) .. (475,310.33) – (475,354.67) .. controls (475,357.61) and (472.61,360) .. (469.67,360) – (340.33,360) .. controls (337.39,360) and (335,357.61) .. (335,354.67) – cycle ;
[color=rgb, 255:red, 128; green, 128; blue, 128   ,draw opacity=1 ][line width=1.5]  (335,310.33) .. controls (335,307.39) and (337.39,305) .. (340.33,305) – (469.67,305) .. controls (472.61,305) and (475,307.39) .. (475,310.33) – (475,354.67) .. controls (475,357.61) and (472.61,360) .. (469.67,360) – (340.33,360) .. controls (337.39,360) and (335,357.61) .. (335,354.67) – cycle ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (405,55) – (405,65) ;

[shading=_4mpioe0og,_zxnng3t9p] (336,20.71) .. controls (336,17.56) and (338.56,15) .. (341.71,15) – (469.29,15) .. controls (472.44,15) and (475,17.56) .. (475,20.71) – (475,49.29) .. controls (475,52.44) and (472.44,55) .. (469.29,55) – (341.71,55) .. controls (338.56,55) and (336,52.44) .. (336,49.29) – cycle ;
[line width=0.75]  (336,20.71) .. controls (336,17.56) and (338.56,15) .. (341.71,15) – (469.29,15) .. controls (472.44,15) and (475,17.56) .. (475,20.71) – (475,49.29) .. controls (475,52.44) and (472.44,55) .. (469.29,55) – (341.71,55) .. controls (338.56,55) and (336,52.44) .. (336,49.29) – cycle ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (405,120) – (405,130) ;

[shading=_oy8slsmjf,_brnspl88w,path fading= _cxthpmt4m ,fading transform=xshift=2] (335,135.33) .. controls (335,132.39) and (337.39,130) .. (340.33,130) – (469.67,130) .. controls (472.61,130) and (475,132.39) .. (475,135.33) – (475,179.67) .. controls (475,182.61) and (472.61,185) .. (469.67,185) – (340.33,185) .. controls (337.39,185) and (335,182.61) .. (335,179.67) – cycle ;
[color=rgb, 255:red, 128; green, 128; blue, 128   ,draw opacity=1 ][line width=1.5]  (335,135.33) .. controls (335,132.39) and (337.39,130) .. (340.33,130) – (469.67,130) .. controls (472.61,130) and (475,132.39) .. (475,135.33) – (475,179.67) .. controls (475,182.61) and (472.61,185) .. (469.67,185) – (340.33,185) .. controls (337.39,185) and (335,182.61) .. (335,179.67) – cycle ;

[shading=_7pxwj6k25,_86pwd4rpv,path fading= _r1hnpvm92 ,fading transform=xshift=2] (335,70.33) .. controls (335,67.39) and (337.39,65) .. (340.33,65) – (469.67,65) .. controls (472.61,65) and (475,67.39) .. (475,70.33) – (475,114.67) .. controls (475,117.61) and (472.61,120) .. (469.67,120) – (340.33,120) .. controls (337.39,120) and (335,117.61) .. (335,114.67) – cycle ;
[color=rgb, 255:red, 128; green, 128; blue, 128   ,draw opacity=1 ][line width=1.5]  (335,70.33) .. controls (335,67.39) and (337.39,65) .. (340.33,65) – (469.67,65) .. controls (472.61,65) and (475,67.39) .. (475,70.33) – (475,114.67) .. controls (475,117.61) and (472.61,120) .. (469.67,120) – (340.33,120) .. controls (337.39,120) and (335,117.61) .. (335,114.67) – cycle ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (245,170) – (245,180) ;

[shading=_jlsylzxhs,_0voh1wtf1] (170,135.71) .. controls (170,132.56) and (172.56,130) .. (175.71,130) – (314.29,130) .. controls (317.44,130) and (320,132.56) .. (320,135.71) – (320,164.29) .. controls (320,167.44) and (317.44,170) .. (314.29,170) – (175.71,170) .. controls (172.56,170) and (170,167.44) .. (170,164.29) – cycle ;
[line width=0.75]  (170,135.71) .. controls (170,132.56) and (172.56,130) .. (175.71,130) – (314.29,130) .. controls (317.44,130) and (320,132.56) .. (320,135.71) – (320,164.29) .. controls (320,167.44) and (317.44,170) .. (314.29,170) – (175.71,170) .. controls (172.56,170) and (170,167.44) .. (170,164.29) – cycle ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (245,235) – (245,245) ;

[shading=_v4eu6y63p,_qvzx3csp3,path fading= _deo07dtt0 ,fading transform=xshift=2] (170,248.88) .. controls (170,246.74) and (171.74,245) .. (173.88,245) – (316.12,245) .. controls (318.26,245) and (320,246.74) .. (320,248.88) – (320,281.12) .. controls (320,283.26) and (318.26,285) .. (316.12,285) – (173.88,285) .. controls (171.74,285) and (170,283.26) .. (170,281.12) – cycle ;
[color=rgb, 255:red, 128; green, 128; blue, 128   ,draw opacity=1 ][line width=1.5]  (170,248.88) .. controls (170,246.74) and (171.74,245) .. (173.88,245) – (316.12,245) .. controls (318.26,245) and (320,246.74) .. (320,248.88) – (320,281.12) .. controls (320,283.26) and (318.26,285) .. (316.12,285) – (173.88,285) .. controls (171.74,285) and (170,283.26) .. (170,281.12) – cycle ;

[shading=_wd7kl0bbl,_oia713dxx,path fading= _9i6rr27oq ,fading transform=xshift=2] (170,185.33) .. controls (170,182.39) and (172.39,180) .. (175.33,180) – (314.67,180) .. controls (317.61,180) and (320,182.39) .. (320,185.33) – (320,229.67) .. controls (320,232.61) and (317.61,235) .. (314.67,235) – (175.33,235) .. controls (172.39,235) and (170,232.61) .. (170,229.67) – cycle ;
[color=rgb, 255:red, 128; green, 128; blue, 128   ,draw opacity=1 ][line width=1.5]  (170,185.33) .. controls (170,182.39) and (172.39,180) .. (175.33,180) – (314.67,180) .. controls (317.61,180) and (320,182.39) .. (320,185.33) – (320,229.67) .. controls (320,232.61) and (317.61,235) .. (314.67,235) – (175.33,235) .. controls (172.39,235) and (170,232.61) .. (170,229.67) – cycle ;

[shading=_ozt2lo4g3,_ofic6u11z] (170,305.71) .. controls (170,302.56) and (172.56,300) .. (175.71,300) – (314.29,300) .. controls (317.44,300) and (320,302.56) .. (320,305.71) – (320,334.29) .. controls (320,337.44) and (317.44,340) .. (314.29,340) – (175.71,340) .. controls (172.56,340) and (170,337.44) .. (170,334.29) – cycle ;
[color=rgb, 255:red, 128; green, 128; blue, 128   ,draw opacity=1 ][dash pattern=on 5.63pt off 4.5pt][line width=1.5]  (170,305.71) .. controls (170,302.56) and (172.56,300) .. (175.71,300) – (314.29,300) .. controls (317.44,300) and (320,302.56) .. (320,305.71) – (320,334.29) .. controls (320,337.44) and (317.44,340) .. (314.29,340) – (175.71,340) .. controls (172.56,340) and (170,337.44) .. (170,334.29) – cycle ;

[color=rgb, 255:red, 128; green, 128; blue, 128   ,draw opacity=1 ][dash pattern=on 5.63pt off 4.5pt][line width=1.5]  (244.33,347.08) .. controls (244.33,345.02) and (246,343.35) .. (248.06,343.35) – (280.27,343.35) .. controls (282.33,343.35) and (284,345.02) .. (284,347.08) – (284,358.27) .. controls (284,360.33) and (282.33,362) .. (280.27,362) – (248.06,362) .. controls (246,362) and (244.33,360.33) .. (244.33,358.27) – cycle ;

[draw opacity=0][shading=_15eu1drzd,_8dtgbzzql] (155.78,296.84) .. controls (155.78,296.14) and (156.34,295.57) .. (157.04,295.57) – (164.51,295.57) .. controls (165.21,295.57) and (165.78,296.14) .. (165.78,296.84) – (165.78,363.31) .. controls (165.78,364.01) and (165.21,364.58) .. (164.51,364.58) – (157.04,364.58) .. controls (156.34,364.58) and (155.78,364.01) .. (155.78,363.31) – cycle ;

[color=rgb, 255:red, 128; green, 128; blue, 128   ,draw opacity=1 ][line width=1.5]  (199.67,347.08) .. controls (199.67,345.02) and (201.34,343.35) .. (203.4,343.35) – (235.6,343.35) .. controls (237.66,343.35) and (239.33,345.02) .. (239.33,347.08) – (239.33,358.27) .. controls (239.33,360.33) and (237.66,362) .. (235.6,362) – (203.4,362) .. controls (201.34,362) and (199.67,360.33) .. (199.67,358.27) – cycle ;

(85.5,35) node  [font=] [align=left] [lt]95.2pt

A. Enhancing DRLs for Network Slicing

(85,91.5) node  [font=] [align=left] [lt]95.2pt

GAN+DDQN tackling DRLs' nonstationarity and over/under estimation <cit.>

(85,154) node  [font=] [align=left] [lt]95.2pt

Dueling GAN+DDQN for dynamic and efficient res. allocation per slice <cit.>

(245,67.5) node  [font=] [align=left] [lt]108.8pt

Sec. <ref> - Network Optimization, Organization,
& Management

(85.5,215) node  [font=] [align=left] [lt]95.2pt

B. Resource & Slicing Alloc. Predict.

(85,269.46) node  [font=] [align=left] [lt]95.2pt

GAN-Slicing to predict dynamic service-oriented SDNs <cit.>

(85,332.5) node  [font=] [align=left] [lt]95.2pt

cGAN to predict HetVNET congestion and dynamicity <cit.>

(405.5,225) node  [font=] [align=left] [lt]95.2pt

C. Self Organizing Networks

(405,275) node  [font=] [align=left] [lt]95.2pt

GAN+Adaboost for cell outage detection <cit.>

(405,332.5) node  [font=] [align=left] [lt]95.2pt

GAN for generating real. synthetic data for pro-active SON functions <cit.>

(405.5,35) node  [font=] [align=left] [lt]95.2pt

D. Adapting Dynamic Environments

(405,92.5) node  [font=] [align=left] [lt]95.2pt

LSTM+GAN to improve  mobility tracking and forecasting of DRLs <cit.>

(405,157.5) node  [font=] [align=left] [lt]95.2pt

Training DRL with GAN generated synthetic data to boost URLLC <cit.>

(245,150) node  [font=] [align=left] [lt]102.26pt

E. Wireless Coverage Estim. & Optim.

(245.25,206.5) node  [font=] [align=left] [lt]101.66pt

GAN to produce rational and acurate wireless coverage map <cit.>

(245,265) node  [font=] [align=left] [lt]102.34pt

GANs for coverage planning and optimization <cit.>

(245,320) node  [font=] [align=left] [lt]102.26pt

VAE for Solving
Assignment Problems <cit.>

(264.67,352.68) node  [font=] [align=left] [lt]27.43pt

(219.33,351.77) node  [font=] [align=left] [lt]28.11pt

GAN applications for network optimization, organization, and management

§ GENAI FOR NETWORK OPTIMIZATION, ORGANIZATION, AND MANAGEMENT

GMs have shown great potential for addressing various challenges in network optimization, organization, and resource management. By generating synthetic data and improving learning algorithms, GANs and other GMs can enhance the performance of wireless networks, mobile network slicing, and self-organization features. In the remainder, we present a range of studies that demonstrate the efficacy of GMs in tackling problems related to resource allocation, network organization and management management, and performance optimization in diverse communication scenarios.

§.§ ENHANCING ML ALGORITHMS
GMs can improve the learning capabilities of RL/DRL agents by generating samples that represent the distribution of state-action values <cit.>. This mitigates the effects of learning from nonstationary environments, avoiding action-value overestimation or underestimation issues, and ultimately leading to better network optimization. Hua et al. <cit.> propose a GAN-based Deep Distributional Q Network (GAN-DDQN) to address the challenge of demand-aware resource allocation in 5G network slicing. The GAN-DDQN model generates samples for each action, representing the distribution of state-action values, and mitigates issues stemming from learning in a nonstationary environment. The authors also introduce a reward-clipping mechanism to handle destabilizing effects during training. In a subsequent paper <cit.>, they develop the Dueling GAN-DDQN, which separates state-value distribution from action-value distribution and combines the action advantage function to obtain action values. Extensive simulations demonstrate the superiority of GAN-DDQN and Dueling GAN-DDQN algorithms over classical DQN in dynamic and efficient resource allocation per slice, leading to better spectrum efficiency and service level agreement satisfaction.

§.§ NETWORK SLICING AND RESOURCE ALLOCATION

GMs can be used to predict user requirements for different resources, enabling efficient and flexible management of network, computing, and storage devices. These predictions
can be employed in dynamic service-oriented network slicing schemes for a better timely and flexible resource provisioning, and improved quality-of-user-experience (QoE). In this regard, Gu et al. <cit.> propose GANSlicing, a dynamic service-oriented software-defined mobile network slicing scheme for resource allocation prediction in IoT applications. This scheme aims to improve users' QoE by allocating resources in a timely and flexible manner. GANSlicing uses GANs to predict user requirements for different resources, allowing for efficient and flexible management of network, computing, and storage devices. GANSlicing outperforms current network slicing schemes, accepting 16% more requests with 12% fewer resources for the same service request batch. Likewise, the work in <cit.> employs a cGAN to augment information related to successful network scenarios, which can be used in the creation of Heterogeneous Vehicular Networks (HetVNETs) slices to address network congestion and dynamicity. The generated data from the cGAN is similar to real data and contributes to enhancing the flexibility and adaptability of the HetVNET. By applying cGAN for the first time in this context, the proposed architecture can intelligently and reliably generate valuable information to create network slices, ultimately avoiding congestion in HetVNETs.

There has been a recent interest in using DL methods to solve assignment problems
<cit.>. Similarly, GMs, specifically VAE variants, can be exploited to solve linear sum assignment problems, which are extensively studied in wireless resource allocation problems. In <cit.>, the authors propose exploiting a VAE variant to solve linear sum assignment problems, commonly exploited in wireless resource allocation problems <cit.>. Simulation results demonstrate that the proposed approach can replace the conventional Hungarian algorithm and quickly and accurately find solutions for large sizes of cost matrices.

§.§ SELF-ORGANIZING NETWORKS (SONs)
GMs can address data scarcity and imbalanced data issues in SONs by generating realistic synthetic data for training ML algorithms powering SON functions <cit.>. This can improve the performance of SONs in detecting cell outages and other network management tasks. In <cit.>, the authors present a novel method for cell outage detection in SONs by combining GAN and Adaboost to address the issue of imbalanced data. GAN is utilized to preprocess the imbalanced data, generating more synthetic samples for the minority class, while Adaboost is employed to classify the balanced dataset. This approach effectively detects cell outages in cellular networks and overcomes the limitations of traditional classification algorithms in dealing with imbalanced data. The proposed method demonstrates significant improvements in classification performance based on various metrics such as receiver operating characteristic, precision, recall rate, and F-value. In <cit.>, the authors propose a method for addressing the data scarcity issue in SON for future mobile 5G networks by utilizing GANs to generate large amounts of realistic synthetic data. GANs are used to create synthetic Call Data Records (CDRs), which can be used as training data for ML algorithms that power SON functions. The study demonstrates that GAN-generated synthetic CDRs can be effectively combined with real data to improve the prediction accuracy of ML algorithms, driving proactive SON functions in future 5G networks. This approach is particularly useful when obtaining more real labeled data is not feasible or efficient, and it represents the first time that GANs have been used to generate synthetic CDRs.

§.§ ADAPTATION TO DYNAMIC ENVIRONMENTS
GMs can enhance the adaptability of algorithms to changes in network conditions, such as UAV mobility patterns <cit.> or rare events in URLLC <cit.>, by generating synthetic data that captures the underlying structure of these patterns. Xu et al. <cit.> propose a RL approach to maximize the sum-rate of multiple-UAV-served MTC by jointly optimizing transmission power, mode, frequency spectrum, relay selection, and trajectory. They combine LSTM and GAN to improve RL's UAV mobility tracking and forecasting efficacy, and overall reward performance. The improved synthetic data generated by the GAN augments the training set for the LSTM network, leading to a more robust model that better captures UAV mobility patterns. Kasgari et al. <cit.> propose an experienced DRL framework for model-free resource allocation for URLLC in 6G networks without user traffic assumptions. By exploiting GANs to pre-train DRL agents with a mix of real and synthetic data, the framework achieves near-optimal performance within the rate-reliability-latency region under extreme network conditions. The GAN-generated dataset also allows the framework to recover faster during unpredicted rare events. GMs can also be integrated into vehicular networks to address challenges such as navigation optimization, traffic prediction, data generation, and evaluation. Zhang et al. <cit.> investigate integrating GenAI technologies into vehicular networks, focusing on applications and challenges such as navigation optimization, traffic prediction, data generation, and evaluation. They propose a multi-modality semantic-aware framework to enhance QoS by utilizing multi-modal and semantic communication technologies. A DRL-based approach for resource allocation in GenAI-enabled V2V communication is also proposed, aiming to maximize the system's QoE within constraints.

§.§ COVERAGE ESTIMATION AND OPTIMIZATION

In <cit.>, the authors propose a weakly-supervised GAN (WS-GAN) with auxiliary information to estimate wireless coverage based on randomly distributed samples of received signal strength. Unlike traditional methods such as kNN or matrix completion, WS-GANs approximate the real distribution of observations by incorporating auxiliary information such as terrain and building data, which significantly impacts the variation of received signal strength, to improve estimation performance. Experiments on a real long-term evolution (LTE) dataset show that WS-GAN outperforms baseline methods in estimation accuracy and produces more rational and practical wireless coverage maps. Moreover, the work in <cit.> proposes a novel approach for small cell coverage planning and performance optimization in HetNets using GAN to derive knowledge and delivers it to each local SDN controller in a simplified form. The key aspect of this approach is effective training of GANs for various topologies, even when there is limited real data about network behavior and performance. In this way, the authors aim to improve network management and self-organizing functionality, ensuring high QoE across different cell radii.

§.§ SUMMARY, INSIGHTS, AND FUTURE DIRECTIONS

Our review reveals that GMs, especially GANs and VAEs, hold considerable promise for various aspects of network management and optimization. From enhanced learning algorithms to solving assignment problems, GMs offer a flexible and effective toolkit for network researchers and practitioners. GMs have demonstrated significant advantages in tackling non-stationary environments, predicting user requirements for resources, dealing with imbalanced and scarce data, and adapting to dynamic network conditions. Techniques such as GAN-DDQN, Dueling GAN-DDQN, GANSlicing, and WS-GAN are among the various innovative approaches that have emerged, providing superior performance over traditional algorithms in tasks like resource allocation, network slicing, and wireless coverage estimation.

One of the crucial insights to draw from this section is that GMs offer a way to make networks more adaptive, efficient, and robust. Through synthetic data generation, GMs can augment real data to create richer datasets for training ML algorithms, thereby improving their performance in tasks like resource allocation and outage detection. Moreover, GMs enhance the capabilities of RL/DRL algorithms, particularly in non-stationary environments, making them more reliable for real-world deployments. The fusion of GMs with existing technologies appears to be not just beneficial but potentially transformative for next-generation networks. Despite the promising advancements, several challenges and opportunities lie ahead:

72 Scalability: As networks grow more complex and large-scale, the scalability of GM-based solutions will be an area requiring rigorous investigation.

72 Security Concerns: The use of synthetic data and automated decision-making could pose new security and privacy challenges that need to be addressed.

72  Real-world Validation: While many studies rely on simulations, there's a pressing need for more empirical studies to validate the effectiveness of GMs in real-world network environments.

72  Interdisciplinary Research: Combining GMs with other emerging technologies like edge computing or quantum computing could lead to breakthroughs that address some of the most pressing challenges in network optimization.

72  Ethical and Regulatory Aspects: As AI and GMs become more integrated into network management, ethical and regulatory considerations such as data privacy and fairness will become increasingly important.

Noting that above works have mainly focused on using GANs and VAEs, without delving into exploring other GMs presented in Section <ref>. Future research should continue to explore and refine these techniques, as well as investigate the integration of various GMs with other emerging technologies to further advance network optimization and resource management.

_jlmpyo6vo/.code =  0 bp   0 bp     0   2

_sz817zhfb/.code =  0 bp   0 bp     0   2

_w7epzq3xm/.code =  0 bp   0 bp     0   2

_gzta5pee7/.code =  0 bp   0 bp     0   2

_gyp949wn6/.code =  0 bp   0 bp     0   2

_wlkerr4ui/.code =  0 bp   0 bp     0   2

_bsoq9hqcn/.code =  0 bp   0 bp     0   2

_l8xswyvw4/.code =  0 bp   0 bp     0   2

_6es6zr2fb/.code =  0 bp   0 bp     0   2

_i3nrrn8xo/.code =  0 bp   0 bp     0   2

_lgxzwadzu/.code =  0 bp   0 bp     0   2

_3lijayp8s/.code =  0 bp   0 bp     0   2

_4lkk5gwx5/.code =  0 bp   0 bp     0   2

_usza90ir8/.code =  0 bp   0 bp     0   2

_fmoh1k8eb/.code =  0 bp   0 bp     0   2

_jdykzyz75/.code =  0 bp   0 bp     0   2

_zbupdvs7q/.code =  0 bp   0 bp     0   2

_ty8zdqdc4/.code =  0 bp   0 bp     0   2

_cyue011ml/.code =  0 bp   0 bp     0   2

_hirtny89a/.code =  0 bp   0 bp     0   2

_b6yj1j6yj/.code =  0 bp   0 bp     0   2

_u8jgs3ogg/.code =  0 bp   0 bp     0   2

_tratgy8sm/.code =  0 bp   0 bp     0   2
every picture/.style=line width=0.75pt

[shading=_qqlbftpg6,_jlmpyo6vo] (-235,19.47) .. controls (-235,14.24) and (-230.76,10) .. (-225.53,10) – (925.53,10) .. controls (930.76,10) and (935,14.24) .. (935,19.47) – (935,215.53) .. controls (935,220.76) and (930.76,225) .. (925.53,225) – (-225.53,225) .. controls (-230.76,225) and (-235,220.76) .. (-235,215.53) – cycle ;
[color=rgb, 255:red, 155; green, 155; blue, 155   ,draw opacity=1 ] (-235,19.47) .. controls (-235,14.24) and (-230.76,10) .. (-225.53,10) – (925.53,10) .. controls (930.76,10) and (935,14.24) .. (935,19.47) – (935,215.53) .. controls (935,220.76) and (930.76,225) .. (925.53,225) – (-225.53,225) .. controls (-230.76,225) and (-235,220.76) .. (-235,215.53) – cycle ;

[shading=_1ev1e81ht,_sz817zhfb] (315,71.35) .. controls (315,67.71) and (317.96,64.75) .. (321.6,64.75) – (458.4,64.75) .. controls (462.04,64.75) and (465,67.71) .. (465,71.35) – (465,213.15) .. controls (465,216.79) and (462.04,219.75) .. (458.4,219.75) – (321.6,219.75) .. controls (317.96,219.75) and (315,216.79) .. (315,213.15) – cycle ;
[color=rgb, 255:red, 155; green, 155; blue, 155   ,draw opacity=1 ] (315,71.35) .. controls (315,67.71) and (317.96,64.75) .. (321.6,64.75) – (458.4,64.75) .. controls (462.04,64.75) and (465,67.71) .. (465,71.35) – (465,213.15) .. controls (465,216.79) and (462.04,219.75) .. (458.4,219.75) – (321.6,219.75) .. controls (317.96,219.75) and (315,216.79) .. (315,213.15) – cycle ;

[shading=_rsb90lxql,_w7epzq3xm] (470,71.35) .. controls (470,67.71) and (472.96,64.75) .. (476.6,64.75) – (613.4,64.75) .. controls (617.04,64.75) and (620,67.71) .. (620,71.35) – (620,213.15) .. controls (620,216.79) and (617.04,219.75) .. (613.4,219.75) – (476.6,219.75) .. controls (472.96,219.75) and (470,216.79) .. (470,213.15) – cycle ;
[color=rgb, 255:red, 155; green, 155; blue, 155   ,draw opacity=1 ] (470,71.35) .. controls (470,67.71) and (472.96,64.75) .. (476.6,64.75) – (613.4,64.75) .. controls (617.04,64.75) and (620,67.71) .. (620,71.35) – (620,213.15) .. controls (620,216.79) and (617.04,219.75) .. (613.4,219.75) – (476.6,219.75) .. controls (472.96,219.75) and (470,216.79) .. (470,213.15) – cycle ;

[shading=_rxu5czdzl,_gzta5pee7] (160,71.35) .. controls (160,67.71) and (162.96,64.75) .. (166.6,64.75) – (303.4,64.75) .. controls (307.04,64.75) and (310,67.71) .. (310,71.35) – (310,213.15) .. controls (310,216.79) and (307.04,219.75) .. (303.4,219.75) – (166.6,219.75) .. controls (162.96,219.75) and (160,216.79) .. (160,213.15) – cycle ;
[color=rgb, 255:red, 155; green, 155; blue, 155   ,draw opacity=1 ] (160,71.35) .. controls (160,67.71) and (162.96,64.75) .. (166.6,64.75) – (303.4,64.75) .. controls (307.04,64.75) and (310,67.71) .. (310,71.35) – (310,213.15) .. controls (310,216.79) and (307.04,219.75) .. (303.4,219.75) – (166.6,219.75) .. controls (162.96,219.75) and (160,216.79) .. (160,213.15) – cycle ;

[shading=_njhfgq9tp,_gyp949wn6] (166,78.75) .. controls (166,73.78) and (170.03,69.75) .. (175,69.75) – (296,69.75) .. controls (300.97,69.75) and (305,73.78) .. (305,78.75) – (305,105.75) .. controls (305,110.72) and (300.97,114.75) .. (296,114.75) – (175,114.75) .. controls (170.03,114.75) and (166,110.72) .. (166,105.75) – cycle ;
[line width=0.75]  (166,78.75) .. controls (166,73.78) and (170.03,69.75) .. (175,69.75) – (296,69.75) .. controls (300.97,69.75) and (305,73.78) .. (305,78.75) – (305,105.75) .. controls (305,110.72) and (300.97,114.75) .. (296,114.75) – (175,114.75) .. controls (170.03,114.75) and (166,110.72) .. (166,105.75) – cycle ;

[shading=_d1yoa3h30,_wlkerr4ui] (165,127.08) .. controls (165,123.03) and (168.28,119.75) .. (172.33,119.75) – (297.67,119.75) .. controls (301.72,119.75) and (305,123.03) .. (305,127.08) – (305,207.42) .. controls (305,211.47) and (301.72,214.75) .. (297.67,214.75) – (172.33,214.75) .. controls (168.28,214.75) and (165,211.47) .. (165,207.42) – cycle ;
(165,127.08) .. controls (165,123.03) and (168.28,119.75) .. (172.33,119.75) – (297.67,119.75) .. controls (301.72,119.75) and (305,123.03) .. (305,127.08) – (305,207.42) .. controls (305,211.47) and (301.72,214.75) .. (297.67,214.75) – (172.33,214.75) .. controls (168.28,214.75) and (165,211.47) .. (165,207.42) – cycle ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (235,114.75) – (235,119.75) ;

[shading=_8t6mi6wk3,_bsoq9hqcn] (320,78.75) .. controls (320,73.78) and (324.03,69.75) .. (329,69.75) – (451,69.75) .. controls (455.97,69.75) and (460,73.78) .. (460,78.75) – (460,105.75) .. controls (460,110.72) and (455.97,114.75) .. (451,114.75) – (329,114.75) .. controls (324.03,114.75) and (320,110.72) .. (320,105.75) – cycle ;
[line width=0.75]  (320,78.75) .. controls (320,73.78) and (324.03,69.75) .. (329,69.75) – (451,69.75) .. controls (455.97,69.75) and (460,73.78) .. (460,78.75) – (460,105.75) .. controls (460,110.72) and (455.97,114.75) .. (451,114.75) – (329,114.75) .. controls (324.03,114.75) and (320,110.72) .. (320,105.75) – cycle ;

[shading=_jwb8o0ffp,_l8xswyvw4] (475,77.42) .. controls (475,73.18) and (478.43,69.75) .. (482.67,69.75) – (607.33,69.75) .. controls (611.57,69.75) and (615,73.18) .. (615,77.42) – (615,107.08) .. controls (615,111.32) and (611.57,114.75) .. (607.33,114.75) – (482.67,114.75) .. controls (478.43,114.75) and (475,111.32) .. (475,107.08) – cycle ;
[line width=0.75]  (475,77.42) .. controls (475,73.18) and (478.43,69.75) .. (482.67,69.75) – (607.33,69.75) .. controls (611.57,69.75) and (615,73.18) .. (615,77.42) – (615,107.08) .. controls (615,111.32) and (611.57,114.75) .. (607.33,114.75) – (482.67,114.75) .. controls (478.43,114.75) and (475,111.32) .. (475,107.08) – cycle ;

[shading=_sgu55z1s3,_6es6zr2fb] (319.5,126.75) .. controls (319.5,122.88) and (322.63,119.75) .. (326.5,119.75) – (452.5,119.75) .. controls (456.37,119.75) and (459.5,122.88) .. (459.5,126.75) – (459.5,207.75) .. controls (459.5,211.62) and (456.37,214.75) .. (452.5,214.75) – (326.5,214.75) .. controls (322.63,214.75) and (319.5,211.62) .. (319.5,207.75) – cycle ;
(319.5,126.75) .. controls (319.5,122.88) and (322.63,119.75) .. (326.5,119.75) – (452.5,119.75) .. controls (456.37,119.75) and (459.5,122.88) .. (459.5,126.75) – (459.5,207.75) .. controls (459.5,211.62) and (456.37,214.75) .. (452.5,214.75) – (326.5,214.75) .. controls (322.63,214.75) and (319.5,211.62) .. (319.5,207.75) – cycle ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (390,114.75) – (390,119.75) ;

[shading=_hc7q4w0um,_i3nrrn8xo] (475,126.75) .. controls (475,122.88) and (478.13,119.75) .. (482,119.75) – (608,119.75) .. controls (611.87,119.75) and (615,122.88) .. (615,126.75) – (615,207.75) .. controls (615,211.62) and (611.87,214.75) .. (608,214.75) – (482,214.75) .. controls (478.13,214.75) and (475,211.62) .. (475,207.75) – cycle ;
(475,126.75) .. controls (475,122.88) and (478.13,119.75) .. (482,119.75) – (608,119.75) .. controls (611.87,119.75) and (615,122.88) .. (615,126.75) – (615,207.75) .. controls (615,211.62) and (611.87,214.75) .. (608,214.75) – (482,214.75) .. controls (478.13,214.75) and (475,211.62) .. (475,207.75) – cycle ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (555,114.75) – (555,119.75) ;

[shading=_ctx4aof9l,_lgxzwadzu] (210,26.5) .. controls (210,24.15) and (211.9,22.25) .. (214.25,22.25) – (565.75,22.25) .. controls (568.1,22.25) and (570,24.15) .. (570,26.5) – (570,50.5) .. controls (570,52.85) and (568.1,54.75) .. (565.75,54.75) – (214.25,54.75) .. controls (211.9,54.75) and (210,52.85) .. (210,50.5) – cycle ;
[line width=0.75]  (210,26.5) .. controls (210,24.15) and (211.9,22.25) .. (214.25,22.25) – (565.75,22.25) .. controls (568.1,22.25) and (570,24.15) .. (570,26.5) – (570,50.5) .. controls (570,52.85) and (568.1,54.75) .. (565.75,54.75) – (214.25,54.75) .. controls (211.9,54.75) and (210,52.85) .. (210,50.5) – cycle ;

[shading=_wkxw0nlpj,_3lijayp8s] (625,21.6) .. controls (625,17.96) and (627.96,15) .. (631.6,15) – (768.4,15) .. controls (772.04,15) and (775,17.96) .. (775,21.6) – (775,213.4) .. controls (775,217.04) and (772.04,220) .. (768.4,220) – (631.6,220) .. controls (627.96,220) and (625,217.04) .. (625,213.4) – cycle ;
[color=rgb, 255:red, 155; green, 155; blue, 155   ,draw opacity=1 ] (625,21.6) .. controls (625,17.96) and (627.96,15) .. (631.6,15) – (768.4,15) .. controls (772.04,15) and (775,17.96) .. (775,21.6) – (775,213.4) .. controls (775,217.04) and (772.04,220) .. (768.4,220) – (631.6,220) .. controls (627.96,220) and (625,217.04) .. (625,213.4) – cycle ;

[shading=_4avbix19v,_4lkk5gwx5] (780,21.6) .. controls (780,17.96) and (782.96,15) .. (786.6,15) – (923.4,15) .. controls (927.04,15) and (930,17.96) .. (930,21.6) – (930,213.4) .. controls (930,217.04) and (927.04,220) .. (923.4,220) – (786.6,220) .. controls (782.96,220) and (780,217.04) .. (780,213.4) – cycle ;
[color=rgb, 255:red, 155; green, 155; blue, 155   ,draw opacity=1 ] (780,21.6) .. controls (780,17.96) and (782.96,15) .. (786.6,15) – (923.4,15) .. controls (927.04,15) and (930,17.96) .. (930,21.6) – (930,213.4) .. controls (930,217.04) and (927.04,220) .. (923.4,220) – (786.6,220) .. controls (782.96,220) and (780,217.04) .. (780,213.4) – cycle ;

[shading=_bs49wfj9k,_usza90ir8] (630,29) .. controls (630,24.03) and (634.03,20) .. (639,20) – (761,20) .. controls (765.97,20) and (770,24.03) .. (770,29) – (770,56) .. controls (770,60.97) and (765.97,65) .. (761,65) – (639,65) .. controls (634.03,65) and (630,60.97) .. (630,56) – cycle ;
[line width=0.75]  (630,29) .. controls (630,24.03) and (634.03,20) .. (639,20) – (761,20) .. controls (765.97,20) and (770,24.03) .. (770,29) – (770,56) .. controls (770,60.97) and (765.97,65) .. (761,65) – (639,65) .. controls (634.03,65) and (630,60.97) .. (630,56) – cycle ;

[shading=_m31sio8yo,_fmoh1k8eb] (785,29) .. controls (785,24.03) and (789.03,20) .. (794,20) – (916,20) .. controls (920.97,20) and (925,24.03) .. (925,29) – (925,56) .. controls (925,60.97) and (920.97,65) .. (916,65) – (794,65) .. controls (789.03,65) and (785,60.97) .. (785,56) – cycle ;
[line width=0.75]  (785,29) .. controls (785,24.03) and (789.03,20) .. (794,20) – (916,20) .. controls (920.97,20) and (925,24.03) .. (925,29) – (925,56) .. controls (925,60.97) and (920.97,65) .. (916,65) – (794,65) .. controls (789.03,65) and (785,60.97) .. (785,56) – cycle ;

[shading=_vks2k5yxy,_jdykzyz75] (630,80.14) .. controls (630,74.54) and (634.54,70) .. (640.14,70) – (759.86,70) .. controls (765.46,70) and (770,74.54) .. (770,80.14) – (770,204.86) .. controls (770,210.46) and (765.46,215) .. (759.86,215) – (640.14,215) .. controls (634.54,215) and (630,210.46) .. (630,204.86) – cycle ;
(630,80.14) .. controls (630,74.54) and (634.54,70) .. (640.14,70) – (759.86,70) .. controls (765.46,70) and (770,74.54) .. (770,80.14) – (770,204.86) .. controls (770,210.46) and (765.46,215) .. (759.86,215) – (640.14,215) .. controls (634.54,215) and (630,210.46) .. (630,204.86) – cycle ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (700,64.75) – (700,70) ;

[shading=_e32ev2u4a,_zbupdvs7q] (785,77.37) .. controls (785,73.3) and (788.3,70) .. (792.37,70) – (917.63,70) .. controls (921.7,70) and (925,73.3) .. (925,77.37) – (925,207.63) .. controls (925,211.7) and (921.7,215) .. (917.63,215) – (792.37,215) .. controls (788.3,215) and (785,211.7) .. (785,207.63) – cycle ;
(785,77.37) .. controls (785,73.3) and (788.3,70) .. (792.37,70) – (917.63,70) .. controls (921.7,70) and (925,73.3) .. (925,77.37) – (925,207.63) .. controls (925,211.7) and (921.7,215) .. (917.63,215) – (792.37,215) .. controls (788.3,215) and (785,211.7) .. (785,207.63) – cycle ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (855,65) – (855,70) ;

[shading=_frl0k4ro7,_ty8zdqdc4] (-230,24.03) .. controls (-230,19.04) and (-225.96,15) .. (-220.97,15) – (145.97,15) .. controls (150.96,15) and (155,19.04) .. (155,24.03) – (155,210.97) .. controls (155,215.96) and (150.96,220) .. (145.97,220) – (-220.97,220) .. controls (-225.96,220) and (-230,215.96) .. (-230,210.97) – cycle ;
[color=rgb, 255:red, 155; green, 155; blue, 155   ,draw opacity=1 ] (-230,24.03) .. controls (-230,19.04) and (-225.96,15) .. (-220.97,15) – (145.97,15) .. controls (150.96,15) and (155,19.04) .. (155,24.03) – (155,210.97) .. controls (155,215.96) and (150.96,220) .. (145.97,220) – (-220.97,220) .. controls (-225.96,220) and (-230,215.96) .. (-230,210.97) – cycle ;

[shading=_wm4ok4m6r,_cyue011ml] (-225,150) .. controls (-225,144.48) and (-220.52,140) .. (-215,140) – (-50,140) .. controls (-44.48,140) and (-40,144.48) .. (-40,150) – (-40,205) .. controls (-40,210.52) and (-44.48,215) .. (-50,215) – (-215,215) .. controls (-220.52,215) and (-225,210.52) .. (-225,205) – cycle ;
(-225,150) .. controls (-225,144.48) and (-220.52,140) .. (-215,140) – (-50,140) .. controls (-44.48,140) and (-40,144.48) .. (-40,150) – (-40,205) .. controls (-40,210.52) and (-44.48,215) .. (-50,215) – (-215,215) .. controls (-220.52,215) and (-225,210.52) .. (-225,205) – cycle ;

[shading=_4a3r9vx1l,_hirtny89a] (-225,70) .. controls (-225,64.48) and (-220.52,60) .. (-215,60) – (-50,60) .. controls (-44.48,60) and (-40,64.48) .. (-40,70) – (-40,125) .. controls (-40,130.52) and (-44.48,135) .. (-50,135) – (-215,135) .. controls (-220.52,135) and (-225,130.52) .. (-225,125) – cycle ;
(-225,70) .. controls (-225,64.48) and (-220.52,60) .. (-215,60) – (-50,60) .. controls (-44.48,60) and (-40,64.48) .. (-40,70) – (-40,125) .. controls (-40,130.52) and (-44.48,135) .. (-50,135) – (-215,135) .. controls (-220.52,135) and (-225,130.52) .. (-225,125) – cycle ;

[color=rgb, 255:red, 74; green, 74; blue, 74   ,draw opacity=1 ][line width=1.5]    (-35,180) – (-40,180) ;