# Efficient Information Extraction in Few-Shot Relation Classification through
Contrastive Representation Learning

Philipp Borchert1,2, Jochen De Weerdt2, Marie-Francine Moens3
1IESEG School of Management, 3 Rue de la Digue, 59000 Lille, France
2Research Centre for Information Systems Engineering, KU Leuven, Belgium
3Department of Computer Science, KU Leuven, Belgium

###### Abstract

Differentiating relationships between entity pairs with limited labeled
instances poses a significant challenge in few-shot relation classification.
Representations of textual data extract rich information spanning the domain,
entities, and relations. In this paper, we introduce a novel approach to
enhance information extraction combining multiple sentence representations and
contrastive learning. While representations in relation classification are
commonly extracted using entity marker tokens, we argue that substantial
information within the internal model representations remains untapped. To
address this, we propose aligning multiple sentence representations, such as
the [CLS] token, the [MASK] token used in prompting, and entity marker tokens.
Our method employs contrastive learning to extract complementary
discriminative information from these individual representations. This is
particularly relevant in low-resource settings where information is scarce.
Leveraging multiple sentence representations is especially effective in
distilling discriminative information for relation classification when
additional information, like relation descriptions, are not available. We
validate the adaptability of our approach, maintaining robust performance in
scenarios that include relation descriptions, and showcasing its flexibility
to adapt to different resource constraints111Our model is available at
https://github.com/pnborchert/MultiRep..

Efficient Information Extraction in Few-Shot Relation Classification through
Contrastive Representation Learning

Philipp Borchert1,2, Jochen De Weerdt2, Marie-Francine Moens3 1IESEG School of
Management, 3 Rue de la Digue, 59000 Lille, France 2Research Centre for
Information Systems Engineering, KU Leuven, Belgium 3Department of Computer
Science, KU Leuven, Belgium

##  1 Introduction

Relation classification (RC) is an important subtask in the relation
extraction, focusing on identifying the types of relations between pairs of
entities within a given textual context. Extracting relevant information is
central to this task. To achieve this, RC models must distill rich information
from sentences, including contextual cues, entity attributes, and relation
characteristics. While language models are essential for extracting text
representations, their use of vector space in sentence representations is
suboptimal (Ethayarajh, 2019). To improve this, recent advances have enhanced
sentence representations through various techniques like flow-based methods
(Li et al., 2020), whitening operations (Huang et al., 2021), prompting (Jiang
et al., 2022), and contrastive learning (Gao et al., 2021; Kim et al., 2021;
Zhou et al., 2022).

Figure 1: Overview of the MultiRep model, which integrates relation
description information. The $\circ$ represents the vector dot product between
the prototypes and the query samples, while the addition operation is denoted
by $\otimes$. Attracting and repelling forces in contrastive learning are
represented by $\rightarrow\leftarrow$ and $\dashleftarrow\dashrightarrow$,
respectively. 1g11g and 1g21g illustrate the representations extracted from
input sentences and relation descriptions, respectively.

Relation extraction faces challenges with limited data for many relation types
and disproportionate data acquisition costs Yang et al. (2021). To address
this challenge, few-shot RC trains models to quickly adapt to new relation
types using only few labeled examples. Common strategies include meta-learning
and prototypical networks, leveraging representation similarity to match
unseen query instances with few labeled support instances (Snell et al.,
2017). Recent studies have enhanced model representations using supplementary
data. Yang et al. (2021) and Qu et al. (2020) incorporate information from
external knowledge bases, augmenting entity-related knowledge. Wang et al.
(2020b) and Yu et al. (2022) utilize linguistic dependencies to integrate
structural sentence information in models. Textual relation descriptions also
improve prototypical network performance by offering additional insights into
relation types (Han et al., 2021; Dong et al., 2021; Liu et al., 2022).

Given the inherent complexity of distinguishing between various relation
types, RC applications commonly combine representations of entity marker
tokens as sentence representations (Baldini Soares et al., 2019; Dong et al.,
2021). Recent work employs contrastive learning in few-shot RC for more
discriminative representations Han et al. (2021); Zhang and Lu (2022); Dong et
al. (2021). Additionally, studies show that representing sentences with the
[MASK] token through prompting improves sentence representations (Jiang et
al., 2022).

In this study, our goal is to enhance sentence representations for few-shot
relation classification by aligning multiple representations, such as entity
markers and the [MASK] token used in prompting. We recognize that individual
model representations, being condensed summaries of the model’s internal
workings, often carry information that is not directly relevant to relation
classification. To address this, we combine various representations to create
more comprehensive sentence embeddings, thereby improving the model’s
discriminative power in low resource settings. While multiple representations
offer diverse perspectives, their informational content tends to overlap
significantly. To align this information, we introduce contrastive learning
objectives providing an effective solution for extracting complementary and
discriminative information. A key advantage in our approach is the efficient
utilization of resources, since all representations are derived from a single
forward pass. Our method is adaptable to different resource constraints by
varying the number of representations used in the sentence embeddings. This
adaptability allows for a balance between performance and resource usage.
Additionally, we extend our approach to include additional information
sources, such as relation descriptions. In summary, our contributions are:

  * •

We introduce a novel method for aligning multiple representations in few-shot
relation classification using contrastive learning.

  * •

Our approach adapts to various resource constraints and extends to additional
information sources, including relation descriptions.

  * •

We emphasize the resource efficiency of our method, improving performance in
low-resource settings.

##  2 Approach

This section provides a detailed overview of our approach, as depicted in
Figure 1.

###  2.1 Sentence Representations

In line with related work, we utilize the BERT-Base model (Devlin et al.,
2019) to encode textual inputs (Han et al., 2021; Liu et al., 2022). We
describe below the techniques used to generate various sentence
representations from the BERT encoder.

Average Pooling computes sentence representations by averaging token
representations. Concurrently, the [CLS] token is used as a sentence
representation during BERT-Base encoder’s pretraining, capturing information
of the entire input sequence. The entity marker technique augments the input
sentence $x$ with markers indicating entities in the text (Baldini Soares et
al., 2019). This augments the input into
$\bar{x}=[x_{0},...,\verb|[E1S]|,x_{i},\verb|[E1E]|,...,x_{n}]$. The sentence
representation is constructed by concatenating the entity start marker
representations $\verb|[E1S]|$ and $\verb|[E2S]|$ (Baldini Soares et al.,
2019). In the prompting approach, the RC task is reformulated as a masked
language modeling problem. With a template $\mathcal{T}$, each input is
transformed into $x_{\text{prompt}}=\mathcal{T}(x)$ containing at least one
[MASK] token. This masked token represents the relation label and is predicted
from the context, e.g., $\bar{x}=\verb|[MASK]|\text{: }x\text{. }$ (Schick and
Schütze, 2021). Gao et al. (2021) introduce using dropout masks for generating
augmented sentence representations. Since entity marker representations are
not suitable for relation descriptions, we employ prompting and [CLS]
representations with varying dropout masks.

###  2.2 Contrastive Representation Learning

The objective of our representation-representation contrastive loss term is
aligning individual representations to extract discriminatory information for
the relation classification task. A key difference to contrastive learning
objectives in related work lies in our method of constructing positive
instance pairs. Related methods like SimCSE (Gao et al., 2021) or CMC (Tian et
al., 2020) derive their representations from different augmented inputs. In
contrast, we extract $M$ different representations from each input sentence in
a single forward pass, and consider these representations as positive pairs.
Consequently, representations from other sentences in the training set serve
as negative instance pairs. For a given representation $r_{i}^{m}$ (where
$m\in M$, $i\in N\times K$), we define positive instances $r_{i}^{+}$ and
negative instances $r_{i}^{-}$ as follows:

$\begin{split}&r_{i}^{+}=\\{r_{i}^{k\neq m}\,|\,k\in M\\}\\\
&r_{i}^{-}=\\{r_{j\neq i}^{m}\,|\,j\in N\times K\\}\end{split}$

This aims to maximize the similarity between different representations of the
same sentence and minimize the similarity to representations obtained from
other sentences (van den Oord et al., 2019; Gao et al., 2021). It ensures that
the differentiating factors encoded in the embeddings primarily reflect the
underlying sentences, regardless of how these representations are derived from
the internal model representations. The representation-representation
contrastive loss is computed as follows:

$\mathcal{L}_{RCL}=\sum_{i=1}^{N\times
K}\sum_{m=1}^{M}{-log\frac{exp\left(\phi(r_{i}^{m},r_{i}^{+})/\tau\right)}{exp\left(\phi(r_{i}^{m},r_{j}^{-})/\tau\right)}},$

where $\tau$ is a temperature scaling parameter, and
$\phi(r_{i}^{m},r_{i}^{+})$ represents the element-wise cosine similarity
$\sum_{k=1}^{M-1}{r_{i}^{m}\cdot r_{i}^{k}/\|r_{i}^{m}\|\|r_{i}^{k}\|}$
between representation $r_{i}^{m}$ and each representation in $r_{i}^{+}$.

In the instance-relation description contrastive loss, we leverage the
relation descriptions to maximize the similarity between instance
representations and corresponding relation description representations. To
construct the instance representations $R_{i}$ and the relation description
representations $D_{i}$, we concatenate all representations extracted from the
encoder $R_{i}=[r_{i}^{1};r_{i}^{2};...;r_{i}^{M}]$ and
$D_{i}=[d_{i}^{1};d_{i}^{2};...;d_{i}^{M}]$. Specifically, $R_{i}$ is composed
of average pooled, [CLS], [MASK], and two entity marker representations. Given
that entity marker representations are not available for relation
descriptions, we apply a 10% dropout to the [CLS] and [MASK] representations,
along with the average pooled, [CLS], and [MASK] representations to form
$D_{i}$. For instance representation $R_{i}$, we select the corresponding
relation description $D^{+}$ based on the label information in the support
set. Non-corresponding relation descriptions $D^{-}$ form negative pairs. The
instance-relation description contrastive loss is computed as follows:

$\mathcal{L}_{RDCL}=\sum_{i=1}^{N\times
K}{-log\frac{exp\left(\phi(R_{i},D^{+})/\tau\right)}{exp\left(\phi(R_{i},D^{-})/\tau\right)}}$

Model | Relation Descriptions | 5-1 | 5-5 | 10-1 | 10-5 | Avg.
---|---|---|---|---|---|---
Proto | - | \- / 80.68 | \- / 89.60 | \- / 71.48 | \- / 82.89 | \- / 81.16
BERT-Pair | - |  85.66 / 88.32 | 89.48 / 93.22 | 76.84 / 80.63 | 81.76 / 87.02 | 83.44 / 87.30
CTEG | - | 84.72 / 88.11 |  92.52 / 95.25 | 76.01 / 81.29 |  84.89 / 91.33 |  84.54 / 89.00
DAPL | - | \- / 85.94 | \- / 94.28 | \- / 77.59 | \- / 89.26 | \- / 86.77
SimpleFSRE | - | 84.77 / 89.33 | 89.54 / 94.13 |  76.85 / 83.41 | 83.42 / 90.25 | 83.64 / 89.28
MultiRep (Ours) | - |  87.13 / 89.20 |  92.93 / 95.09 |  78.42 / 84.18 |  87.29 / 91.65 |  86.44 / 90.03
TD-Proto | ✓ | \- / 84.76 | \- / 92.38 | \- / 74.32 | \- / 85.92 | \- / 84.34
HCRP | ✓ | 90.90 / 93.76 | 93.22 / 95.66 | 84.11 / 89.95 | 87.79 / 92.10 | 89.01 / 92.87
SimpleFSRE | ✓ |  91.29 / 94.42 |  94.05 / 96.37 |  86.09 / 90.73 |  89.68 / 93.47 |  90.28 / 93.75
MultiRep (Ours) | ✓ |  92.73 / 94.18 |  93.79 / 96.29 |  86.12 / 91.07 |  88.80 / 91.98 |  90.36 / 93.38

Table 1: Accuracy on the FewRel validation / test set.

Model | 5-1 | 10-1
---|---|---
MultiRep | 92.73 | 86.12
w/o $\mathcal{L}_{RCL}$ | 92.08 | 85.95
w/o $\mathcal{L}_{RDCL}$ | 90.14 | 84.16
w/o Avg. Pooling | 92.26 | 85.82
w/o Entity Marker | 91.90 | 84.83
w/o [CLS] | 91.35 | 85.51
w/o [MASK] | 91.87 | 85.80
w/ prototype addition | 91.75 | 85.82

Table 2: Model variants with (w/) or without (w/o) indicated representations
and architectural changes evaluated on the FewRel validation set.

###  2.3 Relation Classification

We obtain $N$ class prototypes by averaging the $K$ instance representations
in the support set. We compute the similarity between query instances and
support prototypes using the vector dot product and selecting the most similar
class prototype. For the relation description, we compute the similarity
between query instances and relation description representations $D$. We add
the similarities obtained from the relation descriptions with the similarities
obtained from the class prototypes and select the most similar prototype and
relation description. This is in line with Liu et al. (2022), who instead
directly add the prototype and relation description representations. We
compute the cross-entropy loss $\mathcal{L}_{CE}=-log\left(z_{y}\right)$,
where $z_{y}$ is the probability for class $y$. The total loss is defined as
the sum of the individual loss terms
$\mathcal{L}=\mathcal{L}_{CE}+\mathcal{L}_{RCL}+\mathcal{L}_{RDCL}$.

##  3 Experiments

###  3.1 Task Definition

In the N-way K-shot evaluation setting, episodes are randomly sampled from the
training set. An episode consists of $N\times K$ input sentences $x$ in the
support set $S=\\{(x_{i},rel_{i})\\}^{N\times K}_{i=1}$ and $N\times K$ inputs
from the query set $Q=\\{x_{i}\\}^{N\times K}_{i=1}$. The relations are
randomly sampled from the relation types included in the training dataset.
Importantly, the relation types in the training set are not overlapping with
the test set (and validation set) $rel\textsubscript{train}\cap
rel\textsubscript{test}=\varnothing$ (Gao et al., 2019).

###  3.2 Dataset and Evaluation

Our experiments were conducted using the FewRel dataset, which consists of 700
instances for each of the 100 relation types (Han et al., 2018). Originating
from Wikipedia, the dataset is segmented into training, validation, and test
sets, containing 64, 16, and 20 relation types, respectively. Following the
training and evaluation procedure outlined in Gao et al. (2019), we train
MultiRep on randomly sampled episodes from the training set and subsequently
evaluate the model on previously unseen data from the validation and test
sets. We report classification accuracy, with validation accuracy metrics
being averaged over three random seeds. The test set accuracies are referenced
from the public FewRel leaderboard222Our evaluation results on the FewRel test
set are available on the public leaderboard, listed under the “multirep”
alias.. Our MultiRep model is trained for 30,000 iterations on the FewRel
training set with a batch size of 4 and a learning rate of 2e-5, ensuring our
experiments are consistent with related work (Han et al., 2021; Liu et al.,
2022).

###  3.3 Results

We present the results of our MultiRep approach and compare them to relevant
benchmark models designed for few-shot RC, some of which incorporate relation
descriptions as additional information. For a consistent and fair model
comparison, all benchmarked models exclusively utilize BERT-Base (Devlin et
al., 2019) as the sentence encoder. This approach aligns with recent findings
suggesting that LLMs like GPT-3.5 do not yield superior performance in
relation extraction tasks (Meng et al., 2023). The benchmarked models include
Proto (Gao et al., 2019), BERT-Pair (Gao et al., 2019), TD-Proto (Yang et al.,
2020), CTEG (Wang et al., 2020a), DAPL (Yu et al., 2022), HCRP (Han et al.,
2021), and SimpleFSRE (Liu et al., 2022).

Our model evaluation results are summarized in Table 1. We analyze these
results for two distinct scenarios: (i) models that do not incorporate
additional information, and (ii) models that incorporate relation description
information. We observe that MultiRep outperforms existing models,
particularly in settings where information is limited. Specifically, this
includes scenarios where relation description information is unavailable, as
well as 1-Shot settings in the presence of relation description information.
To validate the importance of individual components in the MultiRep model, we
conducted ablation studies and present the results in Table 2. These results
are based on the MultiRep model that incorporates relation description
information, evaluated on the FewRel validation set in the 5-Way 1-Shot and
10-Way 1-Shot settings. Our findings indicate that removing the contrastive
learning loss terms, $\mathcal{L}_{RCL}$ and $\mathcal{L}_{RDCL}$,
substantially reduces model performance. Furthermore, removing individual
representations from the MultiRep model has a negative impact on performance,
and there are no specific representations that disproportionately affect the
model’s performance. Additionally, we validate our approach of computing
separate instance prototypes and relation description prototypes, as compared
to the direct prototype addition method introduced by Liu et al. (2022).

###  3.4 Resource Adaptability

To assess our method’s adaptability to different resources, we explored how
changes in the number of representations affect relation classification
performance. We experimented with different combinations of individual
representations as outlined in Section 2.1. Figure 2 shows the average
accuracy obtained with differing numbers of representations ($M$) on the
FewRel validation set, employing our MultiRep model that includes relation
descriptions. The analysis revealed a clear trend: increasing the number of
representations leads to improved performance and reduced performance
variability. The results demonstrate that the MultiRep approach optimizes
model performance and indicate that the number of representations can be
adjusted flexibly to accommodate resource constraints.

Figure 2: Average accuracy and standard deviation for varying number of
representations $M$ evaluated on the FewRel validation set.

##  4 Conclusion

In this study, we propose aligning multiple sentence representations in few-
shot RC through contrastive learning. This method efficiently distills
discriminative information from these representations. Our results emphasize
its adaptability and effectiveness in diverse scenarios, particularly in low-
resource and 1-Shot environments, particularly when relation descriptions are
unavailable. A key advantage of our approach lies in its resource efficiency,
deriving all sentence representations from a single forward pass.
Additionally, we demonstrate that the performance of our method improves with
an increasing number of representations, showcasing its resource-adaptability.

##  5 Limitations

Although our approach efficiently utilizes multiple sentence representations
within a single forward pass, it is important to note that this involves
combining these representations into larger vectors. This aggregation process
may require additional memory and computational resources. Moreover, the
application of contrastive learning comes with additional computational
requirements. Our method is specifically designed for few-shot RC tasks, and
its performance might vary when applied to different types of NLP tasks.

## References

  * Baldini Soares et al. (2019) Livio Baldini Soares, Nicholas FitzGerald, Jeffrey Ling, and Tom Kwiatkowski. 2019.  Matching the blanks: Distributional similarity for relation learning.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 2895–2905, Florence, Italy. Association for Computational Linguistics.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Dong et al. (2021) Manqing Dong, Chunguang Pan, and Zhipeng Luo. 2021.  MapRE: An effective semantic mapping approach for low-resource relation extraction.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 2694–2704, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Ethayarajh (2019) Kawin Ethayarajh. 2019.  How contextual are contextualized word representations? Comparing the geometry of BERT, ELMo, and GPT-2 embeddings.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 55–65, Hong Kong, China. Association for Computational Linguistics.
  * Gao et al. (2019) Tianyu Gao, Xu Han, Hao Zhu, Zhiyuan Liu, Peng Li, Maosong Sun, and Jie Zhou. 2019.  FewRel 2.0: Towards more challenging few-shot relation classification.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 6250–6255, Hong Kong, China. Association for Computational Linguistics.
  * Gao et al. (2021) Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021.  SimCSE: Simple contrastive learning of sentence embeddings.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 6894–6910, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Han et al. (2021) Jiale Han, Bo Cheng, and Wei Lu. 2021.  Exploring task difficulty for few-shot relation extraction.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 2605–2616, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Han et al. (2018) Xu Han, Hao Zhu, Pengfei Yu, Ziyun Wang, Yuan Yao, Zhiyuan Liu, and Maosong Sun. 2018.  FewRel: A large-scale supervised few-shot relation classification dataset with state-of-the-art evaluation.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 4803–4809, Brussels, Belgium. Association for Computational Linguistics.
  * Huang et al. (2021) Junjie Huang, Duyu Tang, Wanjun Zhong, Shuai Lu, Linjun Shou, Ming Gong, Daxin Jiang, and Nan Duan. 2021.  WhiteningBERT: An easy unsupervised sentence embedding approach.  In _Findings of the Association for Computational Linguistics: EMNLP 2021_ , pages 238–244, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Jiang et al. (2022) Ting Jiang, Jian Jiao, Shaohan Huang, Zihan Zhang, Deqing Wang, Fuzhen Zhuang, Furu Wei, Haizhen Huang, Denvy Deng, and Qi Zhang. 2022.  PromptBERT: Improving BERT sentence embeddings with prompts.  In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 8826–8837, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Kim et al. (2021) Taeuk Kim, Kang Min Yoo, and Sang-goo Lee. 2021.  Self-guided contrastive learning for BERT sentence representations.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 2528–2540, Online. Association for Computational Linguistics.
  * Li et al. (2020) Bohan Li, Hao Zhou, Junxian He, Mingxuan Wang, Yiming Yang, and Lei Li. 2020.  On the sentence embeddings from pre-trained language models.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 9119–9130, Online. Association for Computational Linguistics.
  * Liu et al. (2022) Yang Liu, Jinpeng Hu, Xiang Wan, and Tsung-Hui Chang. 2022.  A simple yet effective relation information guided approach for few-shot relation extraction.  In _Findings of the Association for Computational Linguistics: ACL 2022_ , pages 757–763, Dublin, Ireland. Association for Computational Linguistics.
  * Meng et al. (2023) Shiao Meng, Xuming Hu, Aiwei Liu, Shuang Li, Fukun Ma, Yawen Yang, and Lijie Wen. 2023.  RAPL: A relation-aware prototype learning approach for few-shot document-level relation extraction.  In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_ , pages 5208–5226, Singapore. Association for Computational Linguistics.
  * Qu et al. (2020) Meng Qu, Tianyu Gao, Louis-Pascal A. C. Xhonneux, and Jian Tang. 2020.  Few-shot relation extraction via bayesian meta-learning on relation graphs.  _International Conference on Machine Learning (ICML)_ , pages 7867–7876.
  * Schick and Schütze (2021) Timo Schick and Hinrich Schütze. 2021.  Exploiting cloze-questions for few-shot text classification and natural language inference.  In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_ , pages 255–269, Online. Association for Computational Linguistics.
  * Snell et al. (2017) Jake Snell, Kevin Swersky, and Richard Zemel. 2017.  Prototypical Networks for Few-shot Learning.  _Advances in Neural Information Processing Systems_ , page 11.
  * Tian et al. (2020) Yonglong Tian, Dilip Krishnan, and Phillip Isola. 2020.  Contrastive multiview coding.
  * van den Oord et al. (2019) Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2019.  Representation learning with contrastive predictive coding.
  * Wang et al. (2020a) Yingyao Wang, Junwei Bao, Guangyi Liu, Youzheng Wu, Xiaodong He, Bowen Zhou, and Tiejun Zhao. 2020a.  Learning to decouple relations: Few-shot relation classification with entity-guided attention and confusion-aware training.  In _Proceedings of the 28th International Conference on Computational Linguistics_ , pages 5799–5809, Barcelona, Spain (Online). International Committee on Computational Linguistics.
  * Wang et al. (2020b) Yuxia Wang, Karin Verspoor, and Timothy Baldwin. 2020b.  Learning from unlabelled data for clinical semantic textual similarity.  In _Proceedings of the 3rd Clinical Natural Language Processing Workshop_ , pages 227–233, Online. Association for Computational Linguistics.
  * Yang et al. (2020) Kaijia Yang, Nantao Zheng, Xinyu Dai, Liang He, Shujian Huang, and Jiajun Chen. 2020.  Enhance prototypical network with text descriptions for few-shot relation classification.  In _Proceedings of the 29th ACM International Conference on Information & Knowledge Management_, CIKM ’20, page 2273–2276, New York, NY, USA. Association for Computing Machinery.
  * Yang et al. (2021) Shan Yang, Yongfei Zhang, Guanglin Niu, Qinghua Zhao, and Shiliang Pu. 2021.  Entity concept-enhanced few-shot relation extraction.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)_ , pages 987–991, Online. Association for Computational Linguistics.
  * Yu et al. (2022) Tianshu Yu, Min Yang, and Xiaoyan Zhao. 2022.  Dependency-aware prototype learning for few-shot relation classification.  In _Proceedings of the 29th International Conference on Computational Linguistics_ , pages 2339–2345, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
  * Zhang and Lu (2022) Peiyuan Zhang and Wei Lu. 2022.  Better few-shot relation extraction with label prompt dropout.  In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 6996–7006, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Zhou et al. (2022) Kun Zhou, Beichen Zhang, Xin Zhao, and Ji-Rong Wen. 2022.  Debiased contrastive learning of unsupervised sentence representations.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 6120–6130, Dublin, Ireland. Association for Computational Linguistics.

##  Appendix A Model Training

The MultiRep model consists of 109.48 million parameters and was trained on a
single NVIDIA A6000 48GB GPU. The combined training and evaluation time for
the 5-Way 1-Shot and 10-Way 5-Shot models, incorporating relation
descriptions, was 16 hours and 25 hours, respectively.

##  Appendix B Input Templates

To obtain various representations for a given input sequence and relation
description within a single forward pass, we augment the inputs with special
tokens. For input sequences, we employ the template “`[CLS]` `[E1]`, `[MASK]`,
`[E2]` `[SEP]` `[TEXT]`”, where `[E1]` and `[E2]` denote the respective
entities in the input sentence, and `[TEXT]` represents the augmented input
sentence with entity markers. For the relation description, we utilize the
template “`[CLS]` `[MASK]`: `[RELATION DESCRIPTION]`”, where the relation
descriptions included in the FewRel dataset are represented by `[RELATION
DESCRIPTION]`. The FewRel dataset provides a relation type category and
textual description of the relation type. We include both the relation type
category and textual description, separated by a comma in the `[RELATION
DESCRIPTION]`.

##  Appendix C Case Study

This case study highlights the contributions of our MultiRep approach, which
integrates representation-representation contrastive learning objective. We
demonstrate this by comparing the embeddings and predictions of the MultiRep
model, trained with these contrastive learning objective, against a variant
trained without them (MultiRep w/o CL). In a 5-Way 1-Shot setting on the
FewRel validation set, the MultiRep w/o CL model achieves an accuracy of
84.83%, substantially lower than the 87.13% achieved by the MultiRep model
with contrastive learning objectives.

Both models combine five individual representations to form sentence
representations, as detailed in Section 2.1. We display selected prediction
results in Table 3. Notably, our MultiRep model demonstrates a tendency to
misclassify more challenging cases, such as instances involving difficult
relation types like "spouse", "mother", and "child". This observation aligns
with the hard classification examples highlighted and addressed by Han et al.
(2021). Conversely, the model variant without contrastive learning objectives
is more prone to errors in simpler classification tasks, leading to an overall
lower performance. To further demonstrate the benefit of our contrastive
representation learning approach, we present a t-SNE projection of sentence
representations from both the MultiRep and MultiRep w/o CL models in Figures 3
and 4. These figures allow for a direct comparison of sentence embeddings for
relation types that were frequently confused, as listed in Table 3. The
projections reveal that sentence embeddings for the same relation type cluster
more closely together, while those from different types are more dispersed.
This clustering pattern suggests that the sentence embeddings obtained from
our MultiRep model effectively extract discriminative information for relation
classification.

(a)

(b)

Figure 3: t-SNE visualization of 120 randomly sampled support embeddings for
hard relation classification examples obtained from MultiRep.

(a)

(b)

Figure 4: t-SNE visualization of 120 randomly sampled support embeddings for
hard relation classification examples obtained from MultiRep w/o CL.

Label |  Prediction MultiRep |  Prediction MultiRep w/o CL |  Sentence
---|---|---|---
sport |  sport |  military rank |  Jürgen Hasler E1 ( born 7 May 1973 ) is a Liechtenstein former alpine skier E2 who competed in the 1994 Winter Olympics , 1998 Winter Olympics and 2002 Winter Olympics.
main subject |  main subject |  constellation |  Comet 252P / LINEAR E1 is a periodic comet and near - Earth object E2 discovered by the LINEAR survey on April 7 , 2000.
part of |  part of |  competition class |  During the Caucasus Campaign E1 of World War I E2, the Russian Empire occupied most of the Armenian - populated regions of the Ottoman Empire.
spouse |  spouse |  mother |  He was the father - in - law of Azerbaijan ’s President Heydar Aliyev E2, who married his daughter Zarifa Aliyeva E1, and maternal grandfather of Azerbaijan ’s current President Ilham Aliyev.
child |  child |  mother |  He was the father of Frank Islacker E1 and the grandfather of Mandy Islacker E2.
child |  mother |  child |  "Me and Liza" is about Wainwright’s relationship with Liza Minnelli E2, who was reportedly upset by his 2006 tribute concerts to her mother , American actress and singer Judy Garland E1.
spouse |  mother |  spouse |  His brother , Fridolin Weber , who died in 1785 , was the father of Mozart E2 ’s wife , Constanze E1.
member of |  part of |  military rank |  Designers Domenico Dolce E1 and Stefano Gabbana ( Dolce & Gabbana E2 ) spoke of working with Minogue for the costumes of the tour .
main subject |  member of |  part of |  His grandmother Elisabeth was a member of the Ephrussi family E2, whose history he chronicled in " The Hare with Amber Eyes E1 ".
part of |  constellation |  constellation |  The SS " Edward Y. Townsend " ( official number 203449 ) was a American E1 Great Lakes freighter that served on the Great Lakes of North America E2.

Table 3: Comparative prediction examples: MultiRep with contrastive learning
objectives (MultiRep) versus MultiRep without contrastive learning objectives
(w/o CL). Examples are obtained from the FewRel validation set.


# Strong Identification Codes for Graphs

Ghurumuruhan Ganesan

Institute of Mathematical Sciences, HBNI, Chennai E-Mail<EMAIL_ADDRESS>
###### Abstract

For any graph $G,$ a set of vertices ${\cal V}$ is said to be dominating if
every vertex of $G$ contains at least one node of $G$ and separating if each
vertex $v$ contains a unique neighbour $u_{v}\in{\cal V}$ that is adjacent to
no other vertex of $G.$ If ${\cal V}$ is both dominating and separating, then
${\cal V}$ is defined to be an identification code. In this paper, we study
strong identification codes with an index $r,$ by imposing the constraint that
each vertex of $G$ contains at least $r$ unique neighbours in ${\cal V}.$ We
use the probabilistic method to study both the minimum size of strong
identification codes and the existence of graphs that allow an identification
code with a given index.

Key words: Strong identification codes, strong neighbourhood graphs,
probabilistic method.

AMS 2000 Subject Classification: Primary: 05C62;

##  1 Introduction

An identification code of a graph $G$ is a set of vertices of $G$ that form
both a dominating and a separating set (Karpovsky, Chakrabarty and Levitin
[7]). Bounds for $\gamma(G),$ the minimum size of an identification code for
$G,$ have been evaluated for different classes of graphs: Gravier and Moncel
[6] describe graphs whose minimum size identification code contains all
vertices apart from exactly one vertex. Foucaud, Klasing, Kosowski and Raspaud
[4] studied constructions of identification codes for triangle-free graphs,
with size at most a constant fraction of the number of vertices. Later Foucaud
and Perarnau [5] obtained bounds for $\gamma(G)$ in terms of the minimum
vertex degree $\delta$ for graphs $G$ whose girth is at least five and also
study the size of the identification codes for random regular graphs.
Recently, [3] have investigated the use of graph spectra as a tool to
characterize the identification codes.

In this paper, we study strong identification codes of a graph $G$ with an
index $r\geq 1,$ satisfying the additional constraint that each vertex
contains at least $r$ unique neighbours in the code. (For formal definitions,
we refer to Section 2). We use the probabilistic method to estimate the
minimum size of an identification code and also to determine existence of
graphs that allow strong identification codes with a given index.

The paper is organized as follows. In Section 2, we state and prove our main
result, Theorem 1, regarding the minimum size of a strong identification code
with index $r.$ Next in Section 3, we state and prove Theorem 2 regarding
existence of graphs that allow strong idenfication codes with a given index.

##  2 Strong Identification Codes

Let $G=(V,E)$ be a graph with vertex set $V=\\{1,2,\ldots,n\\}$ and edge set
$E.$ For a vertex $v\in G,$ we define $N(v)$ to be the set of all vertices
adjacent to $v,$ not including $v$ and let $N[v]=N(v)\cup\\{v\\}.$ We also
denote the elements of $N(v)$ as _neighbours_ of $v.$ Letting $\\#A$ denote
the number of elements in the set $A,$ we have the following definition.

######  Definition 1.

A set of vertices ${\cal C}$ is said to be an _identification code_ for $G$
with _index_ $r$ if for any two distinct vertices $v,u\in G$ we have

$\\#\left(\left(N[v]\setminus N[u]\right)\bigcap{\cal C}\right)\geq r.$ (2.1)

For $r=1,$ the above definition reduces to the concept of identification codes
as studied in Gravier and Moncel [6] and Foucaud and Perarnau [5].

Clearly, one necessary condition for $G$ to have an identification code with
index $r$ is that the neighbourhood of any vertex contains at least $r$
distinct vertices not present in any other vertex, i.e., for any two distinct
vertices $u,v\in G,$ we must have that

$\\#\left(N[v]\setminus N[u]\right)\geq r.$ (2.2)

If (2.2) holds, we say that $G$ has the $r-$strong neighbourhood property. In
the next Section 3, we use the probabilistic method to establish the existence
of graphs satisfying (2.2).

If the $r-$strong neighbourhood property holds, then the whole vertex set of
$G$ is itself an identification code with index $r.$ For graphs with stronger
neighbourhoods, we have the following result regarding the minimum size of an
identification code with a given index.

######  Theorem 1.

Let $G$ be a graph on $n$ vertices satisfying the $(r+d+1)-$strong
neighbourhood property for some integer $d\geq 1.$ If
$\theta_{n}=\theta_{n}(G,r,d)$ is the minimum size of an identification code
of $G$ with index $r,$ then

$\frac{1}{\Delta+1}\leq\frac{\theta_{n}}{n}\leq
1-\frac{c(d,r)}{(\Delta+1)^{(r+2)/d}},$ (2.3)

where $\Delta\geq 2$ is the maximum degree of a vertex in $G$ and
$c(d,r):=d(d+1)^{-1-1/d}(2r)^{-1/d}.$

Thus if $G$ has $(r+d+1)-$strong neighbourhood property for some integer
$d\geq 1,$ then there exists a _nontrivial_ fraction of vertices of $G$ that
act as an identification code with index $r.$

### Proof of Theorem 1

The lower bound in (2.3) is obtained as follows. If ${\cal C}$ is an
identification code with index $r,$ then ${\cal C}$ is also a dominating set
in the sense that $\bigcup_{v\in{\cal C}}N[v]=\\{1,2,\ldots,n\\}.$ Since
$\\#N[v]\leq\Delta+1,$ it follows that $\\#{\cal C}\geq\frac{n}{\Delta+1}.$

To prove the upper bound in (2.3), we use the probabilistic method similar to
Theorem 1.2.2, pp. 4, Alon and Spencer [1], that concerns the minimum size of
a dominating set. We select each vertex of $G$ independently with probability
$q$ to be determined later and let $Z$ be the random set of vertices selected.
Say that a vertex $v\in G$ is _bad_ if one of the following two conditions
hold: either $(a)$ $\\#\left(N[v]\bigcap Z\right)\leq r-1$ or $(b)$
$\\#\left(N[v]\bigcap Z\right)\geq r$ but there exists a vertex $w$ satisfying

$\\#\left((N[v]\setminus N[w])\bigcap Z\right)\leq r-1.$ (2.4)

A vertex of $G$ which is not bad is said to be _good_.

If $Y_{b}$ is the set of all bad vertices and $Z_{b}=\bigcup_{v\in
Y_{b}}N[v],$ then the set $Y:=Z\cup Z_{b}$ is an identification code for $G$
with index $r:$ Indeed, if $v$ is a good vertex, then for any vertex $u\neq v$
we have

$\\#\left((N[v]\setminus N[u])\cap Y\right)\geq\\#\left((N[v]\setminus
N[u])\cap Z\right)\geq r.$

On the other hand if $v$ is a bad vertex, then for any vertex $u\neq v,$ we
have by the $(r+d+1)-$strong neighbourhood property that

$\displaystyle\\#\left((N[v]\setminus N[u])\cap Y\right)$ $\displaystyle\geq$
$\displaystyle\\#\left((N[v]\setminus N[u])\cap Z_{b}\right)$ $\displaystyle=$
$\displaystyle\\#\left(N[v]\setminus N[u]\right)$ $\displaystyle\geq$
$\displaystyle r+d+1.$

We now estimate the size of $Z_{b}$ by bounding the probability that a vertex
$v\in G$ is bad. If $\delta(v):=\\#N[v],$ then the probability that case $(a)$
occurs is

$f_{1}(q,v):=\sum_{l=0}^{r-1}{\delta(v)\choose l}q^{l}(1-q)^{\delta(v)-l}.$
(2.5)

Now if possibility $(b)$ occurs and some vertex $w$ satisfies (2.4), then
either $w$ is a neighbour of $v$ or $w$ is at distance two from $v$ and shares
a common neighbour with $v.$ Letting $\delta(v,w):=\\#(N[v]\setminus N[w]),$
the probability that case $(b)$ occurs is then bounded above by

$f_{2}(q,v):=\max_{w}\sum_{l=0}^{r-1}{\delta(v,w)\choose
l}q^{l}(1-q)^{\delta_{1}(v,w)-l},$ (2.6)

where the maximum is taken over all vertices $w$ at a distance of at most two
from $v.$ Since the maximum degree of any vertex is at most $\Delta,$ there
are at most $\Delta+\Delta(\Delta-1)=\Delta^{2}$ neighbours at a distance of
at most two from $v.$ Consequently, the probability that vertex $v$ is bad is
upper bounded by $f_{1}(q,v)+\Delta^{2}f_{2}(q,v)$ and so the expected number
of vertices in $Y_{b}$ is at most $n(f_{1}(q,v)+\Delta^{2}f_{2}(q,v)).$ This
in turn implies that the expected number of vertices in $Z_{b}=\bigcup_{v\in
Y_{b}}N[v]$ is at most $n(\Delta f_{1}(q,v)+\Delta^{3}f_{2}(q,v)).$

Summarizing, we get from the discussion in the previous paragraph that the
expected number of vertices in $Z\cup Z_{b}$ is at most

$n\left(q+\Delta\max_{v}f_{1}(q,v)+\Delta^{3}\max_{v}f_{2}(q,v)\right).$ (2.7)

To find suitable bounds for $f_{1}(q,v)$ and $f_{2}(q,v),$ we first let

$\delta:=\min_{v}\\#N(v)\geq r+d$

be the minimum degree of a vertex in $G.$ From (2.5) we have that

$f_{1}(q,v)\leq(1-q)^{\delta-r+1}\sum_{l=0}^{r-1}{\delta(v)\choose
l}q^{l}\leq(1-q)^{d+1}\sum_{l=0}^{r-1}{\delta(v)\choose l}q^{l}$

and using ${\delta(v)\choose l}\leq\delta^{l}(v)\leq(\Delta+1)^{l}$ and $q\leq
1,$ we get that

$f_{1}(q,v)\leq(1-q)^{d+1}\sum_{l=0}^{r-1}(\Delta+1)^{l}\leq
r(\Delta+1)^{r-1}(1-q)^{d+1}.$ (2.8)

For estimating (2.6), we argue similarly and use $q\leq 1$ and the fact that

$\delta(v,w)=\\#(N[v]\setminus N[w])\geq r+d+1$

by the $(r+d+1)-$strong neighbourhood property, to get that

$f_{2}(q,v)\leq(1-q)^{d+2}\sum_{l=0}^{r-1}{\delta(v,w)\choose l}\leq
r(\Delta+1)^{r-1}(1-q)^{d+2}.$ (2.9)

Plugging the estimates (2.8) and (2.9) into (2.7), we get that the expected
number of vertices in the identification code $Z\cup Z_{b}$ is at most

$\displaystyle\mathbb{E}\left(\\#(Z\cup Z_{b})\right)$ $\displaystyle\leq$
$\displaystyle nq+r(\Delta+1)^{r}(1-q)^{d+1}+r(\Delta+1)^{r+2}(1-q)^{d+2}$
(2.10) $\displaystyle\leq$ $\displaystyle n(q+2r(\Delta+1)^{r+2}(1-q)^{d+1})$
$\displaystyle=:$ $\displaystyle n\Gamma(q).$

The solution $q_{0}$ to the equation $\Gamma^{\prime}(q)=0$ satisfies

$1-2r(\Delta+1)^{r+2}(d+1)(1-q_{0})^{d}=0$

and so

$\Gamma(q_{0})=q_{0}+\frac{1-q_{0}}{d+1}=1-\frac{d}{d+1}\left(\frac{1}{2r(\Delta+1)^{r+2}(d+1)}\right)^{\frac{1}{d}}.$

Consequently, there exists at least one identification code with index $r$ and
size at most $n\Gamma(q_{0}).$

##  3 Strong Neighbourhood Graphs

In this section, we use the probabilistic method to obtain $r-$strong
neighbourhood graphs for any integer $r\geq 1.$

The cycle $C_{n}$ on $n\geq 4$ vertices satisfies the $1-$strong neighbourhood
property. The following result establishes the existence of bounded degree
graphs with $w-$strong neighbourhoods, for any integer $w\geq 2.$

######  Theorem 2.

Let $w\geq 2$ be fixed. There exists an absolute constant $C>0$ not depending
on $w$ such that if $n\geq\max(C,160(w+1)^{2}+1):=M(w),$ then there exists a
connected graph $G$ on $n$ vertices satisfying the $w-$strong neighbourhood
property and having maximum vertex degree

$\Delta(G)\leq\max(10\log(2M(w)),8(w+1))+1.$

Thus for any $w\geq 2$ and all $n$ large, there are bounded degree graphs on
$n$ vertices satisfying the $w-$strong neighbourhood property.

We use the probabilistic method to prove Theorem 2, beginning with the
following auxiliary Lemma.

######  Lemma 3.

Let $y\geq 3$ be fixed. There exists an absolute constant $C>0$ not depending
on $y$ such that if $n\geq\max(C,160y^{2}+1),$ then there exists a connected
graph $G$ satisfying the $(y-1)-$strong neighbourhood property and having
maximum vertex degree of at most $\max(32\log{n},8y).$

To prove Lemma 3, we use the following standard deviation estimate. Let
$\\{X_{j}\\}_{1\leq j\leq L}$ be independent Bernoulli random variables with
$\mathbb{P}(X_{j}=1)=1-\mathbb{P}(X_{j}=0)>0.$ If
$T_{L}=\sum_{j=1}^{L}X_{j},\theta_{L}=\mathbb{E}T_{L}$ and
$0<\epsilon\leq\frac{1}{2},$ then

$\mathbb{P}\left(\left|T_{L}-\theta_{L}\right|\geq\theta_{L}\epsilon\right)\leq
2\exp\left(-\frac{\epsilon^{2}}{4}\theta_{L}\right).$ (3.1)

For a proof of (3.1), we refer to Corollary A.1.14, pp. 312 of Alon and
Spencer (2008).

_Proof of Lemma 3_: We use the probabilistic method. Let $H=G(n,p)$ be the
random graph obtained by setting each edge of the complete graph $K_{n}$ on
$n$ vertices to be independently present with probability $p,$ where

$p=\frac{\max(16\log{n},4y)}{n-1}$ (3.2)

and absent otherwise.

Letting $N(1)$ denote the neighbours of the vertex $1,$ we have that
$\mathbb{E}\\#N(1)=(n-1)p$ and so using (3.1) with $\epsilon=\frac{1}{2},$ we
get

$\mathbb{P}\left(\frac{(n-1)p}{2}\leq\\#N(1)\leq\frac{3(n-1)p}{2}\right)\geq
1-\exp\left(-\frac{(n-1)p}{4}\right)\geq 1-\frac{1}{n^{4}},$ (3.3)

by (3.2). Letting

$A_{tot}:=\bigcap_{1\leq i\leq
n}\left\\{\frac{(n-1)p}{2}\leq\\#N(1)\leq\frac{3(n-1)p}{2}\right\\},$

we get by the union bound and (3.3) that

$\mathbb{P}(A_{tot})\geq 1-\frac{1}{n^{3}}.$ (3.4)

Thus, with high probability, i.e., with probability converging to one as
$n~{}\rightarrow~{}\infty,$ each vertex in the random graph $H$ contains at
most $2(n-1)p=\max(32\log{n},8y)$ neighbours.

Next, we estimate the number of common neighbours between the two vertices $i$
and $j$ as follows. Letting $T_{ij}:=\\#(N(i)\cap N(j))$ and using the
Chernoff bound, we have for $s,t>0$ that

$\mathbb{P}\left(T_{ij}>t\right)\leq\left(\mathbb{E}e^{sT_{ij}}\right)^{n-2}e^{-st}.$
(3.5)

Each vertex $z\neq i,j$ is adjacent to both $i$ and $j$ with probability
$p^{2},$ independent of the other vertices and so

$1<\mathbb{E}e^{sT_{ij}}=1+(e^{s}-1)p^{2}\leq\exp\left((e^{s}-1)p^{2}\right).$
(3.6)

Substituting (3.6) into (3.5) and using
$(\mathbb{E}e^{sT_{ij}})^{n-2}\leq(\mathbb{E}e^{sT_{ij}})^{n-1},$ we get

$\mathbb{P}\left(T_{ij}>t\right)\leq\exp\left((e^{s}-1)(n-1)p^{2}\right)e^{-st}.$
(3.7)

For $s=2,$ we have

$\displaystyle(e^{s}-1)(n-1)p^{2}$ $\displaystyle=$
$\displaystyle(e^{2}-1)\frac{\max(256(\log{n})^{2},16y^{2})}{n-1}$
$\displaystyle\leq$
$\displaystyle\frac{1}{n-1}\max(2560(\log{n})^{2},160y^{2}),$

which is at most one, provided that $n\geq 1+\max(160y^{2},n_{0}),$ where
$n_{0}$ is the smallest integer $x$ such that $x>2560(\log{x})^{2}.$ For all
$n\geq 1+\max(160y^{2},n_{0}),$ we therefore have from (3.7) that

$\mathbb{P}\left(T_{ij}>\frac{(n-1)p}{4}\right)\leq
e\exp\left(-\;\frac{(n-1)p}{2}\right)\leq\frac{e}{n^{8}},$ (3.8)

since $(n-1)p\geq 16\log{n}$ by (3.2). Thus,

$\mathbb{P}\left(\\#\left(N(i)\cap N(j)\right)\leq\frac{(n-1)p}{4}\right)\geq
1-\frac{e}{n^{8}}$

and, using the union bound, we have

$\mathbb{P}\left(\bigcap_{i\neq j}\left\\{\\#\left(N(i)\cap
N(j)\right)\leq\frac{(n-1)p}{4}\right\\}\right)\geq 1-\frac{e}{n^{6}}.$ (3.9)

From (3.4) and (3.9), we get that with high probability, the maximum degree of
the random graph $H$ is at most $2(n-1)p=\max(32\log{n},8y).$ In addition, for
any two vertices $v\neq u,$ the corresponding neighbourhood sets satisfy

$\\#\left(N[v]\setminus N[u]\right)\geq\frac{(n-1)p}{4}-1\geq y-1.$

Consequently, with high probability, the graph $H$ satisfies the
$(y-1)-$strong neighbourhood property and because $p\geq\frac{16\log{n}}{n}$
(see (3.2)), the graph $H$ is also connected with high probability (pp.
164–165, Bollobás [2]). Summarizing, there exists an absolute constant $C>0$
not depending on $y$ such that if
$n\geq\max(C,160y^{2}~{}+~{}1),$ then with high probability $H$ is connected,
satisfies the $(y-1)-$strong neighbourhood property and has a maximum vertex
degree of at most $\max(32\log{n},8y).$

_Proof of Theorem 2_: Let $w\geq 2$ and $M(w)$ be as in the statement of
Theorem 2. For $n\geq M(w),$ split $\\{1,2,\ldots,n\\}$ into $T$ sets
$V_{1},\ldots,V_{T}$ such that $V_{i},1\leq i\leq T-1,$ has size exactly
$M(w)$ and $V_{T}$ has size at least $M(w)$ and at most $2M(w).$ Using Lemma 3
with $w=y-1,$ we know that there exists a connected graph $G_{i}$ with vertex
set $V_{i}$ satisfying the $w-$strong neighbourhood property and having a
maximum vertex degree of at most
$\Delta_{0}=\max(32\log(2M(w)),8(w+1)).$ We pick one vertex $u_{i}\in V_{i}$
and connect $u_{i}$ to $u_{i+1}$ for $1\leq i\leq T-1.$ The resulting graph
$G$ is connected, satisfies the $w-$strong neighbourhood property and has a
maximum vertex degree of at most $\Delta_{0}+1.$

### Acknowledgement

I thank Professors Rahul Roy, C. R. Subramanian and the referees for crucial
comments that led to an improvement of the paper. I also thank IMSc for my
fellowships.

## References

  * [1] N. Alon and J. Spencer,  The Probabilistic Method,  Wiley Interscience, 2008.
  * [2] B. Bollobás,  Random graphs,  Cambridge University Press, 2001.
  * [3] C. Balbuena, C. Dalfó, B. Martínez-Barona, Characterizing identifying codes from the spectrum of a graph or digraph, Linear Algebra and its Applications, 570 (2019) 138–147.
  * [4] F. Foucaud, R. Klasing, A. Kosowski, A. Raspaud,  On the size of identifying codes in triangle-free graphs,  Discrete Applied Mathematics, 160 (2012) 1532–1546.
  * [5] F. Foucaud, G. Perarnau,  Bounds for identifying codes in terms of degree parameters,  Electronic Journal of Combinatorics, 19 (2012) P32.
  * [6] S. Gravier, J. Moncel,  On graphs having a $V\setminus\\{x\\}$ set as an identifying code,  Discrete Mathematics, 307 (2007) 432–434.
  * [7] M. G. Karpovsky, K. Chakrabarty, L. B. Levitin,  On a new class of codes for identifying vertices in graphs,  IEEE Transactions on Information Theory, 44 (1998) 599–611.


# Channel-Feedback-Free Transmission for Downlink FD-RAN: A Radio Map based
Complex-valued Precoding Network Approach

Jiwei Zhao
School of Electronic Science and Engineering
Nanjing University
Nanjing, China, 710023
<EMAIL_ADDRESS>
&Jiacheng Chen
Department of Strategic and Advanced Interdisciplinary
Research, Peng Cheng Laboratory
Shenzhen 518000, China
<EMAIL_ADDRESS>
&Zeyu Sun
School of Electronic Science and Engineering
Nanjing University
Nanjing, China, 710023
<EMAIL_ADDRESS>
&Yuhang Shi
School of Electronic Science and Engineering
Nanjing University
Nanjing, China, 710023
<EMAIL_ADDRESS>
&Haibo Zhou
School of Electronic Science and Engineering
Nanjing University
Nanjing, China, 710023
<EMAIL_ADDRESS>
&Xuemin (Sherman) Shen
Department of Electrical and Computer Engineering
University of Waterloo
Waterloo, Ontario, N2L 3G1, Canada
<EMAIL_ADDRESS>
Corresponding author.

###### Abstract

As the demand for high-quality services proliferates, an innovative network
architecture, the fully-decoupled RAN (FD-RAN), has emerged for more flexible
spectrum resource utilization and lower network costs. However, with the
decoupling of uplink base stations and downlink base stations in FD-RAN, the
traditional transmission mechanism, which relies on real-time channel
feedback, is not suitable as the receiver is not able to feedback accurate and
timely channel state information to the transmitter. This paper proposes a
novel transmission scheme without relying on physical layer channel feedback.
Specifically, we design a radio map based complex-valued precoding network
(RMCPNet) model, which outputs the base station precoding based on user
location. RMCPNet comprises multiple subnets, with each subnet responsible for
extracting unique modal features from diverse input modalities. Furthermore,
the multi-modal embeddings derived from these distinct subnets are integrated
within the information fusion layer, culminating in a unified representation.
We also develop a specific RMCPNet training algorithm that employs the
negative spectral efficiency as the loss function. We evaluate the performance
of the proposed scheme on the public DeepMIMO dataset and show that RMCPNet
can achieve 16% and 76% performance improvements over the conventional real-
valued neural network and statistical codebook approach, respectively.

_Keywords_ FD-RAN; beamforming; deep learning; complex neural networks

##  1 Introduction

With the explosive growth of mobile communication demand, multi-antenna
systems have been widely deployed to provide spectrum-efficient transmission
and meet more stringent service requirements Liu et al. (2020a); Yu et al.
(2019); Dang et al. (2020); Wang et al. (2023); Qian et al. (2023); Xu et al.
(2021). Currently, the multi-antenna transmission in 5G is based on codebook
and requires feedback to let the transmitter know the channel information in
order to precode original signals loaded into different antennas Tse and
Viswanath (2005). However, as the number of antennas increases, more pilots
are needed to obtain the channel between users and base stations, and the
feedback of the channel has become a significant obstacle to achieving higher
spectral efficiency of wireless communication Wen et al. (2018); Zhou et al.
(2020); Yu et al. (2023). Furthermore, considering the channel variation in
the practical environment, the feedback information used by base stations for
precoding may become outdated and thus invalid Luo et al. (2018). This leads
to a significant decline in the performance of the multi-antenna transmission,
especially when the channel changes fast, such as in mobile scenarios.
Moreover, in fully decoupled radio access networks (FD-RAN), where the uplink
base stations (UBSs) and downlink base stations (DBSs) are physically
decoupled, the transmission resources for feedback will be more scarce because
users in a wide area share the same control base station Zhao et al. (2023).
Hence, exploring a new transmission mechanism for massive MIMO becomes an
indispensable yet challenging task.

Many recent works have attempted to solve the challenges faced by real-time
feedback based communications. With the development of autoencoder, channel
state information (CSI) compression based on deep learning has attracted wide
attention from researchers Wen et al. (2018); Shen et al. (2016); Guo et al.
(2020). Specifically, CSI is compressed from different frequency domain and
spatial domain into bit streams to reduce the amount of feedback data Mashhadi
et al. (2020); Yin et al. (2022); Song et al. (2021); Wang et al. (2021a).
Literature Mashhadi et al. (2020) proposes a fully convolutional neural
network that incorporates quantization and entropy coding blocks into its
design, improving CSI recovery accuracy. Yin et al. propose a CSI compression
and feedback network based on a self-information model Yin et al. (2022).
Literature Song et al. (2021) proposes a network with self-attention and dense
refine mechanisms. These neural network-based methods for CSI compression
improve feedback accuracy and are suitable for different channel compression
ratios. However, these works are still based on real-time feedback of
channels. They cannot solve the challenge of outdated channel feedback caused
by delay and channel variations.

Another effective approach to deal with outdated channel feedback is to use
statistical information of channels for transmission Gao et al. (2009);
Björnson et al. (2014); Kim et al. (2021); Wang et al. (2019); Shi et al.
(2021); Peng et al. (2019); Luo et al. (2020); Alrabeiah and Alkhateeb (2020).
Literature Gao et al. (2009) considers the closed-form upper bound of channel
capacity when only statistical channel state information is available at the
transmitter. Björnson et al. (2014) derives a closed-form expression for
optimal linear precoding. Wang et al. (2019) proposes a downlink transmission
method based on channel statistical information that considers physical layer
security. Muhammad et al. design a learning framework that is an integration
of a convolutional neural network and a long short term with memory network
Alrabeiah and Alkhateeb (2020). Shi et al. exploit the structural
characteristics for the optimal precoding and propose a deep neural network
(DNN) based robust precoding method that learns directly from CSI statistical
knowledge Shi et al. (2021). The online computational complexity of the
precoder is substantially reduced compared with the existing iterative
algorithm while maintaining nearly the same performance.

Location-based precoding techniques have attracted considerable interest from
academia owing to the relatively stable statistical characteristics of the
channel at the user’s location Liu et al. (2020b); Koivisto et al. (2017);
Kela et al. (2016); Miao et al. (2020); Alkhateeb et al. (2018); Zhou et al.
(2019); Liu et al. (2022). By integrating the navigation information and
wireless channel information, Wang et al. enhance the performance of cellular-
enabled UAV communications by predicting the angle of arrival Miao et al.
(2020). In literature Liu et al. (2022), a historical channels based
convolutional long short-term memory network is proposed for predicting
beamforming. In addition, literature Liu et al. (2020b); Alkhateeb et al.
(2018) exploits deep learning models to predict the beamforming vectors and
further support highly mobile users. Most deep learning based communication
techniques are based on real-valued neural networks (RVNN), which may lead to
performance degradation for ignoring the correlation between In-
Phase/Quardrature (IQ) components Trabelsi et al. (2018).

In recent years, complex-valued neural networks (CVNN) Trabelsi et al. (2018);
Fang and Sun (2014); Zhang et al. (2023) have been applied to different
communication scenarios due to their superior performance and ability to
directly process complex-valued data, including channel prediction Yang et al.
(2019), modulation classification Tu et al. (2020), OFDM receiver Zhao et al.
(2021), etc. Literature Zhao et al. (2021); Wang et al. (2021b) validate that
a deep complex-valued neural network has advantages on complex baseband signal
processing. A lightweight CLNet with a complex-valued input layer effectively
extracts complex-valued features of input Ji and Li (2021). Nevertheless, to
the best of our knowledge, CVNN has not been employed for predicting the
precoding for multi-antenna transmission.

The precoding vector/matrix is complex-valued, and the channel statistics are
also typically complex. Considering the advantages of complex neural networks
in handling complex data, as well as the current dilemma of channel feedback
for massive MIMO in 5G Wen et al. (2018); Wang et al. (2021b), we propose a
radio map based complex-valued precoding network (RMCPNet) model, which
predicts the precoding for transmitting to the user with multiple antennas
based on user positions and corresponding complex channel statistics. The
contributions of this paper are summarized as follows:

  * •

We propose a downlink channel-feedback-free transmission approach in the FD-
RAN based on user positions and historical channel statistics in the specified
spatial domain. The new transmission approach will largely reduce the usage of
precious spectrum resources required for channel estimation and feedback.

  * •

Considering that wireless signal processing is usually realized in the complex
domain, we propose a complex-valued precoding network based on the radio map.
The network can achieve fusion perception of multi-modal information,
including user position, transmit/receive correlation matrix, etc., thus
improving achievable spectral efficiency.

  * •

Based on publicly available ray tracing data Alkhateeb (2019), we conduct
extensive performance evaluation and provide result analysis for the proposed
method. Simulation results show that our proposed method can outperform real-
valued neural networks and the statistical codebook scheme.

The rest of the paper is organized as follows. Section II presents the system
model and problem formulation. Then, we elaborate on the proposed RMCPNet
architecture and corresponding training algorithm in Section III. In Section
IV, the simulation scenarios, settings, as well as corresponding simulation
results are explained in detail. Finally, we conclude the paper and point out
some potential research directions in Section V.

##  2 System Model and Problem Formulation

Figure 1 illustrates the downlink transmission in FD-RAN without real-time
physical layer feedback. We assume that the edge cloud has stored the spatial
domain statistical information about the channel, which is periodically
updated to adapt to changes in the environment. The edge cloud deploys RMCPNet
for user downlink precoding. In this paper, we focus on the single-user case
of multi-antenna transmission. First, a user will upload position information
to the edge cloud via the control base station. Note that the position
feedback is less sensitive than the channel feedback. Then, the edge cloud
inputs the user’s position information and corresponding channel statistical
information stored on the cloud server into RMCPNet to generate precoding and
deliver to the serving data base station. Last, the DBS will perform
beamforming to serve the user.

Figure 1: The proposed channel-feedback-free transmission in the downlink FD-
RAN.

According to Alkhateeb (2019) the channel between base station $b$ and user
$k$ on carrier $u$ is expressed by:

$\mathbf{h}_{k}^{b,u}=\sum_{\ell=1}^{L}\sqrt{\frac{\rho_{\ell}}{U}}e^{j\left(\vartheta_{\ell}^{b,k}+\frac{2\pi
u}{U}\tau_{\ell}^{b,k}B\right)}\mathbf{a}\left(\phi_{\mathrm{az}}^{b,k},\phi_{\mathrm{el}}^{b,k}\right),$
(1)

where
$\mathbf{a}\left(\phi_{\mathrm{az}}^{b,u},\phi_{\mathrm{el}}^{b,u}\right)$ is
the array response vector of the BS. Since we only consider a single base
station serving a single user within a subband, for simplicity of analysis, we
denote the channel of user $k$ with

$\mathbf{H}_{k}=\mathbf{U}_{k}\mathbf{G}_{k}\mathbf{V}_{k}^{H}\in\mathbb{C}^{N_{k}\times
M},$ (2)

where $\mathbf{G}_{k}\in\mathbb{C}^{N_{k}\times M}$ represents the beam domain
channel matrix, $N_{k}$ is the number of antennas for user $k$, and $M$
denotes the number of antennas for DBS. As shown in Eq. (2), $\mathbf{V}$ is
independent of the user’s position and only depends on the geometry of the
base station antenna array. When the base station employs a uniform linear
array (ULA) with half-wavelength antenna spacing, we have
$\mathbf{H}_{k}\stackrel{{\scriptstyle
M\rightarrow\infty}}{{=}}\mathbf{U}_{k}\mathbf{G}_{k}\mathbf{V}^{H}$. In this
case, $\mathbf{V}$ can be approximated by a discrete Fourier transform (DFT)
matrix. We assume that the signal transmitted by user $k$ is
$\mathbf{q}_{k}\in R^{M}$, which satisfies

$\mathbb{E}\left\\{\mathbf{q}_{k}\mathbf{q}_{k^{\prime}}^{H}\right\\}=\left\\{\begin{array}[]{cc}0&\left(k^{\prime}\neq
k\right)\\\ 1&\text{ others. }\end{array}\right.$ (3)

For convenience of notation and to simplify the analysis, we will omit the
subscript for user $k$ in the remainder. We assume that the precoding vector
for user $k$ at the base station is $\mathbf{w}$, and the signal sent by the
base station is

$\mathbf{x}=\mathbf{w}\mathbf{q}.$ (4)

We assume that $\rho=P_{\max}/\left(M\sigma_{n}^{2}\right)$ represents the
average signal-to-noise ratio of the user’s transmission, where $P_{\max}$ is
the maximum power. And, we denote the transmit covariance matrix as
$\mathbf{Q}=\mathbb{E}\left\\{\mathbf{x}\mathbf{x}_{k}^{H}\right\\}\in\mathbb{C}^{M\times
M}$. Then, the signal received by the user is

$\mathbf{y}=\mathbf{Hx}+\mathbf{n}\in\mathbb{C}^{N\times 1},$ (5)

where $\mathbf{n}$ represents the circularly symmetric complex Gaussian noise
with zero mean and covariance $\sigma_{n}^{2}$, i.e.,
$\mathbf{n}\sim\mathcal{N}_{\mathbb{C}}\left(0,\sigma^{2}\mathbf{I}_{N}\right)$.

Based on the channel capacity of MIMO channels, we can obtain the ergodic
rate:

$\displaystyle R$
$\displaystyle=\mathbb{E}\left\\{\log\det\left(\mathbf{I}+\rho\mathbf{HQH}^{H}\right)\right\\}$
(6)
$\displaystyle=\mathbb{E}\left\\{\log\det\left(\mathbf{I}+\rho\mathbf{UHQH}^{H}\mathbf{U}^{H}\right)\right\\}\,$
$\displaystyle=\mathbb{E}\left\\{\log\det\left(\mathbf{I}+\rho\mathbf{GV}^{H}\mathbf{QVG}^{H}\right)\right\\}.$

Our problem is to design a precoding strategy for massive MIMO downlink
transmission that maximizes spectral efficiency

$\displaystyle\mathcal{F}_{1}:$
$\displaystyle\max_{\mathbf{Q}}\quad\mathbb{E}\left\\{\log\det\left(\mathbf{I}+\rho\mathbf{GV}^{H}\mathbf{QVG}^{H}\right)\right\\}$
(7)
$\displaystyle\,\,\mathrm{s}.\mathrm{t}.\quad\mathrm{tr}\left(\mathbf{Q}\right)=1,\mathbf{Q}\succeq
0.$

According to Gao et al. (2009), the optimal precoding satisfies

$\mathbf{Q}^{\mathrm{opt}}=\mathbf{V\Lambda V}^{H}.$ (8)

It implies that the downlink transmission signal should be oriented along the
base station-related matrix’s eigenvectors to achieve the system’s maximum
spectral efficiency. Hence, beam domain transmission facilitates spectral
efficiency optimization in downlink massive MIMO. Consequently, problem
$\mathcal{F}_{1}$ can be reformulated as $\mathcal{F}_{2}$

$\displaystyle\mathcal{F}_{2}:$
$\displaystyle\max_{\mathbf{\Lambda}}\quad\mathbb{E}\left\\{\log\det\left(\mathbf{I}+\rho\mathbf{G\Lambda
G}^{H}\right)\right\\}$ (9)
$\displaystyle\,\,\mathrm{s}.\mathrm{t}.\quad\mathrm{tr}\left(\mathbf{\Lambda}\right)=1,\lambda\succeq
0.$

For $\mathcal{F}_{2}$, it is challenging to use the Monte Carlo method to
solve. Based on the property of the det matrix, we obtain the following
equation:

$\det\left(\mathbf{I}+\mathbf{AB}\right)=\det\left(\mathbf{I}+\mathbf{BA}\right).$
(10)

Applying Jensen’s inequality, we can acquire the following:

$\displaystyle C(\boldsymbol{\lambda})\leq C_{u}(\boldsymbol{\lambda})$
$\displaystyle=\log\operatorname{det}\left(\mathbf{I}_{N_{r}}+\rho\mathbb{E}\left\\{\mathbf{G}\boldsymbol{\Lambda}\mathbf{G}^{H}\right\\}\right)$
(11)
$\displaystyle=\log\operatorname{det}\left(\mathbf{I}_{N_{r}}+\rho\boldsymbol{\Lambda}\mathbb{E}\left\\{\mathbf{G}^{H}\mathbf{G}\right\\}\right).$

Therefore, $\mathcal{F}_{2}$ can be converted into

$\displaystyle\mathcal{F}_{3}:\max_{\Lambda}$
$\displaystyle\log\det\left(\mathbf{I}+\rho\mathbf{\Lambda}\mathbb{E}\left\\{\mathbf{G}^{\mathrm{H}}\mathbf{G}\right\\}\right)$
(12) $\displaystyle\,\,\mathrm{s}.\mathrm{t}.$
$\displaystyle\sum_{i=1}^{M}{\lambda_{i}}=1$
$\displaystyle\lambda_{i}>0,\forall i=1,...,M,$

where $\lambda_{i}$ is power, and
$\mathbb{E}\left\\{\mathbf{G}^{\mathrm{H}}\mathbf{G}\right\\}$ is the
statistical information of the channel. Hence, the base station’s precoding
can be derived from the channel’s statistical information.

##  3 Radio Map based channel-feedback-free Transmission

In the following, we will further derive how to achieve feedback-free
transmission.

###  3.1 Radio Map

From $\mathcal{F}_{3}$, we can acquire that the precoding $\mathbf{w}$ can be
derived from the statistical information of the channel
$\mathbb{E}\left\\{\mathbf{G}^{\mathrm{H}}\mathbf{G}\right\\}$. Given that
$\mathbf{H}=\mathbf{UGV}^{\mathrm{H}}$, we define the statistical information
of the channel as $\mathcal{R}_{t}$:

$\begin{array}[]{l}\mathcal{R}_{t}\triangleq\mathbb{E}\left\\{\mathbf{H}^{\mathrm{H}}\mathbf{H}\right\\}\\\
\quad\
\xlongequal{\mathbf{H}=\mathbf{UGV}^{\mathrm{H}}}\mathbb{E}\left\\{\left(\mathbf{U}\mathbf{G}\mathbf{V}^{\mathrm{H}}\right)^{\mathrm{H}}\left(\mathbf{U}\mathbf{G}\mathbf{V}^{\mathrm{H}}\right)\right\\}\\\
\quad\
=\mathbb{E}\left\\{\mathbf{G}^{\mathrm{H}}\mathbf{G}\right\\}\in\mathbb{R}^{M\times
M}.\end{array}$ (13)

We can analogously define the statistical information $\mathcal{R}_{r}$:

$\mathcal{R}_{r}\triangleq\mathbb{E}\left\\{\mathbf{HH}^{\mathrm{H}}\right\\}\in\mathbb{R}^{N\times
N}.$ (14)

Based on reference Gao et al. (2009), we define the statistical power
information

$\mathcal{R}_{p}(p)\triangleq\mathbb{E}\left\\{\mathbf{H}\odot\mathbf{H}^{*}\right\\}\in\mathbb{R}^{N\times
M}$ (15)

that characterizes the channel.

####  3.1.1 The Concept of Radio Map

We divide the area of interest $\mathcal{P}$ into different small regions
$p\in\mathcal{P}$. In this paper, we refer to map information as
$\mathcal{S}$, and the statistical information is denoted as

$\mathcal{S}\triangleq\left\\{\mathbb{E}\left\\{\mathbf{H}^{\mathrm{H}}\mathbf{H}\right\\},\mathbb{E}\left\\{\mathbf{HH}^{\mathrm{H}}\right\\},\mathbb{E}\left\\{\mathbf{H}\odot\mathbf{H}^{*}\right\\}\right\\},$
(16)

which can completely capture the channel’s fading and correlation information.
In the following, the radio map is defined as the mapping from each spatial
region $p$ to the corresponding channel statistical information

$\mathcal{S}(p)=\left\\{\mathcal{R}_{t}(p),\mathcal{R}_{r}(p),\mathcal{R}_{p}(p)\right\\},\forall
p\in\mathcal{P}.$ (17)

####  3.1.2 The Construction of Radio Map

According to the definition of radio map (17), we assumed that each user would
collect channel statistics for their position. After long-term channel
statistics, the user will periodically upload the position $p$ and
corresponding statistical information $\mathcal{S}(p)$ to the edge cloud.
Through broad statistics for all users located at the spatial domain
$\mathcal{P}$, the edge cloud will possess complete radio maps for the area,
i.e., $\mathcal{S}(p),p\in\mathcal{P}$. During the service period, each user
uploads the position $p$, and the edge cloud determines the precoding vector
$w$ based on
$\left\\{\mathcal{R}_{t}(p),\mathcal{R}_{r}(p),\mathcal{R}_{p}(p)\right\\}$.

###  3.2 Radio map based Downlink Transmission

In the following, we define the function
$\mathbf{w}=f(p,\mathcal{S}(\mathbf{p}))$ as the mapping from the channel
statistical information to the corresponding precoding, and our aim is to find
such a function. In this paper, we will reformulate our objective as to how to
leverage the channel’s statistical information $\mathcal{S}(p)$ to obtain the
user’s precoding

$\displaystyle\mathcal{F}_{4}:$
$\displaystyle\underset{f}{\arg\max}\quad\mathbb{E}\Big{\\{}\,\log_{2}\det\left(\mathrm{I}+\right.$
(18)
$\displaystyle\left.\qquad\rho\mathbf{H}f(p,\mathcal{S}(p))f(p,\mathcal{S}(p))^{\mathrm{H}}\mathbf{H}^{\mathrm{H}}\right)\Big{\\}}.$

The function $f$ cannot be derived by theoretical calculation. According to
reference Liu et al. (2022), such a function $f_{\theta}$ can always be
approximated by a neural network when the depth and width of the network reach
a certain level. Therefore, in this paper, we adopt a data-driven approach to
solve the precoding problem. Assuming that $\theta$ is the network parameter,
then we have

$\mathbf{w}_{\theta}\left(p\right)=f_{\theta}\left(p,\mathcal{S}(p)\right),$
(19)

which represents the precoding unit vector when the user is at position $p$.

Consequently, our ultimate goal is to obtain an optimal model parameter
$\theta$ that maximizes the user’s throughput:

$\displaystyle\mathcal{F}_{5}:\underset{{\theta}}{\arg\max}\mathbb{E}\Big{\\{}$
$\displaystyle\log_{2}\det\left(\mathrm{I}+\right.$ (20)
$\displaystyle\left.\rho\mathbf{H}\mathbf{w}_{\theta}\left(p\right)\mathbf{w}_{\theta}\left(p\right)^{\mathrm{H}}\mathbf{H}^{\mathrm{H}}\right)\Big{\\}},$

where $\rho$ denotes the SINR at the receiver. The model parameter $\theta$
ultimately determines the network performance.

###  3.3 Complex Neural Networks

In complex-domain signal processing, the convolution of signals in the time
domain is precisely equivalent to the multiplication of signals in the complex
domain. As both the precoding and the channel of wireless communication are
complex, the complex-domain operation is more appropriate for signal
processing, as it can better capture the relationship between the real and
imaginary components of the channel.

####  3.3.1 Complex layer

For a complex operation $f$, a complex-valued neuron performs the following
processing on a complex input Trabelsi et al. (2018)

$\left(f_{R}\left(z_{I}\right)+f_{I}\left(z_{R}\right)\right)+i\cdot\left(f_{R}\left(z_{R}\right)-f_{I}\left(z_{I}\right)\right),$
(21)

where $\psi_{R}$ and $\psi_{I}$ indicate the neuron’s real and imaginary
processing channels, respectively. For instance, the complex convolution
operation is realized as illustrated in Figure 2, where $M_{I}$ and $M_{R}$
denote the imaginary and real feature maps, and $K_{I}$ and $K_{R}$ denote the
imaginary and real convolution kernels Trabelsi et al. (2018). $M_{I}K_{I}$
signifies the real-valued convolution outcome between the imaginary
convolution kernel $K_{I}$ and the imaginary feature map $M_{I}$.

Figure 2: Illustration of the complex convolution operator.

####  3.3.2 Activation Unit

We employ Complex PReLU (CPReLU) as the complex activation function, which
applies separate PReLUs to both the real and imaginary parts of a neuron, i.e.

$\operatorname{CPReLU}(z)=\operatorname{PReLU}(z_{R})+i\operatorname{PReLU}(z_{I}).$
(22)

CPReLU satisfies the Cauchy-Riemann equations when both the real and imaginary
parts are simultaneously either strictly positive or strictly negative.

####  3.3.3 Complex Differentiation and Chain Rules

When training neural networks, the backpropagation algorithm requires the
calculation of gradients to update the weight parameters. However, the
incorporation of complex values poses challenges for computing gradients of
activation functions and loss functions during the backpropagation process.
Suppose $L$ is a real-valued loss function and $z$ is a complex variable,
where $z=x+iy$ and $x,y\in\mathbb{R}$. In this case, we have the following
equations:

$\displaystyle\nabla_{L}(z)$ $\displaystyle=\frac{\partial L}{\partial
z}=\frac{\partial L}{\partial x}+i\frac{\partial L}{\partial y}$ (23)
$\displaystyle=\frac{\partial L}{\partial\Re(z)}+i\frac{\partial
L}{\partial\Im(z)}$
$\displaystyle=\Re\left(\nabla_{L}(z)\right)+i\Im\left(\nabla_{L}(z)\right)$

Then, given another complex variable $t=r+is$ where $z$ could be expressed in
terms of $t$ and $r$, $s\in\mathbb{R}$, we would have the complex chain rules
(24) as described in the literature Trabelsi et al. (2018).

###  3.4 RMCPNet for Downlink Transmission

In Figure 3, we present the overall architecture of RMCPNet. The network
comprises four stacked layers: the input layer, the feature extraction layer
stack, the feature fusion layer stack, and the output layer.

Figure 3:  The framework of RMCPNet.

$\displaystyle\nabla_{L}(t)$ $\displaystyle=\frac{\partial L}{\partial
t}=\frac{\partial L}{\partial r}+i\frac{\partial L}{\partial s}=\frac{\partial
L}{\partial x}\frac{\partial x}{\partial r}+\frac{\partial L}{\partial
y}\frac{\partial y}{\partial r}+i\left(\frac{\partial L}{\partial
x}\frac{\partial x}{\partial s}+\frac{\partial L}{\partial y}\frac{\partial
y}{\partial s}\right)$ (24)
$\displaystyle=\Re\left(\nabla_{L}(z)\right)\left(\frac{\partial x}{\partial
r}+i\frac{\partial x}{\partial
s}\right)+\Im\left(\nabla_{L}(z)\right)\left(\frac{\partial y}{\partial
r}+i\frac{\partial y}{\partial s}\right)$

$\mathcal{L}\left(p,H\right)=-\log_{2}\det\left(\mathrm{I}+\rho\mathbf{H}f_{\theta}\left(p,\mathcal{S}(p)\right)f_{\theta}\left(p,\mathcal{S}(p)\right)^{\mathrm{H}}\mathbf{H}^{\mathrm{H}}\right)$
(27)

Considering the heterogeneity of different information, we partition the
information $\mathcal{S}(p)$ into four parallel subnetworks to input different
position information and statistical information to the network, which
achieves the feature extraction of heterogeneous information. For the position
encoding subnetwork, we first convert the input two-dimensional position $p$
into a complex number, and then map it to a higher-dimensional feature through
a fully connected layer. The high-dimensional feature then undergoes a
convolution layer, complex batch normalization, and two successive CConv
layers to obtain the two-dimensional encoding of $p$. The transmit correlation
matrix $\mathcal{R}_{t}(p)$, the receive correlation matrix
$\mathcal{R}_{r}(p)$, and the channel fading matrix $\mathcal{R}_{p}(p)$ are
processed by three successive CConv, CBN, and CPReLU layers, respectively, to
obtain the channel transmit coherence encoding, receive coherence encoding and
channel fading encoding. In the feature fusion layer, we use Flatten and CBN
to concatenate and fuse the features obtained by the four subnetworks. After
multi-modal information fusion, we use CLinear CBN and CPReLU to process the
fused features. The network finally produces the precoding output
$f_{\theta}(p,\mathcal{S}(p))$ through CPReLU.

In this paper, our objective is to maximize the spectral efficiency, which is
equivalent to minimizing the negative spectral efficiency. Since we encounter
difficulty obtaining the optimal precoding $\mathbf{w}$ by the Monte Carlo
method, we can acquire the overall spectral efficiency when applying the
precoding $\mathbf{w}_{\theta}(p)$ by equation (3). In the following, we
denote a sample as

$x=\left\\{p,\mathcal{S}(p),\mathbf{H}\right\\}.$ (25)

Then, a batch of samples can be denoted as

$\mathcal{X}_{b}\triangleq\left\\{p_{i},\mathcal{S}(p_{i}),\mathbf{H}_{i}|i\in{1,\dots
B_{s}}\right\\},$ (26)

where $B_{s}$ is batch size. Therefore, the loss function can be calculated as
(27).

Algorithm 1 RMCPNet Trainning

0: The dataset $\mathcal{D}$, model $f_{\theta}(\cdot)$, learning rate
$\gamma$, batch size $B_{s}$, maximum iteration $I_{\max}$, terminate
parameters $N_{ter},N_{lr}$

0: The trained model $f_{\theta}(\cdot)$

1: Initialize the RMCPNet model $f_{\theta}(\cdot)$ with
$\theta\sim\mathcal{CN}(0,0.01)$

2: partition $\mathcal{D}$ according to batch size $B_{s}$

3: epoch counter $i\leftarrow 0$

4: convergence label $\mathcal{C}=\text{False}$

5: while $i\leq I_{\max}$ or $\mathcal{C}$ do

6: for $b={0,..,B-1}$ do

7: sample a batch $\mathcal{X}_{b}$ from $\mathcal{D}$

8: compute loss $\mathcal{L}(\mathcal{X}_{b})$

9: compute the stochastic gradient of $\theta$ according to
$\mathcal{L}(\mathcal{X}_{b})$

10: update model parameters $f_{\theta}$ with corresponding gradients and
learning rate $\gamma$

11: if $L$ converges then

12: $\mathcal{C}=\text{True}$

13: break

14: end if

15: end for

16: $i=i+1$

17: update the learning rate $\gamma$

18: end while

When a batch of training data $\mathcal{X}_{b}$ is fed into the RMCPNet, we
can obtain the average throughput loss $\mathcal{L}(\mathcal{X}_{b})$ at this
point. When training the network, the proposed training algorithm 1 is goal-
oriented and directly trains the network, which updates the network parameters
in the direction of maximizing $\mathcal{L}(\mathcal{X}_{b})$. In this
algorithm, we constrain the maximum epoch $I_{\max}$, and add an algorithm
convergence condition to terminate the training early (whether the
$\mathcal{L}\left(\mathcal{X}_{b}\right)$ obtained by successive $N_{ter}$
epochs decreases). The learning rate $\gamma$ is executed by exponential decay
$\gamma=0.9*\gamma$ by detecting whether the
$\mathcal{L}\left(\mathcal{X}\right)$ obtained by successive $N_{lr}$ epochs
decreases.

##  4 Simulation Results

In this section, we explain the detailed simulation setup and show the
comparison with several benchmark precoding methods.

###  4.1 Simulation scenario and Data Collection

As illustrated in Figure 4, we consider the simulation scenario based on the
DeepMIMO outdoor dynamic scenario ’O2 Dynamic’ Alkhateeb (2019) to study the
performance of our proposed method. The major simulation hyperparameters are
summarized in Table 1. There are two DBSs located in the area, and users are
distributed in the green rectangular area $\mathcal{P}$ in which we sample up
to 100,000 candidate positions to construct the radio map.

Figure 4: The simulation scenario.

We simulate 800 time slots to collect channel data. Specifically, the first
600 slots are used to generate the radio map $\mathcal{S}(p),p\in\mathcal{P}$
and train the RMCPNet $f_{\theta}\left(p,\mathcal{S}(p)\right)$, while the
remaining 200 slots serve to verify the effectiveness of the proposed method.
For each position $p\in\mathcal{P}$, we collect corresponding channels
$\mathbf{H}$, and compute the radio statitical feature $\mathcal{S}(p)$, which
constitutes a training sample
$x_{i}=\left\\{p,\mathcal{S}(p),\mathbf{H}\right\\}$. Then, we can build the
radio map $\mathcal{S}$ and acquire the dataset.

Table 1: Simulation hyperarameters. Parameters | Values
---|---
Simulation scenario | O2 Dynamic (Outdoor 2)
DBS number | 2
Candidate positions | 100000
Simulation slots | total 800, 600 for train, 200 for test
Channel model | Ray tracing
Center frequency | 3.5 GHz
Multipath | 10
Bandwidth | 100MHz
DBS antenna | 8 $\times$ 2 isotropic antenna array
User antenna | 2 $\times$ 1 isotropic antenna array
OFDM | Number of FFT 256, Sub-carrier 30 KHz

In this paper, we employ four benchmark methods to verify the efficacy of our
proposed algorithm.

  * •

No Precoding (NoP). No precoding is applied.

  * •

Statistical Codebook (SCB). A codebook is used to select precoding. The
precoding that has been used most frequently in the training slots at position
$p$ is adopted as the precoding for the user at that position for the test.

  * •

Neural Networks (NN). We train real-valued neural networks to conduct
precoding.

  * •

SVD decomposition (SVD). The theoretically optimal method for calculating
precoding.

In Table 2, we describe the detailed structure of the RMCPNet. Next, we will
simulate and validate the performance of the proposed algorithm from different
perspectives.

Table 2: RMCPNet structure description. Subnetwork | Structure
---|---
position embedding | 128 units; 64 clinear; 16 2*1 CConv filters;
transmit embedding | 16 3*3 CConv; 32 3*3 CConv; 16 2*2 CConv
receive embedding | 16 2*2 CConv; 16 1*2 CConv; 8 2*1 CConv
power embedding | 16 2*3 CConv; 16 2*3 CConv; 8 1*2 CConv
feature fusion | 256 units; 128 units; 16 units

###  4.2 Channel Distribution of Angular Domain

Calculating the power azimuth spectrum (PAS) of the channel, we observed a
strong correlation between the user’s location/angle and the direction of
maximum spectral energy density. Figure 5 presents the PAS of random sample
precoding. We observe that the precoding obtained by SCMPNet can point more
precisely to the angle and energy peaks of the channel, resulting in a higher
received channel power as compared to the SCB method. This can be attributed
to our method’s ability to provide more accurate beam directions. Moreover,
since RMCPNet outputs the precoding vector, it can generate multiple beam
directions in the spatial domain to accommodate the energy distribution of the
channel small-scale fading in different directions. Hence, our proposed method
exhibits better performance and robustness to channel variations.

Figure 5: The PAS for a sample of channel, RMCPNet precoding, and SCB
precoding.

###  4.3 spectral efficiency

Figure 6 shows that our proposed RMCPNet method significantly outperforms the
SCB and NN methods. Specifically, RMCPNet has very few users with spectral
efficiency below 7 bit/Hz/s, while many users of the SCB method have spectral
efficiency below 5 bit/Hz/s. Moreover, we find that the SCB method, which
obtains the optimal precoding based on historical statistics, is challenging
to work in the complex time-varying electromagnetic environment with
multipath. In addition, compared to the NN method, RMCPNet can effectively
reduce the probability of choosing suboptimal precoding. However, the major
drawback of RMCPNet is that it cannot achieve a higher spectral efficiency
level (above 15bit/s/Hz) compared with the optimal precoding scheme based on
SVD. This is because the current RMCPNet does not introduce a multi-stream
mechanism that can achieve spatial multiplexing.

Figure 6: The CDF of the average spectral efficiency for different algorithms.

Figure 7 compares the average spectral efficiency achieved by different
algorithms. Our proposed scheme is far superior to the statistical precoding
method, with a 76% performance improvement. Compared with the NN method, we
still have a 16% performance improvement, which indicates that our network
architecture design is more reliable. Compared with the optimal precoding
based on SVD, RMCPNet can still achieve 82.4% of the optimal performance. This
gap can be bridged by introducing a multi-stream method in the future.

Figure 7: The average spectral efficiency of different algorithms.

In Figure 8, we change the transmit power of the base station from -30dBm to
20dBm. We find that RMCPNet still achieves the best transmission performance
compared to other single-stream transmission schemes, which demonstrates that
our proposed algorithm is robust and can achieve good performance for
different transmit powers.

Figure 8: The average spectral efficiency of different algorithms varies with
the transmission power. Figure 9:  SNR at different positions for RMCPNet and
SCB, respectively. Note that the positions are left blank if there is no data
available.

According to the phenomena illustrated in Figure 6 and Figure 7, we think that
the performance gain mainly comes from three aspects: the radio map can well
characterize the statistical properties of the channel; the proposed multi-
modal information fusion architecture can better extract the intrinsic
property of the channel; the complex neural processing unit is more conducive
to handling complex channel values.

###  4.4 SNR at different positions

Figure 9 compares the signal-to-noise ratio (SNR) of the RMCPNet and SCB
methods at different positions. Our method outperforms the SCB scheme, i.e.,
achieving better signal coverage. Furthermore, we notice that the statistical
codebook scheme forms more distinct beams in areas close to the base station,
where SNR is high. However, in areas far away from the base station where SNR
is low, there is high variability in the positions with high and low SNR
within these areas. This may be due to the increased uncertainty of the
channel energy distribution when low SNR leads to erratic codebook selection
and suboptimal performance. In contrast, our proposed RMCPNet method can
maintain high SNR across the entire spatial domain.

##  5 Conclusions and Future Works

In this paper, we have proposed a transmission mechanism for downlink
transmission in FD-RAN without relying on physical layer channel feedback.
Furthermore, We have designed a complex-valued precoding network called
RMCPNet based on the radio map. To leverage multimodal input information,
including user location, channel transmission/reception correlation matrix,
and channel energy matrix, in the RMCPNet, we have further designed several
different subnets for multimodal feature extraction. Then, the multimodal
encoded features are fused at the feature fusion layer. Considering the
computational complexity of obtaining optimal precoding, we have also proposed
a specific network training algorithm with negative spectral efficiency as the
loss function. The simulation results of RMCPNet on the public DeepMIMO
dataset demonstrate that the proposed RMCPNet delivers significantly improved
performance compared to traditional neural networks and the statistical
precoding method.

In our future work, we plan to further study the impact of feedback delay on
wireless transmission and consider precoding in mobile scenarios with the help
of predicting user location in real time. We will also implement the feedback-
free beamforming method based on the predicted user position and channel
statistical information on USRP hardware to verify the feasibility of the
proposed scheme in a realistic environment. Furthermore, we will extend the
current method to support multi-stream transmission. Additionally, we will
study cooperative precoding of multiple base stations.

## References

  * Liu et al. [2020a] Guangyi Liu, Yuhong Huang, Na Li, Jing Dong, Jing Jin, Qixing Wang, and Nan Li.  Vision, requirements and network architecture of 6G mobile network beyond 2030.  _China Communications_ , 17(9):92–104, 2020a.
  * Yu et al. [2019] Quan Yu, Haibo Zhou, Jiacheng Chen, Ying Li, Jian Jing, Jiwei Jackokie Zhao, Bo Qian, and Jian Wang.  A fully-decoupled RAN architecture for 6G inspired by neurotransmission.  _Journal of Communications and Information Networks_ , 4(4):15–23, 2019.
  * Dang et al. [2020] Shuping Dang, Osama Amin, Basem Shihada, and Mohamed-Slim Alouini.  What should 6g be?  _Nature Electronics_ , 3(1):20–29, 2020.
  * Wang et al. [2023] Xianbin Wang, Jie Mei, Shuguang Cui, Cheng-Xiang Wang, and Xuemin Sherman Shen.  Realizing 6G: The operational goals, enabling technologies of future networks, and value-oriented intelligent multi-dimensional multiple access.  _IEEE Network_ , 37(1):10–17, 2023.
  * Qian et al. [2023] Bo Qian, Ting Ma, Yunting Xu, Jiwei Zhao, Kai Yu, Yuan Wu, and Haibo Zhou.  Enabling Fully-Decoupled Radio Access with Elastic Resource Allocation.  _IEEE Transactions on Cognitive Communications and Networking_ , pages 1–1, 2023.  ISSN 2332-7731.
  * Xu et al. [2021] Yunting Xu, Haibo Zhou, Ting Ma, Jiwei Zhao, Bo Qian, and Xuemin Shen.  Leveraging Multiagent Learning for Automated Vehicles Scheduling at Nonsignalized Intersections.  _IEEE Internet of Things Journal_ , 8(14):11427–11439, July 2021.  ISSN 2327-4662.
  * Tse and Viswanath [2005] David Tse and Pramod Viswanath.  _Fundamentals of wireless communication_.  Cambridge university press, 2005.
  * Wen et al. [2018] Chao-Kai Wen, Wan-Ting Shih, and Shi Jin.  Deep learning for massive MIMO CSI feedback.  _IEEE Wireless Communications Letters_ , 7(5):748–751, 2018.
  * Zhou et al. [2020] Gui Zhou, Cunhua Pan, Hong Ren, Kezhi Wang, and Arumugam Nallanathan.  A framework of robust transmission design for IRS-aided MISO communications with imperfect cascaded channels.  _IEEE Transactions on Signal Processing_ , 68:5092–5106, 2020.  doi:10.1109/TSP.2020.3019666.
  * Yu et al. [2023] Kai Yu, Quan Yu, Zhixuan Tang, Jiwei Zhao, Bo Qian, Yunting Xu, Haibo Zhou, and Xuemin Shen.  Fully-Decoupled Radio Access Networks: A Flexible Downlink Multi-Connectivity and Dynamic Resource Cooperation Framework.  _IEEE Transactions on Wireless Communications_ , 22(6):4202–4214, June 2023.  ISSN 1536-1276.
  * Luo et al. [2018] Changqing Luo, Jinlong Ji, Qianlong Wang, Xuhui Chen, and Pan Li.  Channel state information prediction for 5G wireless communications: A deep learning approach.  _IEEE Transactions on Network Science and Engineering_ , 7(1):227–236, 2018.
  * Zhao et al. [2023] Jiwei Zhao, Quan Yu, Bo Qian, Kai Yu, Yunting Xu, Haibo Zhou, and Xuemin Shen.  Fully-decoupled radio access networks: A resilient uplink base stations cooperative reception framework.  _IEEE Transactions on Wireless Communications_ , pages 1–1, 2023.
  * Shen et al. [2016] Juei-Chin Shen, Jun Zhang, Emad Alsusa, and Khaled B. Letaief.  Compressed CSI acquisition in FDD massive mimo: How much training is needed?  _IEEE Transactions on Wireless Communications_ , 15(6):4145–4156, 2016.
  * Guo et al. [2020] Jiajia Guo, Chao-Kai Wen, Shi Jin, and Geoffrey Ye Li.  Convolutional neural network-based multiple-rate compressive sensing for massive MIMO CSI feedback: Design, simulation, and analysis.  _IEEE Transactions on Wireless Communications_ , 19(4):2827–2840, 2020.
  * Mashhadi et al. [2020] Mahdi Boloursaz Mashhadi, Qianqian Yang, and Deniz Gündüz.  Distributed deep convolutional compression for massive MIMO CSI feedback.  _IEEE Transactions on Wireless Communications_ , 20(4):2621–2633, 2020.
  * Yin et al. [2022] Ziqing Yin, Wei Xu, Renjie Xie, Shaoqing Zhang, Derrick Wing Kwan Ng, and Xiaohu You.  Deep compression for massive MIMO: A self-information model-driven neural network.  _IEEE Transactions on Wireless Communications_ , 21(10):8872–8886, 2022.
  * Song et al. [2021] Xuan Song, Jun Wang, Jie Wang, Guan Gui, Tomoaki Ohtsuki, Haris Gacanin, and Hikmet Sari.  SALDR: Joint self-attention learning and dense refine for massive MIMO CSI feedback with multiple compression ratio.  _IEEE Wireless Communications Letters_ , 10(9):1899–1903, 2021.
  * Wang et al. [2021a] Jie Wang, Guan Gui, Tomoaki Ohtsuki, Bamidele Adebisi, Haris Gacanin, and Hikmet Sari.  Compressive sampledCSI feedback method based on deep learning for FDD massive MIMO systems.  _IEEE Transactions on Communications_ , 69(9):5873–5885, 2021a.
  * Gao et al. [2009] Xiqi Gao, Bin Jiang, Xiao Li, Alex B Gershman, and Matthew R McKay.  Statistical eigenmode transmission over jointly correlated MIMO channels.  _IEEE Transactions on Information Theory_ , 55(8):3735–3750, 2009.
  * Björnson et al. [2014] Emil Björnson, Mats Bengtsson, and Björn Ottersten.  Optimal multiuser transmit beamforming: A difficult problem with a simple solution structure.  _IEEE Signal Processing Magazine_ , 31(4):142–148, 2014.
  * Kim et al. [2021] Junbeom Kim, Hoon Lee, and Seok-Hwan Park.  Learning robust beamforming for MISO downlink systems.  _IEEE Communications Letters_ , 25(6):1916–1920, 2021.
  * Wang et al. [2019] Wenjin Wang, Xu Chen, Li You, Xinping Yi, and Xiqi Gao.  Artificial noise assisted secure massive MIMO transmission exploiting statistical CSI.  _IEEE Communications Letters_ , 23(12):2386–2389, 2019.
  * Shi et al. [2021] Junchao Shi, Wenjin Wang, Xinping Yi, Xiqi Gao, and Geoffrey Ye Li.  Deep learning-based robust precoding for massive MIMO.  _IEEE Transactions on Communications_ , 69(11):7429–7443, 2021.
  * Peng et al. [2019] Wei Peng, Wengang Li, Wei Wang, Xiao Wei, and Tao Jiang.  Downlink channel prediction for time-varying FDD massive MIMO systems.  _IEEE Journal of Selected Topics in Signal Processing_ , 13(5):1090–1102, 2019.
  * Luo et al. [2020] Changqing Luo, Jinlong Ji, Qianlong Wang, Xuhui Chen, and Pan Li.  Channel state information prediction for 5G wireless communications: A deep learning approach.  _IEEE Transactions on Network Science and Engineering_ , 7(1):227–236, 2020.
  * Alrabeiah and Alkhateeb [2020] Muhammad Alrabeiah and Ahmed Alkhateeb.  Deep learning for mmWave beam and blockage prediction using sub-6 GHz channels.  _IEEE Transactions on Communications_ , 68(9):5504–5518, 2020.
  * Liu et al. [2020b] Chang Liu, Weijie Yuan, Zhiqiang Wei, Xuemeng Liu, and Derrick Wing Kwan Ng.  Location-aware predictive beamforming for UAV communications: A deep learning approach.  _IEEE Wireless Communications Letters_ , 10(3):668–672, 2020b.
  * Koivisto et al. [2017] Mike Koivisto, Aki Hakkarainen, Mario Costa, Petteri Kela, Kari Leppanen, and Mikko Valkama.  High-efficiency device positioning and location-aware communications in dense 5G networks.  _IEEE Communications Magazine_ , 55(8):188–195, 2017.
  * Kela et al. [2016] Petteri Kela, Mario Costa, Jussi Turkka, Mike Koivisto, Janis Werner, Aki Hakkarainen, Mikko Valkama, Riku Jantti, and Kari Leppanen.  Location based beamforming in 5G ultra-dense networks.  In _2016 IEEE 84th Vehicular Technology Conference (VTC-Fall)_ , pages 1–7. IEEE, 2016.
  * Miao et al. [2020] Wang Miao, Chunbo Luo, Geyong Min, Yang Mi, and Zhengxin Yu.  Location-based robust beamforming design for cellular-enabled UAV communications.  _IEEE Internet of Things Journal_ , 8(12):9934–9944, 2020.
  * Alkhateeb et al. [2018] Ahmed Alkhateeb, Sam Alex, Paul Varkey, Ying Li, Qi Qu, and Djordje Tujkovic.  Deep learning coordinated beamforming for highly-mobile millimeter wave systems.  _IEEE Access_ , 6:37328–37348, 2018.
  * Zhou et al. [2019] Bingpeng Zhou, An Liu, and Vincent Lau.  Successive localization and beamforming in 5G mmWave MIMO communication systems.  _IEEE Transactions on Signal Processing_ , 67(6):1620–1635, 2019.
  * Liu et al. [2022] Chang Liu, Weijie Yuan, Shuangyang Li, Xuemeng Liu, Husheng Li, Derrick Wing Kwan Ng, and Yonghui Li.  Learning-based predictive beamforming for integrated sensing and communication in vehicular networks.  _IEEE Journal on Selected Areas in Communications_ , 40(8):2317–2334, 2022.
  * Trabelsi et al. [2018] Chiheb Trabelsi, Olexa Bilaniuk, Ying Zhang, Dmitriy Serdyuk, Sandeep Subramanian, João Felipe Santos, Soroush Mehri, Negar Rostamzadeh, Yoshua Bengio, and Christopher J. Pal.  Deep complex networks.  In _6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings_. OpenReview.net, 2018.
  * Fang and Sun [2014] Tao Fang and Jitao Sun.  Stability of complex-valued recurrent neural networks with time-delays.  _IEEE Transactions on Neural Networks and Learning Systems_ , 25(9):1709–1713, 2014.
  * Zhang et al. [2023] Chenwu Zhang, Hancheng Lu, and Jinxue Liu.  Imreconet: Learn to detect in index modulation aided MIMO systems with complex valued neural networks.  _IEEE Transactions on Vehicular Technology_ , 72(2):1791–1805, 2023.
  * Yang et al. [2019] Yuwen Yang, Feifei Gao, Geoffrey Ye Li, and Mengnan Jian.  Deep learning-based downlink channel prediction for FDD massive MIMO system.  _IEEE Communications Letters_ , 23(11):1994–1998, 2019.
  * Tu et al. [2020] Ya Tu, Yun Lin, Changbo Hou, and Shiwen Mao.  Complex-valued networks for automatic modulation classification.  _IEEE Transactions on Vehicular Technology_ , 69(9):10085–10089, 2020.
  * Zhao et al. [2021] Zhongyuan Zhao, Mehmet Can Vuran, Fujuan Guo, and Stephen D Scott.  Deep-waveform: A learned OFDM receiver based on deep complex-valued convolutional networks.  _IEEE Journal on Selected Areas in Communications_ , 39(8):2407–2420, 2021.
  * Wang et al. [2021b] Yu Wang, Guan Gui, Haris Gacanin, Tomoaki Ohtsuki, Octavia A Dobre, and H Vincent Poor.  An efficient specific emitter identification method based on complex-valued neural networks and network compression.  _IEEE Journal on Selected Areas in Communications_ , 39(8):2305–2317, 2021b.
  * Ji and Li [2021] Sijie Ji and Mo Li.  CLNet: Complex input lightweight neural network designed for massive MIMO CSI feedback.  _IEEE Wireless Communications Letters_ , 10(10):2318–2322, 2021.
  * Alkhateeb [2019] A. Alkhateeb.  DeepMIMO: A generic deep learning dataset for millimeter wave and massive MIMO applications.  In _Proc. of Information Theory and Applications Workshop (ITA)_ , pages 1–8, San Diego, CA, Feb 2019.