# Query Expansion and Entity Weighting for Query Reformulation Retrieval in
Voice Assistant Systems

Zhongkai Sun<EMAIL_ADDRESS>Amazon Alexa AISeattleWAUSA ,  Sixing Lu
<EMAIL_ADDRESS>Amazon Alexa AISeattleWAUSA ,  Chengyuan Ma
<EMAIL_ADDRESS>Amazon Alexa AISeattleWAUSA ,  Xiaohu Liu
<EMAIL_ADDRESS>Amazon Alexa AISeattleWAUSA and  Chenlei Guo
<EMAIL_ADDRESS>Amazon Alexa AISeattleWAUSA

(2022)

###### Abstract.

Voice assistants such as Alexa, Siri, and Google Assistant have become
increasingly popular worldwide. However, linguistic variations, variability of
speech patterns, ambient acoustic conditions, and other such factors are often
correlated with the assistants misinterpreting the user’s query. In order to
provide better customer experience, retrieval based query reformulation (QR)
systems are widely used to reformulate those misinterpreted user queries.
Current QR systems typically focus on neural retrieval model training or
direct entities retrieval for the reformulating. However, these methods rarely
focus on query expansion and entity weighting simultaneously, which may limit
the scope and accuracy of the query reformulation retrieval. In this work, we
propose a novel Query Expansion and Entity Weighting method (QEEW), which
leverages the relationships between entities in the entity catalog (consisting
of users’ queries, assistant’s responses, and corresponding entities), to
enhance the query reformulation performance. Experiments on Alexa annotated
data demonstrate that QEEW improves all top precision metrics, particularly 6%
improvement in top10 precision, compared with baselines not using query
expansion and weighting; and more than 5% improvement in top10 precision
compared with other baselines using query expansion and weighting.

query expansion, entity weighting, query reformulation, retrieval system

††copyright: acmcopyright††journalyear: 2022††doi:
10.1145/1122445.1122456††conference: Phoenix ’22: the 15th ACM International
WSDM Conference; Feb 21–25, 2022; Phoenix, AZ††booktitle: Phoenix ’22: the
15th ACM International WSDM Conference, Feb 21–25, 2022, Phoenix, AZ††price:
15.00††isbn: 978-1-4503-XXXX-X/18/06††ccs: Information systems Novelty in
information retrieval

##  1\. Introduction

Virtual voice assistants such as Alexa, Siri, and Google Assistant have become
daily necessities for worldwide users. There are two main components in such
systems, namely Automatic Speech Recognition (ASR) and Natural Language
Understanding (NLU) to extract the semantic information from an input voice
query. However, misinterpretations may occur owing to factors such as
semantic/speech pattern variants and ambient acoustic conditions, resulting in
incorrect responses and unsatisfactory user experience. Therefore, it has
become a vital yet challenging task to enable voice assistants to correctly
understand those misinterpreted user queries.

Many works have been devoted to query reformulating (QR) to correct
misinterpreted user queries (Wang et al., 2021; Fan et al., 2021; Buck et al.,
2017; Cao et al., 2008; Xu and Croft, 2017; He et al., 2016). Among QR
systems, the retrieval and ranking based approaches are efficient and widely-
used in the industry (Fan et al., 2021; Wang et al., 2021; Cho et al., 2021).
These works focus more on retrieval models without additional entity
information. However, in voice assistant systems where task-oriented queries
are usually short and ambiguous, the query intention may not be fully
interpreted and this limitation negatively impact the performance of QR system
whose relevance strongly relies on the query. For instance, a user’s input
query (”play long distance love by Sheena Easton”) contains a correct singer
name (”Sheena Easton”) but also a lyric (”long distance love”) instead of the
correct song name (”telefone”). To reformulate this query so that virtual
voice assistants can find the target song to play, the QR system needs to
utilize the existing entities to obtain the correct song name ”telefone”;
besides, the model also needs to make a decision between two candidates (”play
long distance love by little feat” and ”play telefone by sheena easton”) as
they both contain key words (”long distance love” or ”sheena easton”) in the
original query.

Query expansion is an effective approach to enrich the original oral query.
Some researches consider the correlations between a query and its expansion
candidates to select the most relevant candidate from the candidate pool: (Cui
et al., 2002) proposes a query expansion method based on probabilistic
correlations between query logs; (Imani et al., 2019) uses a DNN model to
directly predict the scores for query expansion terms. Both approaches make
query more complete but do not consider the weighting between original query
and expanded ones. Other works have studied how to directly generate the
expansions: (Gao and Nie, 2012) proposes a concept-based translation method to
generate the expanded query; (Mustar et al., 2020) uses BERT (Devlin et al.,
2018) and BART (Lewis et al., 2019) with a next query prediction task to do
query suggestion; docT5query (Nogueira et al., 2019) implements a powerful T5
model (Raffel et al., 2019) to predict the relevant query for each document.
The generative approaches are able to provide large number of variants, but it
may also bring noise and lead to incorrect reformulations.

Entity (term/phrase) weighting is also a common approach to emphasize the key
words in the query and improve the retrieval performance. To predict the
weight of each entity, conventional methods use the corpus statistics based
methods like TF-IDF, BM25, and SRNM (Jones, 1972; Zamani et al., 2018) to
compute the scores of entities. Recently, neural network based methods have
been widely studied: (Roitman, 2019) uses a supervised query performance
prediction method to learn query term weighting; (Zhang et al., 2017) utilizes
a random-walk model to learn latent representations for candidates to capture
the relative importance between entities; (Bai et al., 2020) uses pre-trained
language models to learn sparse entity representations in order to better
predict the importance of each entity. However, these works haven’t utilized
the expansion and term weighting simultaneously. DeepImapct (Mallia et al.,
2021) first expands the documents using docT5query (Nogueira et al., 2019),
and then uses a contrastive loss to predict the weight of each term in the
query and document. However, DeepImapct focuses on generating the expansions
for long documents (e.g. a paragraph), but it has not been expanded to short
query scenarios.

In this paper, we integrate query expansion and entity weighting together to
propose a novel method called Query Expansion and Entity Weights prediction
(QEEW) for query reformulation. Specifically, QEEW first establishes an entity
Expansion Knowledge Base (EEKB) by categorizing and linking entities in an
entity catalog (which consists of user queries, assistant’s responses and
corresponding entities) to expand queries. An entity weighting prediction
model is then trained using annotated query reformulation pairs with
expansions obtained from the EEKB. By integrating query entity weighting with
query expansion, QEEW can retrieve reformulations more relevant to the user’s
intention. Our evaluation on annotated Alexa data shows that the proposed QEEW
improves 2.6%, 5.3%, 1.1% on precision@50, precision@10, and precision@1
compared with best scores obtained from using other methods.

##  2\. Methodology

This section describes the details of the proposed QEEW method. The training
data are pairs of annotated misinterpreted queries and their corresponding
reformulations, and the EEKB is built from a pre-established user-satisfied
entity catalog. To determine if a user’s query is successfully processed by
the voice assistant, an automatic friction estimation system (Gupta et al.,
2021) is applied, and only the non-frictional queries and their corresponding
assistant’s responses in the catalog will be used to build the EEKB. As shown
in Figure 1, our proposed QEEW model mainly consists of two components:

  * •

Entity Expansion Knowledge Base An EEKB which contains connections between
different entities is built by categorizing entities based on their
occurrences in queries and responses, both of which are obtained form the user
satisfied entity catalog. Once this knowledge base is built, it will be used
to provide expansion candidates in both training and test steps. Details of
the building EEKB and how to expand query with it will be introduced in
section 2.1.

  * •

Entity Weight Prediction Model After expanding the query from the EEKB, an
entity weight prediction model will be trained to predict the weighting scores
of both entities in the original query and those expansions obtained from the
EEKB. Section 2.2 introduces the details of this entity weight prediction
model.

The expanded query with the predicted entity weights can then be used in the
downstream retrieval/ranking QR system to obtain the final reformulation.

Figure 1. The overview of the workflow: 1) entity catalog data is first used
to build the Entity Expansion Knowledge Base; 2) an entity weight prediction
model is trained with annotated Alexa ¡query, reformulation¿ pairs. Figure 2.
Illustration of the building process of the entity connection knowledge base.
First, each entity in one entity catalog entry will be categorized based on
its occurrence in this entry’s query and response. Second, every two entities
in the same entry will be connected with each other. Third, for each edge, the
”overall relevance score” is calculated based on these two entities’
respective scores in all times they appear together. Figure 3. The
illustration of the entity weight prediction model. A hierarchical structure
is applied to predict the relative weights for all entities

###  2.1. Entity Expansion Knowledge Base

The EEKB is built using a comprehensive high-quality entity catalog consisting
of user’s queries, user-satisfied assistant’s responses, and every query-
response pair’s unique entity list. This entity catalog is able to provide
corrections for those inaccurate entities; and the voice assistant’s responses
can also help to indicate entities’ relative importance. By connecting each
entity with others and categorizing each entity according to its existence in
query and response, the EEKB can be established with the relationships between
different types of entities. Therefore, a query can be expanded by finding the
most relevant expansions in this EEKB for every entity in the query.

0: ENTITY CATALOG $\mathbb{C}=\\{(C_{i}\\}$, $1\leq i\leq K$, where
$C_{i}=\\{Query_{i},Response_{i},E_{i}\\}$, where $E_{i}\leftarrow
uniqueentitiesin\\{Query_{i},Response_{i}\\}$

KG $\leftarrow\\{\\}$

for $i\in\\{1,\dots,K\\}$ do

for $e_{j}$ in $E_{i}$ do

if $e_{j}$ in $Query_{i},Response_{i}$ then

$Level_{j}\leftarrow 3$

else if $e_{j}$ in $Response_{i}$ then

$Level_{j}\leftarrow 2$

else

$Level_{j}\leftarrow 1$

end if

end for

for $e_{m},e_{n}$ in $E_{i},m\neq n$ do

KG[$e_{m}$][$e_{n}$] += $level_{m}\times level_{n}$

KG[$e_{n}$][$e_{m}$] += $level_{n}\times level_{m}$

end for

end for

Algorithm 1 Building the entity expansion knowledge base

Figure 2 demonstrates the building process of the EEKB. Nodes in the EEKB are
entities from the entity catalog, and each node is connected with others when
both nodes appear in the same entry in the entity catalog. The score on the
edge indicates the overall relevance level of two nodes. For an entity in one
entry, its current relevance level is based on its occurrence in query and
response. Level 1 means that this entity only exists in the query; level 2
represents that it only exists in the response; and level 3 indicates that it
exists in both query and response. The motivation is that the higher the
relevance level, the more relevant the corresponding entity with the query is.
In the example in Figure 2, the entity ”Sheena Easton” has a relevance level 3
because it exists in both query and response. Every two entities’ relevance
levels are also recorded. For instance, in Figure 2, $"telefone"$ and
$"SheenaEaston"$ will have a relevance level pair $[level2,level3]$. The
overall relevance level score between two connected nodes is defined as the
sum of the product of the their respective scores each time they appear
together. The detailed algorithm is presented in Algorithm 1.

Once EEKB is established, it can be used to expand a specific query. For each
entity in the original target query, its K most relevant nodes (ranked
according to the overall relevance score) is obtained from the knowledge base.
However, since typically only limited expansions are useful for reformulation
retrieval, the weighting model trained in 2.2 can help to polish the
expansion. For example, some of the expanded phases may be assigned weight=0
and will be removed from the expansion.

###  2.2. Entity Weights Prediction Model

The weight prediction model is trained for entities in the original query as
well as expansions obtained from the EEKB in section 2.1. The training data is
the labelled Alexa query reformulation pairs, and the weight to be predicted
for each entity is a specific weighting score level based on the
reformulation.

As shown in the workflow in Figure 1, the task is defined as predicting the
weighting score level of all entities (both original and expanded) for a given
query. Entities exist in the reformulation will have higher weighting score
levels while those not in the reformulation will have lower score levels. The
weighting score level of each entity is assigned according to the existence in
the labelled Alexa query and reformulation pairs: 1) if this entity exist
neither in the original defective query nor in the reformulation, the score
level is labelled as 0; 2) the score level of entities in original query will
be labeled as 1; 3) the score level of entities in reformulation (either in
request or not) will be labeled as 2, which indicates the most important
weight.

The weight training task is defined as: give a query $Q$ with $m$ entities
$E_{1},E_{2},...,E_{m}$ and each entity $E_{i}$ has $k$ expansions
$E_{i}^{1},E_{i}^{2},...,E_{i}^{k}$, the target is to predict the importance
score $l_{p}$ for every entity $E_{p}$ (both original and expanded) where
$1\leq p\leq m(k+1)$. In this work, we train a hierarchical model to predict
each entity’s weight. Figure 3 demonstrates the structure of the hierarchical
model, which can better learning the relationships between each entity and its
expansions instead of predicting all of the weights in one step.

As shown in Figure 3, a pretrained language model (PLM) is used to encode each
entity $E_{p}$. Specifically, each $E_{p}$ is concatenated with the original
query and its entity type (e.g., ”SongName”, ”AuthorName”) to form the input
text ”Query [SEP] Entity [SEP] Type”. This input text will be encoded to an
embedding by the PLM. After that, $M$ first-layer cross-attentions modules are
applied to $M$ raw entity and their expansions. So that each first-layer
cross-attention module can learn the relationships between each raw entity and
its expansions. At last, each entity’s representation $rep_{p}$ (where $1\leq
p\leq m(k+1)$) from all first cross-attention modules are jointly input to the
second cross attention module. Therefore, the interactions between all
entities can be learned and the outputs $rep_{p}^{{}^{\prime}}$ (where$1\leq
p\leq m(k+1)$) will then be input to the final classification layer to predict
the importance weight logits $g_{p}$. Cross-entropy is used to calculate the
loss between each entity’s predicted logits $g_{p}$ and its true label
$l_{p}$, the final loss is defined as the average of these losses:

(1)
$\mathcal{L}_{final}=\frac{1}{m(k+1)}\sum_{p=1}^{m(k+1)}\text{CrossEntropy}(l_{p},g_{p})$

##  3\. Experiments and Evaluation

We extract 2000 hours of successfully processed Alexa data from an aggregated
and anonymized datalog (which has no user specific information), and use it to
build the EEKB and the retrieval index. An annotated query-reformulation
dataset is split into 60K/15k/27k, to be used for training, evaluation, and
testing, respectively.

The Robert-Base(Liu et al., 2019) model is utilized as the encoder and a
multi-head attention module is used as the cross-attention layer in Figure 3.
Hyper-parameters of the model can be found in Appendix A. P@K (Precision of
top-K retrieved candidates) is used as the evaluation metrics, and four
baselines are used for the comparison.
1) The ElasticSearch111Version 7.10 is used in this work, which is a popular
string match retrieval method based on the inverted indexing and BM25 (Jones,
1972);
2) The SimCSE (Gao et al., 2021), which is a neural network based model that
leverages the contrastive learning to build a representation encoder;
3) DocT5query (Nogueira et al., 2019), which learns to expand the document by
predicting its relevant query using the T5 model. The expanded document can be
used in a specific retrieval system, e.g. Elasticsearch.
4) DeepImpact (Mallia et al., 2021), which performs both document expansion
and semantic importance estimation using DocT5Query and a contrastive learning
based term prediction.

Method | P@50 | P@10 | P@1
---|---|---|---
DeepImapct | 63.8% | 32.4% | 16.3%
Elasticsearch | 85.4% | 67.6% | 20.6%
docT5query on Elasticsearch | 85.6% | 68.3% | 20.5%
QEEW on Elasticsearch | 88.2% | 73.6% | 25.1%
SimCSE | 72.7% | 64.0% | 33.7%
QEEW on SimCSE | 75.9% | 64.1% |  34.8%
Table 1. Main result table. QEEW on Elasticsearch is able to achieve significantly better performance on P@50 and P@10; QEEW on SimCSE achieves the best performane on P@1. Method | P@50 | P@10 | P@1
---|---|---|---
Elasticsearch | 85.4% | 67.6% | 20.6%
Elasticsearch with expansion | 86.3% | 68.5% | 21.1%
Elasticsearch with weight | 87.4% | 73.1% | 25.0%
QEEW on Elasticsearch | 88.2% | 73.6% | 25.1%
Table 2. The ablation study on Elasticsearch demonstrates the usefulness of
both expansion and weight prediction.

To evaluate the effectiveness of QEEW, we applied it to 1) string-match based
Elasticsearch and 2) the neural embedding based SimCSE. On Elasticsearch, we
first build the reformulation index from reformulation candidates set, and
append each query with its expanded entities to form a new query. The
predicted weights are applied to the retrieval score function so that the
higher weighted entities contribute to higher scores for ranking; On SimCSE,
the retrieval of ranking reformulations is based on the similarity score
between query and reformulation strings, and the relevance score is the
Euclidean Distance between the embeddings of query and reformulations. To
apply the QEEW, we also adjust the similarity scores of retrieval
reformulations based on the weights so that the reformulations that contain
higher weighted entities will be with higher ranking scores.

Table 1 shows that the proposed QEEW method can achieve the highest P@50 and
P@1 when applied to Elasticsearch, and achieves the best P@1 when applied to
the SimCSE. Specifically, QEEW can improve P@50, P@10, and P@1 by 2.8%, 6%,
and 4.5% respectively in Elasticsearch and by 3.2%, 0.1%, and 1.1%
respectively in SimCSE. Notice that the SimCSE achieves higher P@1 but lower
P@10/P@50 compared with Elasticsearch, this is because that the SimCSE is
trained with a semantic matching objective so that it performs better on
precise matching, while Elasticsearch enables a wider search and thus achieves
higher P@50 and P@10. When comparing with (3) DocT5Query and (4) DeepImapct,
our proposed method still achieves better performance. Specifically,
Elasticsearch on QEEW can improve 2.6%, 5.3%, and 4.6% on P@50, P@10, and P@1
respectively compared to docT5query on Elasticsearch; QEEW on SimCSE can
improve 12.2%, 31.7%, and 18.5% on P@50, P@10, and P@1 respectively compared
to DeepImapct. The reason why (3) and (4) cannot achieve as good performance
as our proposal is that these models are more suitable for longer paragraphs
but fail to perfectly expand to the short and ambiguous spoken queries.

Table 2 demonstrates the ablation study result of applying QEEW to
Elasticsearch. Elasticsearch with predicted expansions singly is able to gain
0.9%, 0.9%, 0.5% on P@50, P@10, and P@1; while with predicted weights singly
can improve 2%, 5.5%, 4.4% on P@50, P@10, and P@1. This result demonstrates
weighting can significantly improve the performance compared to query
expansion alone.

##  4\. Conclusions and Future Work

In this work, we propose a novel method named as QEEW for query expansion and
entity weighting predicion. Experiments on annotated Alexa data demonstrates
that 1) QEEW can increase retrieval precision when applying to both
conventional string-matching retrieval system (e.g., Elasticsearch) and novel
neural network retrieval system (e.g., SimCSE); 2) QEEW achieves better
performance compared with SOTA retrieval based QR systems (e.g., DocT5Query,
DeepImpact).

For the future work, we would like to investigate more effective ways to learn
entity extension/ weighting and how to leverage the EEKB and predicted entity
weights to improve other tasks such as recommendation and personalization.

## References

  * (1)
  * Bai et al. (2020) Yang Bai, Xiaoguang Li, Gang Wang, Chaoliang Zhang, Lifeng Shang, Jun Xu, Zhaowei Wang, Fangshan Wang, and Qun Liu. 2020.  SparTerm: Learning Term-based Sparse Representation for Fast Text Retrieval.  _arXiv preprint arXiv:2010.00768_ (2020).
  * Buck et al. (2017) Christian Buck, Jannis Bulian, Massimiliano Ciaramita, Wojciech Gajewski, Andrea Gesmundo, Neil Houlsby, and Wei Wang. 2017.  Ask the right questions: Active question reformulation with reinforcement learning.  _arXiv preprint arXiv:1705.07830_ (2017).
  * Cao et al. (2008) Huanhuan Cao, Daxin Jiang, Jian Pei, Qi He, Zhen Liao, Enhong Chen, and Hang Li. 2008.  Context-aware query suggestion by mining click-through and session data. In _Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining_. 875–883.
  * Cho et al. (2021) Eunah Cho, Ziyan Jiang, Jie Hao, Zheng Chen, Saurabh Gupta, Xing Fan, and Chenlei Guo. 2021.  Personalized Search-based Query Rewrite System for Conversational AI. In _Proceedings of the 3rd Workshop on Natural Language Processing for Conversational AI_. 179–188.
  * Cui et al. (2002) Hang Cui, Ji-Rong Wen, Jian-Yun Nie, and Wei-Ying Ma. 2002\.  Probabilistic query expansion using query logs. In _Proceedings of the 11th international conference on World Wide Web_. 325–332.
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_ (2018).
  * Fan et al. (2021) Xing Fan, Eunah Cho, Xiaojiang Huang, and Chenlei Guo. 2021\.  Search based Self-Learning Query Rewrite System in Conversational AI. In _2nd International Workshop on Data-Efficient Machine Learning (De-MaL)_.
  * Gao and Nie (2012) Jianfeng Gao and Jian-Yun Nie. 2012.  Towards concept-based translation models using search logs for query expansion. In _Proceedings of the 21st ACM international conference on Information and knowledge management_. 1–10.
  * Gao et al. (2021) Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021.  SimCSE: Simple Contrastive Learning of Sentence Embeddings.  _arXiv preprint arXiv:2104.08821_ (2021).
  * Gupta et al. (2021) Saurabh Gupta, Xing Fan, Derek Liu, Benjamin Yao, Yuan Ling, Kun Zhou, Tuan-Hung Pham, and Chenlei Guo. 2021\.  RoBERTaIQ: An Efficient Framework for Automatic Interaction Quality Estimation of Dialogue Systems. In _2nd International Workshop on Data-Efficient Machine Learning (DeMaL)_.
  * He et al. (2016) Yunlong He, Jiliang Tang, Hua Ouyang, Changsung Kang, Dawei Yin, and Yi Chang. 2016\.  Learning to rewrite queries. In _Proceedings of the 25th ACM International on Conference on Information and Knowledge Management_. 1443–1452.
  * Imani et al. (2019) Ayyoob Imani, Amir Vakili, Ali Montazer, and Azadeh Shakery. 2019.  Deep neural networks for query expansion using word embeddings. In _European Conference on Information Retrieval_. Springer, 203–210.
  * Jones (1972) Karen Sparck Jones. 1972\.  A statistical interpretation of term specificity and its application in retrieval.  _Journal of documentation_ (1972).
  * Lewis et al. (2019) Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. 2019\.  Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension.  _arXiv preprint arXiv:1910.13461_ (2019).
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.  _arXiv preprint arXiv:1907.11692_ (2019).
  * Mallia et al. (2021) Antonio Mallia, Omar Khattab, Torsten Suel, and Nicola Tonellotto. 2021.  Learning passage impacts for inverted indexes. In _Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval_. 1723–1727.
  * Mustar et al. (2020) Agnès Mustar, Sylvain Lamprier, and Benjamin Piwowarski. 2020\.  Using BERT and BART for Query Suggestion.. In _CIRCLE_.
  * Nogueira et al. (2019) Rodrigo Nogueira, Jimmy Lin, and AI Epistemic. 2019.  From doc2query to docTTTTTquery.  _Online preprint_ (2019).
  * Raffel et al. (2019) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2019.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _arXiv preprint arXiv:1910.10683_ (2019).
  * Roitman (2019) Haggai Roitman. 2019\.  Query Term Weighting based on Query Performance Prediction.  _arXiv preprint arXiv:1902.10371_ (2019).
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need. In _Advances in neural information processing systems_. 5998–6008.
  * Wang et al. (2021) Zhuoyi Wang, Saurabh Gupta, Jie Hao, Xing Fan, Dingcheng Li, Alexander Hanbo Li, and Chenlei Guo. 2021.  Contextual Rephrase Detection for Reducing Friction in Dialogue Systems. In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_. 1899–1905.
  * Xu and Croft (2017) Jinxi Xu and W Bruce Croft. 2017.  Quary expansion using local and global document analysis. In _Acm sigir forum_ , Vol. 51. ACM New York, NY, USA, 168–175.
  * Zamani et al. (2018) Hamed Zamani, Mostafa Dehghani, W Bruce Croft, Erik Learned-Miller, and Jaap Kamps. 2018\.  From neural re-ranking to neural ranking: Learning a sparse representation for inverted indexing. In _Proceedings of the 27th ACM international conference on information and knowledge management_. 497–506.
  * Zhang et al. (2017) Yuxiang Zhang, Yaocheng Chang, Xiaoqing Liu, Sujatha Das Gollapalli, Xiaoli Li, and Chunjing Xiao. 2017.  Mike: keyphrase extraction by integrating multidimensional information. In _Proceedings of the 2017 ACM on Conference on Information and Knowledge Management_. 1349–1358.

##  Appendix A Model Hyperparameter

For the weight predcition model in Figure 3, Robert-
Base222https://huggingface.co/roberta-base is used as the pre-trained language
model. Multi-head attention (Vaswani et al., 2017) is used as the cross-
Attention module, and we set $head=6$ and $dropout=0.3$ in this work. A single
linear layer is used as the classifier, with a $dropout=0.5$. Cross-Entropy is
used to calculate the loss between each predicted logits and true labels.
AdamW is used as the optimizer, with $learningrate=3e-5$ and $eps=1e-8$.
Training epoch is set as 20, and an early stopping mechanism is applied on the
validation step to select the best model.

##  Appendix B Evaluation Methods Details

Figure 4. Illustration of using ElasticSearch to measure the model’s
performance.

Figure 4 demonstrates how we use the ElasticSearch to evaluate the retrieval
performance. Each reformulation candidate’s body text is input to the
ElasticSearch to build the index; given an input query, it will be first
concatenated with the expansion to form a new ”query + expansion”, and this
”query + expansion” will be used to retrieval the relevant reformulation
candidates. The matching score is calculated based on BM25. After the
retrieval, each retrieved reformulation candidate’s matching score will be
adjusted according to the predicted weight from model 3: if the candidate
contains the entity that is predicted as ”2”, the candidate’s retrieval score
will be multiplied by a constant $\alpha$. We set $\alpha=1.5$ in this work.

The SimCSE method is a contrastive learning based method. A Roberta-base model
is trained using Alexa labelled data with the objective to minimize the
distance between positive query-reformulation pairs and maximize the distance
between negative pairs. After the training, the model can be used to encode
both the query and reformulation candidates to embeddings. We then used the
FAISS 333https://github.com/facebookresearch/faiss to retrieve the most
similar reformulation embeddings for each query embedding. Similarly, we
adjust each retrieval’s score by dividing the original matching distance by
$\alpha=1.2$ if this reformulation contains an important entity.


# Evolution of 3GPP Standards Towards True Extended Reality (XR) Support in 6G
Networks

Ali A. Esswie, and Morris Repeta
Standards & Industry Forums, Advanced Wireless Technology, Dell Technologies

###### Abstract

Extended reality (XR) is a key innovation of 5G-advanced and beyond networks.
The diverse XR use-cases, including virtual reality, augmented reality, and
mixed reality, transform the way humans interact with surrounding
environments. Thus, XR technology enables true immersive experiences of novel
services spanning, e.g., e-commerce, healthcare, and education, respectively.
However, the efficient support of XR services over existing and future
cellular systems is highly challenging and requires multiple radio design
improvements, due to the unique XR traffic and performance characteristics.
Thus, this article surveys the state-of-art 3GPP standardization activities
(release-18) for integrating the XR service class into the 5G-advanced
specifications, highlighting the major XR performance challenges. Furthermore,
the paper introduces valuable insights and research directions for supporting
true XR services over the next-generation 6G networks, where multiple novel
radio design mindsets and protocol enhancements are proposed and evaluated
using extensive system level simulations, including solutions for application-
native dynamic performance reporting, traffic-dependent control channel
design, collaborative device aggregation for XR capacity boosting and offload,
respectively.

Index Terms— Extended Reality (XR); 3GPP; 5G New Radio; 6G; Device
aggregation.

##  I Introduction

Extended reality (XR) is one of the most promising service classes of future
cellular systems. XR use-cases, including virtual reality (VR), augmented
reality (AR), mixed reality, complemented by advanced cloud gaming (CG), offer
immersive human experiences for collaborative gaming, telepresence and hybrid
workplaces [1]. However, the success of the mass XR adoption depends on the
efficient support of future cellular systems to the challenging XR quality of
service (QoS) and computing demands. Therefore, XR requires substantial
specification efforts from various standardization bodies, and specially, the
3rd generation partnership project (3GPP), due to its unique performance and
traffic characteristics [2].

Particularly, the XR QoS requirements and traffic characteristics do not fit
within the existing 5G-native QoS classes such as the ultra-reliable and low-
latency communications (URLLC) and enhanced mobile broadband (eMBB). That is,
XR applications demand an eMBB-alike extreme capacity with URLLC-alike tight
radio latency and reliability targets [3]. This makes fulfilling the XR radio
performance targets highly challenging and many of the existing radio
operations (e.g., radio resource scheduling and radio performance reporting)
become sub-optimal due to the native design of the 5G radio, which is
optimized towards URLLC-eMBB coexistence [4].

Accordingly, 3GPP partners have been extensively studying XR service and
traffic characteristics, and therefore, identifying respective radio
challenges and potential solutions for XR capacity and latency optimization.
Specifically, the standardization of XR via 5G has started in 2016, where the
3GPP service and system aspects (SA1) working group (WG) initiated the
specification of the 5G target service requirements for high-capacity and low-
latency applications [5]. Following SA conclusions, the radio access network
(RAN1) WG has completed a study item, during 5G release-17, on the traffic
characteristics of the various XR applications [6]. Later, during the current
5G release-18 (to be tentatively completed by 2023), the corresponding
normative work has started within both RAN1 and RAN2 WGs [7], in order to
specify the needed radio enhancements to support basic XR services, including
aspects for capacity enhancements, scheduling enhancements, and XR-specific
power saving [8-10], respectively.

In this paper, we survey the state-of-the-art 3GPP specification efforts for
XR service support via the 5G new radio. The major XR-specific performance
challenges over 5G are reported, and which are currently under 3GPP study and
improvement. This includes multiple radio aspects as XR radio resource
management, XR traffic Jitter adaptation, and the XR multi-flow multi-priority
traffic, respectively. In the second part of the paper, we propose and
evaluate multiple novel radio designs and solutions for a more efficient XR
support over 5G-adavnced and beyond radio interfaces, and which transform the
current 3GPP specification mindset. Those are, an application-dependent radio
performance management, traffic-aware control channel design, and
collaborative multi-device aggregation for XR capacity offload and boosting,
respectively. Finally, the paper offers insightful conclusions on how the 3GPP
standardization is changing its basic specification mindset to efficiently
support XR services over future cellular systems.

The paper is organized as follows. Section II presents the current XR
performance challenges, which are under ongoing 3GPP study during the active
release-18. Section III discusses the proposed solutions and corresponding
simulation results for true XR service support over future cellular networks.
Conclusions and recommendations are drawn in Section IV.

##  II XR Performance Challenges in 5G New Radio

XR applications demand a highly perceived data rate with strict packet delay
budget (PDB) and reliability levels, respectively, placing it between the eMBB
and URLLC service classes. Moreover, XR has unique traffic characteristics,
which are dependent on the specific XR use-case (where more than 20 XR use
cases are identified [11] by 3GPP SA WGs). The video streaming is the basic
traffic type of XR applications in the downlink direction (and in uplink
direction as well for AR applications). This leads to an enormous amount of XR
traffic, available for immediate scheduling, and which is associated with
stringent radio latency and reliability targets. Accordingly, the network
spectral efficiency is highly degraded due to the fundamental trade-off
between data rate, latency, and reliability. Subsequently, The vast XR traffic
characteristics and performance requirements make existing radio operations,
e.g., radio resource management, become sub-optimal to efficiently integrate
XR services as follows.

###  II-A Sub-Optimal XR Scheduling Procedures

Scheduling is a vital radio function where the network dynamically allocates
the available radio resources to different active devices, in order to meet
their different QoS metrics. There are generally two scheduling categories:
Dynamic and semi-static scheduling, respectively [12], as follows.

  * •

Dynamic scheduling: it implies the network dynamically schedules radio
resources to devices when either downlink or uplink traffic is available for
transmission. This requires multiple exchanges of control channel information,
for the network to be aware of how much uplink traffic is available at the
devices, and/or for indicating devices with the scheduled downlink/uplink
resources and associated transmission configurations, e.g., adopted modulation
and coding schemes, and the payload re-transmission modes.

  * •

Semi-static scheduling: namely as downlink semi-persistent scheduling or
uplink configured grant scheduling, it denotes that network pre-schedules a
certain amount of downlink or uplink resources for different devices, and
which are periodically repeated in the time domain over a configured
scheduling validity period. Accordingly, the straightforward advantage of
semi-static scheduling is avoiding the transmission of multiple control
channel information, which leaves more valuable radio resources for useful
payload transmissions.

Dynamic scheduling is best utilized when the traffic inter-packet arrival rate
and traffic volume are not consistent in time, i.e., time-varying or sporadic;
though, semi-static scheduling is appropriate for almost periodic traffic
arrivals with a pre-determined payload size. The XR traffic characteristics
have been deemed to fall in a middle class between these two traffic classes.
For instance, for virtual reality services with view-port dependent streaming,
traffic volume and inter-packet arrivals are highly time-variant due to the
dynamic selection and switching of the adopted video codecs, even for the same
downlink or uplink traffic flow. However, for augmented reality applications,
downlink and uplink traffic are deterministic. For instance, after each
received downlink payload, an uplink control update follows within a strict
latency budget.

Thus, with dynamic scheduling, XR traffic is independently scheduled in
downlink and uplink directions, with multiple control channel information
exchanges prior to the actual payload scheduling. This adapts the resource
scheduling to the frequent changes in the traffic arrival rate, periodicity
and volume; however, it leads to a significantly degraded channel capacity and
an increased packet buffering delay, which may violate the stringent XR
latency QoS. With semi-static scheduling, periodic resource grants are
allocated to XR devices to avoid the repeated control message exchanges.
However, due to the dynamic downlink Jitter, which is an XR-application and
XR-server specific, packet availability in the time domain may deviate from
the time instant the semi-statistically granted resources become active, and
thus, clearly degrading the network spectral efficiency [13].

###  II-B Application-Unaware Radio Performance Reporting

Since the era of the third generation (3G), the 3GPP mindset has dictated
specifying radio capabilities that meet the performance requirements of target
use cases and services, instead of specifying use-case-specific solutions and
procedures. Such mindset is making the efficient support of XR services quite
challenging. For instance, during an active session, a device needs to
continuously compile several radio measurement and performance reports, and
accordingly, send those back to the serving network node. Examples are (a) the
channel quality indication (CQI) reporting [14] for indicating the network
with the actual channel and interference conditions, (b) buffer status
reporting (BSR) [15], where it indicates the network about the size of the
buffered uplink traffic, available at the device, and accordingly, the network
can schedule the sufficient amount of uplink resources, and (c) hybrid
automatic repeat request (HARQ) feedback reporting [16], where it signals the
network whether the received downlink payload is successfully decoded or not.

Clearly, those performance reports are vital for the network to efficiently
schedule resources and to select the appropriate transmission configurations
for each traffic flow. However, particularly for XR services, and due to
application-unawareness of the performance reporting procedures, those schemes
may mislead the network and further influence a sub-optimal resource
scheduling as follows.

  * •

Application-outage-unaware HARQ feedback: HARQ reports indicate the network
whether formerly transmitted packets are successfully decoded or not. Upon
receiving a negative HARQ feedback, the network triggers a packet re-
transmission accordingly. For reducing the radio trip latency, HARQ packet re-
transmissions are always prioritized for transmission over new packet
arrivals. For a variety of XR applications, not all traffic flows are equally
important nor equally impacting end user experience, e.g., view-port dependent
XR streaming [6]. Hence, the network scheduler may be dictated by non-
important HARQ packet re-transmissions, which are always prioritized over
possibly more vital new payload arrivals.

  * •

Application-independent BSR reporting precision: BSR reporting is key for
devices to signal the network nodes of the size of their buffered uplink
traffic. Accordingly, network can sufficiently schedule the proper amount of
uplink resources. BSR reporting follows a predefined structure, where a BSR
table is specified such that devices select a row index from the BSR table
which corresponds to a range buffered traffic size, satisfying their actual
buffered traffic volume. The BSR quantization precision is predefined and
designed in such way to offer a higher quantization precision (with less
quantization errors) over the lower range of the traffic volume and a lower
precision over the higher range. The lower quantization precision denotes that
a single BSR index covers a large range of the buffered traffic volume. Hence,
the network shall not identify exactly how much traffic is buffered at
devices, leading to either over-scheduling or under-scheduling of resources
(and thus, a degraded spectral efficiency or increased traffic buffering
latency). However, such design fits perfectly within existing URLLC-eMBB
coexistence deployments. Specifically, the URLLC traffic is always small
sized, and therefore, utilizing the higher BSR quantization precision over the
lower traffic volume range, while eMBB applications have no stringent latency
targets, and thus, they can efficiently coexist with under-scheduling network
strategies, i.e., requiring multiple scheduling grants for the entire buffer
to be transmitted over a longer duration. For XR services, the traffic volume,
inter-packet arrivals, frame size, and radio latency targets (to be satisfied)
are highly time variant and dependent on the XR application itself. This leads
the current BSR reporting procedure to likely offer a higher BSR reporting
precision over a buffered traffic range that may not be experienced by the XR
application, leading to a poor uplink scheduling performance.

###  II-C Semi-Static Control Channel Design

Control channels are critical to achieve reliable 5G communications [17].
Exchange of control information, in both downlink and uplink directions, serve
multiple purposes including exchange of resource allocation information,
sharing of transmission configurations, and carrying performance related key
performance indicators (KPIs), respectively. Thus, without a timely decoding
of the control information, multiple radio procedures are significantly
disturbed, e.g., resource scheduling and dynamic link adaptation. Hence, due
to the vital impact of control channels on the achievable radio latency and
reliability, they are designed in a semi-static and extreme reliable way such
that its corresponding decoding probability is always maximized. That is,
control channels are statically modulated using a highly conservative
modulation scheme and with a strong channel coding rate, e.g., QPSK modulation
to increase the decoding ability, since without a successful decoding of
control channels, devices or the network may not even be aware of, for
instance, granted data resources or critical performance-related KPIs.

Due to the conservative modulation order and strong encoding, the capacity of
control channels is highly limited. Specifically, transmitting a small amount
of control information towards a single device takes over a large set of
valuable resources. With multiple simultaneously active devices, where each
must be assigned a dedicated control channel for continuous monitoring, the
control channel overhead can significantly consume the network capacity, i.e.,
leaving less resources for useful data transmissions. Accordingly,
optimization of control channel capacity is vital for enhancing the aggregate
cell spectral efficiency. With XR services, where an enormous amount of
traffic volume per device is available for transmission, resources consumed by
control channels must be highly limited such that to leave the remaining
resources for the data transmissions; however, due to the semi-static design
of the current 3GPP control channel, control channel capacity can not be
flexibly adapted without impacting either the total network spectral
efficiency (when increasing the control channel resources) or the XR critical
QoS targets (when reducing the amount of control channel resources, and
therefore, devices may exhibit delayed or even canceled control channel
transmissions, leading to a degraded latency performance). Therefore, the
current 3GPP control channel design mindset needs to be upgraded to introduce
capacity- and reliability-adaptive control channels, which are dependent on
the diverse importance of the XR traffic flows, for which the control
information is transmitted.

Therefore, those radio inefficiencies lead to a significantly degraded network
capacity. During the current 3GPP release-18 study on XR [3], the user
satisfaction ratio per cell is adopted as a key XR KPI, and is defined as the
number of XR users, compared to the total number of active XR devices, for
which their latency and capacity QoS targets are satisfied. As depicted by the
exemplary system-level evaluation in Fig. 1 [18], the XR user satisfaction
ratio is presented for three XR use-cases, namely AR, VR, and CG, with 30 and
45 Mbps as the target data rate. The deployment setup is: Dense Urban of 21
macro cells, each equipped with 64 Tx antennas, 100 MHz bandwidth per cell.
Obviously, the 95%ile of the satisfied XR user ratio is highly limited for the
three XR services, despite the large 100 MHz bandwidth allocation per cell.
Specifically, the 95%ile network capacity of VR/AR with a target rate of 30
Mbps is 4.2 users per cell while it drops to 1.6 users per cell with 45 Mbps.
Also, increasing the number of active XR devices clearly reduces the effective
satisfied user ratio due to the additional buffering delay.

Figure 1: User satisfaction ratio of AR, VR and CG. Figure 2: Proposed
application-driven performance reporting.

##  III Evolution of 3GPP Standards Towards 6G XR: Insights and Research
Directions

We address the XR-specific performance challenges, detailed in Section II, and
propose three novel radio design improvements, towards 5G-advanced and beyond
systems, to enable an efficient support of the stringent XR services.

###  III-A Application-Aware Performance Reporting

Application-awareness, augmented in cellular radio performance reporting, is
not aligned with current 3GPP specification mindset, where 3GPP bodies
typically seek to specify performance KPIs and reporting procedures that are
related to basic radio capabilities or services, rather than an application.
However, based on the performance findings of the 3GPP release-17 study on XR,
such mindset is deemed to critically limit the flexibility and efficiency of
cellular systems to support the emerging XR services, where various XR
applications adopt different traffic profiles, QoS demands, and device
behaviors, respectively. With existing application-agnostic performance
reporting, the network fails to perform proper resource scheduling and adopt
adequate transmission configurations for the active XR application.

Therefore, as depicted by Fig. 2, we propose introducing a radio application
programmable interface (R-API) for carrying XR application-specific
information and real-time KPIs. Therefore, the network becomes able to
dynamically request a certain active device to calculate and report a novel
application KPI. Examples can be diverse and not limited to XR services, e.g.,
average traffic volume, and real-time outage latency among each successive
uplink control/update transmission and downlink reception. Therefore, the
network can fine tune its scheduling policies and transmission configurations
specifically to the real-time needs of the stringent applications. Aside from
existing standardized KPI reporting, the introduced R-API procedure allows the
network and devices to dynamically coordinate on calculating and reporting new
and non-standardized performance and application KPIs, on-the-go, depending on
the processing ability of both entities.

As an exemplary implementation of the proposed R-API, an application-driven
BSR precision reporting is depicted in Fig. 3. Unlike the current fixed-
precision BSR reporting procedures (Detailed in Section II), proposed BSR
precision adaptation allows the network to position a more precise BSR
reporting over the real-time buffered traffic volume that is expected from the
end application, and hence, performing more resource- and latency-friendly
uplink scheduling. Specifically, based on the R-API concept, the network
dynamically requests a device to track, calculate and report the real-time
average and standard deviation of the XR application uplink traffic volume,
and hence, the BSR tables can adopt more precise BSR quantization steps, only
over the indicated traffic volume range, i.e., saving the unnecessary BSR
reporting overhead from the high precision BSR reporting over the non-probable
traffic range, while always satisfying high precision reporting over the
expected application traffic volume. This way, an application-specific BSR
reporting is utilized, instead of existing network-specific BSR standardized
reporting.

Figure 3: Dynamic precision BSR design.

To validate the performance of the proposed application-driven BSR design, we
develop extensive system level simulations, following the major 3GPP
simulation guidelines. Table I lists the major simulation settings. A 3GPP-
compliant 5G Macro deployment is simulated, with 21 cells, each is equipped
with 8 transmit antennas. There are 10 UEs on average per cell, each with 2
receive antennas. The FTP3 traffic model is adopted in both downlink and
uplink directions, respectively, with sporadic payload arrival rate (i.e.,
following a Poisson distribution) and equal offered traffic load, following
the latest 3GPP RAN1 methodology for XR evaluation [6]. The effective time-
variant packet arrival rate emulates both high-rate video and low-rate audio
XR streams. Herein, device mobility is only considered for channel fading
calculations; however, no inter-cell handover is utilized, i.e., handover cut-
off delays are ruled out in this evaluation. The uplink performance indicators
including the BSR and CQI indications are calculated and assumed available to
the cells for downlink and uplink scheduling.

Table I: Simulation parameters. Parameter | Value
---|---
Deployment | 3GPP Macro, 7 sites, 21 cells, Wraparound
Inter-site distance | 500 meters
Channel model | 3GPP-3D channel
Spectrum | 2.4 GHz
Bandwidth | 40 MHz
Antenna setup | Tx = 8, Rx = 2
Traffic model | FTP3, packet size = 1 Mbytes
UE dropping | Uniform, 10 UEs/cell
Link adaptation | Adaptive MCS
TTI | 7-OFDM symbols
Numerology | 30 KHz
UE speed | 3 Km/hr
DL and UL scheduler | Proportional fair
CQI mode | Sub-band of 8 PRBs
Duplexing mode | FDD
DL and UL receiver | Linear MMSE
Transmit power | BS: 43 dBm, UE: 23 dBm
HARQ mode | Async with Chase Combining

Fig. 4 depicts the empirical cumulative distribution function (ECDF) of the
achievable uplink throughput per UE, with three BSR reporting schemes: (1) the
standardized 3GPP 8-bit BSR reporting, (2) the proposed application-driven BSR
design, with 8-bits of reporting overhead, and (3) a reference BSR design,
with 10-bits of reporting overhead, respectively. The three BSR schemes cover
the same buffered traffic range, i.e., from 0 Bytes to 81.3 Mbytes; although,
with various quantization steps and precision. Specifically, the 8-bit 3GPP
reporting scheme follows a predefined BSR quantization, where the BSR
precision is higher over the lower traffic range and looser over the larger
traffic range, e.g., for a buffered traffic volume below 100 bytes, a single
BSR index indicates a range of traffic that is only couple of bytes wide;
however, for buffered traffic volume above 15 Mbytes, a single BSR index
indicates a range of traffic that is a couple of Mbytes wide. The 10-bit
reference BSR design implies a fixed and more precise BSR quantization over
the entire supported traffic volume as the 3GPP 8-bit BSR scheme, i.e.,
$BSR_{step}=(81.3*1000)/(2^{10})=79.3$ Kbytes over the entire BSR table.

Finally, the proposed BSR scheme adopts variable BSR quantization steps over
the different traffic volume ranges, where the steps are determined based on
the proposed R-API and the additional XR application-specific assistance
signaling of the expected average traffic volume. For the sake of evaluation,
we assume a generated application-specific average traffic volume of 10 Mbytes
per second (with a packet inter-arrival rate of 10 packets per second), and
accordingly, the BSR quantization precision/step is refined over the traffic
volume range of $10\pm\alpha$, where $\alpha=5$ Mbytes is a design parameter
and determines the dynamic traffic range of the refined BSR steps. Thus, the
BSR quantization precision is 3x refined over the determined range than the
entire BSR table, i.e., the BSR step size over the application-specific
traffic range (5 to 15 Mbytes) is 3x less than the step sizes adopted over the
remaining traffic range in the BSR table.

As can be observed from Fig. 4, the proposed BSR design outperforms the
current 3GPP BSR reporting, while exhibiting the same BSR reporting overhead
of 8-bits. That is, over the 90%ile level, a 66.6% increase in the uplink
throughput per UE is achieved with the proposed BSR reporting design, compared
to the 3GPP BSR design. This is attributed to the proposed BSR framework
refining the BSR quantization precision only over the dynamic traffic range of
the XR application, and thus, the accuracy of the uplink scheduling is
improved by avoiding the sub-optimal over-scheduling, leaving more resources
for other devices. The reference 10-bit BSR design offers the best achievable
throughput per device due to the always-refined BSR precision regardless of
the application traffic characteristics; however, at the expense of the
increased reporting overhead which limits the number of supported devices.

Figure 4: ECDF of the uplink throughput per UE (Mbps).

###  III-B Traffic-Aware Control Channel Design

The capacity of control channels is always deemed to be a critical limitation
of the stringent cellular services. This is due to several fundamental factors
as: (a) the semi-static and conservative transmission configurations of
control channels, for a higher control reliability, and (b) the mandatory
allocation of control channel search space for each active device, for
continuous monitoring and detection of new resource grants, respectively. With
extreme capacity demanding applications, such as XR, the offered spectral
efficiency of the network is highly degraded by the control channels, and
subsequently, the network is only able to support a small number of XR
devices.

Current control channel design mindset ignores the fact that not all XR
traffic is equally important, and hence, not equally impacting the end user
experience. For instance, for view-port dependent XR streaming, packets that
are contributing to the streaming portion in the pose directions are much
vital than others contributing to the streaming edges. Thus, those must be
reliably and rapidly transmitted and successfully decoded. However, with
existing specifications, the control information for such multi-priority
traffic flows are transmitted the same conservative way, leading to a larger
control channel resource overhead per device, and therefore, reducing the
total number of supported users due to resource starvation.

Figure 5: Traffic-aware control channel design.

Thus, as depicted by Fig. 5, a traffic-aware control channel design is
proposed for next generation cellular systems. The control channel per device
is divided into two stages. Stage-1 control channel is device-specific, and is
dynamically associated with the respective latency and reliability critical
traffic flows. Thus, stage-1 control channel information is conservatively
transmitted, i.e., transmitted with a highly conservative modulation and
coding scheme (MCS) [19]. However, stage-2 control channel is dynamically
configured to carry control information associated with best effort traffic
flows. Thus, this allows for: (1) adopting aggressive and capacity-friendly
MCS levels for transmitting the stage-2 control channel information, and (2)
multiplexing the control information of multiple devices on the same resource
sets (i.e., best effort devices and/or best effort traffic flow’s scheduling
information of critical devices), and thus, boosting the control channel
capacity. The degraded decoding reliability of stage-2 control channels (due
to aggressive MCS adoption and device multiplexing), carrying the control
information of best effort traffic flows or devices, is therefore dynamically
traded-off versus the enhanced control channel capacity, detection reliability
and latency of stringent traffic flows, respectively.

To assess the performance of the proposed multi-stage control channel design,
we adopt the same simulation setup, described in Section III.A, and simulation
parameters in Table I, where dropped UEs are randomly categorized into low and
high priority devices. Thus, Fig. 6 depicts the ECDF of the scheduled
transport block (TB) sizes in KBytes, averaged over downlink and uplink
directions for all scheduled UEs, with the existing 5G and proposed control
channel designs, respectively. The average scheduled transport block size is a
vital performance indicator of the control channel starvation since it
reflects the realistic scheduled traffic size, based on actual decoded control
channel, which are carrying the resource scheduling information. In case of a
control channel starvation, i.e., there are not resources available for
transmitting a new scheduling control channel towards a UE, the transmission
of the respective transport blocks will be delayed until sufficient control
channel sources are freed. As shown by Fig. 6, the proposed traffic-aware
control channel design obviously enables the control channel to simultaneously
schedule more UEs, and thus, a larger average transport block sizes, compared
to existing traffic-neutral control channel. The achievable capacity gain is
attributed to the low-priority inter-UE multiplexing of the proposed control
channel framework (i.e., scheduling more UEs), while high-priority UEs are
still offered dedicated control channels. It is also observed that the
achievable scheduling gain over the lower-percentiles, i.e., cell-edge
devices, is much less than that is of the higher-percentiles, i.e., cell-
center UEs. This is due to the degraded decoding reliability of the low-
priority control channels (which is particularity impacting the coverage-poor
cell-edge UEs), and hence, cell-edge UEs may miss detecting the scheduling
information and accordingly, demanding another control channel transmission,
which leads to delaying the transmission of the corresponding transport
blocks. This behavior can be resolved by enforcing cell-edge UEs to be always
part of the high-priority UE group, based on the received coverage level.

Figure 6: ECDF of the scheduled TB sizes (KBytes). Figure 7: Collaborative
device aggregation for dynamic XR capacity boosting and offload.

###  III-C Collaborative Device Aggregation For Capacity Boosting

The majority of XR services demands extreme perceived capacity, due to the
augmentation of the immersive video streaming in both downlink and uplink
directions, respectively. This calls for novel and collaborative capacity-
boosting procedures, specifically designed for XR traffic. Device aggregation
[20] is introduced to allow devices, in proximity of each other, to coordinate
and dynamically perform radio functions over proprietary and non-3GPP links,
e.g., WIFI, and WIGIG [21]. Therefore, device aggregation offers multiple
performance benefits including: (a) enhanced capacity in case the primary
device, aggregating traffic from in-proximity secondary devices, has better
radio conditions to the network than secondary devices, and (b) power/energy
saving of the secondary devices in case the execution of a subset of the radio
functions is delegated to a nearby primary device. Device aggregation is
typically transparent to the network, i.e., the network is not aware about the
formation of the device aggregation setup. However, such full network
transparency may limit the sustainability of the primary devices to perform
efficient traffic and function aggregation. For instance, with network-unaware
device aggregation, primary devices have to monitor and attempt decoding the
independent downlink control channels of each of the connected secondary
devices, which is battery consumption inefficient.

As can be clearly observed, device aggregation can be vital to XR adoption
success since the extreme demanded capacity can always be handled over higher
capacity links, e.g., high capacity 3GPP radio links to primary devices, and
extreme capacity non-3GPP links from primary to intended secondary devices.
However, this requires the efficient sustainability of the processing and
battery capacities of the primary devices, and therefore, partial network
awareness can greatly enhance the overall device aggregation performance.

As shown by Fig. 7, an exemplary network-aware device aggregation procedure is
introduced, for XR downlink traffic delegation and offload. The traffic-heavy
XR flows can be dynamically distributed across multiple direct and indirect
radio links towards intended secondary XR devices. Specifically, the critical
downlink traffic flows are transmitted towards the primary devices over the
3GPP radio interface, and accordingly, being forwarded towards secondary XR
devices over non-3GPP and extreme-capacity links. However, the best effort
downlink traffic flows can be transmitted directly towards the secondary
devices over the lower-capacity radio links. Obviously, the critical traffic
flows are always served on faster radio links; although, without stressing the
processing and energy capacities of the primary devices, due to only
delegating the critical flows to primary device’s links while other non-
significant flows are transmitted over the original radio links. This
certainly requires the partial network awareness about which downlink flows,
of the secondary XR devices, to transmit towards the primary devices instead,
utilizing the better channel conditions and transmission configurations, e.g.,
MCS, of the primary devices.

##  IV Concluding Remarks

In this work, the state-of-the-art 3GPP specifications, on emerging extended
reality (XR) service integration into the 5G new radio, are surveyed. The key
performance challenges are identified and presented, spanning the major 5G
radio aspects such as the radio resource scheduling, control channel design,
performance reporting and exchange procedures. For the efficient support of
the XR service class into 5G and beyond systems, several radio design
improvements have been proposed, and evaluated by extensive system level
simulations, which suggest upgrading the current 3GPP specification mindset,
through introducing XR programmable and application-dependent performance
reporting, traffic-aware control channel design, and collaborative device
aggregation, respectively. A detailed XR study will be performed in a future
work, considering other radio aspects of XR mobility, XR power saving, and XR
scheduling, respectively.

The key recommendations and insights of this paper are as follows: (1) The
emerging XR service class does not fit within existing 5G native quality of
service classes (QoS), e.g., URLLC and eMBB, due to the stringent and joint
performance requirements on reliability, latency, and capacity, respectively,
leading the efficient support of XR services to be further challenging, (2) XR
applications, even those belonging to the same XR service category (virtual
reality, augmented reality, and mixed reality) require a diverse set of QoS
targets, and utilize various traffic profiles. However, the real-time and
application-specific demands are not typically handled over the radio
interface, leading to highly inefficient radio operations, (3) Thus, current
3GPP specification mindset should be upgraded to introduce true application-
awareness mechanisms over the next-generation radio interfaces, and (4)
Finally, based on presented simulation results, solutions for programmable and
application-native performance reporting, traffic-aware control channel
designs, collaborative device and traffic aggregation become vital in
supporting the stringent QoS requirements of emerging XR services.

## References

  * [1] G. Minopoulos and K. E. Psannis, "Opportunities and challenges of tangible XR applications for 5G networks and beyond," IEEE Consumer Electronics Magazine, Mar. 2022.
  * [2] F. Hu et al., “Cellular-connected wireless VR: requirements, and solutions,” IEEE Commun. Mag., vol. 58, no. 5, May 2020, pp. 105-111.
  * [3] 3GPP; “Study on XR (Extended Reality) evaluations for NR,” TR 38.838, v.17.0.0, Jan. 2022.
  * [4] A. A. Esswie and K. I. Pedersen, "Opportunistic spatial preemptive scheduling for URLLC and eMBB coexistence in multi-user 5G networks," IEEE Access, vol. 6, pp. 38451-38463, July 2018.
  * [5] 3GPP; “Study on scenarios and requirements for next generation access technologies”, TR 38.913, v.16.0.0, July 2020.
  * [6] 3GPP; “Typical traffic characteristics of media services on 3GPP networks,” TR 26.925, v.17.0.0, Sep. 2021
  * [7] 3GPP; New SID for Release 18; “Study on XR enhancements for NR”, RP-213587, Dec. 2021.
  * [8] D. Li et.al., “Enhanced power saving schemes for extended reality,” in Proc. IEEE PIMRC, Sep. 2021, pp. 1-6.
  * [9] A. A. Esswie, "Power saving techniques in 3GPP 5G new radio: a comprehensive latency and reliability analysis," in Proc. IEEE WCNC, 2022, pp. 66-71.
  * [10] D. G. Morï¿œn, P. Pï¿œrez and A. G. Armada, "Toward the distributed implementation of immersive augmented reality architectures on 5G networks," IEEE Commun. Mag., vol. 60, no. 2, pp. 46-52, Feb. 2022.
  * [11] 3GPP; “Extended reality (XR) in 5G,” TR 26.928, v.16.1.0, Dec. 2020.
  * [12] A. A. Esswie and K. I. Pedersen, "On the ultra-reliable and low-latency communications in flexible TDD/FDD 5G networks," in Proc. IEEE CCNC, 2020, pp. 1-6.
  * [13] 3GPP; “CG scheduling enhancements for XR,” R2- 2211276, RAN2 #120, Dell Technologies, Nov. 2022.
  * [14] G. Pocovi, A. A. Esswie and K. I. Pedersen, "Channel quality feedback enhancements for accurate URLLC link adaptation in 5G systems," in Proc. IEEE VTC, 2020, pp. 1-6.
  * [15] 3GPP; “BSR feedback enhancements for XR,” R2- 2211275, RAN2 #120, Dell Technologies, Nov. 2022.
  * [16] J. Yeo, H. Ji, J. Bang, Y. Kim and J. Lee, "A novel group retransmission scheme for IIoT over 5G," in Proc IEEE Globecom, 2019, pp. 1-5.
  * [17] V. Braun, et al., "5G PHY downlink control channel: design, performance and enhancements," in Proc. IEEE WCNC, 2019, pp. 1-6.
  * [18] 3GPP; “Performance evaluation results for XR and cloud gaming,” R1-2110811, RAN1 #107-e , Huawei, HiSilicon, Nov. 2021.
  * [19] Y. Wang, W. Liu and L. Fang, "Adaptive modulation and coding technology in 5G system," in Proc. IEEE IWCMC, 2020, pp. 159-164.
  * [20] T. Salam, W. U. Rehman and X. Tao, "Cooperative data aggregation and dynamic Resource allocation for massive machine type communication," IEEE Access, vol. 6, pp. 4145-4158, 2018.
  * [21] C. J. Hansen, "WiGiG: multi-gigabit wireless communications in the 60 GHz band," IEEE Wireless Commun., vol. 18, no. 6, pp. 6-7, Dec. 2011.