# Large Language Model based Long-tail Query Rewriting in Taobao Search

Wenjun Peng<EMAIL_ADDRESS><EMAIL_ADDRESS>University of
Science and Technology of ChinaHefeiAnhuiCHN ,  Guiyang Li, Yue Jiang  ,
Zilong Wang  liguiyang.lgy, jy270069<EMAIL_ADDRESS>Taotian
GroupHangzhouZhejiangCHN ,  Dan Ou  ,  Xiaoyi Zeng  oudan.od,
<EMAIL_ADDRESS>Taotian GroupHangzhouZhejiangCHN ,  Derong Xu, Tong Xu† and
Enhong Chen  derongxu.mail, tongxu<EMAIL_ADDRESS>University of Science
and Technology of ChinaHefeiAnhuiCHN

(2023)

###### Abstract.

In the realm of e-commerce search, the significance of semantic matching
cannot be overstated, as it directly impacts both user experience and company
revenue. Along this line, query rewriting, serving as an important technique
to bridge the semantic gaps inherent in the semantic matching process, has
attached wide attention from the industry and academia. However, existing
query rewriting methods often struggle to effectively optimize long-tail
queries and alleviate the phenomenon of “” caused by semantic gap. In this
paper, we present , a comprehensive framework that Bridges the sEmantic gap
for long-tail QUEries. In detail, comprises three stages: multi-instruction
supervised fine tuning (SFT), offline feedback, and objective alignment. We
first construct a rewriting dataset based on rejection sampling and auxiliary
tasks mixing to fine-tune our large language model (LLM) in a supervised
fashion. Subsequently, with the well-trained LLM, we employ beam search to
generate multiple candidate rewrites, and feed them into Taobao offline system
to obtain the partial order. Leveraging the partial order of rewrites, we
introduce a contrastive learning method to highlight the distinctions between
rewrites, and align the model with the Taobao online objectives. Offline
experiments prove the effectiveness of our method in bridging semantic gap.
Online A/B tests reveal that our method can significantly boost gross
merchandise volume (GMV), number of transaction (#Trans) and unique visitor
(UV) for long-tail queries. has been deployed on Taobao, one of most popular
online shopping platforms in China, since October 2023.

Query reformulation, large language models, semantic matching

††copyright: acmcopyright††journalyear: 2023††doi:
XXXXXXX.XXXXXXX††conference: Make sure to enter the correct conference title
from your rights confirmation emai; May 13 - 17, 2024; Singapore††price:
15.00††isbn: 978-1-4503-XXXX-X/18/06††ccs: Information systems Query
reformulation††ccs: Computing methodologies Natural language processing Figure
1. Framework of Taobao search engine.

##  1\. Introduction

Past decades have witnessed the exceptionally rapid growth of e-commerce
platforms. Leading e-commerce companies, such as Taobao, JD and Amazon, have
amassed hundreds of millions of users, generating billions of gross
merchandise volume (GMV) annually. To facilitate the quick retrieval of
related products for these users, a well-established search paradigm has been
proposed, as illustrated in Figure 1, Specifically, this paradigm involves
several steps, i.e., “semantic understanding - retrieval - rank”. Among them,
_semantic understanding_ serves as the foundation of entire system, ensuring
accurate matching of user intent. However, due to the variations for how users
express their preferences for products, semantic gaps often exist between
their queries and the product keywords, even worse with long-tail queries
where retrieval system may fail to provide any relevant products. For
instance, a user with personal expression habits may input a long-tail query
like ”self-building blind box”, which will lead to more retrieval results if
with its synonymous query like ”DIY blind box”. Unfortunately, traditional
term-matching solutions like inverted index could probably fail to match the
commonly-used “DIY” with non-customary term “self-building”, which limit the
retrieval results and significantly impair the user experience. Therefore, it
is urgently required to solve the semantic gap challenge for long-tail
queries, and address the problem of “” in e-commerce platform.

Traditionally, prior arts (Huang et al., 2020; Li et al., 2021; Zhang et al.,
2020; Wang et al., 2023a) mainly focus on the “embedding based retrieval”
paradigm, which initially map the queries and products into a common semantic
space, and then support the Top-$K$ retrieval with approximate nearest
neighbor (ANN) methods. However, the retrieval outcomes might be difficult to
interpret, which severely limit the performance. To enhance the
controllability of retrieval outcomes, some efforts have been made on the
“query rewriting & exact match” paradigm. On the one hand, the _discriminative
methods_ (Zheng et al., 2020; Li et al., 2022) attempt to “rewrite” queries
via finding similar terms from a query reformulation set, and then utilize
them to search for relevant products using sparse retrieval. Although these
approaches could effectively expand the semantic of hot queries, long-tail
queries may not be adequately optimized, thus no related rewrite can be
generated. On the other hand, the _generative methods_ (Qiu et al., 2021; Wang
et al., 2021) involve supervised training on ¡query, rewrite¿ pair data to
empower the model with rewriting capabilities, and the alignment process
(Mohankumar et al., 2021; Agrawal et al., 2023) is further incorporated to
enhance the metric preference. Although these methods partially address the
semantic gap problem, they typically rely on small generative models with
limited comprehension of long-tail queries, which significantly constrained
the rewriting capability. Recently, with the development of LLM techniques,
some efforts (Wang et al., 2023c; Jagerman et al., 2023; Anand et al., 2023;
Wang et al., 2023b) solely utilize LLMs as retrieval data augmentation
generators without additional training to expand query semantics. However,
these methods, even with carefully curated prompts, may still constrain the
ability to specialize for query rewriting task, leading to the poor alignment
with objectives of e-commerce search.

To effectively bridge the semantic gap for long-tail queries, and solve the
above challenges via producing controlled and aligned outcomes with
integrating the knowledge of LLMs, we propose , a novel framework that
involves three stages of fine-tuning LLMs, namely _multi-instruction
supervised fine-tuning_ (SFT), _offline system feedback_ , and _objective
alignment_. Specifically, in the first stage, we utilize the rejection
sampling to collect ¡query, rewrite¿ pairs with desired quality distribution,
and then combine these pairs with data from quality classification, product
title prediction and chain of thought (CoT) tasks to construct the multi-
instruction rewriting dataset for fine-tuning our LLM. Next, with the well-
trained LLM, we employ beam search to generate multiple candidate rewrites for
each query. These candidate rewrites are fed into the Taobao offline system to
retrieve a collection of related products. We calculate the quality score of
retrieved products for the rewrites and use them as rewards for candidates
ranking. To calibrate generation probability of the candidate rewrites, we
introduce a Bradley-Terry based contrastive learning method that considers the
partial order among the these rewrites. Ultimately, the model training
objective is aligned with the online goal of the Taobao search, ensuring that
the generated rewrites yield the desired search results.

The main contributions of this work are listed as follows:

  * •

We have analyzed long-tail queries in e-commerce search and identified the
semantic gap problem associated with such queries. To the best of our
knowledge, we are the first to fine-tune LLMs for industrial query rewriting
task.

  * •

We propose a three-stage fine-tuned framework called to address the issue of
semantic gap in long-tail queries. This framework is designed to generate
rewrites that align with the objectives of Taobao search.

  * •

The effectiveness of our model is demonstrated through both offline and online
experiments, showcasing its ability to significantly improve e-commerce
revenue.

##  2\. Related Works

###  2.1. Query Rewriting

Query rewriting, also known as query expansion or query reformulation, plays a
pivotal role in e-commerce search technology and has a profound impact on the
user’s shopping experience and the revenue of e-commerce platforms. This
technique can be broadly categorized into discriminative and generative
methods.

Discriminative methods treat query rewriting as a retrieval process that
expand the semantics of the original query by selecting appropriate terms from
the candidate set. For example, pseudo-relevance (Cao et al., 2008; Tao and
Zhai, 2006; Xu and Croft, 2017) selects the top k documents from the initial
retrieval as semantic extensions. These approaches, however, often pose
challenges in effectively controlling the semantic scope and ensuring
retrieval relevance. To address these challenges, one potential solution is to
utilize a well-built thesaurus (Mandal et al., 2019; Bhogal et al., 2007) as a
candidate rewrite set. However, it is important to note that the effectiveness
of these methods highly depends on the quality of the thesaurus. Inadequate
quality may result in query semantic drift, where the intended meaning of the
query is compromised. Furthermore, alternative approaches (Cui et al., 2002;
Antonellis et al., 2008; Manchanda et al., 2019; Li et al., 2022) involve
generating candidate rewrites based on search logs, incorporating similar
terms from users’ search history as extensions. Unfortunately, due to the
Matthew effect, search logs naturally exhibit a bias towards popular queries,
resulting in that the training data collected through this approach may not
sufficiently meet the optimization needs for less frequently searched long-
tail queries.

Generative methods (Lee et al., 2018; Qiu et al., 2021; Song et al., 2021;
Wang et al., 2021) formulate the rewriting task as a generative process, where
a transformer-like model is utilized to generate candidate terms for the
original query. Some methods (Agrawal et al., 2023; Mohankumar et al., 2021)
incorporate reinforcement learning or contrastive learning to align with human
preferences or offline metrics. These methods have the ability to generate
related rewrites for each query, but usually employ a model with limited
number of parameters, which makes them less effective in processing long-tail
queries that require a deeper level of semantic understanding. Besides, their
generated rewrites are often inconsistent with the optimization goals of the
actual search engine. On the other hand, LLM-based rewriting methods (Wang et
al., 2023c; Jagerman et al., 2023; Anand et al., 2023; Wang et al., 2023b)
provide a deeper understanding and can generate appropriate expansion for
long-tail queries. Nevertheless they do not receive ad-hoc training on fine-
tuning and goal alignment, and lack of specialization and controllability to
the query rewriting task, which may potentially introduce illusions and noise
to the original query.

###  2.2. Preference Alignment

In recent years, with the increase in number of parameters, language models
(Ouyang et al., 2022; Touvron et al., 2023; Du et al., 2022; Yang et al.,
2023) have demonstrated incredible semantic understanding and zero-shot
capabilities on one hand. However, they have also faced the challenges of
model hallucinations and ethical issues (Ouyang et al., 2022; Zhang et al.,
2023). These models have the potential to fabricate facts with their extensive
background knowledge, resulting in misleading to users. To align the model’s
outputs with human morals and preferences, reinforcement learning (RL) has
been introduced to force the model to learn the partial order among different
outputs (Stiennon et al., 2020; Christiano et al., 2017). For instance, OpenAI
made a groundbreaking application (Ouyang et al., 2022) of RL to the training
process of large language models, specifically with ChatGPT, which received
tremendous attention. Besides, LLama2 (Touvron et al., 2023) designed a multi-
objective reward function that not only ensures the safety of model outputs
but also enhances their helpfulness. Unfortunately, these methods often rely
on complex training processes, and abundant high-quality data, making them
difficult to tune with vast number of hyperparameters. Therefore, rejection
sampling-like methods (Deng et al., 2020; Bai et al., 2022; Gulcehre et al.,
2023; Dong et al., 2023) have been proposed to continue training the model by
collecting outputs with high rewards from the previous rounds, aligning LLMs
with human preferences. Moreover, contrastive learning methods (Song et al.,
2023; Yuan et al., 2023; Rafailov et al., 2023) directly rank the outputs
based on their rewards and utilize ranking loss to adjust the output
probabilities, explicitly learning the partial order of outputs. Building upon
previous findings, we further propose three custom-designed Taobao metrics as
ranking rewards to calibrate generation probabilities of candidate rewrites,
forcing model to align with online objectives of Taobao search.

Figure 2. Framework of .

##  3\. Method

###  3.1. Framework Overview

Long-tail query rewriting aims to expand the original query semantics to
address the problem of semantic gap while ensuring relevance. To this end, as
shown in Figure 2, we propose a three-stage rewriting framework, which
consists of: multi-instructions supervised fine-tuning (SFT), offline feedback
and objective alignment. 1) First, with rejeciton sampling, we constructed a
multi-instructions SFT dataset based on online logs that focuses on rewriting
tasks, mixed with quality classification, query correction and chain of
thought (CoT) tasks to train rewriting-specific LLMs. 2) After that, we use
the well-trained LLM obtained in the first stage to generate multiple
candidate rewrites for each sampled query. In order to obtain the partial
order of these candidate rewrites, we construct a taobao offline system to
obtain search results for these rewrites. The quality scores of the search
results are used to rank the candidates. 3) Based on the partial order of
candidate rewrites, we calibrate the generation probability of these rewrites
using Bradley-Terry based contrastive learning to maximize the probability of
rewrites that can obtain the desired search results.

###  3.2. Multi-instruction SFT

Given that no publicly available LLMs are specifically designed for e-commerce
query rewriting, direct utilization of general LLMs to address the long-tail
query semantic gap issue is likely to introduce inaccuracies and noise.
Consequently, we have embarked on an approach wherein we gather various
rewriting-related tasks to fine tune LLMs, enhancing their ability to
comprehend and rewrite e-commerce queries effectively.

Query Rewriting Dataset: we initially source rewrites from Taobao previous-
generation rewriting policy. This process yields the initial rewriting
dataset. Specifically, when a user initiates a query $x$ in Taobao search, old
rewriting policy generates a list of rewrites queries
$Y=\\{y_{1},y_{2},\cdots,y_{n}\\}$. From this list, we select the top-ranked
$y_{1}$ as the gold standard candidate to construct our initial rewriting
dataset $\mathcal{D}$ with $N$ samples:

(1) $\mathcal{D}=\left\\{\left.\left(x^{i},y^{i}\right)\right|_{i=1}^{N}\text{
such that }x^{i}\sim p(x),y^{i}\sim\pi_{old}\left(y\mid
x=x^{i},\theta_{old}\right)\right\\},$

where $p(x)$ denotes the query distribution in Taobao search engine,
$\pi_{old}$ and $\theta_{old}$ is the previous-generation rewriting policy of
Taobao and it’s parameters.

It’s important to highlight that e-commerce query rewriting differs from other
text generation tasks. In this context, semantic similarity between query and
rewrite does not necessarily guarantee retrieval of similar sets of products.
What we aim to achieve is a high relevance between the products retrieved by
rewrite $y$ and the original query $x$. To attain this, we apply a relevance
filter to $\mathcal{D}$ through rejection sampling:

(2)
$\mathcal{D}_{r}=\left\\{\left.\left(x^{i},y^{i}\right)\right|_{i=1}^{N_{r}}\text{
such that }x^{i},y^{i}\in\mathcal{D},rele(x^{i},y^{i})>\tau^{rele}\right\\},$

where $rele(\cdot)$ and $\tau^{rele}$ denote the relevance method and its
threshold of ¡query, rewrite¿ pair. The detail of the function $rele(\cdot)$
is discussed in the Section 3.3.

Furthermore, Taobao’s previous-generation of rewriting models primarily lacks
optimization for long-tail queries. As we work on the new generation of
rewriting models, our goal is to maintain the relevance of retrievals while
expanding the semantics. This expansion is aimed at alleviating the issue of
long-tail queries leading to “” results. As a result, we utilize rejection
sampling once again to filter $\mathcal{D}_{r}$ by considering the retrieval
increment. Additionally, we include the most recent interacted product title
of $x$ as supplementary information to better address long-tail queries:

(3) $\displaystyle\mathcal{D}_{sft}$
$\displaystyle=\\{(concat(x^{i},\mathcal{E}_{x}),y^{i})|_{i=1}^{N_{sft}}$
$\displaystyle\text{ such that
}x^{i},y^{i}\in\mathcal{D}_{r},incr(x^{i},y^{i})>\tau^{incr}\\},$

where, $\mathcal{E}_{x}$ is the interacted product title list of query $x$,
$incr(\cdot)$ and $\tau^{incr}$ denote the increment method and its threshold
of query-rewrite pair, respectively. The detail of the function $incr(\cdot)$
is discussed in the Section 3.3.

Auxiliary Task Datasets: In order to further enhance LLMs’ ability of
comprehending long-tail queries, we have gathered three high-related task
datasets in the context of query rewriting. These tasks encompass quality
classification, product title prediction, and Chain-of-thought. 1) To tackle
the quality classification task, our approach began with the extraction of
query pairs from online logs. These query pairs were then subjected to human
annotation to determine if they met the data requirements specified for SFT.
2) For the product title prediction task, we chose the most recent interacted
product under the query as the reference, forming ¡query, product title¿
pairs. 3) As for the CoT task, we employed the original online queries to
construct prompts for human evaluators. It’s noteworthy that these evaluators
were not only tasked with providing query rewrites aimed at improving the
quality of query retrieval but were also expected to articulate their thought
processes, explaining the rationale behind their specific revisions. The
details prompt design for above auxiliary tasks are shown in Table 1. These
datasets were subsequently incorporated into the rewriting task to construct
the dataset for the SFT stage.

Supervised Fine Tuning: The process of generating text with a condition
language model can be viewed as a constrained auto-regressive sampling
strategy. Given a prompt $x$ and its gold standard $y$, The training objective
is to maximize the conditional probability $p(y|x)$. Considering our multi-
instruction SFT dataset and assuming that
$p(y|x)=\prod_{i=1}{p(y_{i}|y_{0:i-1},x)}$, the training objective of
rewriting model involves minimizing the negative log likelihood:

(4)
$\mathcal{L}_{\mathrm{SFT}}(\theta)=-\mathbb{E}_{(x,y)\sim\mathcal{D}_{msft}}\sum_{i=1}\log\pi\left(y_{i}\mid
y_{0:i-1},x;\theta\right),$

where $\mathcal{D}_{msft}$ denotes multi-instruction SFT data, which consists
of a mixture of query rewriting dataset $\mathcal{D}_{sft}$ and a variety of
auxiliary task datasets, $\pi(\cdot)$ and $\theta$ denote our query rewriting
model and its parameters. It is worth mentioning that LLMs typically have
fixed prefixes in the prompt $x$. To avoid introducing noise, we disregarded
the losses coressponding to $x$.

Table 1. Prompt examples of different instructions. Task | Prompt Example
---|---
Quality Classification |  Is this a good e-commerce query rewrite? Query: {query} Rewrite: {rewrite} System: {Yes or No}
Title Prediction |  Please generate product titles that match input query Query: {query} System: {Yes or No}
Chain of Thought |  Your task is to rewrite the input query into a query that makes it easier to search for related products, and you are required to give the tho- -ught process and then the query rewriting re- -sult. The thought process and the query rewr- -iting result are separated by a semicolon. Query: {query} System: {CoT}; {Rewrite}
|

###  3.3. Offline Feedback

Currently, most alignment methods (Gulcehre et al., 2023; Rafailov et al.,
2023; Mohankumar et al., 2021; Agrawal et al., 2023) rely on manual annotation
and training-based reward models. However, we argue that these approaches can
be easily influenced by the quality of annotations and the effectiveness of
the reward model training, which often leads to inaccurate reflection of
response scores and compromises the learning of the generation model. To
address this issue, we propose a feedback system based on the Taobao search
engine, which provides more accurate rewrite scores.

The main structure of the Taobao online service process is illustrated in
Figure 1. When a query is received by the Taobao search engine, it undergoes
preprocessing and is then passed to the query understanding. This module
comprehends the semantic meaning of the query and send it to the retrieval
module. The retrieval module retrieves a large number of candidate products
from a massive Taobao product dataset. After deduplication and filtering, the
candidate product sets from different retrieval paths are merged into an
unordered and non-repetitive product set. These products are presented to the
user through a complex ranking system.

Similarly, when our offline system receives a rewrite, it simulates the
process of the Taobao online service to retrieve the corresponding products
for the rewrite. Based on the product set, our system provides us with a
quality score. It is important to note that we mainly address the semantic gap
issue caused by long-tail queries in exact match. Therefore, our rewriting
module only operates on the inverted index matching of the retrieval module,
and the product set considered for rewrite retrieval is related only to the
inverted index path. We propose three scores to measure the quality of the
rewrite, namely relevance, increment, and hitrate.

As mentioned in Section 3.2 , even if the query and rewrite are semantically
similar, it does not guarantee that the retrieved product sets will also be
related. To prevent the model from retrieving products that are completely
different from the user’s original intent, we introduce the relevance score.
It is calculated as follows:

(5)
$rele(x,y)=\frac{\sum_{i}\mathbbm{1}_{f(x,z^{i}_{y})>\tau^{\prime}}}{|\mathcal{Z}_{y}|},z^{i}_{y}\in\mathcal{Z}_{y},$

where $\mathbbm{1}$ is the indicator function, $f(\cdot)$ is Taobao relevance
function which is design for evaluate the relevance between item title and
query text, $\tau^{\prime}$ denotes the semantic relevance threshold of query-
item pair, $\mathcal{Z}_{y}$ denote the offline retrieval product list of text
$y$. These notations represent the same meaning in all the following formulas.

Furthermore, we need to ensure that the rewrite can expand the semantic
meaning of the original query to some extent, avoiding the issue of “.”
Therefore, we introduce the increment score, calculated as follows:

(6)
$incr(x,y)=\frac{\sum_{i}\mathbbm{1}_{f(x,z_{xy}^{i})>\tau^{\prime}}}{\sum_{i}\mathbbm{1}_{f(x,z_{x^{\prime}}^{i})>\tau^{\prime}}},z_{xy}^{i}\in\mathcal{Z}_{e}\cap(\mathcal{Z}_{x}\cup\mathcal{Z}_{y}),z_{x^{\prime}}^{i}\in\mathcal{Z}_{e}\cap\mathcal{Z}_{x},$

where, $\mathcal{Z}_{x}$ denotes the offline retrieval product set of text
$x$, and $\mathcal{Z}_{e}$ is the excellent product set maintained by Taobao
search group.

Lastly, we define the hitrate, which measures to what extent the rewrite can
compensate the semantic gap of the original query. The calculation process is
as follows:

(7)
$hitrate(x,y)=\frac{|\mathcal{E}\cap(\mathcal{Z}_{x}\cup\mathcal{Z}_{y})|}{\sum_{i}\mathbbm{1}_{f(x,e^{i})>\tau^{\prime}}},e^{i}\in\mathcal{E},$

where $\mathcal{E}$ is the collection of products that users have transacted
outside of the search scenario, $\mathcal{Z}_{x}$ and $\mathcal{Z}_{y}$ denote
the offline retrieval product set of text $x$ and $y$. Assuming that a
product, which is semantically related to the current user’s query, is not
transacted in the search. This indicates that the original query fails to
retrieve the product. However, if the rewrite is able to return the product,
then it implies that the product exists in the
$\mathcal{Z}_{x}\cup\mathcal{Z}_{y}$. This proves that the rewrite can indeed
compensate for the semantic gap of the original query. As a result, the
hitrate can effectively reflect the model’s ability to compensate for this
semantic gap.

Overall, our proposed feedback system based on the Taobao search engine
provides more accurate rewrite scores by considering relevance, increment, and
hitrate. This helps improve the alignment process and ensures better learning
of the generation model.

###  3.4. Objective Alignment

To avoid introducing bias through the reward model, we introduce the
Preference Rank Optimization (PRO) (Song et al., 2023), based on the Bradley
Terry Model. This method aims to enforce the model to learn the rewrite
partial order provided by offline feedback. According to the Bradley Terry
Model, the probability of selecting an policy should be proportional to its
corresponding reward. Given the partial order: $y_{1}\succ y_{2}$, the
preference probability can be expressed as:

(8) $P_{BT}=\frac{\exp(r(y_{1},x))}{\exp(r(y_{1},x))+\exp(r(y_{2},x))},$

where $r(\cdot)$ is the reward function, which is defined as the normalized
log probability of the rewrite generated by the LLM in PRO.

PRO expands this pairwise partial order to a more general list-wise partial
order. Additionally, a temperature is introduced in order to capture the
significance of the ranking based on the reward. PRO loss is represented by
the following equation:

(9)
$\mathcal{L}_{PRO}(\theta)=-\mathbb{E}_{(\boldsymbol{x},\boldsymbol{y})\sim\mathcal{D}_{PRO}}\sum_{k=1}^{n-1}\log\frac{\exp(\frac{\pi_{PRO}(y_{k}|x;\theta)}{\mathcal{T}^{k}_{k}})}{\sum_{i=k}^{n}\exp(\frac{\pi_{PRO}(y_{i}|x;\theta)}{\mathcal{T}^{i}_{k}})},$

where $\mathcal{T}^{i}_{k}=1/(r(y_{k})-r(y_{i}))$ and
$\mathcal{T}^{k}_{k}=\min_{i>k}(\mathcal{T}^{i}_{k})$for ranking difference,
$\mathcal{D}_{PRO}$ denotes the dataset for alignment, $\pi_{PRO}(\cdot)$ and
$\theta$ denote the policy model and its parameters, $n$ denotes the number of
candidate rewrites. Additionally, a SFT loss is added on top of the PRO loss
with weight $\lambda$ to maintain the ability of the model to generate normal
outputs:

(10) $\mathcal{L}_{ALIGN}=\mathcal{L}_{SFT}+\lambda\mathcal{L}_{PRO}.$

###  3.5. Online Serving

Due to the constraints imposed by numerous parameters and the autoregressive
prediction mode of large language models, it is almost impractical to deploy
online and meet the latency requirements of the Taobao search system. To
address this issue, we utilize offline to perform rewriting inference on torso
and tail queries, which are defined as queries with retrieval results
containing less than 70% related products or less than ten thousand products.
The rewrites generated from these queries are stored in an online key-value
graph, enabling quick online response. When a long-tail query matches the key-
value graph, the search engine retrieves the corresponding rewrite.
Subsequently, both the query and rewrite are tokenized into terms and used as
keywords for inverted index matching to obtain a set of related products. The
union of the query and rewrite retrieval sets forms the final candidate set of
products for the ranking system. The offline inference of covers 27% of the
page views (PV) in Taobao’s main search and has a minimal impact on the
latency of the online retrieval system.

##  4\. Experiments

###  4.1. Datasets

Training Dataset: For the SFT training process, we extracted approximately 20
million records from the online rewriting logs on Taobao prior to September
2023 as the initial training data. To enhance the quality of the training
data, we conducted two rounds of rejection sampling to obtain the query
rewriting dataset, which consists of 419,806 pairs of ¡query, rewrite¿.
Additionally, we included 155,662 manually rewriting data in the dataset to
ensure that the SFT model’s rewriting adheres to human preferences. Finally,
we combined 50,000 samples each from quality classification, product title
prediction and CoT task with the query rewriting dataset to construct the
Multi-instruction SFT dataset.

For the objective alignment training process, we randomly selected 10,000
queries from the query rewriting dataset to generate candidates. For each
query, we used the SFT Model to generate 5 candidate queries. These 50,000
rewrites were then scored using the Taobao offline system. After removing any
outliers, the alignment training dataset consisted of a total of 45,350
candidate rewrites.

Test Dataset: For the offline test set, we selected 14,981 queries from Taobao
search logs to evaluate the model’s performance. Among them, 50% of the
queries were randomly sampled proportionally to the actual queries.
Furthermore, to assess the model’s capability to rewrite long-tail queries, we
sampled the remaining 50% of the data from the long-tail queries.

###  4.2. Evaluation

Offline Metrics: We employ relevance (rele), increment (incr) and hitrate as
offline metrics to valid the effectiveness of our method. These metrics are
defined in Section 3.3.

Online Metrics: We introduce three key online metrics: pay amount, number of
transactions, and number of unique IP of visitors, to evaluate the model’s
online performance. These metrics are represented in abbreviated form as
follows:

(11) $\displaystyle GMV$ $\displaystyle:=\\#\text{pay amount},$
$\displaystyle\\#Trans$ $\displaystyle:=\\#\text{transaction},$ $\displaystyle
UV$ $\displaystyle:=\\#\text{unique IP of visitors.}$

###  4.3. Implementation Details

Unless stated otherwise, the optimizer used for model training is AdamW.
During the multi-instruction SFT stage, the model is trained for one epoch
with the learning rate set to 1e-5. In the objective alignment stage, the
model is trained for four epochs with the learning rate set to 1e-6.
Additionally, the maximum length of the prompt for the rewriting task is set
to 64, while for the rest of the tasks it is set to 256.

###  4.4. Offline Experiments

####  4.4.1. Results of Different LLMs

Table 2. Comparison of different LLMs trained on multi-instruction SFT dataset. Method | #params |  rele (%) |  incr (%) |  hitrate (%)
---|---|---|---|---
ChatGLM | 6b | 63.5 | 143.4 | 15.88
ChatGLM2.0 | 6b | 63.2 | 105.0 | 14.39
Baichuan | 7b | 66.0 | 114.3 | 15.70
Qwen | 7b | 62.6 | 133.2 | 15.63
|  |  |  |

In this section, we present the comparison results of different LLMs as base
models trained on the multi-instruction SFT dataset, as shown in Table 2. The
base models include ChatGLM (Du et al., 2022; Zeng et al., 2022), ChatGLM2.0
(Du et al., 2022; Zeng et al., 2022), Baichuan (Inc., 2023), and Qwen (Team,
2023). The variability in the advantages of these base models can be
attributed to disparities in their pre-training data and model architectures.
Notably, ChatGLM exhibits superior performance in both increment and hitrate
compared to the other three models. However, due to copyright constraints, we
have to utilize Qwen as the base model to fulfill our business objectives.
Furthermore, Qwen’s pre-training incorporates e-commerce data, augmenting its
comprehension of the e-commerce domain. This indicates a higher potential for
enhancing Qwen’s ability to rewrite long-tail queries.

####  4.4.2. Ablation Study of Multi-instruction SFT data

To investigate the impact of the multi-instruction SFT dataset on the
rewriting ability of LLMs, an ablation study was conducted in this section.
First, we randomly extracted query pairs from online logs, with a size similar
to that of the multi-instruction SFT dataset, which were not subjected to
rejection sampling and were not mixed with other task data. Then, we used the
data to train the model and obtained Qwen w/o MI. As depicted in Table 3, it
is evident that the multi-instruction SFT dataset significantly enhances the
model’s performance in terms of relevance, increment, and hitrate. This
improvement can be attributed to the substantial enhancement in the quality of
query pairs after rejection sampling. Furthermore, auxiliary tasks closely
associated with query rewriting, such as quality classification, product title
prediction, CoT, etc., further augment the model’s comprehension of query
semantics, subsequently elevating the quality of the produced rewriting
results.

Table 3. Comparison of Qwen with/without auxiliary tasks. MI means multi-instruction Method |  rele (%) |  incr (%) |  hitrate (%)
---|---|---|---
Qwen w/o MI | 61.4 | 109.6 | 14.58
Qwen | 62.6 | 133.2 | 15.63
|  |  |

####  4.4.3. Results of Different Contrast Number

In Table 4, we present the impact of varying contrastive numbers of candidates
on model performance during the objective alignment stage. It can be observed
that, as the number of candidates increases, the relevance shows a consistent
improvement, while the increment and hitrate decrease. This outcome can be
attributed to our modified PRO, where all candidates are treated as gold
standard and SFT Loss is calculated for each of them. Consequently, a larger
candidate pool implies an increase in the SFT loss weights and a decrease in
the partial order learning weights, improving relevance of generated rewrites.
Furthermore, there exists a trade-off between relevance and increment, where
an increase in one metric necessitates sacrificing the other, leading to a
negative impact on the increment metric. In addition, it is important to
mention that hitrate can be regarded as the increment with weak relevance
constraint, which explains its decrease in this context. Considering the
balance of the three metrics, we select the model with contrast number of 4 to
be best checkpoint for overall comparison.

Table 4. Comparison of with different number of contrast candidates. Number |  rele (%) |  incr (%) |  hitrate (%)
---|---|---|---
2 | 53.3 | 215.6 | 17.66
3 | 56.6 | 205.4 | 17.36
4 | 57.7 | 198.7 | 17.27
5 | 58.8 | 190.3 | 17.21
|  |  |
Table 5. Overall performance of with multiple baselines. The best results are
in bold, and the second-best results are underlined. “Top Queries” are defined
as queries with retrieval results containing more than 70% related products;
“Torso Queries” are defined as queries with retrieval results containing
10%-70% related products; “Tail Queries” are defined as queries with retrieval
results containing less than 10% related products. All the metrics are the
larger the better.

Method | Top Queries | Torso Queries | Tail Queries | All Queries
---|---|---|---|---
rele | incr | hitrate | rele | incr | hitrate | rele | incr | hitrate | rele | incr | hitrate
CLE-QR | 73.4 | 90.0 | 13.16 | 24.7 | 36.4 | 15.77 | 10.0 | 17.2 | 12.36 | 69.6 | 90.0 | 12.95
BART | 67.5 | 106.0 | 13.26 | 27.9 | 130.0 | 17.70 | 8.4 | 27.5 | 13.59 | 62.2 | 100.0 | 13.56
Q2D (ChatGPT) | 71.7 | 47.3 | 12.96 | 35.3 | 66.7 | 16.86 | 12.8 | 15.9 | 17.21 | 66.7 | 45.5 | 14.73
Qwen (SFT) | 67.1 | 117.4 | 14.18 | 25.5 | 56.4 | 17.49 | 7.2 | 34.9 | 14.91 | 61.4 | 109.6 | 14.58
RL (rele) | 74.7 | 48.6 | 12.66 | 39.4 | 2.1 | 15.60 | 14.5 | 8.1 | 11.36 | 70.0 | 45.1 | 12.42
RL (incr) | 46.9 | 76.1 | 13.33 | 17.7 | 134.1 | 18.00 | 4.4 | 60.9 | 15.20 | 42.5 | 75.9 | 14.22
RL (hitrate) | 56.0 | 176.8 | 15.10 | 6.2 | 117.3 | 20.16 | 2.2 | 51.0 | 18.22 | 49.6 | 162.8 | 15.75
(rele) | 69.3 | 174.5 | 15.04 | 27.3 | 160.9 | 19.29 | 4.1 | 66.4 | 18.27 | 62.3 | 164.0 | 16.43
(incr) | 64.6 | 212.3 | 15.45 | 20.5 | 207.7 | 21.45 | 2.4 | 71.7 | 19.62 | 57.7 | 198.7 | 17.27
(hitrate) | 69.3 | 177.0 | 15.39 | 18.4 | 204.0 | 19.78 | 5.0 | 62.5 | 18.22 | 62.1 | 167.0 | 16.64
|  |  |  |  |  |  |  |  |  |  |  |

####  4.4.4. Main Results

We compare with multiple baselines, including CLE-QR, query2doc (Q2D), BART,
Qwen, and RL-based LLM. CLE-QR (Li et al., 2022) is the previous-generation
query rewriter of Taobao search that generates semantic representations and
retrieve related rewrites for each query based on contrastive learning. BART
(Lewis et al., 2020) is a powerful pre-trained generation model based on the
encoder-decoder structure. We fine-tune it with query pairs from online logs
to enhance its ability to rewrite e-commerce queries. Qwen (Team, 2023) is a
large-scale language model based on the decoder-only structure that contains 7
Billion parameters. Similarly, we fine-tune it with query pairs from online
logs to enhance its ability to rewrite e-commerce queries. Furthermore,
following the settings of (Agrawal et al., 2023), we introduce an RL-based LLM
and utilize relevance, increment, and hitrate as rewards to encourage the RL
model to align with the Taobao offline metrics, respectively. From analyzing
the data presented in Table 5, the following conclusions can be drawn:

Generative models outperform discriminative models when rewriting “Torso” and
“Tail” queries. For instance, when considering CLE-QR and BART, both models
exhibit similar performance on “Top Queries” across three metrics. However,
BART significantly outperforms CLE-QR in terms of hitrate and increment on
“Torso Queries” and “Tail Queries” while maintaining relevance. This
discrepancy arises because discriminative models like CLE-QR rely on existing
queries in the search system as rewrite candidates, which are often biased
towards top queries. As a result, torso and tail queries, which lack
semantically similar top rewrites, do not receive related search candidates
from CLE-QR. In contrast, BART’s rewriting process is not restricted by the
semantic scope of online queries, enabling it to generate rewrites that are
not present in the search system’s history. This allows BART to overcome the
limitations of discriminative models and optimize torso and tail query
rewriting problem.

LLMs exhibit superior long-tail semantic understanding capabilities compared
to small models.  Qwen and BART serve as examples, where Qwen, with its
extensive parameter size, demonstrates stronger semantic expansion than BART
in terms of hitrate and increment of “All Queries”. Analyzing individual query
slices, Qwen’s improvement in hitrate and increment primarily occurs in the
“Tail Queries”, further validating the suitability of LLMs for long-tail query
rewriting tasks.

Retrieval augmentation methods demonstrate limited semantic expansion
capabilities. Comparing Q2D (ChatGPT) and , Q2D (ChatGPT) maintains good
retrieval relevance across all query slices but lacks sufficient semantic
expansion capabilities, resulting in subpar increment and hitrate performance.
Conversely, our , which is specifically optimized for semantic expansion in
rewriting, significantly enhances these two metrics.

The reinforcement learning (RL) may introduce bias and impact the
effectiveness of the rewriting LLMs. Examining RL and , RL process introduces
a reward model to guide the base model’s training. However, calculating the
reward requires offline search system simulation, and the reward model may not
accurately capture the search system’s features, leading to reduced
performance of RL models. In contrast, our employs contrastive learning to
explicitly learn the partial order of candidates, circumventing potential bias
caused by the reward model. Ultimately, while minimizing the adverse impact on
retrieval relevance, substantially improves the model’s increment and hitrate.

Different offline metrics work differently as rewards. For instance, when the
framework prioritizes relevance as its training objective, it demonstrates a
more cautious approach to bridging the semantic gap. The improvements in both
increment and hitrate tend to be challenging to achieve in this context.
However, when the primary objective shifts to maximizing increment, the model
demonstrates a significant capacity to enhance both the increment and hitrate
of retrieval, effectively addressing the issue of “”. In such cases, a
marginal decrease in relevance becomes an acceptable trade-off. When hitrate
becomes the target, the model also can effectively enhance both the increment
and hitrate. Nevertheless, owing to the intricacies of the hitrate computation
process, the model encounters difficulties in capturing the partial order
among candidates. Consequently, the model’s ability to expand semantic is
diminished in comparison to the that focuses on increment.

###  4.5. Online Experiments

Table 6. Online A/B test of on Mobile Taobao Search. “all queries”: every query in the test bucket counts, regardless of whether it has been rewritten or not. “covered queries”: only rewritten queries count. “long-tail queries”: rewritten and long-tail queries count. “queries”: rewritten and queries count. Online Traffic | GMV | #Trans | UV
---|---|---|---
all queries | +0.40% | +0.34% | +0.33%
covered queries | +2.96% | +1.36% | +1.22%
long-tail queries | +1.57% | +2.52% | +2.32%
“” queries | +18.66% | +5.90% | +6.25%
|  |  |

To assess the actual online performance of , we deployed it on Taobao search
for a 14-day online test, during which we recorded the three key metrics in
the Taobao search scene: GMV, #Trans, and UV. Table 6 reveals that surpassed
the previous-generation rewriting model CLE-QR by 0.4%, 0.34%, and 0.33% in
terms of GMV, #Trans, and UV, respectively. This implies that contributes
millions of GMV to Taobao search. It’s important to note that the overall
performance mentioned here refers to all queries in the testing buckets. Since
we inference offline, there are about 70% of online queries that do not hit
our rewriting table. Even in these cases, our model still delivers remarkable
enhancements. Additionally, for the queries covered (rewritten) by
(approximately 27% of total PV), there were noteworthy increases of 2.96%,
1.36%, and 1.22% in GMV, #Trans, and UV, respectively. These findings indicate
that effectively rewrites queries and addresses potential semantic gaps in the
semantic matching process. Moreover, significantly improved online #Trans and
UV for long-tail queries and “” queries, although we disregarded the GMV
fluctuation for this subset due to its low proportion. This improvement can be
attributed to our specialized optimization for long-tail queries. During the
first-stage SFT of , rejection sampling and auxiliary task data enhanced the
model’s performance in terms of retrieval increment and relevance, and also
deepened its understanding of long-tail queries. The alignment process in the
second and third stages effectively compelled the model to align with online
objectives of Taobao search.

##  5\. Conclusion

In this paper, we introduce , a framework specifically designed for e-commerce
query rewriting. The main objective of is to address the semantic gap that
occurs during the semantic matching process, particularly for long-tail
queries. Initially, we improve the quality of the rewriting dataset by
employing rejection sampling and auxiliary task mixing. We then train a LLM
using this refined dataset, which enhances the model’s query understanding and
enables effective rewriting of e-commerce queries. Utilizing the well-trained
LLM, we generate multiple candidate rewrites for each sampled query. To
establish a partial order among these candidates, we create an offline
feedback system based on online Taobao search. This feedback system accurately
evaluates the retrieval quality of the candidate rewrites from various
perspectives, such as relevance, increment, and hitrate. Finally, by
incorporating the partial order of rewriting retrieval quality, we introduce
PRO, which aligns the model’s objectives with those of Taobao. This ensures
that our approach generates rewriting results that yield high-quality
retrieval outcomes. Through multiple experiments, we have demonstrated the
effectiveness of our approach in improving offline metrics. Additionally,
online A/B experiments have substantiated a significant increase in Taobao
Search’s GMV, #Trans, and UV, particularly for long-tail queries.

## References

  * (1)
  * Agrawal et al. (2023) Sanjay Agrawal, Srujana Merugu, and Vivek Sembium. 2023\.  Enhancing e-commerce product search through reinforcement learning-powered query reformulation. In _CIKM_.
  * Anand et al. (2023) Avishek Anand, Abhijit Anand, Vinay Setty, et al. 2023\.  Query Understanding in the Age of Large Language Models.  _arXiv preprint arXiv:2306.16004_ (2023).
  * Antonellis et al. (2008) Ioannis Antonellis, Hector Garcia-Molina, and Chi-Chao Chang. 2008\.  Simrank++ query rewriting through link analysis of the clickgraph (poster). In _TheWebConf_. 1177–1178.
  * Bai et al. (2022) Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. 2022\.  Constitutional ai: Harmlessness from ai feedback.  _arXiv preprint arXiv:2212.08073_ (2022).
  * Bhogal et al. (2007) Jagdev Bhogal, Andrew MacFarlane, and Peter Smith. 2007\.  A review of ontology based query expansion.  _Information processing & management_ 43, 4 (2007), 866–886.
  * Cao et al. (2008) Guihong Cao, Jian-Yun Nie, Jianfeng Gao, and Stephen Robertson. 2008.  Selecting good expansion terms for pseudo-relevance feedback. In _SIGIR_. 243–250.
  * Christiano et al. (2017) Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017\.  Deep reinforcement learning from human preferences.  _NIPS_ 30 (2017).
  * Cui et al. (2002) Hang Cui, Ji-Rong Wen, Jian-Yun Nie, and Wei-Ying Ma. 2002\.  Probabilistic query expansion using query logs. In _TheWebConf_. 325–332.
  * Deng et al. (2020) Yuntian Deng, Anton Bakhtin, Myle Ott, Arthur Szlam, and Marc’Aurelio Ranzato. 2020.  Residual energy-based models for text generation.  _arXiv preprint arXiv:2004.11714_ (2020).
  * Dong et al. (2023) Hanze Dong, Wei Xiong, Deepanshu Goyal, Rui Pan, Shizhe Diao, Jipeng Zhang, Kashun Shum, and Tong Zhang. 2023\.  Raft: Reward ranked finetuning for generative foundation model alignment.  _arXiv preprint arXiv:2304.06767_ (2023).
  * Du et al. (2022) Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, and Jie Tang. 2022.  GLM: General Language Model Pretraining with Autoregressive Blank Infilling. In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_. 320–335.
  * Gulcehre et al. (2023) Caglar Gulcehre, Tom Le Paine, Srivatsan Srinivasan, Ksenia Konyushkova, Lotte Weerts, Abhishek Sharma, Aditya Siddhant, Alex Ahern, Miaosen Wang, Chenjie Gu, et al. 2023\.  Reinforced self-training (rest) for language modeling.  _arXiv preprint arXiv:2308.08998_ (2023).
  * Huang et al. (2020) Jui-Ting Huang, Ashish Sharma, Shuying Sun, Li Xia, David Zhang, Philip Pronin, Janani Padmanabhan, Giuseppe Ottaviano, and Linjun Yang. 2020.  Embedding-based retrieval in facebook search. In _SIGKDD_. 2553–2561.
  * Inc. (2023) Baichuan Inc. 2023\.  A large-scale 7b pretraining language model developed by baichuan-inc.  https://github.com/baichuan-inc/Baichuan-7B
  * Jagerman et al. (2023) Rolf Jagerman, Honglei Zhuang, Zhen Qin, Xuanhui Wang, and Michael Bendersky. 2023.  Query Expansion by Prompting Large Language Models.  _arXiv preprint arXiv:2305.03653_ (2023).
  * Lee et al. (2018) Mu-Chu Lee, Bin Gao, and Ruofei Zhang. 2018.  Rare query expansion through generative adversarial networks in search advertising. In _SIGKDD_. 500–508.
  * Lewis et al. (2020) Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020.  BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension. In _ACL_. 7871–7880.
  * Li et al. (2022) Sen Li, Fuyu Lv, Taiwei Jin, Guiyang Li, Yukun Zheng, Tao Zhuang, Qingwen Liu, Xiaoyi Zeng, James Kwok, and Qianli Ma. 2022\.  Query Rewriting in TaoBao Search. In _CIKM_. 3262–3271.
  * Li et al. (2021) Sen Li, Fuyu Lv, Taiwei Jin, Guli Lin, Keping Yang, Xiaoyi Zeng, Xiao-Ming Wu, and Qianli Ma. 2021\.  Embedding-based product retrieval in taobao search. In _SIGKDD_. 3181–3189.
  * Manchanda et al. (2019) Saurav Manchanda, Mohit Sharma, and George Karypis. 2019\.  Intent term weighting in e-commerce queries. In _CIKM_. 2345–2348.
  * Mandal et al. (2019) Aritra Mandal, Ishita K Khan, and Prathyusha Senthil Kumar. 2019\.  Query Rewriting using Automatic Synonym Extraction for E-commerce Search.. In _eCOM@ SIGIR_.
  * Mohankumar et al. (2021) Akash Kumar Mohankumar, Nikit Begwani, and Amit Singh. 2021\.  Diversity driven query rewriting in search advertising. In _SIGKDD_. 3423–3431.
  * Ouyang et al. (2022) Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022\.  Training language models to follow instructions with human feedback, 2022.  _arXiv preprint arXiv:2203.02155_ 13 (2022).
  * Qiu et al. (2021) Yiming Qiu, Kang Zhang, Han Zhang, Songlin Wang, Sulong Xu, Yun Xiao, Bo Long, and Wen-Yun Yang. 2021\.  Query Rewriting via Cycle-Consistent Translation for E-Commerce Search. In _ICDE_. IEEE, 2435–2446.
  * Rafailov et al. (2023) Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D Manning, and Chelsea Finn. 2023.  Direct preference optimization: Your language model is secretly a reward model.  _arXiv preprint arXiv:2305.18290_ (2023).
  * Song et al. (2023) Feifan Song, Bowen Yu, Minghao Li, Haiyang Yu, Fei Huang, Yongbin Li, and Houfeng Wang. 2023.  Preference ranking optimization for human alignment.  _arXiv preprint arXiv:2306.17492_ (2023).
  * Song et al. (2021) Zhenqiao Song, Jiaze Chen, Hao Zhou, and Lei Li. 2021\.  Triangular Bidword Generation for Sponsored Search Auction. In _WSDM_. 707–715.
  * Stiennon et al. (2020) Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. 2020.  Learning to summarize with human feedback.  _NIPS_ 33 (2020), 3008–3021.
  * Tao and Zhai (2006) Tao Tao and ChengXiang Zhai. 2006.  Regularized estimation of mixture models for robust pseudo-relevance feedback. In _SIGIR_. 162–169.
  * Team (2023) Qwen Team. 2023\.  _QWEN TECHNICAL REPORT_.  Technical Report. Alibaba Group.  https://qianwen-res.oss-cn-beijing.aliyuncs.com/QWEN_TECHNICAL_REPORT.pdf
  * Touvron et al. (2023) Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023\.  Llama 2: Open foundation and fine-tuned chat models.  _arXiv preprint arXiv:2307.09288_ (2023).
  * Wang et al. (2023a) Binbin Wang, Mingming Li, Zhixiong Zeng, Jingwei Zhuo, Songlin Wang, Sulong Xu, Bo Long, and Weipeng Yan. 2023a.  Learning Multi-Stage Multi-Grained Semantic Embeddings for E-Commerce Search. In _TheWebConf_. 411–415.
  * Wang et al. (2023c) Liang Wang, Nan Yang, and Furu Wei. 2023c.  Query2doc: Query Expansion with Large Language Models.  _arXiv preprint arXiv:2303.07678_ (2023).
  * Wang et al. (2023b) Shuai Wang, Harrisen Scells, Bevan Koopman, and Guido Zuccon. 2023b.  Can ChatGPT write a good boolean query for systematic review literature search?  _arXiv preprint arXiv:2302.03495_ (2023).
  * Wang et al. (2021) Yaxuan Wang, Hanqing Lu, Yunwen Xu, Rahul Goutam, Yiwei Song, and Bing Yin. 2021\.  QUEEN: Neural query rewriting in e-commerce.  (2021).
  * Xu and Croft (2017) Jinxi Xu and W Bruce Croft. 2017.  Quary expansion using local and global document analysis. In _Acm sigir forum_ , Vol. 51. ACM New York, NY, USA, 168–175.
  * Yang et al. (2023) Aiyuan Yang, Bin Xiao, Bingning Wang, Borong Zhang, Chao Yin, Chenxu Lv, Da Pan, Dian Wang, Dong Yan, Fan Yang, et al. 2023\.  Baichuan 2: Open Large-scale Language Models.  _arXiv preprint arXiv:2309.10305_ (2023).
  * Yuan et al. (2023) Zheng Yuan, Hongyi Yuan, Chuanqi Tan, Wei Wang, Songfang Huang, and Fei Huang. 2023\.  Rrhf: Rank responses to align language models with human feedback without tears.  _arXiv preprint arXiv:2304.05302_ (2023).
  * Zeng et al. (2022) Aohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang, Hanyu Lai, Ming Ding, Zhuoyi Yang, Yifan Xu, Wendi Zheng, Xiao Xia, et al. 2022\.  Glm-130b: An open bilingual pre-trained model.  _arXiv preprint arXiv:2210.02414_ (2022).
  * Zhang et al. (2020) Han Zhang, Songlin Wang, Kang Zhang, Zhiling Tang, Yunjiang Jiang, Yun Xiao, Weipeng Yan, and Wen-Yun Yang. 2020\.  Towards personalized and semantic retrieval: An end-to-end solution for e-commerce search via embedding learning. In _SIGIR_. 2407–2416.
  * Zhang et al. (2023) Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, et al. 2023\.  Siren’s Song in the AI Ocean: A Survey on Hallucination in Large Language Models.  _arXiv preprint arXiv:2309.01219_ (2023).
  * Zheng et al. (2020) Zhi Zheng, Kai Hui, Ben He, Xianpei Han, Le Sun, and Andrew Yates. 2020.  BERT-QE: Contextualized Query Expansion for Document Re-ranking. In _EMNLP_. 4718–4728.


2.5D FRADO model in BAL QSOs Proceedings of the International Astronomical
Union: LaTeX Guidelines for authors

17 2023 10.1017/xxxxx

Proceedings IAU Symposium

# BAL effect in quasars due to source orientation

M. Sniegowska1,2 M. H<EMAIL_ADDRESS>M. L. Martinez-Aldama3,4
P. Marziani5 S. Panda6222CNPq Fellow B. Czerny1 1Center for Theoretical
Physics, Polish Academy of Sciences, Lotnikow 32/46, 02-668 Warsaw, Poland
2Nicolaus Copernicus Astronomical Center, Polish Academy of Sciences, Bartycka
18, 00-716 Warsaw, Poland  3Instituto de Física y Astronomía, Facultad de
Ciencias, Universidad de Valparaíso, Gran Bretaña 1111, Valparaíso, Chile
4Departamento de Astronomía, Universidad de Chile, Casilla 36D, Santiago,
Chile  5INAF-Astronomical Observatory of Padova, Vicolo dell’Osservatorio, 5,
35122 Padova PD, Italy  6Laboratório Nacional de Astrofísica - Rua dos Estados
Unidos 154, Bairro das Nações. CEP 37504-364, Itajubá, MG, Brazil

###### Abstract

We investigated a scenario where the presence of a broad absorption line (BAL)
feature in quasars (QSOs) is contingent upon the line of sight being situated
within an outflow cone emanating from the source. We examined the mechanism of
dust-driven winds based on the failed radiatively accelerated dusty outflow
(FRADO) model proposed by Czerny & Hryniewicz, letting it be responsible for
the formation of massive outflow. We calculated the probability of observing
the BAL effect from the geometry of outflow which is a function of global
parameters of black hole mass ($M_{\bullet}$), Eddington ratio ($\alpha_{\rm
Edd}$), and metallicity (Z). We then compared the results with prevalence of
BAL QSOs in a sample of observational data from SDSS. The consistency of our
model with the data supports the interpretation of the BAL phenomenon as a
result of source orientation, rather than a transitory stage in AGN evolution.

###### keywords:

Active Galaxies, Broad Absorption Lines, Quasars, Dust, Radiation Pressure

††editors: C. Sterken, J. Hearnshaw & D. Valls-Gabaud, eds.

##  1 Introduction

BAL QSOs represent a unique and enigmatic class of quasars that exhibit
prominent blue-shifted absorption features in their spectra (Lynds, 1967;
Trump et al., 2006) which evidence the presence of strong and massive outflows
from the source (Allen et al., 2011; Hamann et al., 2013). They exhibit a
continuous and wide absorption trough encompassing velocities reaching several
times $10^{4}$ km/s (Risaliti et al., 2005; Gibson et al., 2009). The study of
BAL QSOs holds significant importance in understanding the intricate interplay
between the accretion disk, black hole, and surrounding environment (Hopkins
et al., 2009; Hamann et al., 2019).

BAL QSOs are divided into three types based on the absorption lines present in
the spectrum (Hall et al., 2002). High-ionization BAL quasars (HBAL QSOs),
constituting approximately 85% of BAL quasars, exhibit absorption features
only from highly ionised atoms. Low-ionization BAL quasars (LBAL QSOs) have
high-ionization absorption plus displaying absorption troughs in low-
ionisation ions, making up approximately 15% of the entire BAL population.
FeLBAL QSOs, the rarest subtype among BAL QSOs, exhibit absorption from both
high- and low-ionization species, as well as absorption from Iron excited
states.

Observationally, a comparison of multiple wavelengths reveals minimal
intrinsic distinctions between BALs and non-BALs, except for the tendency of
BAL QSOs to exhibit redder ultraviolet continua compared to non-BAL QSOs
(Weymann et al., 1991; Lewis et al., 2003). BALs are commonly associated with
outflows from the accretion disk of active galactic nuclei (AGNs) (Murray et
al., 1995). The origin of BALs in quasars is subject to two theoretical
scenarios. The first suggests that BAL QSOs are essentially normal QSOs
observed from a line of sight that passes through the outflowing gas (Elvis,
2000). The second scenario involves an evolutionary process (Williams et al.,
1999) which is no longer preferred (Turnshek, 1988). The first scenario relies
on the massive outflow of gas due to line-driving mechanism (Elvis, 2000;
Risaliti et al., 2011). However, there are shreds of evidence in the spectra
of BAL sources hinting at the presence of dust (Dunn et al., 2010; Borguet et
al., 2013). We, therefore, aimed at testing whether there is any consistency
or correlation between the properties of a dust-driven outflow and the
probability of observing BAL feature in a given source. For the comprehensive
version of this study, see Naddaf et al. (2023).

##  2 Methodology

Figure 1: An example of outflow in 2.5D FRADO model for a source at Eddington
rate with $M_{\bullet}$ of $10^{8}M_{\odot}$ and metalicity of $5Z_{\odot}$.
Dotted lines show the path of the trajectory of a clump. The area shaded in
blue/red corresponds to the dusty/dustless (lower/higher ionized) actual
status of outflowing material. The two black solid lines represent the maximum
and minimum angles (to symmetry axis, $z$) confining the outflow cone. The BAL
feature is seen if the line of sight is within the outflow cone.

We use the enhanced 2.5D version of the failed radiatively accelerated dusty
outflow model (FRADO) (Czerny & Hryniewicz, 2011; Naddaf & Czerny, 2022).
FRADO model predicts that radiation pressure acting on dust lifts the clumps
of dusty material from the surface of an accretion disk. Depending on the
launching location and the global parameters, i.e. black hole mass
($M_{\bullet}$), Eddington ratio ($\alpha_{\rm Edd}$), and metallicity ($Z$),
the material may then come back to the disk surface or it may escape from the
gravitational potential. Also, the launched material may or may not lose the
dust content depending on the trajectory (for more details of the 2.5D model,
see Naddaf et al., 2021; Naddaf & Czerny, 2022). An example of the outflow in
the 2.5D FRADO model is shown in figure 1 which in this case includes both
highly- and lowly-ionized regions shaded in red (dustless) and blue (dusty),
respectively. The geometry of the outflow is a function of $M_{\bullet}$,
$\alpha_{\rm Edd}$, and $Z$. We calculate the probability of observing BAL
feature as

$P_{\rm BAL}=\dfrac{{\rm
fraction~{}of~{}the~{}sky~{}covered~{}by~{}outflow~{}cone}}{{\rm
unit~{}sphere}}=\cos(\theta_{\rm min})-\cos(\theta_{\rm max})$ (1)

As for the observational data, we collected a sample from the quasar catalogue
of the Sloan Digital Sky Survey DR7 (Shen et al., 2011). The selected sample
consists of sources with a median S$/$N$>10$ per pixel in the rest-frame
2700-2900Å region in which the black hole masses are estimated from Mgii
$\lambda$2800 . The full sample contains 42,349 objects that 3% of which are
detected as BAL QSOs. Black hole masses and Eddington ratios for Mgii were
obtained from the SDSS catalog. Dividing the parameter space of black hole
mass and Eddington ratio, $M_{\bullet}$-$\alpha_{\rm Edd}$, into bins centered
at integer values of $\log M_{\bullet}$ and $\log$$\alpha_{\rm Edd}$with the
step size of one in log scale, the median values for the distribution of HBAL
and LBAL sources of the sample in each bin are presented in Figure 2. The
value from the sample of observational data which can be compared with $P_{\rm
BAL}$ from 2.5D FRADO is the prevalence ratio, $r_{\rm BAL}$, defined as

$r_{\rm BAL}=\dfrac{N_{\rm BAL}}{(N_{\rm BAL}+N_{\rm nonBAL})}$ (2)

where $N$ stands for the corresponding number of sources in each bin.

Figure 2: The $M_{\bullet}$-$\alpha_{\rm Edd}$ space for non-BAL (black
$\smallstar$), BAL (blue $\square$), HBAL (orange $\diamond$ in the left
panel) and LBAL (green $\diamond$ in the right panel) QSOs. Symbols represent
the median values of $\log{M_{\bullet}}$ and log $\alpha_{\rm Edd}$ in each
bin. Contours in black, orange, and green mark the distribution for non-BAL,
HBAL, and LBAL QSOs, respectively. Figure 3: Comparison of $r_{\rm BAL}$ from
the sample of observational data with $P_{\rm BAL}$ from 2.5D FRADO, for the
range of $M_{\bullet}$, and $\alpha_{\rm Edd}$. Solid lines show the best
exponential fit to the corresponding data. The values of $r_{\rm BAL}$ and the
corresponding fit in each panel are shown in black. Results from 2.5D FRADO,
$P_{\rm BAL}$, for metallicities of 1, 2.5, 5, and 10 $Z_{\odot}$, and their
corresponding fits are color coded in green, violet, blue, and red,
respectively.

##  3 Result & Discussion

Utilizing the 2.5D FRADO model, we obtained the geometry of outflow for a
range of global parameters of $M_{\bullet}$, $\alpha_{\rm Edd}$, and $Z$. We
then quantified the associated parameter $P_{\rm BAL}$. Concurrently, we
analyzed an observational sample to extract the corresponding $r_{\rm BAL}$.
The figure 3 presents a comparative analysis between the simulation results
generated by 2.5D FRADO (colored data points) and observational data (black
data points). Remarkably, exponential functions emerged as optimal fits that
effectively captured the trends exhibited by the data points in all cases.
This striking correlation between theoretical predictions and observational
data lends substantial support to the hypothesis that the BAL effect is not a
temporary phenomenon but rather arises from the inherent orientation of the
source relative to the observer. Specifically, our findings indicate that the
BAL effect, both in our model and in the empirical data, is predominantly
associated with sources possessing $M_{\bullet}$ exceeding $10^{8}M_{\odot}$,
and also the effect gets amplified as the accretion rate of the source rises.
Here we presented a subset of our results in which the presence of torus is
not taken into account for computing the $P_{\rm BAL}$. In this particular
scenario, characterized by the absence of a torus, our analysis reveals a
preference for lower metallicities, as evident from the observed trends in the
empirical data. However, through further investigations incorporating the
torus (see Naddaf et al., 2023, for details), we found that higher
metallicities are more favored especially when the accretion rate of the
source is also higher, as frequently concluded in other studies (see e.g.
Martínez-Aldama et al., 2018; Panda et al., 2019, 2020; Panda, 2021;
Śniegowska et al., 2021; Panda, 2022; Garnica et al., 2022).

## References

  * Allen et al. (2011) Allen, J. T., et al., _MNRAS_ 410, 2, 860 (2011)
  * Borguet et al. (2013) Borguet, B. C. J., et al., _ApJ_ 762, 1, 49 (2013)
  * Czerny & Hryniewicz (2011) Czerny, B., Hryniewicz, K., _A &A_ 525, L8 (2011)
  * Dunn et al. (2010) Dunn, J. P., et al., _ApJ_ 709, 2, 611 (2010)
  * Elvis (2000) Elvis, M., _ApJ_ 545, 1, 63 (2000)
  * Garnica et al. (2022) Garnica, K., et al., _A &A_ 667, A105 (2022)
  * Gibson et al. (2009) Gibson, R. R., et al., _ApJ_ 692, 1, 758 (2009)
  * Hall et al. (2002) Hall, P. B., et al., _ApJS_ 141, 2, 267 (2002)
  * Hamann et al. (2019) Hamann, F., Herbst, H., Paris, I., Capellupo, D., _MNRAS_ 483, 2, 1808 (2019)
  * Hamann et al. (2013) Hamann, F., et al., _MNRAS_ 435, 1, 133 (2013)
  * Hopkins et al. (2009) Hopkins, P. F., Murray, N., Thompson, T. A., _MNRAS_ 398, 1, 303 (2009)
  * Lewis et al. (2003) Lewis, G. F., Chapman, S. C., Kuncic, Z., _ApJL_ 596, 1, L35 (2003)
  * Lynds (1967) Lynds, C. R., _ApJ_ 147, 396 (1967)
  * Martínez-Aldama et al. (2018) Martínez-Aldama, M. L., et al., _A &A_ 618, A179 (2018)
  * Murray et al. (1995) Murray, N., Chiang, J., Grossman, S. A., Voit, G. M., _ApJ_ 451, 498 (1995)
  * Naddaf & Czerny (2022) Naddaf, M. H., Czerny, B., _A &A_ 663, A77 (2022)
  * Naddaf et al. (2021) Naddaf, M. H., Czerny, B., Szczerba, R., _ApJ_ 920, 1, 30 (2021)
  * Naddaf et al. (2023) Naddaf, M. H., et al., _A &A_ (2023)
  * Panda (2021) Panda, S., _A &A_ 650, A154 (2021)
  * Panda (2022) Panda, S., _Frontiers in Astronomy and Space Sciences_ 9, 850409 (2022)
  * Panda et al. (2019) Panda, S., Marziani, P., Czerny, B., _ApJ_ 882, 2, 79 (2019)
  * Panda et al. (2020) Panda, S., Marziani, P., Czerny, B., _Contributions of the Astronomical Observatory Skalnate Pleso_ 50, 1, 293 (2020)
  * Risaliti et al. (2005) Risaliti, G., et al., _ApJL_ 630, 2, L129 (2005)
  * Risaliti et al. (2011) Risaliti, G., et al., _MNRAS_ 410, 2, 1027 (2011)
  * Shen et al. (2011) Shen, Y., et al., _ApJS_ 194, 2, 45 (2011)
  * Śniegowska et al. (2021) Śniegowska, M., et al., _ApJ_ 910, 2, 115 (2021)
  * Trump et al. (2006) Trump, J. R., et al., _ApJS_ 165, 1, 1 (2006)
  * Turnshek (1988) Turnshek, D. A., QSO Absorption Lines, Cambridge Uni. Press (1988)
  * Weymann et al. (1991) Weymann, R. J., Morris, S. L., Foltz, C. B., Hewett, P. C., _ApJ_ 373, 23 (1991)
  * Williams et al. (1999) Williams, R. J. R., Baker, A. C., Perry, J. J., _MNRAS_ 310, 4, 913 (1999)