#  Ktrl+F: Knowledge-Augmented In-Document Search

Hanseok Oh1 Haebin Shin1,2∗ Miyoung Ko1 Hyunji Lee1 Minjoon Seo1
1KAIST AI 2Samsung Research
{hanseok, haebin.shin, miyoungko, hyunji.amy.lee<EMAIL_ADDRESS>
indicates equal contribution

###### Abstract

We introduce a new problem Ktrl+F, a knowledge-augmented in-document search
task that necessitates real-time identification of all semantic targets within
a document with the awareness of external sources through a single natural
query. This task addresses following unique challenges for in-document search:
1) utilizing knowledge outside the document for extended use of additional
information about targets to bridge the semantic gap between the query and the
targets, and 2) balancing between real-time applicability with the
performance. We analyze various baselines in Ktrl+F and find there are
limitations of existing models, such as hallucinations, low latency, or
difficulties in leveraging external knowledge. Therefore we propose a
Knowledge-Augmented Phrase Retrieval model that shows a promising balance
between speed and performance by simply augmenting external knowledge
embedding in phrase embedding. Additionally, we conduct a user study to verify
whether solving Ktrl+F can enhance search experience of users. It demonstrates
that even with our simple model, users can reduce the time for searching with
less queries and reduced extra visits to other sources for collecting
evidence. We encourage the research community to work on Ktrl+F to enhance
more efficient in-document information access.111Code,Chrome extension plugin,
and dataset are available at https://github.com/kaistAI/KtrlF

##  1 Introduction

Despite significant advancement in many Natural Language Processing
applications, facilitated by transformer-based models (Devlin et al., 2019;
Raffel et al., 2019), real-time in-document search still leans heavily on
conventional lexical matching tools like the "Find" function (Ctrl+F) and
regular expressions. These tools, while fast, have clear limitations,
especially with ambiguous keywords or multiple targets.

Machine Reading Comprehension (MRC) seems a promising solution to these
issues. It reads documents, comprehends their context, and answers questions
(Rajpurkar et al., 2016). However, MRC focuses on explicit contents, limiting
its value when users need broader context not directly in the document
(Trischler et al., 2017; Rajpurkar et al., 2018; Joshi et al., 2017).
Consider, for instance, a scenario in which a user is reading a news article
and searching for information related to "Social network platform of China."
(Figure 1). Typically, users would need to consult external sources, such as
Wikipedia, to gather additional details about candidate keywords that are not
explicitly mentioned in the news, such as WeChat, Baidu, and Twitter. An
alternative approach is harnessing the capabilities of powerful pre-trained
language models (Brown et al., 2020; Touvron et al., 2023). However, their
generative nature poses challenges in real-time search task.

To overcome the limitations of previous methods and enhance the efficiency and
comprehensiveness of in-document search, we present a new problem Ktrl+F
(knowledge-augmented in-document search). This task aims to reduce redundancy
and better meet the requirements of real users. Given a natural language query
and a long input document, Ktrl+F is designed to fulfill three key criteria:
(req 1) Find all semantic targets. (req 2) Utilizes external knowledge. (req
3) Operates in real-time. In the absence of a suitable dataset to evaluate
Ktrl+F, we curate a new dataset comprising unique queries that demand matching
external evidence. To measure model performance in Ktrl+F, we introduce a set
of reformulated metrics tailored to measure processing speed while maintaining
robust and high performance.

Figure 1: Comparison between in-document search and Ktrl+F task. In-document
search accesses the information in documents by either lexical search (Ctrl+F,
Regular expression) or semantic search (MRC). Lexical search suffers from
finding semantically matching keywords, and semantic search does not consider
external knowledge. Ktrl+F requires an efficient way to utilize external
knowledge to find all semantic targets in real-time.

We conduct an extensive analysis of various baselines for Ktrl+F and find
several limitations, including issues such as hallucination, slow speed with
generative models, and challenges in incorporating external knowledge into MRC
models (see Section 6.2 for details). To strike a balance between real-time
processing speed and achieving high performance through effective utilization
of additional knowledge, we introduce a simple yet effective extension of
phrase retrieval (Lee et al., 2021): Knowledge-Augmented Phrase Retrieval.
This model seamlessly extends the phrase retrieval to cater to in-document
search scenarios, all while integrating external knowledge without the need
for additional training steps. Our experimental results support that by simply
adding the knowledge embedding and the phrase embedding, Knowledge-Augmented
Phrase Retrieval exhibits the potential to reflect external knowledge without
sacrificing latency.

Furthermore, we conduct a user study to show the necessity of Ktrl+F utilizing
a Chrome extension plugin that operates in the real web environments, built
upon our model. Results of the study demonstrate that search experience of
users can be enhanced even with our simple model with seamless access to
external knowledge during in-document searches. We encourage the research
community to take on the unique challenge of solving Ktrl+F requiring balance
between performance and speed to enhance more efficient and effective
information access.

##  2 Related Works

#### Machine Reading Comprehension (MRC)

is a task to find the answer to a question in the provided context. Most MRC
datasets assess the ability of context understanding of the model by
extracting a single span for the query only grounding on the information
within a provided context (Rajpurkar et al., 2016; Trischler et al., 2017;
Joshi et al., 2017; Rajpurkar et al., 2018; Fisch et al., 2019; Kwiatkowski et
al., 2019). Few works explore the identification of multiple targets for a
query in the input document evaluating the model’s comprehension of the given
context (Dasigi et al., 2019; Zhu et al., 2020; Li et al., 2022) . Some
studies tackle information-seeking problem by utilizing external information
missing from input document to gap knowledge (Ferguson et al., 2020; Dasigi et
al., 2021). This external information aids in enhancing the understanding of
the context. However, Ktrl+F relies on external knowledge beyond the context
to establish a connection between queries and target within the given context.
Since much of the target information is not available in the given context, it
is essential to explicitly ground external knowledge about the target.
Consequently, the evaluation of Ktrl+F focuses not on the understanding of the
given context, but on information obtained from outside the given context.

#### Knowledge augmented information retrieval

can be seen as an approach to enhance external contextual information for the
embedding. Lee et al. (2023) uses contextualized embeddings as vocab
embeddings for text tokens in the generative retrieval model to enhance
contextual information for naive text tokens. Raina et al. (2023) focuses on
the retrieval augmented text embedding to find a way to efficiently reuse
prebuilt dense representation with lightweight representation, and also
discusses the necessity of systems for utilizing external contextual
information to include contextual information outside the given context. Lin
et al. (2022) tries to aggregate contextualized token embedding with [CLS]
embedding. They focus on the augmenting contextual information using the
embeddings from the given context, not focused on external knowledge.

##  3 Ktrl+F: Knowledge-Augmented In-Document Search

In this section, we define Ktrl+F, which is knowledge-augmented in-document
search task and its unique characteristics (section 3.1). Then we describe the
evaluation metrics to measure each requirement (section 3.2).

###  3.1 Task Definition

Ktrl+F is a task that requires finding all semantic targets from a given input
document in real-time with the awareness of external knowledge, when given a
natural language query. As illustrated in Figure 1, when presented with a
natural language query and a input document, Ktrl+F is designed to meet three
essential criteria.

####  req 1: Find all semantic targets.

Ktrl+F requires finding all relevant targets within a given document. The term
"all" refers to multiple aspects: finding all multiple answers (baidu, wechat,
weibo), all occurrences of each answer (baidu appears two times in the
document), and all lexical variations of mentions for each answer (Weibo,
Sina).

####  req 2: Utilize external knowledge.

Expanding the matching space from lexical to semantic introduces a
comprehensive connection between query and target units. However, in many
cases, the targets contain additional information beyond the semantic
information present in the input document. By effectively leveraging this
additional information through utilization of external knowledge, we can
further bridge the semantic gap between the query and the targets.

####  req 3: Search in real-time.

Ktrl+F inherits the practicality of in-document search, such as Ctrl+F, which
emphasizes real-time search to minimize the time on finding targets within the
input document. The complexity of Ktrl+F lies in effectively balancing real-
time applicability with the performance of finding all matching targets by
leveraging external knowledge.

###  3.2 Evaluation Metrics

To assess various aspects of Ktrl+F, we employ a range of metrics that
collectively measure the overall balance of performance and speed. Evaluating
the appropriateness of relevant external knowledge sources (req 2) is
challenging as there is no definite gold standard answer. Therefore, following
Izacard and Grave (2021), we indirectly assess the impact of utilizing
external knowledge by comparing the overall performance of the system with and
without its incorporation.

#### List EM, List Overlap F1, Robustness Score.

The three metrics measure if the model finds all semantic targets, and
fulfills req 1. List EM and List Overlap F1 assess the model’s ability to
generate an accurate list of targets. List EM considers correct only when the
prediction list is exactly the same as the ground truth list, whereas List
Overlap F1 allows partial matches between individual elements of the predicted
and the ground truth list. Note that List EM is different from Set EM, a
commonly used metric in Machine Reading Comprehension Rajpurkar et al. (2016),
in that List EM aims to identify all occurrences of targets within a input
document. Inspired by Zhong et al. (2022), we adjust robustness score to
assess the robustness of system in predicting target answer entities as
queries change within a given input document. By treating queries associated
with the same input document as a cohesive cluster, we compute the robustness
score by averaging the minimum score within each cluster; robustness score of
List EM shows the average of minimum List EM score of each cluster and the
same for List Overlap F1. This approach enhances the comprehensive evaluation
of Ktrl+F task, given that the knowledge-augmented design of Ktrl+F allows for
various queries with different target answers for in-document searches.

#### Latency.

Latency is a metric for assessing real-time applicability, therefore
satisfying req 3. We measure in ms/Q (millisecond per query) which is widely
used in retrieval systems to represent query inference speed (Khattab and
Zaharia, 2020; Santhanam et al., 2022).

##  4 Ktrl+F Dataset

Figure 2: Overview of Ktrl+F Dataset construction pipeline. We utilize real
news articles as input documents (Step 1), and automatically generate queries
and targets (Step 2). To improve the reliability of targets, we identify
whether each entity is a target by referring to external knowledge, and
finalize entities that differ from the Step 2 results (Step 3-1). We remove
queries that do not satisfy req 2 using the MRC model (Step 3-2).

In the absence of suitable benchmarks for evaluating the Ktrl+F, we introduce
a data construction pipeline that aims to collect sets of key components for
Ktrl+F: input document, query, corresponding targets, and external knowledge.
We employ a pipeline to generate an evaluation set for Ktrl+F and subsequently
assess its quality through human verification. The overall process of pipeline
is shown in Figure 2.

###  4.1 Dataset Construction Pipeline

#### Step 1. Select Real News Articles.

To simulate real-world document scenarios, we randomly sample 100 English news
articles from the publicly available Common Crawl (Raffel et al., 2019) after
preprocessing them according to two criteria. Initially, to remove abnormal
articles, we collect 6,936 articles from the 13,863 articles in the Common
Crawl validation set, with lengths ranging from 991 to 3,298, covering the
lower to upper quartiles. Subsequently, we utilize an entity linking
API222https://cloud.google.com/natural-language/docs/analyzing-entities to
identify all entities within the article and extract external knowledge333We
consider Wikipedia as an external knowledge source, wherein the acquisition of
external knowledge for targets can be equated to utilizing the corresponding
Wiki page linked to a particular entity (Wu et al., 2019). linked to the
entities. To ensure diversity of questions and quality of documents, we
collect 3,910 articles with 4 to 11 entities, covering the lower to upper
quartiles.

#### Step 2. Generate Pairs of (Query, Targets).

Using the entities extracted from each input document (Step 1), we utilize
LLAMA-2 (Touvron et al., 2023) to generate diverse queries and targets (prompt
in Figure 5). We generate 10 questions for each input document. To satisfy req
2 of Ktrl+F, we provide only the extracted entities into the model, excluding
the input document. This is done to remove the dependency on the document
itself, as Ktrl+F prioritizes queries that cannot be answered solely with the
document and requires the integration of external knowledge.

#### Step 3-1. Target Filtering.

To mitigate the potential problem of false positive and false negative in the
generated targets by LLAMA-2 (Touvron et al., 2023), we implement an
additional process inspired by Zhong et al. (2022). This process determines
whether each entity is the answer to the query, leveraging external knowledge
(prompt in Figure 6). Initially, we utilize GPT-3.5 (OpenAI,
2022)444gpt-3.5-turbo-0613 to identify entities judged as potential answer
targets. Subsequently, GPT-4 (OpenAI, 2023)555gpt-4-0613 makes the final
decision for entities where there is a disagreement between GPT-3.5 and the
results of Step 2. Detailed statistics of the results by each model are
available in the Appendix A.

#### Step 3-2. Query Filtering.

Though we prioritize queries that require integrating external knowledge in
Step 2, there are still many queries that do not meet req 2. To further reduce
the number of such queries, we utilize a DeBERTaV3-large (He et al.,
2023)666https://huggingface.co/deepset/deberta-v3-large-squad2, finetuned
using the SQuAD 2.0 (Rajpurkar et al., 2018). We specifically exclude queries
that the MRC model can answer solely based on the input document, leaving only
suitable queries for req 2. Finally, 512 queries are collected out of the
1,000 queries generated in Step 2. See Appendix A for detailed scoring
criteria of the MRC model.

###  4.2 Dataset Analysis

#### Human verification setup.

To report the quality of the auto-generated dataset, we conduct human
verification on a randomly sampled subset of the generated dataset. Four
annotators evaluate a total of 32 queries, representing approximately 6% of
the entire dataset 777We plan to expand more examples in future versions.. The
human verification is designed to assign three annotators to each sample. Each
sample is evaluated by responding to three specific questions.

The first question aims to evaluate how well the generated query meets req 2.
Annotators are instructed to identify evidence for each target to answer the
query. The ideal result is for annotators to respond that evidence cannot be
found in the input document for all targets. The second question evaluates the
naturalness of the generated query by choosing the type of unnatural query:
"Ambivalent or subjective expressions", "Lack of factual basis", "Logical
errors", "etc". The ideal result is for annotators to select "None of these
options", indicating a naturalness in the generated queries. The third
question focuses on evaluating the reliability of auto-generated targets. The
annotator is instructed to select the correct target for the query by
referring to Wikipedia. This is the same process as in target filtering (Step
3-1) in the dataset construction pipeline, to identify the reliability between
the response by the annotator and the dataset. The user interface and detailed
instructions for each question are shown in Figure 7.

Q1. Is it possible to answer using only the input doc? |
---|---
Need more external knowledge | 71.9%
Don’t need external knowledge | 28.1%
% of answered targets | 51.6%
Q2. Is it unnatural query? |
Natural Query | 84.4%
Subjective Query | 9.3%
etc. | 6.3%
Q3. Reliability of Target determination |
kappa coefficient ($\kappa$) | 0.6232

Table 1: Human Verification Results

| Avg. | Min. | Max.
---|---|---|---
Length of Input Document | 1974 | 999 | 3254
Queries per Input Document | 5.2 | 1 | 10
Answer Mentions per Query | 4.2 | 1 | 30
Answer Entities per Query | 1.8 | 1 | 8

Table 2: Statistics of Ktrl+F Dataset

#### Dataset quality and statistics.

Since all samples are evaluated by three annotators, final human judgment is
determined through majority voting. For the first question, 71.9% of samples
are considered unable to answer the target solely based on the input document.
While 28.1% of samples can be found using only an input document, only 51.6%
of all targets could be inferred. This indicates that there are still
limitations to finding all targets based on input documents alone. For the
second question, asking about the naturalness of the query, 84.4% of samples
are considered natural queries, while 9.4% of the samples are subjective
queries. About 6.3% of the samples contain unnatural queries for other
reasons, such as entities being directly mentioned in the query. For the third
question, we find a kappa coefficient (Cohen, 1960) of $\kappa=0.6232$ between
humans and the dataset. Following the interpretation by Landis and Koch
(1977), this indicates substantial agreement between human judgment and the
criteria used by the data construction pipeline to determine targets. In
total, the Ktrl+F dataset has 512 queries for 98 input documents with an
average of 4.2 mentions per query as shown in Table 2. More examples of the
Ktrl+F dataset are available in Table 5.

##  5 Knowledge-Augmented Phrase Retrieval

Figure 3: Overview of Knowledge-Augmented Phrase Retrieval. We redefine the
phrase retrieval architecture originally proposed by DensePhrases (Lee et al.,
2021) within the setting of in-document search and enrich external knowledge
about potential targets. The original design of DensePhrases lacks the
capability to integrate external knowledge. To address this limitation, we
explicitly augment knowledge embedding, extracted with an entity linker, into
the phrase embedding. This serves to bridge the semantic gap between the query
and the targets (req 2).

The challenge of Ktrl+F lies in effectively balancing real-time applicability
and high performance while utilizing efficient use of external knowledge. To
comprehensively address this challenge and fulfill three requirements of
Ktrl+F (req 1, req 2, req 3), we introduce Knowledge-Augmented Phrase
Retrieval, which consists of multiple modules, as illustrated in Figure 3.
Notably, our model doesn’t need an additional training step; it represents a
straightforward approach that combines and utilizes the modules as they are.

###  5.1 External Knowledge Linking Module

The external knowledge linking module skims the target text and identifies
entities, which can be potential candidate targets, mapping each of them to
the relevant Wikipedia knowledge base. The output of this module is a list of
candidate targets along with Wikipeidia page linked to each target, which
serves as external knowledge about the targets. To focus on building a model
that can integrate external knowledge, we use existing entity linking models
for this purpose. While there are various entity-linking models available for
this purpose, we opt to utilize a Wikifier API (Brank et al., 2017) as an
entity linker for ease of use.

###  5.2 Query and Phrase Encoder

The phrase and query encoder modules are responsible for encoding the
candidate entity and the query, respectively. We employ the frozen
DensePhrases model (Lee et al., 2021), which is trained on multiple question
answering datasets, to extract phrase embeddings. For the query embedding, we
extract the embedding of the special token [CLS] from the output embeddings of
the query encoder. We use two distinct query encoders to extract the start and
end position embeddings for the query, following Lee et al. (2021).
Subsequently, we concatenate the corresponding token embeddings, denoted as
$[q_{start};q_{end}]\in\mathbb{R}^{2d}$, to create a query embedding. For the
phrase encoder, we similarly utilize concatenated token-level embeddings of
the entity’s boundary tokens, namely the start and end token embedding denoted
as $[e_{start},e_{end}]$, as the entity embedding.

###  5.3 Knowledge Aggregation Module

To incorporate external knowledge related to the entity, we utilize the same
phrase encoder that is used to extract embeddings for candidate entities.
Following the approach in Lee et al. (2023), we generate knowledge embedding
for the linked entity by concatenating the text of the name of entity and its
corresponding Wikipedia page (see details in Figure 8 of Appendix). By
adopting this approach, we can effectively encode pertinent knowledge about
the entity into its embedding. To combine this external knowledge embedding
with the entity embedding and create an in-document phrase index, we employ a
simple element-wise addition operation. This demonstrates promising results in
our experiments enabling the system to capture the contextual knowledge for
more accurate and comprehensive search and retrieval within the document
without requiring further tuning. With Maximum Inner Product Search (MIPS)
operation, Knowledge-Augmented Phrase Retrieval can find all matching targets
in real time.

##  6 Experiments

|  | Speed | Performance
---|---|---|---
Type | Model | Latency (ms/Q) ($\downarrow$) | List EM ($\uparrow$) | (R) List EM ($\uparrow$) | List Overlap ($\uparrow$) | (R) List Overlap ($\uparrow$)
Generative | GPT-4 | - | 30.457 | 7.452 | 37.402 | 12.898
GPT-3.5 | - | 30.346 | 8.284 | 41.929 | 19.446
LLAMA-2-Chat-7B | 2359 | 28.529 | 8.947 | 40.546 | 20.008
LLAMA-2-Chat-13B | 3176 | 28.846 | 8.024 | 37.098 | 14.367
VICUNA-7B-v1.5 | 1951 | 17.831 | 3.694 | 31.216 | 12.532
VICUNA-13B-v1.5 | 2420 | 24.490 | 6.977 | 39.278 | 20.401
Extractive | SequenceTagger | 26 | 7.239 | 0.612 | 8.614 | 1.211
Retrieval | Ours (w/ Wikifier) | 15 | 23.152 | 7.091 | 40.718 | 23.107
Ours (w/ Gold) | 14 | 46.170 | 22.426 | 53.689 | 32.285

Table 3: Overall speed and performance evaluation results for Ktrl+F dataset.
Note that as we cannot measure the API-based models, we skip the speed
evaluation for GPT-3.5 and GPT-4. Robustness score is noted with (R) with
corresponding metric. Ours denotes Knowledge-Augmented Phrase Retrieval and
the best results excluding Ours (w/ Gold) are in bold, while second-best ones
are underlined.

In section 6.1, we describe the baselines used in our experiment. In section
6.2, we present experimental results and conduct a comprehensive analysis of
various aspects to solve Ktrl+F.

###  6.1 Setup

We compare various baselines: generative, extractive, and retrieval-based
models. Note that all models in the experiment are evaluated in a zero-shot
manner, which haven’t been trained with the dataset for Ktrl+F. All speed
measurements except LLM-API are done using a A6000 GPU on a server with two
AMD EPYC 7513 CPUs, each with 32 physical cores.

#### Generative baselines

solve Ktrl+F as a text generation problem, where the model takes instructions,
a input text, and a query as input and sequentially produces matching targets
(see Appendix B). The parametric space of Large Language Models (LLM) serves
as an implicit source of general knowledge (Yu et al., 2023). To explore the
knowledge within the parametric space, we utilize various LLM models, such as
the LLM API versions GPT-3.5 (OpenAI, 2022) and GPT-4 (OpenAI, 2023), as well
as open-source models like LLAMA-2 (Touvron et al., 2023) and VICUNA v1.5
(Chiang et al., 2023), ranging in size from 7B to 13B. In this experiment, we
do not consider retrieval-augmented methods that use retrieval results by
appending them to the input. We additionally post-process generated outputs of
models to only extract targets for evaluation.

#### Extractive baseline

is similar to extraction-based model for Machine Reading Comprehension task.
This approach uses the internal knowledge within the target text to directly
locate the answer spans. In order to find all relevant spans in the target
text, we follow the previous works (Segal et al., 2020; Li et al., 2022) that
helps identify multiple entities. We utilize a BERT based sequence tagging
model which is fine-tuned using MultiSpanQA (Li et al., 2022) dataset, denoted
as SequenceTagger.

Entity Linker | Model | List EM ($\uparrow$) | (R)List EM ($\uparrow$) | List Overlap ($\uparrow$) | (R)List Overlap ($\uparrow$)
---|---|---|---|---|---
Gold (GCP API) | Ours | 46.170 | 22.426 | 53.689 | 32.285
\- External | 34.582 | 14.178 | 43.758 | 26.406
\- Internal | 47.345 | 23.097 | 54.308 | 30.599
Wikifier | Ours | 23.152 | 7.091 | 40.718 | 23.107
\- External | 15.620 | 4.742 | 31.805 | 18.823
\- Internal | 22.851 | 7.773 | 39.391 | 20.812

Table 4: Ablation study on the impact of existence and quality of external
knowledge. We measure the performance when using different entity linkers
(Gold w/ GCP API, Wikifier API). Also, we evaluate the impact of contextual
phrase embedding and external embedding by removing the related part.

###  6.2 Results

#### Generative and extractive baselines

show difficulties in balancing real-time applicability and performance as
shown in Table 3. Generative baselines demonstrate superior performances in
addressing the Ktrl+F, leveraging parametric knowledge of pre-trained language
models to bridge the semantic gap between queries and targets. However,
scaling up model capacity doesn’t consistently improve performance and leads
to increased latency. The generative nature of these models often causes
problems in constraining the output to targets within the input document, such
as hallucination and difficulty restricting the output to targets within the
input document which need post-processing step. Conversely, the
SequenceTagger, an extractive baseline using the same model capacity (BERT-
base) as our model, falls short in Ktrl+F. Its inability to utilize external
knowledge highlights the importance of incorporating such knowledge beyond the
input document for successful Ktrl+F resolution. For a comprehensive baseline
understanding, we additionally report extra experiments in Table 7 and Table 8
of Appendix E. Detailed prediction example for each model is available in
Table 9 of Appendix D.

#### Knowledge-Augmented Phrase Retrieval

demonstrates a balance between latency and achieving overall performance. By
simply incorporating knowledge embedding into the phrase retrieval process,
our model (Ours w/ Wikifier) shows competitive performance in List Overlap
metrics, despite having a significantly smaller model capacity compared to
other generative baselines. When provided gold entity linking information used
in the dataset construction pipeline, our model can achieve the best
performance among all (Ours w/ Gold). To compare with other baselines, we
threshold the prediction results from top 4 according to the data distribution
888To provide a comprehensive understanding of the model, we additionally
report MAP metrics in Table 6 of Appendix C.. Not only for performance we can
see that retrieval-based design of our model is proper design for real-time
applicability, which shows way faster latency than other baselines. While our
model requires additional time to index the long input documents into
searchable formats9992.863 sec for ours (w/ Wikifier) and 0.955 sec for ours
(w/ Gold). Note that the indexing process only needs to occur once for each
input document. , the subsequent querying of the indexed text incurs real-time
latency.

###  6.3 Ablation Study

By integrating external knowledge into our models, we aim to enhance the
knowledge of the model about candidate targets, thereby bridging the semantic
gap between query and the targets beyond the information present in the input
document. To measure the importance of using high-quality external knowledge
in addressing Ktrl+F, we compare the performance between our model with
Wikifier and Gold entity linker utilized in the dataset construction pipeline.
The one with the Gold entity linker (Ours w/ Gold) exhibits a significant
performance improvement despite employing all other identical modules.

We also evaluate the importance of the knowledge aggregation design in our
model. Our model utilizes in-document phrase index by adding external
knowledge embedding from Wikipedia and phrase embedding from the input
document. In Table 4, (-External) signifies the exclusion of external
knowledge embedding, while (-Internal) indicates the removal of phrase
embedding. Results indicate a notable performance drop with (-External) when
both Gold and Wikifier entity linkers are used. When phrase embedding is
removed (-Internal), the model with the Gold entity linker shows better
performance overall, while the model with the Wikifier entity linker shows
lower results overall compared to using both embeddings. However, robustness
of List Overlap scores consistently remains higher than when partial
components are removed. This highlights that internal knowledge about
candidate targets might play a crucial role in constructing a robust
embedding, especially when external information is of sub-optimal quality.

##  7 User Study

(a) Number of queries

(b) Number of visited websites

(c) Spent time (sec)

(d) List EM F1 score

Figure 4: A Comprehensive comparison of in-document search systems: Ktrl+F
plugin outperforms with fewest queries, minimal web surfing, least time spent,
and best List EM F1 performance. We use abbreviation for regular expression as
Regex in the figure.

To verify whether solving Ktrl+F can enhance search experience of users in the
real web environments, we build Chrome extension plugin (Ktrl+F plugin) built
on our model to compare with other in-document search systems. We compare
search experience of users using Ktrl+F and other in-document search systems,
specifically Ctrl+F and Regular expression101010We exclude MRC system, because
it is not designed for seeking information without reading the text, which is
hard to use for our user study.. The main objective is to analyze efficiency
and convenience of each system for users.

###  7.1 Setup

We recruit 6 participants for an in-depth experiment, where they are tasked
with using the given system to find information from 10 different examples on
a given website within a limited time 111111We plan to expand more examples in
future versions. To ensure a fair comparison, each user is assigned, on an
example-by-example basis, to use only a specific system among Ktrl+F plugin,
Ctrl+F, and Regular expression to help them find all parts of targets that
match given search intent. The participants manually annotate the targets in
the PDFs using the respective system. To capture the entire process, we record
the screens of participants throughout the experiment. During the analysis
phase, we focus on several key elements. These include the time each
participant spent finding information using the specified system, the number
of queries used to find the target parts, the count of additional websites
visited to verify the identified targets for each example, and an overall
score for the prediction results using List EM F1 to evaluate the
effectiveness of searching for multiple targets.

###  7.2 Findings

For a comprehensive comparison of the usefulness and efficiency of the Ktrl+F
plugin with other in-document search systems, we present the results of the
conducted user study in Figure 4. More detailed results are available in
Appendix F.

#### Less search time with Ktrl+F plugin.

As depicted in Figure 4LABEL:sub@fig:user_study_time, Ktrl+F plugin exhibits
the shortest median average time when searching for answers. Its ability to
find multiple semantic targets in a single query, thus reducing the need for
additional searches to verify results, contributes to its superior efficiency
compared to other in-document search systems. It’s worth noting that regular
expressions can also search for multiple targets at once, which can be
considered efficient. However, creating the appropriate regular expression for
a given search intent can be complex and often challenging to debug, as
demonstrated in the example queries for each system in Figure 12.

#### Fewer queries to find targets.

Figure 4LABEL:sub@fig:user_study_num_query illustrates the average number of
queries used per example to find answers. Regular expressions and Ctrl+F are
lexical matching systems that rely on user-generated candidate lexical
prefixes to find answers. Transforming natural language search intent into the
format supported by lexical matching systems increases query usage. While
verifying each query using Ctrl+F is a quick process, users typically cannot
predict which keywords will appear in unknown text prior to reading it all.
Regular expressions can consolidate multiple simple searches into one, but
crafting complex expressions dynamically can be challenging and debugging
erroneous code compounds the difficulty.

#### Fewer visits for extra sources.

The Ktrl+F plugin’s ability to enhance external knowledge beyond the current
web page diminishes the necessity of consulting additional document sources
for result verification, as Figure 4LABEL:sub@fig:user_study_num_visit. The
capability to augment external information beyond the document at hand, such
as accessing Wikipedia page, enables knowledge-augmented in-document search
and results in improved performance (Figure 4LABEL:sub@fig:user_study_score).
When a search intent encompasses multiple targets, as in the query "List all
cities from California from the web page", the need for additional visits to
external sources increases, as demonstrated by the generally higher values
recorded in lexical matching systems.

##  8 Conclusion and Future Work

In this paper, we introduce a new problem Ktrl+F, a knowledge-augmented in-
document search which requires identifying all semantic targets with a single
natural query in real-time. Ktrl+F tackles unique challenge for in-document
search that requires capturing targets containing additional information
beyond the input document by utilizing external knowledge while balancing
between speed and performance. We investigate various baselines for Ktrl+F and
find that existing models have limitations, such as hallucinations, high
latency, or difficulties to incorporate external knowledge. Our experimental
results show that our Knowledge-Augmented Phrase Retrieval, simple extension
of phrase retrieval architecture for in-document search scenarios can be a
robust model for Ktrl+F. Furthermore, results of the study demonstrate that
search experience of users can be enhanced even with our simple model with
seamless access to external knowledge during in-document searches. Future work
can explore how to build an end-to-end trainable architecture that retrieves
external knowledge and integrates it into a searchable index, all while
maintaining real-time processing capabilities. Additionally, Ktrl+F can be
extended to reflect updated knowledge, like news, which cannot be easily
addressed by Large Language Models alone (Ram et al., 2023; Peng et al., 2023;
Kaddour et al., 2023). The scalability and practicality of Ktrl+F will open up
opportunities for various future advancements in the field of information
retrieval and knowledge augmentation.

## Limitations

The system design for Ktrl+F can incorporate various forms of external
knowledge, not limited to the Wikipedia page associated with the entity. It
can also identify a wide range of target spans within the target text,
including dates and numbers, without being restricted to entities. However,
the primary focus of this paper revolves around addressing Ktrl+F,
specifically emphasizing entities as the primary search targets. By narrowing
our focus to entities, we make effective use of entity linking information as
external knowledge. Furthermore, due to the inherent nature of retrieval
systems, our Knowledge-Augmented Phrase Retrieval model requires an extra
indexing stage whenever a change in the input document, which requires
additional time to use. Also it relies on thresholding to truncate predicted
results, which we employ top-k results based on the data distribution in our
experiment. Exploring more efficient methods for enhancing external knowledge
while reducing the time needed for the indexing stage is a potential avenue.

## References

  * Brank et al. (2017) Janez Brank, Gregor Leban, and Marko Grobelnik. 2017.  Annotating documents with relevant wikipedia concepts.  In _SiKDD_.
  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020.  Language models are few-shot learners.  In _NeurIPS_.
  * Chiang et al. (2023) Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023.  Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality.
  * Cohen (1960) Jacob Cohen. 1960.  A coefficient of agreement for nominal scales.  _Educational and Psychological Measurement_.
  * Dasigi et al. (2019) Pradeep Dasigi, Nelson F. Liu, Ana Marasović, Noah A. Smith, and Matt Gardner. 2019.  Quoref: A reading comprehension dataset with questions requiring coreferential reasoning.  In _EMNLP_.
  * Dasigi et al. (2021) Pradeep Dasigi, Kyle Lo, Iz Beltagy, Arman Cohan, Noah A. Smith, and Matt Gardner. 2021.  A dataset of information-seeking questions and answers anchored in research papers.  In _NAACL_.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  Bert: Pre-training of deep bidirectional transformers for language understanding.  In _NAACL_.
  * Ferguson et al. (2020) James Ferguson, Matt Gardner, Hannaneh Hajishirzi, Tushar Khot, and Pradeep Dasigi. 2020.  Iirc: A dataset of incomplete information reading comprehension questions.  In _EMNLP_.
  * Fisch et al. (2019) Adam Fisch, Alon Talmor, Robin Jia, Minjoon Seo, Eunsol Choi, and Danqi Chen. 2019.  MRQA 2019 shared task: Evaluating generalization in reading comprehension.  In _Proceedings of the 2nd Workshop on Machine Reading for Question Answering_.
  * He et al. (2023) Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2023.  Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing.  In _ICLR_.
  * Izacard and Grave (2021) Gautier Izacard and Edouard Grave. 2021.  Leveraging passage retrieval with generative models for open domain question answering.  In _EACL_.
  * Joshi et al. (2017) Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017.  TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension.  In _ACL_.
  * Kaddour et al. (2023) Jean Kaddour, Joshua Harris, Maximilian Mozes, Herbie Bradley, Roberta Raileanu, and Robert McHardy. 2023.  Challenges and applications of large language models.  _arXiv_.
  * Khattab and Zaharia (2020) Omar Khattab and Matei Zaharia. 2020.  Colbert: Efficient and effective passage search via contextualized late interaction over bert.  In _SIGIR_.
  * Kwiatkowski et al. (2019) Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019.  Natural questions: A benchmark for question answering research.  _Transactions of the Association for Computational Linguistics_.
  * Landis and Koch (1977) J Richard Landis and Gary G. Koch. 1977.  The measurement of observer agreement for categorical data.  _Biometrics_.
  * Lee et al. (2023) Hyunji Lee, Jaeyoung Kim, Hoyeon Chang, Hanseok Oh, Sohee Yang, Vladimir Karpukhin, Yi Lu, and Minjoon Seo. 2023.  Nonparametric decoding for generative retrieval.  In _Findings of ACL_.
  * Lee et al. (2021) Jinhyuk Lee, Mujeen Sung, Jaewoo Kang, and Danqi Chen. 2021.  Learning dense representations of phrases at scale.  In _ACL_.
  * Li et al. (2022) Haonan Li, Martin Tomko, Maria Vasardani, and Timothy Baldwin. 2022.  Multispanqa: A dataset for multi-span question answering.  In _ACL_.
  * Lin et al. (2022) Sheng-Chieh Lin, Minghan Li, and Jimmy Lin. 2022.  Aggretriever: A simple approach to aggregate textual representations for robust dense passage retrieval.  _Transactions of the Association for Computational Linguistics_.
  * OpenAI (2022) OpenAI. 2022.  Chatgpt: Optimizing language models for dialogue.
  * OpenAI (2023) OpenAI. 2023.  Gpt-4 technical report.
  * Peng et al. (2023) Baolin Peng, Michel Galley, Pengcheng He, Hao Cheng, Yujia Xie, Yu Hu, Qiuyuan Huang, Lars Liden, Zhou Yu, Weizhu Chen, et al. 2023.  Check your facts and try again: Improving large language models with external knowledge and automated feedback.  _arXiv_.
  * Raffel et al. (2019) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2019.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _JMLR_.
  * Raina et al. (2023) Vatsal Raina, Nora Kassner, Kashyap Popat, Patrick Lewis, Nicola Cancedda, and Louis Martin. 2023.  Erate: Efficient retrieval augmented text embeddings.  In _First Workshop on Insights from Negative Results in NLP_.
  * Rajpurkar et al. (2018) Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018.  Know what you don’t know: Unanswerable questions for SQuAD.  In _ACL_.
  * Rajpurkar et al. (2016) Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016.  Squad: 100,000+ questions for machine comprehension of text.  In _EMNLP_.
  * Ram et al. (2023) Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. 2023.  In-context retrieval-augmented language models.  _TACL_.
  * Santhanam et al. (2022) Keshav Santhanam, Omar Khattab, Jon Saad-Falcon, Christopher Potts, and Matei Zaharia. 2022.  ColBERTv2: Effective and efficient retrieval via lightweight late interaction.  In _NAACL_.
  * Segal et al. (2020) Elad Segal, Avia Efrat, Mor Shoham, Amir Globerson, and Jonathan Berant. 2020.  A simple and effective model for answering multi-span questions.  In _EMNLP_.
  * Touvron et al. (2023) Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. 2023.  Llama 2: Open foundation and fine-tuned chat models.  _arXiv_.
  * Trischler et al. (2017) Adam Trischler, Tong Wang, Xingdi Yuan, Justin Harris, Alessandro Sordoni, Philip Bachman, and Kaheer Suleman. 2017.  NewsQA: A machine comprehension dataset.  In _Proceedings of the 2nd Workshop on Representation Learning for NLP_.
  * Wu et al. (2019) Ledell Yu Wu, Fabio Petroni, Martin Josifoski, Sebastian Riedel, and Luke Zettlemoyer. 2019.  Scalable zero-shot entity linking with dense entity retrieval.  In _EMNLP_.
  * Yu et al. (2023) Wenhao Yu, Dan Iter, Shuohang Wang, Yichong Xu, Mingxuan Ju, Soumya Sanyal, Chenguang Zhu, Michael Zeng, and Meng Jiang. 2023.  Generate rather than retrieve: Large language models are strong context generators.  In _ICLR_.
  * Zhong et al. (2022) Victor Zhong, Weijia Shi, Wen-tau Yih, and Luke Zettlemoyer. 2022.  Romqa: A benchmark for robust, multi-evidence, multi-answer question answering.  _arXiv_.
  * Zhu et al. (2020) Ming Zhu, Aman Ahuja, Da-Cheng Juan, Wei Wei, and Chandan K. Reddy. 2020.  Question answering with long multiple-span answers.  In _EMNLP_.

##  Appendix A Details for Dataset Construction Pipeline

#### Step 3.1. Target Filtering.

In this step, given a (query, entity, external knowledge) triple, we follow
Zhong et al. (2022) to derive whether an entity is an answer to a query or
not. We utilize the first 10 sentences from the Wikipedia article as an
external knowledge, which covers more than 99% of the total sample within
4,096 tokens of GPT-3.5. GPT-3.5 processes a total of 7,060 triple samples,
and the final judgment is made by GPT-4 on 1,226 samples that show different
results from the target generated by LLAMA-2 in Step 2. On average, 1.6
entities disagreed per query, which is an average of 22% of the candidate
entities per query. After the final judgment, queries with all targets
determined to be false are discarded. As a result, 816 queries remained out of
the total 1,000 queries generated by Step 2, and the average number of
entities in a target increased slightly from 1.4 to 1.9.

#### Step 3.2. Query Filtering.

In this step, we exclude a query if the MRC model answers any of the target
entities. The MRC model is considered correct when it scores over 0.9 in F1
score, following the human performance described in Rajpurkar et al. (2018).
As a result, 512 queries were collected from the 816 queries derived in Step
3-1.

##  Appendix B Implementation Details for Baselines

#### Generative baselines.

To convert Ktrl+F as generation problem, we use following instructions for
generative models and then post-process the output text to only utilize the
answer part. We use temparture 0.5, max new token 512.

⬇

Find all mentions from the article

below that correspond to the query.

Only generate mentions with comma

separate.

Article: {Input Document}

Query: {Query}

Mentions:

⬇

You are an expert of query generation for entity search.

You must follow this requirements.

Requirements:

- Your task is to generate queries that retrieve entities in a given list.

- The generated query must be able to list the multiple entities.

- The answers must be countable.

- The answers have to be entities.

- Make sure your questions are unambiguous and based on facts rather than temporal information.

- Do not specify the number in a query.

- Do not start with ’What’ in a query.

- Do not start with ’Which’ in a query.

- Do not include an expression in your query that tells it to find from a given list.

The example is as below.

Generate 4 queries from the following list and extract subset list.

Candidate List:

[Apple, Microsoft, Samsung Electronics, Alphabet, AT&T, Amazon, Verizon
Communications, China Mobile, Walt Disney, Facebook, Alibaba, Intel, Softbank,
IBM, Tencent Holdings, Nippon Telegraph & Tel, Cisco Systems, Oracle, Deutsche
Telkom, Taiwan Semiconductor, KDDi , HP, Legend Holding, Lenovo Group, ebay]

Query:

1. IT companies in Computer Hardware industry

=> Apple, HP, Legend Holding, Lenovo Group

2. Find all IT companies that have software as main business.

=> Microsoft, Oracle

3. Companies that is known for retail service

=> Amazon, Alibaba, ebay

4. Name all IT companies that have license in USA

=> Apple, Microsoft, Alphabet, AT&T, Amazon, Verizon Communications, Walt
Disney, Facebook, Intel, IBM, Cisco Systems, Oracle, HP, ebay

Figure 5: Prompt for generating queries and targets

⬇

You are a QA system to identify the given entity is the answer.

The inputs are entity, query and evidence.

You must follow this requirements.

Requirements:

- Output have to be either ’true’ or ’false’

- Do not say anything except ’true’ or ’false’

The example is as below.

Entity: Google

Query: Find all IT companies in Computer industry

Evidence: Google LLC is an American multinational technology company focusing
on artificial intelligence, online advertising, search engine technology,
cloud computing, computer software, quantum computing, e-commerce, and
consumer electronics. It has often been considered "the most powerful company
in the world" and as one of the world’s most valuable brands due to its market
dominance, data collection, and technological advantages in the field of
artificial intelligence. Its parent company Alphabet is often considered one
of the Big Five American information technology companies, alongside Amazon,
Apple, Meta, and Microsoft.

Output: true

Entity: Samsung

Query: Find all companies in United States

Evidence: Samsung Group, or simply Samsung, is a South Korean multinational
manufacturing conglomerate headquartered in Samsung Town, Seoul, South Korea.
It comprises numerous affiliated businesses, most of them united under the
Samsung brand, and is the largest South Korean chaebol (business
conglomerate). As of 2020, Samsung has the eighth highest global brand value.

Output: false

Figure 6: Prompt for target filtering

(a) The Q1 requests the identification of evidence for each target to evaluate
whether the query satisfies req 2.

(b) The Q2 requests the selection of options to evaluate the naturalness of
the query.

(c) The Q3 requests the selection of targets to evaluate the reliability of
target determination.

Figure 7: User Interface for Human Verification.

[Query] Social network platform of China
---
[Input Document]
It is a highly competitive market with many local competitors who already
understand the shopping habits of the Chinese, which are very different to
those of consumers in the Western world. Chinese platforms such as Taobao and
Tmall dominate the shopping world … successfully. BAIDU DOMINATES ONLINE
Currently, there are an estimated 900 million internet users across China,
with most users spending 1.5 hours a day just browsing. Baidu is the most
popular search engine across China. Think of it as ‘the Google of China’. …
time. Baidu also brings the … social media app across China, it is imperative
that your company becomes familiar with the inner workings of Wechat. Wechat
is fundamentally a social media… in store through Wechat pay are all possible.
Many local and foreign companies already successful in China have official
accounts on Wechat. An official … Wechat wallet. Many companies now also offer
customer service through Wechat. Again, this is highly advisable as this is a
service many Chinese consumers will now look for as it is quick and direct.
Weibo is another popular social media app used across China. Think of Weibo as
‘the Twitter of China’. Weibo is an open network site so users can see posts
from anyone without being their friend or following them. Similar to Twitter,
Weibo can be an excellent way to market your company by sharing the latest
updates, offers, promotions etc. Your followers can also start to share your
content helping your company’s reputation spread by word of mouth.
[Ground Truth] [’Wechat’, ’Wechat’, ’Wechat’, ’Wechat’, ’Wechat’, ’Wechat’,
’Weibo’, ’network site’, ’Weibo’, ’Weibo’, ’app’, ’Weibo’, ’Baidu’, ’Baidu’,
’BAIDU’]
[Query] Find all players who played in Premier League
[Input Document]
McClaren banks on Mitrovic’s influence. LONDON • If he performs half as well
as the players he idolises, Aleksandar Mitrovic will not prove to be much of a
gamble, even at £13 million (S$27.7 million), the fourth-highest incoming
transfer in Newcastle United’s history. The Serbia striker on Tuesday became
the club’s second expensive new arrival this summer, leaving Anderlecht for a
five-year contract at St James’ Park. The arrival of Mitrovic, 20, was hailed
for its significance by Newcastle manager Steve McClaren. Mitrovic scored 23
goals in 44 appearances for the Belgian club last season, including a goal
against Arsenal in the 3-3 Champions League draw at the Emirates. … But
Mitrovic’s signing on top of that of Georginio Wijnaldum, a £14.5 million
addition from PSV Eindhoven, has restored a positive atmosphere at Tyneside.
Only Michael Owen (£17.5 million) and Alan Shearer (£14.7 million) have cost
more. … Moreover, Chancel Mbemba, Mitrovic’s former team-mate in Brussels, is
also expected to join McClaren’s team. Mitrovic is hoping to follow in some
illustrious footsteps. "I am a real No. 9, the penalty box is my place," he
said. "My position is through the middle. I’m like strikers who have played
here before - from Alan Shearer to Papiss (Cisse). They are big names and I
hope I can make a similar impact. "Alan Shearer is a real legend and one of my
heroes. That and (Didier) Drogba is how I like to play and it’s an honour to
play at this club." THE TIMES, LONDON
[Ground Truth] [’Aleksandar Mitrovic’, ’Mitrovic’, ’Mitrovic’, ’Mitrovic’,
’Georginio Wijnaldum’, ’Michael Owen’, ’Alan Shearer’, ’Chancel Mbemba’,
’Mitrovic’, ‘’Mitrovic’, ’Alan Shearer’, ’Papiss’, ’Alan Shearer’, ’(Didier)
Drogba’]
[Query]  Smaller geographical divisions within Monmouth County, New Jersey
[Input Document]
New Jersey family of four was killed and their mansion set on fire, officials
say. (CNN) — A family of four found dead at their mansion in Colts Neck, New
Jersey, were the victims of homicide and their house was subsequently set on
fire, Monmouth County Prosecutor Christopher Gramiccioni said Wednesday. …
Gramiccioni said authorities are investigating a separate house fire in nearby
Ocean Township as arson and are looking into whether the two fires are
related. The other house is the residence of Paul Caneiro, the brother of
Keith Caneiro, Gramiccioni said. … Gramiccioni cautioned against leaping to
conclusions about the fires until authorities investigate further. “Both of
those homes that we are talking about, they’re owned by — under public records
— by common family members,” Grdamiccioni said. … The call for the Colts Neck
fire came in at 12:38 p.m. on Tuesday — more than seven hours after the Ocean
Township fire — and more than 20 fire departments responded. Authorities said
they were not aware of any law enforcement calls to the home before that time.
… A 2007 filing indicates that Keith was the president of the company. The
brothers were best friends who talked almost daily, Honecker said Saturday,
adding he had no reason to believe there was animosity between them.
[Ground Truth] [’Colts Neck’, ’Colts Neck’, ’Monmouth County’, ’Ocean
Township’, ’Colts Neck’, ’Ocean Township’]

Table 5: Example of Ktrl+F evaluation dataset.

#### Extractive baseline.

We solve Ktrl+F using sequence tagging model following (Li et al., 2022). It
can be regarded as a model without utilizing external knowledge. We reproduce
the model trained on MultiSpanQA (Li et al., 2022) for 3 epochs.

Model | Indexing time (Sec) ($\downarrow$) | ms/Q ($\downarrow$) |<EMAIL_ADDRESS>($\uparrow$) |<EMAIL_ADDRESS>($\uparrow$)
---|---|---|---|---
Ours w/ Wikifier | 3.555 | 14 | 0.464 | 0.209
w/o INT | 3.027 | 14 | 0.494 | 0.220
w/o EXT | 3.145 | 14 | 0.335 | 0.153
Ours w/ Gold | 0.955 | 14 | 0.716 | 0.380
w/o INT | 0.912 | 14 | 0.776 | 0.408
w/o EXT | 0.799 | 14 | 0.508 | 0.213

Table 6: MAP metric for retrieval approach. The result shows the effectiveness
of phrase retrieval architecture. When using MAP as a metric, it reflect
retrieved ranks of results and ours show slightly performance drop than ours
w/o internal knowledge.

Model | List EM ($\uparrow$) | Set EM ($\uparrow$) | List Overlap ($\uparrow$) | Set Overlap ($\uparrow$)
---|---|---|---|---
GPT-4 | 30.457 | 36.422 | 37.402 | 51.071
GPT-3.5 | 30.346 | 36.668 | 41.929 | 56.334
LLAMA-2-Chat-7B | 28.529 | 34.235 | 40.546 | 52.843
LLAMA-2-Chat-13B | 28.846 | 35.206 | 37.098 | 51.672
VICUNA-7B-v1.5 | 17.831 | 22.265 | 31.216 | 42.460
VICUNA-13B-v1.5 | 24.490 | 29.223 | 39.278 | 49.449
SequenceTagger | 7.239 | 9.041 | 8.614 | 15.648
Knowledge-Augmented Phrase Retrieval (w/ Wikifier) | 23.152 | 24.793 | 40.718 | 46.841
Knowledge-Augmented Phrase Retrieval (w/ Gold) | 46.170 | 50.254 | 53.689 | 63.230

Table 7: We additionally report Set-based scores with our List-based scores,
which doesn’t necessitate recognizing every target occurrences.

Model | List EM ($\uparrow$) | (R) List EM ($\uparrow$) | List Overlap ($\uparrow$) | (R) List Overlap ($\uparrow$)
---|---|---|---|---
GPT-4 (w/ Gold) | 52.937 | 22.479 | 55.765 | 25.183
GPT-3.5 (w/ Gold) | 44.697 | 22.048 | 56.615 | 35.874
LLAMA-2-Chat-7B (w/ Gold) | 40.225 | 17.738 | 50.466 | 30.140
LLAMA-2-Chat-13B (w/ Gold) | 45.674 | 19.329 | 50.172 | 23.291
VICUNA-7B-v1.5 (w/ Gold) | 27.374 | 8.651 | 41.466 | 21.611
VICUNA-13B-v1.5 (w/ Gold) | 39.898 | 17.065 | 54.695 | 33.814
Knowledge-Augmented Phrase Retrieval (w/ Gold) | 46.170 | 22.426 | 53.689 | 32.285

Table 8: Results for when generative models use candidate entities from input
document as additional input for instruction (denoted as w/ Gold). We evaluate
the results by giving gold entity linking information version.

##  Appendix C Analysis of Retrieval Approach for Ktrl+F

Determining a proper threshold for retrieval is challenging, especially when
the number of targets varies. Therefore, we additionally measure the Mean
Average Precision (MAP), which calculates the mean value per query Q of the
Area under the Curve (AUC) of the precision-recall graph in Table 6. This
metric provides a comprehensive measure of the system’s ability to quantify
the overall effectiveness.

##  Appendix D Prediction Examples

Table 9 shows the results of various approaches on same query and input
document for qualitative analysis.

##  Appendix E Baseline Analysis from Different Perspectives

For a comprehensive baseline understanding, we additionally present set-base
scores which doesn’t require recognizing every target occurrences in Table 7.
We can see the Set Overlap score gets a higher result than List Overlap
overall, and especially generative models show major performance gain in
Overlap score when using Set score, which shows finding all matching target is
hard for generative models. Given that our model leverages entity linking
information to identify targets from a restricted pool of candidates, we
conduct an additional experiment by supplying additional information about
potential targets for generative models (refer to Table 8). When adding extra
information about potential targets for generative models, it proves to
enhance the overall performance of generative models. Notably, in the case of
LLM-API (GPT-4, GPT-3.5), it even outperforms our model with gold-standard
information. However, it’s important to note that enhancing information for
generative models comes with increased costs and slower latency, making it
impractical for real-time applicability.

[Query] Social network platform of China
---
[Input Document]
It is a highly competitive market with many local competitors who already
understand the shopping habits of the Chinese, which are very different to
those of consumers in the Western world. Chinese platforms such as Taobao and
Tmall dominate the shopping world … successfully. BAIDU DOMINATES ONLINE
Currently, there are an estimated 900 million internet users across China,
with most users spending 1.5 hours a day just browsing. Baidu is the most
popular search engine across China. Think of it as ‘the Google of China’. …
time. Baidu also brings the … social media app across China, it is imperative
that your company becomes familiar with the inner workings of Wechat. Wechat
is fundamentally a social media… in store through Wechat pay are all possible.
Many local and foreign companies already successful in China have official
accounts on Wechat. An official … Wechat wallet. Many companies now also offer
customer service through Wechat. Again, this is highly advisable as this is a
service many Chinese consumers will now look for as it is quick and direct.
Weibo is another popular social media app used across China. Think of Weibo as
‘the Twitter of China’. Weibo is an open network site so users can see posts
from anyone without being their friend or following them. Similar to Twitter,
Weibo can be an excellent way to market your company by sharing the latest
updates, offers, promotions etc. Your followers can also start to share your
content helping your company’s reputation spread by word of mouth.
[Ground Truth] [’Wechat’, ’Wechat’, ’Wechat’, ’Wechat’, ’Wechat’, ’Wechat’,
’Weibo’, ’network site’, ’Weibo’, ’Weibo’, ’app’, ’Weibo’, ’Baidu’, ’Baidu’,
’BAIDU’]
GPT-4: [’Wechat’, ’Weibo’]
GPT-3.5: [’Taobao’, ’Tmall’, ’Wechat’, ’Weibo’]
Vicuna-13B-v1.5: [’WeChat (official accounts, WeChat pay, customer service)’,
’Taobao (dominates shopping world)’, ’Tmall (dominates shopping world)’,
’Baidu (most popular search engine, brings WeChat)’,’Weibo (open network,
excellent way to market, Twitter of China)’]
SequenceTagger: [’taobao, ’tmall’]
Ours (w/o INT): [’service’, ’Wechat’, ’Wechat’, ’Wechat’, ’Wechat’, ’Wechat’,
’Wechat’, ’way’, ’Weibo’, ’network site’, ’Weibo’, ’Weibo’, ’app’, ’Weibo’,
’Taobao’, ’Tmall’, ’Twitter’, ’Twitter’, ’China’, ’China’, ’China’, ’China’,
’China’, ’China’, ’China’, ’Baidu’, ’Baidu’, ’BAIDU’, ’Chinese’, ’Chinese’,
’Chinese’]
Ours (w/o EXT): [’Weibo’, ’Weibo’, ’Wechat’, ’Wechat’, ’Weibo’, ’BAIDU’,
’Weibo’, ’Baidu’, ’Wechat’, ’Twitter’, ’Taobao’, ’Tmall’, ’Wechat’, ’Wechat’,
’Baidu’, ’China’, ’Wechat’, ’China’, ’app’, ’Twitter’, ’China’, ’China’,
’Chinese’, ’network site’, ’China’, ’Chinese’, ’China’, ’Chinese’, ’China’,
’way’, ’service’]
Ours: [’Wechat’, ’Wechat’, ’Weibo’, ’Weibo’, ’Wechat’, ’Wechat’, ’Weibo’,
’Wechat’, ’Weibo’, ’Wechat’, ’Taobao’, ’app’, ’network site’, ’Tmall’,
’Twitter’, ’BAIDU’, ’Baidu’, ’service’, ’China’, ’China’, ’Twitter’, ’Baidu’,
’China’, ’way’, ’China’, ’China’, ’China’, ’China’, ’Chinese’, ’Chinese’,
’Chinese’]

Table 9: Prediction result per different approaches. Note that our model uses
thresholding for find proper points per query. In this result we show all
ranking results.

##  Appendix F Details for User Study

We additionally compare existing in-document search systems in Table 10. The
criteria are matching type, ability to search multiple targets, search
intention of the system, and ability to augment external knowledge. Also, we
add examples what queries users use with different in-document search systems
for finding same targets in Table 12.

| Matching Type | Search Mulitple Targets | Search Intention | External Knowledge-Augmented |
---|---|---|---|---|---
Ctrl+F | Lexical | NO | Skimming | Manual |
Regular Expression | Lexical | YES | Skimming | Manual |
MRC | Semantic | YES | After Understanding | NO |
Ktrl+F | Semantic | YES | Skimming | Automatic |

Table 10: Comparing characteristics of Ktrl+F with other systems.

| Time(s) | # of Queries | # of visited Websites | Performance(List EM F1)
---|---|---|---|---
Ctrl+F | 235(248) | 7.47(8) | 3.95(4.12) | 58.64(61.79)
Regular Expression | 265(275) | 3.4(2) | 3.54(4) | 54.31(55.74)
Ktrl+F plugin | 211(217) | 1.41(1.25) | 1.08(1) | 72.70(71.60)

Table 11: Evaluation table for comparing Ktrl+F plugin with other systems. Averaged value is reported and median value are noted within bracket. Search Intetion |  Query per System |  Result
---|---|---
List the cities from California |  Ktrl+F : List the cities from California |  SAN JOSE , Calif. - Paramount to the … they played smarter than they did Sunday in Anaheim, … The Rangers signed 23-year-old defenseman Vince Pedrie out of Penn State, for whom he had 30 points in 39 games this season.
|  Ctrl+F : [San jose, California, Anaheim] |  SAN JOSE , Calif. - Paramount to the … they played smarter than they did Sunday in Anaheim, … The Rangers signed 23-year-old defenseman Vince Pedrie out of Penn State, for whom he had 30 points in 39 games this season.
|  Regex: (SAN JOSE | Calif | Anaheim) |  SAN JOSE , Calif. - Paramount to the … they played smarter than they did Sunday in Anaheim, … The Rangers signed 23-year-old defenseman Vince Pedrie out of Penn State, for whom he had 30 points in 39 games this season.
List all football teams |  Ktrl+F : List all football teams |  LIVERPOOL star Fabinho has been caught on camera appearing to sneeze on Chelsea’s Eden Hazard. Liverpool took back top spot in the Premier League after beating Chelsea at Anfield earlier today. The Reds now have four games … leading Manchester City by … “He’s a fantastic player. Chelsea is …
|  Ctrl+F : [Liverpool, Chelsea, Manchester City] |  LIVERPOOL star Fabinho has been caught on camera appearing to sneeze on Chelsea’s Eden Hazard. Liverpool took back top spot in the Premier League after beating Chelsea at Anfield earlier today. The Reds now have four games … leading Manchester City by … “He’s a fantastic player. Chelsea is …
|  Regex: (LIVERPOOL | Chelsea | Manchester City) |  LIVERPOOL star Fabinho has been caught on camera appearing to sneeze on Chelsea’s Eden Hazard. Liverpool took back top spot in the Premier League after beating Chelsea at Anfield earlier today. The Reds now have four games … leading Manchester City by … “He’s a fantastic player. Chelsea is …
Table 12: The figure above illustrates how each system handles the same search
intention. It is worth noting that Ctrl+F and Regex require additional search
engines to convert natural language search intentions, such as "List the
cities from California," into candidate keywords like "Los Angeles, San Diego,
San Jose, San Francisco, etc." which consist of over a thousand cities.
Moreover, there is no guarantee that these cities will appear on the web page.
The highlighted text in yellow represents potential correct targets based on
the query, while the red indicates possible false negative failures when using
lexical search systems like Ctrl+F and Regex, which need to be highlighted.
Figure 8: The figure demonstrates how to extract entity embedding, which is
used for external knowledge for Knowledge-Augmented Phrase Retrieval. We
utilize the frozen pre-trained phrase retrieval model (Lee et al., 2021),
which shows good at encoding contextual information. The idea of using
concatenated text with title and context and only extracting title embedding
are following (Lee et al., 2023)


# Magnetic-field oscillation model of pulsar radio emission: prediction of an
observable effect

Zhu-Xing Liang1 and Yi Liang2⋆
118-4-102 Shuixiehuadu, Zhufengdajie, Shijiazhuang, Hebei 050035, China
255-1-302 Shuixiehuadu, Zhufengdajie, Shijiazhuang, Hebei 050035, China
<EMAIL_ADDRESS>(LZX<EMAIL_ADDRESS>(LY)

###### Abstract

Since the discovery of pulsars, the rotating-lighthouse model has been the
choice of model to explain the radiation pulsation of pulsars. After
discovering that some main sequence stars (e.g., CU Virginis) and ultracool
dwarfs (e.g., TVLM 513-46546) also behave like pulsars, the lighthouse model
was again adopted to explain their pulse signals. Our research found that if
we use the magnetic-field oscillation (MO) model, we can explain the pulse
radiation results better regardless of whether its source is a neutron star, a
main-sequence star or an ultracool dwarf. We propose a verifiable prediction
that can be used to evaluate the MO model. Our prediction is that there is a
90$\degr$ phase lag between the magnetic field and radio signal of TVLM
513-46546, and the zero-crossing point of the magnetic field is the moment
when the direction of the light’s circular polarization is reversed. No new
observations are needed to check this prediction, but certain existing data
needs to be re-mined.

###### keywords:

polarization – methods: data analysis – pulsars: general – stars: magnetic
fields.

††pagerange: Magnetic-field oscillation model of pulsar radio emission:
prediction of an observable effect–LABEL:lastpage††pubyear: 2002

##  1 Introduction

In 1967, Jocelyn Bell and Antony Hewish discovered the first pulsar. Shortly
thereafter, the rotating-lighthouse model was employed to explain the periodic
radiation of pulsars (Gold, 1968; Radhakrishnan & Cooke, 1969). To date, the
lighthouse model has become the only trusted model. This model suggests that a
pulsar is a rapidly rotating, highly magnetized neutron star, and the magnetic
poles can emit beams with high directionality. These beams of light sweep
through the vast universe like beams from a rotating lighthouse; the pulsar is
seen when the beam sweeps across Earth.

To produce a highly directional beam, an extreme magnetic field, as high as
$10^{8}$ – $10^{12}$ Gauss is required around the neutron star (Lyne & Graham-
Smith, 2006). However, observations in the past 20 years found that some non-
compact stars with weak magnetic fields (thousands of Gauss) can also emit
periodic radiation like pulsars; they include the AP star CU Virginis (Leto et
al., 2006; Lo et al., 2012), the BP star HD 133880 (Das, Chandra & Wade,
2018), the ultracool dwarfs TVLM 513-46546 (Hallinan et al., 2007), LSR
1835+3259 (Hallinan et al., 2008), 2MASSW J0746425+200032 (Berger et al.,
2009), 2MASS J13142039+130011 (McLean et al., 2011), 2MASS J00361617+1821104
(Berger et al., 2005) and WISE J062309.94-045624.6 (Rose et al., 2023). Except
for the difference in the timescale of the pulse period, there is no
significant difference between the radiation signals of these star systems and
that of pulsars. For convenience in later discussions, we call these objects
that emit periodic signals like pulsars but are not neutron stars ‘quasi-
pulsars’.

Although the pulse periods of quasi-pulsars are several hours long, their
pulse profiles, duty cycles, and coherence characteristics are remarkably like
those of pulsars. Given these similarities, several authors have employed the
lighthouse model to explain their pulse signal. Meanwhile, they consider the
underlying radiation mechanism of quasi-pulsars to be coherent electron
cyclotron maser emission (Trigilio et al., 2008; Hallinan et al., 2007).

We previously published a method that can test the lighthouse model (Liang,
Liang & Weisberg, 2014). Using our method, Pol et al. (2018) reported a result
that demonstrated the validity of the lighthouse model. However, when re-
examining their routine, we found that the pulse frequency of PSR J0737-3039B
used by them was 0.3605 Hz, reaching a cumulative error of 0.0039 Hz after 64
accumulations of this low-precision frequency value. This error has far
exceeded the orbital frequency of the binary system (0.000113 Hz), making the
calculation results meaningless. After replacing the low-precision value with
a high-precision value (0.36056035506 Hz), we could not reconstruct their
results. Therefore, to confirm the lighthouse model, a reassessment is needed.
We also found that even using a high-precision frequency value, the three
frequency points (corresponding to prograde rotation, no rotation, or
retrograde rotation) still deviated from the extreme value (maximum) of the
power spectrum, thus making it impossible to draw reliable conclusions. There
are two possible reasons: first, the available data spanned only 344 s long,
whereas the orbital period was as long as 8820 s. To extract the influence of
the orbital frequency, the amount of data may be insufficient. Second, a
centroid correction was required in the calculation, and therefore the
correction procedure may have generated systematic errors. The exact cause
remains to be further investigated by experts.

We have proposed a magnetic-field oscillation model (MO model) (Liang & Liang,
2007) as a competing model to the lighthouse model. Although the MO model
provides concise explanations for most of the radiation features of pulsars,
it was never recognized in the astrophysics community. The emergence of quasi-
pulsars provides new evidence supporting the MO model. Quasi-pulsars not only
provide radio pulse signals but also magnetic field strength signals. The
correlation between these two periodic signals provides additional information
for our study, leading to some advancements. Here, we introduce our new
findings and propose a verifiable prediction for testing the MO model.

##  2 Introduction to the MO Model

Historically, the Schuster-Wilson hypothesis (Schuster, 1911; Wilson, 1923)
proposed that if the rotational speed of a star is high enough, the star can
generate and maintain a magnetic field. However, this hypothesis was refuted
by the anti-dynamo theorem and an experiment (Cowling, 1933; Blackett, 1952).
Our analysis found that both Cowling’s theorem and Blackett’s experiment
negated stable magnetic fields only, but did not negate alternating magnetic
fields. If the requirement of a stable magnetic field is replaced by that of
an alternating magnetic field, the Schuster-Wilson hypothesis may be
resurrected.

A recognized fact is that the oscillation period of the solar magnetic field
is 22 years. Regardless of its mass, if the Sun were to collapse and become a
neutron star, during the collapse, its magnetic-field oscillation may not stop
but instead quicken. Its period may shorten to seconds or even milliseconds,
and finally become a pulsar. This is the genesis of the MO model.

The MO model uses four basic assumptions:

  1. 1.

The flip oscillation is a common attribute of the magnetic fields of all
stars. First, the solar magnetic field exhibits oscillatory behavior. Second,
Babcock (1951, 1958) reported several stars with a magnetic period of
approximately one week. After a thorough study of those stars, he hypothesized
that their magnetic fields undergo oscillations. On these observations, we
believe that flipping oscillations may be a common attribute of the magnetic
fields of all stars.

  2. 2.

The oscillation frequency of the magnetic field is positively correlated with
its rotational frequency. Neutron stars are formed by the collapse of main
sequence stars. With the conservation of angular momentum, their rotational
speeds are inevitably much higher than that of the Sun. Their pulse
frequencies (pulse arrival rate) are also much higher than the oscillation
frequency of the Sun’s magnetic field. Therefore, we assume that the
oscillation frequency is positively correlated with the rotational frequency,
but they are no longer equal.

  3. 3.

The strength of the stellar magnetic field is positively correlated with its
oscillation frequency. The reported pulse frequencies of the quasi-pulsars are
higher than the oscillation frequency of the solar magnetic field, and their
magnetic field strengths are also higher than that of the Sun. This may be a
universal law.

  4. 4.

A magnetic field oscillation induces a oscillating electric field, which, at
its peak, produces a coherent radiation pulse. The higher the induced electric
field, the higher the electron acceleration; therefore, coherent radiation is
more favored to occur.

When a dipole magnetic field oscillates, a doughnut-shaped radiation region is
formed parallel to the equatorial plane. In the radiation region, an
alternating electric field is established around the star. This electric field
accelerates electrons. At every location at the same altitude, the electric
field strength is the same, therefore, the highly ordered distribution of
magnetic field, electric field, electron speed, and electron acceleration
creates the best conditions for coherent radiation.

Based on the second and third assumptions above, the induced electric field is
negatively correlated with the period of oscillation _P_. Therefore, both the
acceleration and speed of the electrons are also negatively correlated with
_P_. That is, quasi-pulsars feature long periods and low electron speeds,
whereas pulsars exhibit short periods and high electron speeds.

According to current particle radiation theory, cyclotron radiation occurs
with low electron speeds whereas synchrotron radiation occurs with high
electron speeds. Therefore, quasi-pulsars with lower electron speeds produce
cyclotron radiation, whereas pulsars produce synchrotron radiation.

###  2.1 Cyclotron radiation of quasi-pulsars

Without considering coherent radiation, the characteristic of cyclotron
radiation is that the radiation parallel to the magnetic axis is fully
circularly polarized whereas the radiation perpendicular to the magnetic axis
is fully linear polarized, with the circular-polarized light being twice as
strong as the linear-polarized light. In the MO model, the notion of magnetic
inclination angle is not needed, and the rotation axis is always aligned with
the magnetic axis. The equatorial plane where the pulse radiation occurs is
like a huge planar radar antenna. This antenna lets coherent light concentrate
along the rotation axis and forms a light beam with high directionality. If
our line-of-sight coincides with the rotation axis, we see very bright
circular polarized light. In this process, the coherence property is of great
significance for the beam’s directionality.

If our line-of-sight is parallel to the equatorial plane, we see fully linear-
polarized light. However, the radiation near the equatorial plane is only
energy that leaks around the antenna and is not directional. The radiation
flux is dispersed uniformly in all directions, encompassing a full 360$\degr$.
Hence, only a small fraction of light is emitted towards Earth. As a result,
only circular-polarized light is easily observed from quasi-pulsars.

###  2.2 Synchrotron radiation of pulsars

Figure 1: Polarization and brightness from synchrotron radiation under the MO
model. The radiation region is toroidal in shape parallel to the equatorial
plane. Only the flux inside the radiation cone can reach Earth. The radiation
cone does not co-rotate with the neutron star.

According to the MO model and synchrotron radiation theory, the radiation
region is toroidal in shape lying symmetrically to the equatorial plane. The
radiation flux is concentrated in a circular space with a field angle of
2$\alpha$ (Fig. 1); the axial flux disappears.

In the MO model, the trajectory plane of the electron motion is always
parallel to the equatorial plane. With this premise, all the observational
effects listed below can be inferred from the theory of synchrotron radiation.

  1. 1.

The centre of the radiation cone has the highest flux density, whereas that of
the edge is lowest.

  2. 2.

When the line-of-sight is parallel to the equatorial plane, fully linear-
polarized light is observed.

  3. 3.

When the line-of-sight deviates from the equatorial plane, namely
$\alpha>\beta\neq 0$, elliptical-polarized light is observed, here, $\beta$ is
the angle between the line of sight and the equatorial plane, as shown in
Figure 1. The larger the angle $\beta$, the higher the circular polarization
fraction and the lower the flux observed.

Based on the above three points, a statistical law can be derived: brighter
pulsars exhibit a higher degree of linear polarization.

  4. 4.

Only a part of the radiation region (the visible radiation region) contributes
to the observed flux; the remainder propagates in other directions. The ratio
of the visible radiation region and the entire radiation region equals
$\varphi/360$, where $\varphi$ is the visible radiation angle in Fig. 1.

  5. 5.

The higher the electron speed, the smaller the half-cone angle $\alpha$ and
the visible radiation angle $\varphi$.

  6. 6.

The alternating electric field results in an alternating electron speed, and
hence the angles $\alpha$ and $\varphi$ expand and contract with period _P_.
If $\beta\neq 0$, what we see is elliptically polarized light. Because the
angle $\alpha$ determines all polarization characteristics, including the
polarization position angle (PA), when $\alpha$ changes, PA also changes. The
reversal of the magnetic-field direction may be the reason for the jump of PA.

  7. 7.

The hollow radiation cone described in the lighthouse model can be explained
with the MO model: a dip in the synchrotron radiation occurs when the magnetic
field crosses zero.

  8. 8.

The fluctuation in the density of electrons in the visible radiation region
creates the complexity in the pulsar signal. Because the rotation period is
not equal to the magnetic period, and the electron density in the visible
radiation region may fluctuate with the rotation period, the coupling between
the two periods may cause pulse null, drift, and even quasi-period patterns
(beat frequency).

##  3 USING THE MO MODEL TO EXPLAIN THE POLARIZATION CHARACTERISTICS

Each pulsar and quasi-pulsar have distinct linear and circular polarization
fractions. To study the relative contributions of these polarization
fractions, Oswald et al. (2023) introduced a helpful metric termed the
‘circular contribution’, $\theta$. According to their definition, $\theta$ is
the magnitude of the angle of latitude on the Poincaré sphere. Fully linear
polarization corresponds to $\theta$ = 0 whereas fully circular polarization
corresponds to $\theta=90\degr$.

Oswald et al. (2023) investigated the relationship between $\theta$ and radio
frequency. However, we consider the relationship between $\theta$ and pulse
period _P_ is also noteworthy. If the MO model is correct, based on the
analysis above, $\theta$ should be positively correlated statistically with
_P_ ; specifically, the longer the pulse period, the larger the circular
contribution $\theta$. To verify this relationship, we plotted the available
data of pulsar and quasi-pulsar in Fig. 2. The data from eight quasi- pulsars
are from relational references cited in the Introduction. The data of 1097
normal pulsars are taken from Oswald et al. (2023) and Posselt et al. (2023);
any overlapping data between the two sources were merged. The data of 189
millisecond pulsars are taken from Spiewak et al. (2022). The periods _P_ were
obtained from the ATNF pulsar catalogue (ATNF
https://www.atnf.csiro.au/research/pulsar/psrcat) (Manchester et al., 2005)
and most of the $\theta$ values are calculated from the definition:

$\theta=\arctan\frac{|V|}{L}$ (1)

where $|V|$ is the absolute circular polarization fraction, $L$ is the linear
polarization fraction.

From Fig. 2, we note four observations:

  1. 1.

All quasi-pulsars exhibit fully circular polarization, with no exceptions so
far. All of them thus lie in the upper-right corner region.

  2. 2.

The overall trend of the whole graph is that $\theta$ is positively correlated
with _P_.

  3. 3.

Within the normal pulsar population, $\theta$ is positively correlated with
_P_. This result is in line with our expectations. Based on the third
assumption in Section 2, we make the following inference: period long
$\rightarrow$ magnetic field weak $\rightarrow$ induced electric field weak
$\rightarrow$ electron speed low. Then according to the theory of synchrotron
radiation, electron speed low $\rightarrow$ $\alpha$ large $\rightarrow$
circularly polarized fraction high $\rightarrow$ $\theta$ large. Thus, we
expect a positive correlation between $\theta$ and _P_.

  4. 4.

Millisecond pulsars are special, possibly because they are not driven purely
by rotational energy; accretion (or past accretion) may have had an impact on
the $\theta-P$ relationship.

Roughly speaking, the pulsars are indeed in the lower-left corner of Fig. 2
and the $\theta-P$ relationship is consistent with the expectations of the MO
model.

The relationship between $\theta$ and the rate of change in period $\dot{P}$
was plotted in Fig. 3.

Although $\dot{P}$ and spin-down energy $\dot{E}$ are derived from the
lighthouse model, they are also important parameters in the MO model. The
lighthouse model and the MO model are based on the same assumption: the energy
of electromagnetic radiation comes from the rotational energy.

Under the lighthouse model, $\dot{E}=4\pi^{2}I\dot{P}P^{-3}$ (Posselt et al.
2023); here, _P_ is the rotation period and _I_ the estimated moment of
inertia.

In the MO model, _P_ is the electromagnetic oscillation period rather than the
rotation period, and the rotation period is unknown. Therefore, we cannot
directly calculate $\dot{E}$ using the above formula. If we use a positive
correlation coefficient to convert the magnetic period into a rotational
period (according to the second assumption in Section 2), we can calculate
$\dot{E}$ using the same formula again.

In both models, $\dot{P}$ and $\dot{E}$ are related to the power of energy
conversion (the rotational energy is converted into electromagnetic radiation
energy). High $\dot{P}$ and $\dot{E}$ mean that the energy and speed of the
electrons are also high. According to the analysis in Section 2.2, the higher
the electron speeds, the lower the value of $\theta$. The justification is:
high $\dot{P}$ $\rightarrow$ high $\dot{E}$ $\rightarrow$ high power of energy
conversion $\rightarrow$ high speed of electrons $\rightarrow$ low $\theta$,
which simply answers the question why is $\theta$ negatively correlated with
$\dot{P}$ as shown in Fig. 3.

The MO model clearly explains the relationships (Figs. 2 and 3), which should
be considered as evidence in its favor. Although this result cannot completely
refute the lighthouse model, the explanation provided by the MO model is
simple and unmatched by the lighthouse model.

Desvignes et al. (2019) provide evidence for the lighthouse model. However,
their results simultaneously provide evidence for the MO model. We infer from
Fig. 1 that in a binary system, if the rotation axis of the pulsar precesses,
the radiation cone may gradually leave our line-of-sight and disappear. Before
disappearing, the circular polarization fraction inevitably and gradually
strengthens, whereas the linear polarization gradually weakens. The results of
Desvignes et al. (2019) help to confirm this trend (see panels A and C in Fig.
4 in their paper). Those two panels show the evolution of both linear and
circular polarizations before the disappearance of the main pulse (MP) of PSR
J1906+0746. They perfectly confirmed the inference of the MO model and
provided support for the MO model.

The trend lines in Figs. 2 and 3 are linear fits. For the normal pulsars, the
correlation coefficient r between $\theta$ and _P_ is 0.16, and that between
$\theta$ and $\dot{P}$ is -0.1. Two weak correlations imply that the influence
factors of $\theta$ may be very complex. Perhaps, factors such as the radius
of the star, temperature, and charge density can all affect the speed of
electrons, thereby affecting $\theta$ and the correlation coefficients.

Figure 2: Correlation between circular contribution $\theta$ and pulse period
_P_. Figure 3: Correlation between circular contribution $\theta$ and the
derivative of period $\dot{P}$.

##  4 A PREDICTION: THERE IS A $90\degr$ PHASE LAG BETWEEN THE RADIO AND
MAGNETIC FIELD SIGNALS OF TVLM 513-46546

Figure 4: Phase relationship predicted by us and based on the MO model.

If the oscillating magnetic field of a star is sinusoidal, the induced
electric field and radio radiation reach their peaks at the zero-crossing
point of the magnetic field. Therefore, the radio signal lags the magnetic
field by a $90\degr$ phase; this phase lag is an important feature of the MO
model. The MO model was proposed in 2004 and first made public at the
$36^{th}$ COSPAR scientific assembly in Beijing in July 2006. At the time, we
gave no attention to reports on quasi-pulsars. Later, we found that three
quasi-pulsars indeed exhibited a $90\degr$ phase lag and therefore provided
potent support for the MO model. The three quasi-pulsars were CU Virginis
(Leto et al., 2006), 2MASSW J0746425+200032 (Berger et al., 2009), and HD
133880 (Das, Chandra & Wade, 2018).

Do all quasi-pulsars show a $90\degr$ phase lag? This requires more
investigation. We suggest that the investigation starts with TVLM 513-46546
because this does not require new observations. Hallinan et al. (2007)
observed periodic radio radiation signals from TVLM 513-46546. Berger et al.
(2008) acquired its periodic changes in magnetic field, and Wolszczan & Route
(2014) determined its period, $P=7054.468\pm 0.007$ s. The only remaining task
is to extrapolate the magnetic field curve back to approximately 335 days and
then use period _P_ to superimpose the magnetic and radio curves (as in Fig.
4).

Based on the MO model, we predict that point A in Fig. 4 is both the zero
point of the magnetic field and the point where the direction of radio
circular polarization is reversed.

If this prediction is confirmed, the $90\degr$ phase lag may be a universal
law of quasi-pulsars, which would provide fresh support for the MO model. At
the same time, the inversion of the direction of polarization (PL and PR in
Fig. 4) can be explained simply by the MO model. The direction of the magnetic
field has reversed, so the direction of circular polarization has also
reversed.

Of course, even if this prediction is confirmed by subsequent investigations,
it only adds evidence favoring the MO model but does not imply a complete
substantiation for this model. If additional assumptions can be added, the
lighthouse model may also be viable in explaining the results. Therefore, the
decisive method remains that proposed by Liang, Liang & Weisberg (2014).

##  5 Discussion

Desvignes et al. (2019) reported fascinating results from their study of PSR
J1906+0746 and provided staunch support for the lighthouse model. However,
they modeled the system using at least seven parameters, which is inevitably
worrisome. Using too many parameters sometimes reduces the credibility of the
fitted results.

We acknowledge that the MO model is nascent. Even if correct, there are still
many issues that need further research. We present below a few unsolved
problems and preliminary thoughts.

Under the MO model framework, the magnetic field, electric field, and electron
speed distribution are highly ordered, which creates conditions most favorable
for coherent radiation. However, why the radiation occurs only when the
electric field is strongest and the magnetic field is near its weakest remains
unsolved.

The radiation mechanism can no longer be explained simply by cyclotron or
synchrotron radiation because only the Lorentz force affects the electrons in
these two processes. In the MO model, besides the Lorentz force, the induced
electric field can force electrons to relativistic speeds within a few
milliseconds. Therefore, it is appropriate to consider this radiation process
as completely novel. Does this impact the discussions above and the prediction
made in Section 4? This prediction was formed after considering this electric
field, and the emergence of this alternating electric field and the novel
radiation process is a necessary condition for the $90\degr$ phase lag.
Therefore, the new radiation process does not impact this prediction. The
addition of this electric field also does not impact our discussion of the
polarization given above because the polarization is determined by the
trajectory plane of the electron motion, and the added electric field does not
change this plane (here the equatorial plane). The addition of an alternating
electric field does affect for example the radio frequency spectrum, pulse
phase, breathing-like changes in the radiation cone, and coherence
characteristics.

The magnetic field curves reported contain both alternating and persistent
components (Berger et al., 2008; Babcock, 1951). The zero point of the
magnetic field in Fig. 4 only corresponds to the zero point of the alternating
component, with no consideration of the persistent component. Our preliminary
thought is that the alternating component may represent the dipole magnetic
field of the star, whereas the persistent component may be a mixing effect
generated by local magnetic fields on the star’s surface. Periodic coherent
radiation may occur at a distance from the surface of the star where the
dipole magnetic field dominates.

The MO model admits magnetic periods of some stars to be close to millisecond
level. If correct, we must abandon fossil magnetic field models and various
dynamo models that rely on internal turbulence. The new dynamo model must
generate purely electromagnetic oscillations driven by rotational energy. The
period depends on the distributed capacitance, inductance, and rotational
speed of the star. The smaller the volume and the faster the star rotates, the
higher the oscillation frequency. Other influencing factors include
temperature, stellar wind flux, and relativistic effects, among others.

If the MO model is ultimately confirmed, future pulsars might be divided into
four types: hour pulsar (non-compact star), minute pulsar (white dwarf),
normal pulsar (neutron star), and millisecond pulsar (recycled pulsar).

## Acknowledgements

We would like to thank the anonymous reviewer for the helpful comments. We
thank Richard Haase, PhD, from Liwen Bianji (Edanz)
(https://www.liwenbianji.cn) for editing the English text of a draft of this
manuscript.

## Data availability

All data sources have been listed in the text.

## References

  * Babcock (1951) Babcock, H.W., 1951, ApJ, 114, 1
  * Babcock (1958) Babcock, H.W., 1958, ApJ, 128, 228
  * Berger et al. (2005) Berger, E., et al., 2005, ApJ, 627(2), 960
  * Berger et al. (2008) Berger, E., et al., 2008, ApJ, 673, 1080
  * Berger et al. (2009) Berger, E., et al., 2009, ApJ, 695, 310
  * Blackett (1952) Blackett, P. M. S., 1952, RSPTA, 245, 309
  * Cowling (1933) Cowling, T.G., 1933, MNRAS, 94, 39
  * Das, Chandra & Wade (2018) Das, B., Chandra, P., Wade, G.A., 2018. MNRAS, 474(1), L61
  * Desvignes et al. (2019) Desvignes G., et al., 2019, science, 365,1013
  * Gold (1968) Gold, T., 1968., Nature, 218(5143), 731
  * Hallinan et al. (2007) Hallinan, G., et al., 2007, ApJ, 663(1), L25
  * Hallinan et al. (2008) Hallinan, G., Antonova, A., Doyle, J.G., Bourke, S., Lane, C., Golden, A., 2008, ApJ, 684(1), 644
  * Leto et al. (2006) Leto, P., Trigilio, C., Buemi, C. S, Umana, G., Leone, F., 2006, A&A 458(3), 831
  * Liang & Liang (2007) Liang, Z.X., Liang, Y., 2007, arXiv preprint arXiv:0709.4315
  * Liang, Liang & Weisberg (2014) Liang, Z.X., Liang, Y., Weisberg, J.M., 2014, MNRAS, 439(4), 3712
  * Lo et al. (2012) Lo, K.K., et al., 2012., MNRAS, 421(4), 3316
  * Lyne & Graham-Smith (2006) Lyne, A., Graham-Smith, F., 2006, Pulsar astronomy, Cambridge University Press
  * Manchester et al. (2005) Manchester, R. N., Hobbs, G.B., Teoh, A. & Hobbs, M., 2005, AJ, 129, 1993
  * McLean et al. (2011) McLean, M., Berger, E., Irwin, J., Forbrich, J., Reiners, A., 2011, ApJ, 741(1), 27
  * Oswald et al. (2023) Oswald, L.S., et al., 2023, MNRAS 520(4), 4961
  * Pol et al. (2018) Pol, N., McLaughlin, M., Kramer, M., Stairs, I., Perera, B.B., Possenti, A., 2018, ApJ, 853(1), 73
  * Posselt et al. (2023) Posselt, B., et al., 2023, MNRAS, 520, 4582
  * Radhakrishnan & Cooke (1969) Radhakrishnan, V., Cooke, D. J., 1969, ApJ, 3, L225
  * Rose et al. (2023) Rose, K., et al., 2023, ApJ, 951, L43
  * Schuster (1911) Schuster, A., 1911, Proc. Phys. Soc. London, 24, 121
  * Spiewak et al. (2022) Spiewak, R., et al., 2022, PASA, 39, e027
  * Trigilio et al. (2008) Trigilio, C., Leto, P., Umana, G., Buemi, C. S., Leone, F., 2008, MNRAS, 384(4), 1437
  * Wilson (1923) Wilson, H. A., 1923, Proc. Roy. Soc. London, A, 104, 451
  * Wolszczan & Route (2014) Wolszczan, A., Route, M., 2014, ApJ, 788(1), 23