#  CooK: Empowering General-Purpose Language Models with Modular and
Collaborative Knowledge

Shangbin Feng1, Weijia Shi1, Yuyang Bai2, Vidhisha Balachandran3, Tianxing
He1, Yulia Tsvetkov1
1University of Washington 2Xi’an Jiaotong University 3Carnegie Mellon
University
<EMAIL_ADDRESS>
###### Abstract

Large language models (LLMs) are increasingly adopted for knowledge-intensive
tasks and contexts. Existing approaches improve the knowledge capabilities of
general-purpose LLMs through retrieval or generated knowledge prompting, but
they fall short of reflecting two key properties of knowledge-rich models:
knowledge should be _modular_ , ever-growing, sourced from diverse domains;
knowledge acquisition and production should be a _collaborative_ process,
where diverse stakeholders contribute new information. To this end, we propose
CooK, a novel framework to empower general-purpose large language models with
modular and collaboratively sourced knowledge. We first introduce _specialized
language models_ , autoregressive models trained on corpora from a wide range
of domains and sources. These specialized LMs serve as parametric knowledge
repositories that are later prompted to generate background knowledge for
general-purpose LLMs. We then propose three knowledge filters to dynamically
select and retain information in generated documents by controlling for
_relevance_ , _brevity_ , and _factuality_. Finally, we propose bottom-up and
top-down knowledge integration approaches to augment general-purpose LLMs with
the curated (relevant, factual) knowledge from community-driven specialized
LMs that enable multi-domain knowledge synthesis and on-demand knowledge
requests. Through extensive experiments, we demonstrate that CooK achieves
state-of-the-art performance on six benchmark datasets. Our results highlight
the potential of enriching general-purpose LLMs with evolving and modular
knowledge – relevant knowledge that can be continuously updated through the
collective efforts of the research community.

##  1 Introduction

Large language models (LLMs) have demonstrated an impressive ability to encode
world knowledge in model parameters (Petroni et al., 2019; Roberts et al.,
2020). However, they still face various challenges in knowledge-intensive
tasks and contexts: they suffer from hallucination (Kryściński et al., 2020;
Pagnoni et al., 2021; Ji et al., 2023), struggle to encode long-tail facts
(Kandpal et al., 2022; Mallen et al., 2022), and could not be easily updated
with new and emerging knowledge (De Cao et al., 2021; Hase et al., 2021).
Existing works propose addressing these limitations through retrieval
augmentation or generated knowledge prompting. _Retrieval-augmented LMs_ (Guu
et al., 2020; Borgeaud et al., 2022; Shi et al., 2023) employ retrieval
systems to fetch relevant documents from a general and fixed retrieval corpus
(e.g., Wikipedia or the Pile (Gao et al., 2020)), leveraging external
knowledge from non-parametric sources to aid LLM generation. _Generated
knowledge prompting_ approaches (Shin et al., 2020; Liu et al., 2022a; Sun et
al., 2022) prompt LLMs to incorporate and generate contextual documents to
encourage knowledge-aware generation.

While these two lines of research have achieved some success, they struggle to
reflect two key properties of knowledge. Knowledge is _modular_
(Stuckenschmidt et al., 2009): it is an “archipelago” rather than a single
“continent”, encapsulating information that exist in diversified forms,
domains, sources, perspectives, and more. The lack of knowledge modularity has
made generalization to new domains and targeted updates of knowledge stored in
LMs difficult. Knowledge is _collaborative_ (Cayzer, 2004): LLMs should be
able to represent and incorporate diverse and evolving knowledge, from multi-
faceted sources and perspectives, while enabling collaborative contribution
from various stakeholders. Community-driven knowledge could aggregate new
knowledge from domain experts and enable the development of specialized LLMs,
tailored to specific industries or applications. That being said, neither
retrieval LMs nor generated prompting employ _modular_ or _collaborative_
knowledge sources that enable the plug-and-play updates and contributions from
various stakeholders. Instead, they only use static knowledge sources, such as
Wikipedia dumps at a fixed date for retrieval augmentation or the LLM itself
as the knowledge repository in generated knowledge prompting. These static
knowledge sources often fail to capture evolving domain-specific or structured
knowledge and do not facilitate community-driven efforts to enhance LLMs’
knowledge abilities.

To this end, we propose CooK, a novel framework to empower general-purpose
LLMs with modular and Collaborative Knowledge through the integration of
smaller, but specialized language models. As an increasing amount of powerful
LLMs are released behind API calls, not directly accessible, and are
prohibitively expensive to train or adapt, CooK specifically focuses on
augmenting black-box LLMs to enrich their knowledge capabilities. We first
curate _specialized LMs_ trained on corpora from diverse sources and domains
to serve as modular knowledge repositories (§2.1). Compared to using retrieval
or LLM itself as knowledge sources, specialized LMs enable flexible and
targeted information access, searching over domains, and employing private and
personalized knowledge sources. These specialized LMs are later prompted to
generate background information to support general-purpose LLMs. We then
propose three levels of _knowledge filters_ to dynamically select and refine
generated documents and control for topic relevance, document brevity, and
knowledge factuality (§2.2). Finally, we propose _bottom-up_ and _top-down_
—two approaches to empower general-purpose LLMs by integrating outputs from
specialized LMs (§2.3). Specifically, the _bottom-up_ approach starts by
prompting all specialized LMs to generate multiple documents, then performs
selection with the three knowledge filters, while concatenating the final
knowledge paragraph with the query for LLM generation. While the bottom-up
approach uniquely enables multi-domain knowledge synthesis, it also presents
the risk of presenting irrelevant information to LLM in contexts where
external information is not needed. This motivates us to propose the _top-
down_ approach, where the general-purpose LLM itself decides whether external
knowledge is necessary for the given query, then relevant specialized LMs are
selectively activated for knowledge integration; this process is repeated
until the general-purpose LLM has enough confidence to generate a response.

Extensive experiments demonstrate that CooK outperforms vanilla LLMs,
retrieval-augmented LMs, and generated prompting approaches on three tasks
across six datasets. For _general-purpose knowledge QA_ , CooK improves Codex
performance by 6.6% on MMLU and even outperforms the 3-times larger Flan-PaLM.
For _misinformation analysis_ that tests multi-domain knowledge integration,
CooK outperforms all baseline approaches by at least 15.8% and 10.0% balanced
accuracy scores on two- and four-way classification settings. In the third
task, to evaluate the ability to update the knowledge of general-purpose LLMs,
we curate MidtermQA, a QA dataset focusing on the 2022 U.S. midterm elections
while the knowledge cutoff of LLMs is generally 2021 or earlier. Experiments
demonstrate that CooK outperforms all baselines by at least 55.6% on exact
match scores, showcasing the ability for temporal knowledge update while only
adding one specialized LM trained on midterm election news with 100x fewer
parameters than the general-purpose LLM. Our findings demonstrate the
potential of empowering general-purpose LLMs by integrating modular and
collaborative knowledge from small, independently trained, and specialized
LMs. We envision CooK as an initiative to encourage LM developers to
collaborate in expanding the knowledge of large language models while reducing
the carbon footprint from retraining gigantic LMs from scratch.

##  2 Methodology

We introduce CooK, a novel framework to empower general-purpose LLMs with
modular and collaborative knowledge (Figure 1). We train various specialized
LMs on knowledge corpora from diversified domains and sources (§2.1). We then
use them to produce background knowledge for the general-puporse LLMs, while
employing three _knowledge filters_ to ensure quality in knowledge synthesis
(§2.2). Finally, we propose _bottom-up_ and _top-down_ , two approaches to
condition the LLM on the content sourced from specialized LMs and post-
processed using the knowledge filters (§2.3).

###  2.1 Specialized Language Models

While existing approaches rely on one fixed source of knowledge to improve
LLMs (one retrieval corpus (Guu et al., 2020; Borgeaud et al., 2022; Shi et
al., 2023), one knowledge graph (Wang et al., 2021; Zhang et al., 2022a; Feng
et al., 2022a), or one pretrained LLM itself (Shin et al., 2020; Liu et al.,
2022a; Sun et al., 2022)), we hypothesize that since knowledge is _modular_ ,
general-purpose LLMs should be augmented with modular plug-and-play knowledge
repositories that allow users to collaboratively add, remove, edit, or update
information. In addition, different communities might have different
definitions and requirements for knowledge. Wikipedia factoids, biomedical
literature, mathematical formulae, and commonsense knowledge graphs are all
valuable knowledge components in various contexts, thus LLMs should be able to
represent and incorporate knowledge contributed by stakeholders across multi-
faceted domains and industries.

To this end, we propose to curate specialized LMs that are much smaller than
black-box LLMs, trained on diversified knowledge corpora from a wide range of
domains and sources. Concretely, we obtain $n$ specialized LMs
$\mathcal{C}=\\{\boldsymbol{c}_{1},\boldsymbol{c}_{2},\cdots,\boldsymbol{c}_{n}\\}$,
each starting from an existing LM checkpoint and further trained on a specific
knowledge corpora $\mathcal{D}_{i}$ with the causal language modeling
objective. Given a query to the LLM, these specialized LMs are selectively
activated and used with prompted generation. Formally, given the query
$\boldsymbol{q}$, specialized LM $\boldsymbol{c}$ defines a mapping
$\boldsymbol{c}(\boldsymbol{q}):\boldsymbol{q}\rightarrow\boldsymbol{d}_{\boldsymbol{q}}$
where $\boldsymbol{q}$ is used as prompt to generate a continuation as the
knowledge document $\boldsymbol{d}_{\boldsymbol{q}}$, which are later
prepended into the context of general-purpose LLMs through various mechanisms
(§2.3).

In this way, the modularity of knowledge is demonstrated through the
effortless addition, removal, or selective activation of various specialized
LMs during the LLM generation process. Similarly, the collaborative nature of
knowledge is reflected by enabling individuals to contribute trained
specialized LMs on their desired knowledge source to CooK, expanding the
knowledge of general-purpose LLMs through community-driven efforts.

Figure 1: Overview of CooK. We train specialized LMs on various knowledge
domains and employ three knowledge filters for quality control. We propose
_bottom-up_ and _top-down_ to integrate general-purpose LLMs with modular and
specialized LMs to enable multi-domain knowledge synthesis (_bottom-up_) and
prompt LLMs to proactively seek external knowledge (_top-down_).

###  2.2 Knowledge Filters

While it is possible to directly adopt $\boldsymbol{d}_{\boldsymbol{q}}$ as
relevant knowledge, we identify three key challenges in the successful
integration of modular specialized LMs and general-purpose LLMs: relevance,
brevity, and factuality. We design three respective filters to control for
such factors.

#### Relevance Filter

While we expect specialized LMs to generate background information that is
relevant and helpful to the query $\boldsymbol{q}$, LMs sometimes deviate from
the query (Holtzman et al., 2019). Furthermore, only a handful of specialized
LMs would be relevant for a given query. To this end, we propose to select and
retain knowledge documents based on relevance. Concretely, given a set of $m$
generated documents $\\{\boldsymbol{d}_{1},\cdots,\boldsymbol{d}_{m}\\}$ and
the query $\boldsymbol{q}$, we aim to retain the top-$k$ relevant documents
and discard irrelevant information. We adopt a separate encoder-based LM
$\mathrm{enc(\cdot)}$ that maps a token sequence to a feature vector and
cosine similarity $\mathrm{sim}(\cdot,\cdot)$ to measure relevance. Formally,
we retain $\boldsymbol{d}_{i}$ if $i\in\mathrm{top\mathchar 45\relax
k}_{j}(\mathrm{sim}(\mathrm{enc}(\boldsymbol{d}_{j}),\mathrm{enc}(\boldsymbol{q})))$
where $\mathrm{top\mathchar 45\relax k}$ is the top-$k$ $\mathrm{argmax}$
operation.

#### Pruning Filter

Existing works mostly integrate one piece of external knowledge into LLMs (Sun
et al., 2022; Shi et al., 2023), while tasks requiring integration of multiple
domains of information, such as misinformation detection (Karimi et al., 2018)
and multi-hop QA (Nishida et al., 2019), are not well supported by existing
paradigms. To effectively incorporate generated documents from multiple LMs
while fitting into the LLM context length limit, we propose to prune knowledge
documents. Formally, given $m$ documents
$\\{\boldsymbol{d}_{1},\cdots,\boldsymbol{d}_{m}\\}$, we adopt a pruning model
$\mathrm{prune}(\cdot)$, operationalized most simply as a summarization system
(Zhang et al., 2020; Liu et al., 2022b), to obtain the condensed versions
separately
$\\{\tilde{\boldsymbol{d}_{1}},\cdots,\tilde{\boldsymbol{d}_{m}}\\}$. This
pruning method allows for the integration into the main LLM of information
from multiple domains while preserving space for in-context learning.

#### Factuality Filter

Language models are prone to hallucination (Ji et al., 2023) and the
specialized LMs are no exception. Given a set of $m$ pruned knowledge
documents
$\\{\tilde{\boldsymbol{d}_{1}},\cdots,\tilde{\boldsymbol{d}_{m}}\\}$, their
original versions $\\{\boldsymbol{d}_{1},\cdots,\boldsymbol{d}_{m}\\}$, and
the query $\boldsymbol{q}$, we filter out the non-factual knowledge and retain
$\ell$ documents. Specifically, we evaluate the factuality of knowledge
documents with two measures.

We first evaluate _summarization factuality_ , ensuring that the pruned
version $\tilde{\boldsymbol{d}_{i}}$ factually captures the important points
in the original $\boldsymbol{d}_{i}$. Concretely, we adopt factuality
evaluation models (Kryściński et al., 2020; Feng et al., 2023a) as a scoring
function $\mathrm{sum\mathchar 45\relax fact}(\cdot,\cdot)$, where each
knowledge document $\boldsymbol{d}$ is assigned a summarization factuality
score $s_{\boldsymbol{d}}^{\textit{sum}}=\mathrm{sum\mathchar 45\relax
fact}(\tilde{\boldsymbol{d}}\mid\boldsymbol{d})\in[0,1]$.

We then propose to evaluate whether the generated knowledge document is well-
supported by real-world knowledge through _retrieval-augmented fact checking_.
Specifically, given a knowledge document $\boldsymbol{d}$, we retrieve $k$
documents from a retrieval corpus
$\boldsymbol{t}_{1},\ldots,\boldsymbol{t}_{k}$, then employ a fact-checking
model (Schuster et al., 2021) as a scoring function $\mathrm{fact\mathchar
45\relax check}(\cdot,\cdot)$. We then assign a fact-checked factuality score
to each $\boldsymbol{d}$ based on the retrieved document that _most_ supports
$\boldsymbol{d}$, formally
$s_{\boldsymbol{d}}^{\textit{fact}}=\mathrm{max}_{1\leq i\leq k}\
\mathrm{fact\mathchar 45\relax
check}(\boldsymbol{d}\mid\boldsymbol{t}_{i})\in[0,1]$. We then average the
summarization factuality score and the fact-checking score for each document
to obtain $s_{\boldsymbol{d}}$.

While it is straightforward to greedily select $\ell$ knowledge documents with
the highest $s_{\boldsymbol{d}}$ scores, new and more recent knowledge might
not be well-supported by existing fact checking tools. As a result, we propose
_top- $k$ factuality sampling_ to allow for flexibility while remaining
stringent towards knowledge documents that are clearly wrong. Formally, we
first obtain $\mathcal{D}^{k}$ as the set of knowledge documents with the
top-$k$ factuality scores where $k>\ell$ is a hyperparameter. We then define a
sampling probability distribution over all $m$ knowledge documents:

$\displaystyle
p(\tilde{\boldsymbol{d}_{i}}\mid\boldsymbol{q})=\begin{cases}\mathrm{exp}(s_{\boldsymbol{d}_{i}})/\sum_{\boldsymbol{d}_{j}\in\mathcal{D}^{k}}\mathrm{exp}(s_{\boldsymbol{d}_{j}}),&\text{if
$\tilde{\boldsymbol{d}_{i}}\in\mathcal{D}^{k}$.}\\\ \mathrm{0},&\text{if
$\tilde{\boldsymbol{d}_{i}}\notin\mathcal{D}^{k}$.}\end{cases}$

We sample $\ell$ knowledge documents from
$\\{\tilde{\boldsymbol{d}_{1}},\cdots,\tilde{\boldsymbol{d}_{m}}\\}$ with
probabilities
$\\{p(\tilde{\boldsymbol{d}_{1}}\mid\boldsymbol{q}),\cdots,p(\tilde{\boldsymbol{d}_{m}}\mid\boldsymbol{q})\\}$.
In this way, knowledge documents with very low factuality scores are strictly
removed while flexibility is built in through sampling from the knowledge with
factuality scores near the top.

###  2.3 Knowledge Integration

After defining the modular components in CooK (a general-purpose LLM,
specialized LMs and knowledge filters), we propose two approaches, _bottom-up_
and _top-down_ , to integrate the general-purpose LLM with external knowledge
sources, which are filtered outputs of our specialized LMs. Specifically,
_bottom-up_ activates all available specialized LMs at once and employs the
three knowledge filters to control for knowledge quality. _Bottom-up_ enables
multi-domain knowledge synthesis across all available sources, but these might
occasionally introduce irrelevant information which may adversely impact LLM
inference. To this end, we additionally propose a _top-down_ approach, in
which the LLM proactively seeks external information from selected specialized
LMs. _top-down_ is advantageous in tasks and domains where external knowledge
is not always necessary.

#### Bottom-Up Approach

_Bottom-up_ starts by prompting available specialized LMs, then progressively
goes through the three knowledge filters, and these outputs are incorporated
into the LLM via the prompt context. Formally, given $n$ specialized LMs
$\mathcal{C}=\\{\boldsymbol{c}_{1},\cdots,\boldsymbol{c}_{n}\\}$ and the query
$\boldsymbol{q}$, we generate $n_{1}$ documents with each specialized LM
through temperature sampling (Holtzman et al., 2019) to obtain
$\\{\boldsymbol{d}_{1},\cdots,\boldsymbol{d}_{n\times n_{1}}\\}$. We first
apply the relevance filter to retain $n_{2}$ most relevant documents
$\\{\boldsymbol{d}_{1},\cdots,\boldsymbol{d}_{n_{2}}\\}$, then conduct
knowledge pruning through the pruning filter
$\\{\tilde{\boldsymbol{d}_{1}},\cdots,\tilde{\boldsymbol{d}_{n_{2}}}\\}$, and
finally leverage the factuality filter to obtain $n_{3}$ high-quality
knowledge documents
$\\{\tilde{\boldsymbol{d}_{1}},\cdots,\tilde{\boldsymbol{d}_{n_{3}}}\\}$.

The final prompt for the LLM is a concatenation of knowledge documents and the
query, formally [“Knowledge: ”
$\parallel\tilde{\boldsymbol{d}_{1}}\parallel\tilde{\boldsymbol{d}_{2}}\parallel\cdots\parallel\tilde{\boldsymbol{d}_{n_{3}}}\parallel\boldsymbol{q}$]
where $\parallel$ denotes concatenation. We expect the bottom-up approach to
be strong in multi-domain knowledge synthesis since multiple specialized LMs
could be activated at once to provide background knowledge from diverse
perspectives. In addition, hyperparameters $n_{1}$, $n_{2}$, and $n_{3}$
enable fine-grained control over the knowledge synthesis process.

#### Top-Down Approach

In _bottom-up_ , we assume that every query would benefit from external
knowledge generated by specialized LMs. However, this could introduce
unnecessary information in the LLM’s prompt context (Zhao et al., 2022).
Following Kadavath et al. (2022), who showed that LLMs possess preliminary
abilities to identify their inherent knowledge limitations, we propose the
_top-down_ approach, putting the LLM in charge to iteratively identify whether
external knowledge is needed and selectively activate relevant specialized LMs
through various strategies.

Concretely, for the $n$ specialized LMs
$\mathcal{C}=\\{\boldsymbol{c}_{1},\cdots,\boldsymbol{c}_{n}\\}$, we also ask
the specialized LM contributors to submit a textual description of LMs
$\mathcal{S}=\\{\boldsymbol{s}_{1},\cdots,\boldsymbol{s}_{n}\\}$ such as
“biomedical literature”, “college calculus”, or “commonsense knowledge graph”.
We first ask the LLM a yes/no question to determine whether external knowledge
is needed for the given query $\boldsymbol{q}$, specifically “Do you need more
information? (Yes or No)”. We encourage better-calibrated answers to the
yes/no question through in-context learning (Wei et al., ; Press et al.,
2022): specifically, we introduce a set of in-context learning examples that
encompass two distinct categories of questions posed to the LLM. The first
category consists of questions that the LLM is capable of answering accurately
without the need for any extra information. For these questions, the response
to the query “Do you need more information (Yes or No)?” is “No.” The second
category comprises questions that the LLM cannot answer correctly without the
provision of additional information. In this case, the corresponding output
label for the query is “Yes.” In this way, we prompt the LLM to learn to
request external knowledge through in-context learning; we analyze the
effectiveness of this approach in Section 5. If the LLM answers “No”, we
directly prompt the LLM to generate based on the query, without resorting to
specialized LMs. If the LLM requests external knowledge by answering “Yes”, we
employ two strategies (Algoithm 2) to select a relevant specialized LM and
generate background knowledge.

  * •

Automatic Selection (auto) We further prompt the LLM with “What kind of
information do you need?” and select one specialized LM based on its response
$\boldsymbol{r}_{\boldsymbol{q}}$. Concretely, we identify which LM
description $\\{\boldsymbol{s}_{1},\cdots,\boldsymbol{s}_{n}\\}$ is most
relevant to $\boldsymbol{r}_{\boldsymbol{q}}$ with the relevance filter (§2.2)
and activate the corresponding LM to generate multiple knowledge documents,
then select one with the highest factuality score based on the factuality
filter (§2.2) to obtain $\boldsymbol{d}$.

  * •

Explicit Selection (exp) Alternatively, we ask the LLM to directly select one
specialized LM by prompting with “$\textit{Choose an information source from
the following: }\boldsymbol{s}_{1},\ldots,\boldsymbol{s}_{n}$”. If the LLM
responds with $\boldsymbol{s}_{i}$, we activate the corresponding specialized
LM $\boldsymbol{c}_{i}$ to generate multiple knowledge documents and select
one with the factuality filter (§2.2) to obtain $\boldsymbol{d}$.

Upon obtaining the document, we append “$\textit{Knowledge: }\boldsymbol{d}$”
to the LLM context. We then iteratively ask “Do you need more information?
(Yes or No)” again, repeat the above process, until the LLM answers “No” and
generates a knowledge-informed response. We expect _top-down_ to perform
better when external knowledge is not always necessary. In this way, the top-
down approach enables LLMs to take charge in identifying their inherent
knowledge limitations and seeking help from external specialized LMs
proactively. We provide prompt examples in Tables 9 and 10 in the Appendix.

##  3 Experiment Settings

#### Implementation

For _specialized LMs_ , we use OPT-1.3B (Zhang et al., 2022b) as the starting
point and separately train 25 distinct specialized LMs on a wide range of
knowledge sources and domains, including corpora in the Pile (Gao et al.,
2020), branch-train-merge (Li et al., 2022), knowledge graphs (Speer et al.,
2017; West et al., 2022; Vrandečić and Krötzsch, 2014; Pellissier Tanon et
al., 2020; Feng et al., 2021; Zhang et al., 2022a), news and social media (Liu
et al., 2022c; Feng et al., 2023b), and more. (Appendix E) We use MPNet (Song
et al., 2020) as the encoder in the _relevance filter_ , Pegasus (Zhang et
al., 2020) as the summarization model in the _pruning filter_ , the WikiSearch
API as the retrieval system in the _factuality filter_ , and FactKB (Feng et
al., 2023a) and VitaminC (Schuster et al., 2021) as the summarization and
fact-checking factuality scoring functions. We use Codex (code-davinci-002)
(Chen et al., 2021) as the default, general-purpose, black-box LLM.

Type | Model | Human. | Social | STEM | Other | All
---|---|---|---|---|---|---
Vanilla LM | Codex | 74.2 | 76.9 | 57.8 | 70.1 | 68.3
PaLM | 77.0 | 81.0 | 55.6 | 69.6 | 69.3
Flan-PaLM | - | - | - | - | 72.2
Retrieval | Atlas | 46.1 | 54.6 | 38.8 | 52.8 | 47.9
RePlug | 76.0 | 79.7 | 58.8 | 72.1 | 71.4
RePlug LSR | 76.5 | 79.9 | 58.9 | 73.2 | 71.8
Generate | GKP | 73.3 | 74.5 | 59.5 | 71.4 | 70.0
Recitation | 76.9 | 78.1 | 59.0 | 74.0 | 71.9
CooK | Bottom-Up | 77.2 | 76.7 | 57.9 | 72.2 | 70.7
Top-Down auto | 77.7 | 78.9 | 59.2 | 73.0 | 72.0
Top-Down exp | 78.6 | 80.9 | 59.6 | 74.3 | 72.8

Table 1: Model performance on the MMLU Benchmark. CooK improves Codex by at
least 3.5% while _top-down_ outperforms all baselines.

Type | Model | Two-Way | Four-Way
---|---|---|---
BAcc | MaF | BAcc | MaF
Vanilla LM | Codex | 65.6 | 51.0 | 52.8 | 44.0
Retrieval | RePlug | 78.8 | 67.8 | 55.8 | 53.0
RePlug LSR | 78.8 | 68.5 | 57.5 | 54.4
Generate | GKP | 73.5 | 60.3 | 61.1 | 46.3
Recitation | 65.0 | 47.7 | 64.2 | 48.6
GRTR | 66.1 | 49.1 | 51.6 | 36.9
CooK | Bottom-Up | 89.8 | 87.3 | 70.6 | 67.3
Top-Down auto | 86.4 | 78.7 | 63.0 | 60.2
Top-Down exp | 91.3 | 86.0 | 69.4 | 65.5

Table 2: Performance on misinformation detection. BAcc and MaF are balanced
accuracy and macro F1. _bottom-up_ performs best due to multi-domain knowledge
integration.

Type | Model | Open-Book | Multiple-Choice
---|---|---|---
EM | F1 | 2-way | 4-way
Vanilla LM | Codex | 55.1 | 57.9 | 90.9 | 60.8
Retrieval | RePlug | 44.8 | - | 85.7 | 62.8
RePlug LSR | 37.2 | - | 86.9 | 65.3
Si et al. | 52.1 | 54.5 | 84.7 | 61.4
Generate | GKP | 45.0 | 46.9 | 89.1 | 53.5
Recitation | 44.4 | 46.4 | 89.3 | 52.3
GRTR | 55.6 | 58.4 | 77.4 | 59.0
CooK | Bottom-Up | 83.6 | 85.6 | 81.6 | 64.5
Top-Down auto | 87.5 | 89.3 | 89.5 | 63.0
Top-Down exp | 75.3 | 75.7 | 91.9 | 67.6

Table 3: Performance on MidtermQA. CooK successfully updates the knowledge of
Codex by adding a single specialized LM.

#### Tasks and Datasets

1) For _general-purpose QA_ , we adopt MMLU (Hendrycks et al., 2020), a
multiple-choice QA dataset covering 57 tasks in humanities, STEM, social
sciences, and others. Following previous works (Si et al., 2022; Shi et al.,
2023), we adopt a 5-shot in-context learning setting. 2) To evaluate _multi-
domain knowledge synthesis_ , we adopt misinformation detection, since news
articles often encompass facts and opinions at the intersection of different
domains and perspectives. We leverage the widely adopted LUN misinformation
detection dataset (Rashkin et al., 2017) with both 2-way and 4-way
classification settings. All models are evaluated based on 16-shot in-context
learning. 3) To evaluate _temporal knowledge update_ , we curate MidtermQA, a
QA benchmark focusing on the 2022 U.S. midterm elections since the knowledge
cutoff of black-box LLMs is often 2021 or earlier. MidtermQA presents three
evaluation datasets and settings: open-book, 2-way, and 4-way multiple choice.
5-shot in-context learning is adopted to evaluate CooK and baselines. We did
not consider existing temporal QA datasets (Dhingra et al., 2022; Kasai et
al., 2022) since they do not focus on any specific event or knowledge domain.

#### Baselines

We compare CooK with a wide range of baseline methods in three categories. 1)
vanilla black-box LLMs: Codex (Chen et al., 2021), PaLM (Chowdhery et al.,
2022), and Flan-PaLM (Chung et al., 2022); 2) generated knowledge prompting
approaches: GKP (Liu et al., 2022a), recitation (Sun et al., 2022), GRTR (Yu
et al., 2022) (Note that we apply these methods to the same LLM Codex (Chen et
al., 2021) for a fair comparison); 3) retrieval-augmented language models:
Atlas (Izacard et al., 2022), Si et al. (2022), RePlug, and RePlug LSR (Shi et
al., 2023).

##  4 Results

#### MMLU

For general-purpose knowledge QA, we use the MMLU benchmark (Hendrycks et al.,
2020). As shown in Table 1, all three configurations of CooK significantly
improves vanilla Codex. Among them, the top-down approach with explicit
selection performs best, improving Codex by 6.6% overall accuracy.
Concurrently, top-down approaches surpasses all baselines, including Flan-PaLM
with a few hundred billion more parameters. This results suggest that we
present an effective approach for making general-purpose LLMs better in
knowledge-intensive contexts. In addition, _top-down_ generally outperforms
_bottom-up_ likely because that MMLU contains math related questions that do
not necessitate external knwoledge. This observation suggests that _top-down_
approaches are better at tasks where external knowledge is not always
necessary.

#### Misinformation Detection

To examine whether CooK successfully integrates multi-faceted knowledge from
diversified sources, we adopt the LUN misinformation dataset (Rashkin et al.,
2017) with two- and four-way classification settings. Table 2 demonstrates
that CooK significantly improves Codex by at least 31.7% and 19.4% in balanced
accuracy scores for both settings. In addition, _bottom-up_ outperforms both
variants of _top-down_ , thanks to its methodology to jointly activate
specialized LMs from various domains and enable multi-domain knowledge
synthesis.

#### MidtermQA

To examine whether CooK could update the parametric knowledge of LLMs, we
train an additional specialized LM on news articles regarding the 2022 U.S.
midterm elections and plug it into CooK. We present model performance on
MidtermQA in Table 3, which demonstrates that CooK substantially outperforms
all baselines in the open-book setting by as much as 57.3% in exact match
scores (EM). This indicates that one specialized LM with 1.3B parameters
successfully updates the parametric knowledge of the 175B Codex through CooK,
showcasing its potential for efficient knowledge update. CooK also outperforms
Si et al. (Codex + Contriever) that uses the same midterm election news as
retrieval corpora. In addition, generated knowledge prompting approaches (GKP,
recitation, GRTR) underperform vanilla Codex, showing that probing LLMs for
explicit knowledge is counterproductive when internal LLM knowledge is
outdated or wrong.

Figure 2: Performance on misinformation detection when each specialized LM is
separately added. CooK enables modular patching of LLMs while in-domain LMs
help the most.

Figure 3: Ablation study of the three knowledge filters on misinformation
detection. While the three filters all contribute to model performance, the
_factuality_ filter is most crucial.

##  5 Analysis

#### Patching LLM Knowledge

When general-purpose LLMs struggle at tasks due to their knowledge
limitations, CooK could serve as an efficient approach to patch LLM weaknesses
by adding modular and specialized language models. To this end, we evaluate
the change in performance when five specialized LMs are separately added to
augment Codex with the top-down approach. Results in Figure 3 demonstrate that
patching the LLM with all five LMs leads to various levels of performance
gains on misinformation detection, while the most in-domain LMs (Wikipedia and
news) lead to greater improvements. This suggests that when LLMs perform
poorly on knowledge-intensive tasks, an additional specialized LM trained on
in-domain corpora could help thanks to CooK.

Model | Two-Way | Four-Way
---|---|---
BAcc | MaF | BAcc | MaF
RePlug | 78.8 | 67.8 | 55.8 | 53.0
RePlug LSR | 78.8 | 68.5 | 57.5 | 54.4
Bottom-Up | 90.0 | 87.0 | 65.3 | 63.3
Top-Down auto | 80.7 | 70.9 | 60.1 | 56.8
Top-Down exp | 80.6 | 70.0 | 59.7 | 56.5

Table 4: CooK outperforms sota retrieval LM RePlug in the Wikipedia-only
setting, suggesting that modular LMs present a better knowledge repository
than retrieval.

#### Knowledge Filter Study

In Section 2.2, we propose three levels of knowledge filters to control for
various factors and ensure knowledge quality. We conduct ablation studies to
remove each knowledge filter in the bottom-up approach and re-evaluate on
misinformation detection. Figure 3 demonstrates that while all three knowledge
filters are helpful, the factuality filter contributes most to model
performance and thus plays a crucial role in ensuring the quality of generated
knowledge documents.

#### Retrieval vs. Specialized LMs

In order to assess the effectiveness of modular specialized LMs as compared to
non-parametric sources like retrieval,, we exclusively use the Wikipedia LM in
CooK and compare with the state-of-the-art retrieval LM RePlug that also uses
Wikipedia as the retrieval knowledge source. Table 4 demonstrates that CooK
outperforms RePlug on both settings of misinformation detection, suggesting
that modular and specialized LMs present a better knowledge repository. CooK
is compatible with multiple knowledge formats while they could be
complementary. (Appendix A)

Figure 4: Investigating the impact of $n_{1}$, $n_{2}$, and $n_{3}$, which
govern the knowledge stream from modular specialized LMs to general-purpose
LLMs. These hyperparameters enable fine-grained control over the knowledge
synthesis process.

Figure 5: CooK is compatible with other general-purpose LLMs, specifically
text-davinci-003 and gpt-3.5-turbo.

#### Knowledge Stream Analysis

In _bottom-up_ , three hyperparameters (§2.3) govern the “knowledge stream”
from modular specialized LMs to the general-purpose LLMs. Specifically,
$n_{1}$ controls how many documents each LM generates, $n_{2}$ controls how
many are retained after the three knowledge filters, and $n_{3}$ controls how
many are put into the context of LLMs. We investigate these control measures
and report model performance in Figure 5. It is illustrated that: 1) $n_{1}$
has a marginal impact, suggesting that specialized LMs generate largely
homogeneous knowledge even with temperature sampling (Caccia et al., 2018); 2)
larger $n_{2}$ leads to performance drops, suggesting that the three knowledge
filters ensure knowledge quality; 3) $n3=1$, where only one knowledge document
is adopted at a time (as in previous works (Sun et al., 2022; Shi et al.,
2023)) is worse than larger values, showing the advantage of multi-domain
knowledge synthesis uniquely enabled by CooK.

Figure 6: Confusion matrices of yes/no and correctness in _top-down_ ,
enabling fine-grained error analysis.

#### LLM Compatibility

While we follow previous works (Sun et al., 2022; Shi et al., 2023) and adopt
Codex as the default black-box LLM, CooK is compatible with different models.
We additionally evaluate CooK with two other LLMs, text-davinci-003 and
gpt-3.5-turbo, and present results in Figure 5, which shows that both _bottom-
up_ and _top-down_ consistently improve different LLMs across various datasets
and evaluation metrics.

Race | Codex | CooK
---|---|---
AL, senate | Doug Jones ✗ | Katie Britt ✓
PA, senate | Bob Casey ✗ | John Fetterman ✓
CA, 3rd | Mike Thompson ✗ | Kevin Kiley ✓
IN, 2nd | Jackie Walorski ✗ | Jim Banks ✗
NV, governor | Steve Sisolak ✗ | Joe Lombardo ✓

Table 5: While vanilla Codex falsely claims that these incumbents won again in
the 2022 elections, CooK successfully updates the knowledge of black-box LLMs.

#### Yes/No in _Top-Down_

In the top-down approach (§2.3), we begin by asking LLMs if they might need
external knowledge for the given query and adopt in-context examples to
encourage well-calibrated answers. We illustrate LLM’s responses along with
the correctness of their answer in Figure 6. The vast majority of queries are
mapped to the “yes, correct” and “no, correct” categories, suggesting that
LLMs have preliminary abilities to “know what they know” and seek external
information if necessary. However, this ability is far from perfect, evident
in the non-negligible category of “no, incorrect”, suggesting that prompting
LLMs to acknowledge knowledge limitations require further research (Kadavath
et al., 2022; Zhao et al., 2022), while new approaches to abstain could be
easily integrated into CooK. In addition, the “yes, incorrect” categories
suggest that specialized LMs occasionally fail to provide enough information.
In summary, these confusion matrices provide fine-grained error analysis and
guidance as to whether the general-purpose LLM, the yes/no question, or
specialized LMs require further improvements.

#### Qualitative Analysis

We curated MidtermQA to evaluate whether CooK enables efficient knowledge
update. We examine the 88 races where the incumbent was not re-elected: Codex
answered 1 out of the 88 questions correctly, while _bottom-up_ and _top-down_
with automatic and explicit selection answered 63, 77, and 42 correctly. Table
5 shows that Codex states the incumbents would win again in 2022, while CooK
successfully updates the knowledge of LLMs with 100x more parameters.

##  6 Related Work

#### Retrieval-Augmented Language Models

Augmenting language models with retrieval has become a promising direction,
advancing the state-of-the-art in open-domain QA (Guu et al., 2020; Izacard et
al., 2022; Lewis et al., 2020; Hu et al., 2022), text classification (Zhao et
al., 2022), and language modeling (Hu et al., 2022; Borgeaud et al., 2022; Min
et al., 2022). The retrieval system could be integrated into encoder-decoder
(Izacard et al., 2022) and decoder-only models (Borgeaud et al., 2022; Shi et
al., 2022; Rubin et al., 2022), or leveraged to interpolate the next token
probability distributions (Khandelwal et al., 2019; Zhong et al., 2022).
Recent advances incorporated frozen (Mallen et al., 2022; Si et al., 2022;
Khattab et al., 2022) and trainable retrievers (Shi et al., 2023) as well as
search engines (Press et al., 2022) to augment LLMs. Compared to retrieval
models and search engines, CooK enables flexible information seeking,
searching over knowledge domains, and employing private knowledge sources. In
addition, these works often leverage only _one_ retrieval corpora and assume
that it’s “omniscient” while suffering from various issues such as domain
coverage and knowledge update. In contrast, we propose to reflect the
modularity and community-driven nature of knowledge by integrating plug-and-
play specialized LMs with general-purpose LLMs.

#### Generated Knowledge Prompting

Language models acquire a considerable amount of knowledge through training on
gargantuan textual corpora (Petroni et al., 2019; Dhingra et al., 2022; He et
al., 2021). Generated knowledge prompting (Liu et al., 2022a) is one of the
early approaches to tap into the parametric knowledge of LLMs by prompting
them to generate background information and re-using it for question
answering. Related works also propose to use LM parametric knowledge for
retrieval (Tay et al., 2022), answer commonsense questions with self-talk
(Shwartz et al., 2020), generate queries (Wang et al., 2022; Zhuang et al.,
2022) or token sequences (Bevilacqua et al., 2022) for document augmentation.
In addition, recitation-augmented language models (Sun et al., 2022) propose
to augment QA examples with diversified knowledge recitations, while (Yu et
al., 2022) shows that generated knowledge prompting approaches are, under
certain circumstances, better than retrieval-augmented models. However, this
line of work assumes that the encoded knowledge in LLM parameters is all we
need, while LLM knowledge suffers from hallucination (Ji et al., 2023),
struggles to encode long-tail facts (Mallen et al., 2022), and can not be
efficiently updated (De Cao et al., 2021). While recent works propose to edit
and update LLM knowledge (Meng et al., 2022; Hernandez et al., 2023), they are
hardly compatible with black-box LLMs. In addition, parametric knowledge in
LLMs is far from modular and collaborative, while language models should be
able to represent and incorporate knowledge and information contributed by all
stakeholders in LLM research and applications. To this end, we propose CooK as
a community-driven initiative to empower general-purpose LLMs with modular and
collaborative knowledge through the sharing and re-using of specialized LMs.

#### Modular Language Models

Mixture-of-Experts (MoE) (Masoudnia and Ebrahimpour, 2014) aims to activate
only one expert network based on the input instance, which has been adopted in
language model research (Gururangan et al., 2022; Roller et al., 2021; Lewis
et al., 2021; Kudugunta et al., 2021; Pfeiffer et al., 2022). Adapters,
modular plug-ins for the LM architecture, are also proposed for task transfer
and parameter-efficient fine-tuning (Houlsby et al., 2019; Pfeiffer et al.,
2020; Zaken et al., 2022). In addition, parameter averaging (Matena and
Raffel, 2022; McMahan et al., 2017; Izmailov et al., 2018; Wortsman et al.,
2022; Li et al., 2022; Gururangan et al., 2023), model fusion (Don-Yehiya et
al., 2022; Borzunov et al., 2022), and other collaborative approaches (Köpf et
al., 2023; Sha, 2023) have also shed light on the possibility of distributed
training of large language models. However, existing proposals for modular
language models only operate in the white-box setting, _i.e._ assuming full
access to the model parameters, token probabilities, and more. Since
increasingly powerful LLMs are only released as a black box and behind API
calls, we propose CooK as a modular solution with the aim of empowering black-
box general-purpose LLMs with community-driven and collaborative knowledge.

##  7 Conclusion

We propose CooK, a novel framework to empower general-purpose LLMs with
modular and collaborative knowledge. We first present specialized language
models trained on various domains and sources of knowledge, and propose three
knowledge filters to ensure knowledge quality. We then propose _bottom-up_ and
_top-down_ approaches to integrate modular specialized LMs with general-
purpose LLMs to enable multi-domain knowledge synthesis and grounding in
external information when necessary. Extensive experiments demonstrate that
CooK outperforms vanilla LLMs, retrieval LMs, and generated knowledge
prompting approaches across three tasks and six datasets, showcasing its
ability to integrate multiple sources of information, efficiently update LLM’s
knowledge, and more. We envision CooK as a community-driven initiative to
empower general-purpose LLMs with modular and collaborative knowledge.

## References

  * Petroni et al. (2019) Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller.  Language models as knowledge bases?  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 2463–2473, 2019.
  * Roberts et al. (2020) Adam Roberts, Colin Raffel, and Noam Shazeer.  How much knowledge can you pack into the parameters of a language model?  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 5418–5426, 2020.
  * Kryściński et al. (2020) Wojciech Kryściński, Bryan McCann, Caiming Xiong, and Richard Socher.  Evaluating the factual consistency of abstractive text summarization.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 9332–9346, 2020.
  * Pagnoni et al. (2021) Artidoro Pagnoni, Vidhisha Balachandran, and Yulia Tsvetkov.  Understanding factuality in abstractive summarization with frank: A benchmark for factuality metrics.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 4812–4829, 2021.
  * Ji et al. (2023) Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung.  Survey of hallucination in natural language generation.  _ACM Computing Surveys_ , 55(12):1–38, 2023.
  * Kandpal et al. (2022) Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace, and Colin Raffel.  Large language models struggle to learn long-tail knowledge.  _arXiv preprint arXiv:2211.08411_ , 2022.
  * Mallen et al. (2022) Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Hannaneh Hajishirzi, and Daniel Khashabi.  When not to trust language models: Investigating effectiveness and limitations of parametric and non-parametric memories.  _arXiv preprint arXiv:2212.10511_ , 2022.
  * De Cao et al. (2021) Nicola De Cao, Wilker Aziz, and Ivan Titov.  Editing factual knowledge in language models.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 6491–6506, 2021.
  * Hase et al. (2021) Peter Hase, Mona Diab, Asli Celikyilmaz, Xian Li, Zornitsa Kozareva, Veselin Stoyanov, Mohit Bansal, and Srinivasan Iyer.  Do language models have beliefs? methods for detecting, updating, and visualizing model beliefs.  _arXiv preprint arXiv:2111.13654_ , 2021.
  * Guu et al. (2020) Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang.  Retrieval augmented language model pre-training.  In _International conference on machine learning_ , pages 3929–3938. PMLR, 2020.
  * Borgeaud et al. (2022) Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George Bm Van Den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, et al.  Improving language models by retrieving from trillions of tokens.  In _International conference on machine learning_ , pages 2206–2240. PMLR, 2022.
  * Shi et al. (2023) Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Rich James, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih.  Replug: Retrieval-augmented black-box language models.  _arXiv preprint arXiv:2301.12652_ , 2023.
  * Gao et al. (2020) Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, et al.  The pile: An 800gb dataset of diverse text for language modeling.  _arXiv preprint arXiv:2101.00027_ , 2020.
  * Shin et al. (2020) Taylor Shin, Yasaman Razeghi, Robert L Logan IV, Eric Wallace, and Sameer Singh.  Autoprompt: Eliciting knowledge from language models with automatically generated prompts.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 4222–4235, 2020.
  * Liu et al. (2022a) Jiacheng Liu, Alisa Liu, Ximing Lu, Sean Welleck, Peter West, Ronan Le Bras, Yejin Choi, and Hannaneh Hajishirzi.  Generated knowledge prompting for commonsense reasoning.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 3154–3169, 2022a.
  * Sun et al. (2022) Zhiqing Sun, Xuezhi Wang, Yi Tay, Yiming Yang, and Denny Zhou.  Recitation-augmented language models.  _arXiv preprint arXiv:2210.01296_ , 2022.
  * Stuckenschmidt et al. (2009) Heiner Stuckenschmidt, Christine Parent, and Stefano Spaccapietra.  _Modular ontologies: concepts, theories and techniques for knowledge modularization_ , volume 5445.  Springer, 2009.
  * Cayzer (2004) Steve Cayzer.  Semantic blogging and decentralized knowledge management.  _Communications of the ACM_ , 47(12):47–52, 2004\.
  * Wang et al. (2021) Xiaozhi Wang, Tianyu Gao, Zhaocheng Zhu, Zhengyan Zhang, Zhiyuan Liu, Juanzi Li, and Jian Tang.  Kepler: A unified model for knowledge embedding and pre-trained language representation.  _Transactions of the Association for Computational Linguistics_ , 9:176–194, 2021.
  * Zhang et al. (2022a) X Zhang, A Bosselut, M Yasunaga, H Ren, P Liang, C Manning, and J Leskovec.  Greaselm: Graph reasoning enhanced language models for question answering.  In _International Conference on Representation Learning (ICLR)_ , 2022a.
  * Feng et al. (2022a) Shangbin Feng, Zhaoxuan Tan, Wenqian Zhang, Zhenyu Lei, and Yulia Tsvetkov.  Kalm: Knowledge-aware integration of local, document, and global contexts for long document understanding.  _arXiv preprint arXiv:2210.04105_ , 2022a.
  * Holtzman et al. (2019) Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi.  The curious case of neural text degeneration.  In _International Conference on Learning Representations_ , 2019.
  * Karimi et al. (2018) Hamid Karimi, Proteek Roy, Sari Saba-Sadiya, and Jiliang Tang.  Multi-source multi-class fake news detection.  In _Proceedings of the 27th international conference on computational linguistics_ , pages 1546–1557, 2018.
  * Nishida et al. (2019) Kosuke Nishida, Kyosuke Nishida, Masaaki Nagata, Atsushi Otsuka, Itsumi Saito, Hisako Asano, and Junji Tomita.  Answering while summarizing: Multi-task learning for multi-hop qa with evidence extraction.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 2335–2345, 2019.
  * Zhang et al. (2020) Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter Liu.  Pegasus: Pre-training with extracted gap-sentences for abstractive summarization.  In _International Conference on Machine Learning_ , pages 11328–11339. PMLR, 2020.
  * Liu et al. (2022b) Yixin Liu, Pengfei Liu, Dragomir Radev, and Graham Neubig.  Brio: Bringing order to abstractive summarization.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 2890–2903, 2022b.
  * Feng et al. (2023a) Shangbin Feng, Vidhisha Balachandran, Yuyang Bai, and Yulia Tsvetkov.  Factkb: Generalizable factuality evaluation using language models enhanced with factual knowledge, 2023a.
  * Schuster et al. (2021) Tal Schuster, Adam Fisch, and Regina Barzilay.  Get your vitamin c! robust fact verification with contrastive evidence.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 624–643, 2021.
  * Zhao et al. (2022) Xinran Zhao, Hongming Zhang, Xiaoman Pan, Wenlin Yao, Dong Yu, and Jianshu Chen.  Thrust: Adaptively propels large language models with external knowledge.  2022\.
  * Kadavath et al. (2022) Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield Dodds, Nova DasSarma, Eli Tran-Johnson, et al.  Language models (mostly) know what they know.  _arXiv preprint arXiv:2207.05221_ , 2022.
  * (31) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed H Chi, Quoc V Le, Denny Zhou, et al.  Chain-of-thought prompting elicits reasoning in large language models.  In _Advances in Neural Information Processing Systems_.
  * Press et al. (2022) Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A Smith, and Mike Lewis.  Measuring and narrowing the compositionality gap in language models.  _arXiv preprint arXiv:2210.03350_ , 2022.
  * Zhang et al. (2022b) Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al.  Opt: Open pre-trained transformer language models.  _arXiv preprint arXiv:2205.01068_ , 2022b.
  * Li et al. (2022) Margaret Li, Suchin Gururangan, Tim Dettmers, Mike Lewis, Tim Althoff, Noah A Smith, and Luke Zettlemoyer.  Branch-train-merge: Embarrassingly parallel training of expert language models.  _arXiv preprint arXiv:2208.03306_ , 2022.
  * Speer et al. (2017) Robyn Speer, Joshua Chin, and Catherine Havasi.  Conceptnet 5.5: An open multilingual graph of general knowledge.  In _Proceedings of the AAAI conference on artificial intelligence_ , volume 31, 2017.
  * West et al. (2022) Peter West, Chandra Bhagavatula, Jack Hessel, Jena Hwang, Liwei Jiang, Ronan Le Bras, Ximing Lu, Sean Welleck, and Yejin Choi.  Symbolic knowledge distillation: from general language models to commonsense models.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 4602–4625, 2022.
  * Vrandečić and Krötzsch (2014) Denny Vrandečić and Markus Krötzsch.  Wikidata: a free collaborative knowledgebase.  _Communications of the ACM_ , 57(10):78–85, 2014\.
  * Pellissier Tanon et al. (2020) Thomas Pellissier Tanon, Gerhard Weikum, and Fabian Suchanek.  Yago 4: A reason-able knowledge base.  In _The Semantic Web: 17th International Conference, ESWC 2020, Heraklion, Crete, Greece, May 31–June 4, 2020, Proceedings 17_ , pages 583–596. Springer, 2020.
  * Feng et al. (2021) Shangbin Feng, Zilong Chen, Wenqian Zhang, Qingyao Li, Qinghua Zheng, Xiaojun Chang, and Minnan Luo.  Kgap: Knowledge graph augmented political perspective detection in news media.  _arXiv preprint arXiv:2108.03861_ , 2021.
  * Liu et al. (2022c) Yujian Liu, Xinliang Frederick Zhang, David Wegsman, Nicholas Beauchamp, and Lu Wang.  Politics: Pretraining with same-story article comparison for ideology prediction and stance detection.  In _Findings of the Association for Computational Linguistics: NAACL 2022_ , pages 1354–1374, 2022c.
  * Feng et al. (2023b) Shangbin Feng, Chan Young Park, Yuhan Liu, and Yulia Tsvetkov.  From pretraining data to language models to downstream tasks: Tracking the trails of political biases leading to unfair nlp models, 2023b.
  * Song et al. (2020) Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu.  Mpnet: Masked and permuted pre-training for language understanding.  _Advances in Neural Information Processing Systems_ , 33:16857–16867, 2020.
  * Chen et al. (2021) Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al.  Evaluating large language models trained on code.  _arXiv preprint arXiv:2107.03374_ , 2021.
  * Hendrycks et al. (2020) Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt.  Measuring massive multitask language understanding.  In _International Conference on Learning Representations_ , 2020.
  * Si et al. (2022) Chenglei Si, Zhe Gan, Zhengyuan Yang, Shuohang Wang, Jianfeng Wang, Jordan Boyd-Graber, and Lijuan Wang.  Prompting gpt-3 to be reliable.  _arXiv preprint arXiv:2210.09150_ , 2022.
  * Rashkin et al. (2017) Hannah Rashkin, Eunsol Choi, Jin Yea Jang, Svitlana Volkova, and Yejin Choi.  Truth of varying shades: Analyzing language in fake news and political fact-checking.  In _Proceedings of the 2017 conference on empirical methods in natural language processing_ , pages 2931–2937, 2017.
  * Dhingra et al. (2022) Bhuwan Dhingra, Jeremy R Cole, Julian Martin Eisenschlos, Daniel Gillick, Jacob Eisenstein, and William W Cohen.  Time-aware language models as temporal knowledge bases.  _Transactions of the Association for Computational Linguistics_ , 10:257–273, 2022.
  * Kasai et al. (2022) Jungo Kasai, Keisuke Sakaguchi, Yoichi Takahashi, Ronan Le Bras, Akari Asai, Xinyan Yu, Dragomir Radev, Noah A Smith, Yejin Choi, and Kentaro Inui.  Realtime qa: What’s the answer right now?  _arXiv preprint arXiv:2207.13332_ , 2022.
  * Chowdhery et al. (2022) Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al.  Palm: Scaling language modeling with pathways.  _arXiv preprint arXiv:2204.02311_ , 2022.
  * Chung et al. (2022) Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al.  Scaling instruction-finetuned language models.  _arXiv preprint arXiv:2210.11416_ , 2022.
  * Yu et al. (2022) Wenhao Yu, Dan Iter, Shuohang Wang, Yichong Xu, Mingxuan Ju, Soumya Sanyal, Chenguang Zhu, Michael Zeng, and Meng Jiang.  Generate rather than retrieve: Large language models are strong context generators.  _arXiv preprint arXiv:2209.10063_ , 2022.
  * Izacard et al. (2022) Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi-Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave.  Few-shot learning with retrieval augmented language models.  _arXiv preprint arXiv:2208.03299_ , 2022.
  * Caccia et al. (2018) Massimo Caccia, Lucas Caccia, William Fedus, Hugo Larochelle, Joelle Pineau, and Laurent Charlin.  Language gans falling short.  In _International Conference on Learning Representations_ , 2018.
  * Lewis et al. (2020) Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al.  Retrieval-augmented generation for knowledge-intensive nlp tasks.  _Advances in Neural Information Processing Systems_ , 33:9459–9474, 2020.
  * Hu et al. (2022) Yushi Hu, Hang Hua, Zhengyuan Yang, Weijia Shi, Noah A Smith, and Jiebo Luo.  Promptcap: Prompt-guided task-aware image captioning.  _arXiv preprint arXiv:2211.09699_ , 2022.
  * Min et al. (2022) Sewon Min, Weijia Shi, Mike Lewis, Xilun Chen, Wen-tau Yih, Hannaneh Hajishirzi, and Luke Zettlemoyer.  Nonparametric masked language modeling.  _arXiv preprint arXiv:2212.01349_ , 2022.
  * Shi et al. (2022) Weijia Shi, Julian Michael, Suchin Gururangan, and Luke Zettlemoyer.  Nearest neighbor zero-shot inference.  _arXiv preprint arXiv:2205.13792_ , 2022.
  * Rubin et al. (2022) Ohad Rubin, Jonathan Herzig, and Jonathan Berant.  Learning to retrieve prompts for in-context learning.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 2655–2671, 2022.
  * Khandelwal et al. (2019) Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis.  Generalization through memorization: Nearest neighbor language models.  In _International Conference on Learning Representations_ , 2019.
  * Zhong et al. (2022) Zexuan Zhong, Tao Lei, and Danqi Chen.  Training language models with memory augmentation.  _arXiv preprint arXiv:2205.12674_ , 2022.
  * Khattab et al. (2022) Omar Khattab, Keshav Santhanam, Xiang Lisa Li, David Hall, Percy Liang, Christopher Potts, and Matei Zaharia.  Demonstrate-search-predict: Composing retrieval and language models for knowledge-intensive nlp.  _arXiv preprint arXiv:2212.14024_ , 2022.
  * He et al. (2021) Tianxing He, Jun Liu, Kyunghyun Cho, Myle Ott, Bing Liu, James Glass, and Fuchun Peng.  Analyzing the forgetting problem in pretrain-finetuning of open-domain dialogue response models.  In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_ , pages 1121–1133, Online, April 2021. Association for Computational Linguistics.  doi: 10.18653/v1/2021.eacl-main.95.  URL https://aclanthology.org/2021.eacl-main.95.
  * Tay et al. (2022) Yi Tay, Vinh Tran, Mostafa Dehghani, Jianmo Ni, Dara Bahri, Harsh Mehta, Zhen Qin, Kai Hui, Zhe Zhao, Jai Gupta, et al.  Transformer memory as a differentiable search index.  _Advances in Neural Information Processing Systems_ , 35:21831–21843, 2022.
  * Shwartz et al. (2020) Vered Shwartz, Peter West, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi.  Unsupervised commonsense question answering with self-talk.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 4615–4629, 2020.
  * Wang et al. (2022) Yujing Wang, Yingyan Hou, Haonan Wang, Ziming Miao, Shibin Wu, Qi Chen, Yuqing Xia, Chengmin Chi, Guoshuai Zhao, Zheng Liu, et al.  A neural corpus indexer for document retrieval.  _Advances in Neural Information Processing Systems_ , 35:25600–25614, 2022.
  * Zhuang et al. (2022) Shengyao Zhuang, Houxing Ren, Linjun Shou, Jian Pei, Ming Gong, Guido Zuccon, and Daxin Jiang.  Bridging the gap between indexing and retrieval for differentiable search index with query generation.  _arXiv preprint arXiv:2206.10128_ , 2022.
  * Bevilacqua et al. (2022) Michele Bevilacqua, Giuseppe Ottaviano, Patrick Lewis, Scott Yih, Sebastian Riedel, and Fabio Petroni.  Autoregressive search engines: Generating substrings as document identifiers.  _Advances in Neural Information Processing Systems_ , 35:31668–31683, 2022.
  * Meng et al. (2022) Kevin Meng, Arnab Sen Sharma, Alex Andonian, Yonatan Belinkov, and David Bau.  Mass-editing memory in a transformer.  _arXiv preprint arXiv:2210.07229_ , 2022.
  * Hernandez et al. (2023) Evan Hernandez, Belinda Z Li, and Jacob Andreas.  Measuring and manipulating knowledge representations in language models.  _arXiv preprint arXiv:2304.00740_ , 2023.
  * Masoudnia and Ebrahimpour (2014) Saeed Masoudnia and Reza Ebrahimpour.  Mixture of experts: a literature survey.  _The Artificial Intelligence Review_ , 42(2):275, 2014.
  * Gururangan et al. (2022) Suchin Gururangan, Mike Lewis, Ari Holtzman, Noah A Smith, and Luke Zettlemoyer.  Demix layers: Disentangling domains for modular language modeling.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 5557–5576, 2022.
  * Roller et al. (2021) Stephen Roller, Sainbayar Sukhbaatar, Jason Weston, et al.  Hash layers for large sparse models.  _Advances in Neural Information Processing Systems_ , 34:17555–17566, 2021.
  * Lewis et al. (2021) Mike Lewis, Shruti Bhosale, Tim Dettmers, Naman Goyal, and Luke Zettlemoyer.  Base layers: Simplifying training of large, sparse models.  In _International Conference on Machine Learning_ , pages 6265–6274. PMLR, 2021.
  * Kudugunta et al. (2021) Sneha Kudugunta, Yanping Huang, Ankur Bapna, Maxim Krikun, Dmitry Lepikhin, Minh-Thang Luong, and Orhan Firat.  Beyond distillation: Task-level mixture-of-experts for efficient inference.  In _Findings of the Association for Computational Linguistics: EMNLP 2021_ , pages 3577–3599, 2021.
  * Pfeiffer et al. (2022) Jonas Pfeiffer, Naman Goyal, Xi Lin, Xian Li, James Cross, Sebastian Riedel, and Mikel Artetxe.  Lifting the curse of multilinguality by pre-training modular transformers.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 3479–3495, 2022.
  * Houlsby et al. (2019) Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly.  Parameter-efficient transfer learning for nlp.  In _International Conference on Machine Learning_ , pages 2790–2799. PMLR, 2019.
  * Pfeiffer et al. (2020) Jonas Pfeiffer, Ivan Vulić, Iryna Gurevych, and Sebastian Ruder.  Mad-x: An adapter-based framework for multi-task cross-lingual transfer.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 7654–7673, 2020.
  * Zaken et al. (2022) Elad Ben Zaken, Yoav Goldberg, and Shauli Ravfogel.  Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , pages 1–9, 2022.
  * Matena and Raffel (2022) Michael S Matena and Colin A Raffel.  Merging models with fisher-weighted averaging.  _Advances in Neural Information Processing Systems_ , 35:17703–17716, 2022.
  * McMahan et al. (2017) Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas.  Communication-efficient learning of deep networks from decentralized data.  In _Artificial intelligence and statistics_ , pages 1273–1282. PMLR, 2017.
  * Izmailov et al. (2018) P Izmailov, AG Wilson, D Podoprikhin, D Vetrov, and T Garipov.  Averaging weights leads to wider optima and better generalization.  In _34th Conference on Uncertainty in Artificial Intelligence 2018, UAI 2018_ , pages 876–885, 2018.
  * Wortsman et al. (2022) Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, et al.  Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time.  In _International Conference on Machine Learning_ , pages 23965–23998. PMLR, 2022.
  * Gururangan et al. (2023) Suchin Gururangan, Margaret Li, Mike Lewis, Weijia Shi, Tim Althoff, Noah A Smith, and Luke Zettlemoyer.  Scaling expert language models with unsupervised domain discovery.  _arXiv preprint arXiv:2303.14177_ , 2023.
  * Don-Yehiya et al. (2022) Shachar Don-Yehiya, Elad Venezian, Colin Raffel, Noam Slonim, Yoav Katz, and Leshem Choshen.  Cold fusion: Collaborative descent for distributed multitask finetuning.  _arXiv preprint arXiv:2212.01378_ , 2022.
  * Borzunov et al. (2022) Alexander Borzunov, Dmitry Baranchuk, Tim Dettmers, Max Ryabinin, Younes Belkada, Artem Chumachenko, Pavel Samygin, and Colin Raffel.  Petals: Collaborative inference and fine-tuning of large models.  _arXiv preprint arXiv:2209.01188_ , 2022.
  * Köpf et al. (2023) Andreas Köpf, Yannic Kilcher, Dimitri von Rütte, Sotiris Anagnostidis, Zhi-Rui Tam, Keith Stevens, Abdullah Barhoum, Nguyen Minh Duc, Oliver Stanley, Richárd Nagyfi, et al.  Openassistant conversations–democratizing large language model alignment.  _arXiv preprint arXiv:2304.07327_ , 2023.
  * Sha (2023) Sharegpt.  https://sharegpt.com/, 2023.  Accessed: 2023-05-03.
  * Wolf et al. (2020) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al.  Transformers: State-of-the-art natural language processing.  In _Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations_ , pages 38–45, 2020.
  * Salemi et al. (2023) Alireza Salemi, Sheshera Mysore, Michael Bendersky, and Hamed Zamani.  Lamp: When large language models meet personalization.  _arXiv preprint arXiv:2304.11406_ , 2023.
  * Agarwal et al. (2021) Oshin Agarwal, Heming Ge, Siamak Shakeri, and Rami Al-Rfou.  Knowledge graph based synthetic corpus generation for knowledge-enhanced language model pre-training.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 3554–3565, 2021.
  * Chen et al. (2020) Wenhu Chen, Yu Su, Xifeng Yan, and William Yang Wang.  Kgpt: Knowledge-grounded pre-training for data-to-text generation.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 8635–8648, 2020.
  * Bagdasaryan and Shmatikov (2022) Eugene Bagdasaryan and Vitaly Shmatikov.  Spinning language models: Risks of propaganda-as-a-service and countermeasures.  In _2022 IEEE Symposium on Security and Privacy (SP)_ , pages 769–786. IEEE, 2022.
  * Perez et al. (2022) Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving.  Red teaming language models with language models.  _arXiv preprint arXiv:2202.03286_ , 2022.
  * Chelba et al. (2013) Ciprian Chelba, Tomas Mikolov, Mike Schuster, Qi Ge, Thorsten Brants, Phillipp Koehn, and Tony Robinson.  One billion word benchmark for measuring progress in statistical language modeling.  _arXiv preprint arXiv:1312.3005_ , 2013.
  * Lo et al. (2020) Kyle Lo, Lucy Lu Wang, Mark Neumann, Rodney Kinney, and Daniel S Weld.  S2orc: The semantic scholar open research corpus.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 4969–4983, 2020.
  * Zhu et al. (2015) Yukun Zhu, Ryan Kiros, Rich Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler.  Aligning books and movies: Towards story-like visual explanations by watching movies and reading books.  In _Proceedings of the IEEE international conference on computer vision_ , pages 19–27, 2015.
  * Rae et al. (2019) Jack W Rae, Anna Potapenko, Siddhant M Jayakumar, Chloe Hillier, and Timothy P Lillicrap.  Compressive transformers for long-range sequence modelling.  In _International Conference on Learning Representations_ , 2019.
  * Wang et al. (2023) Heng Wang, Wenqian Zhang, Yuyang Bai, Zhaoxuan Tan, Shangbin Feng, Qinghua Zheng, and Minnan Luo.  Detecting spoilers in movie reviews with external movie knowledge and user networks.  _arXiv preprint arXiv:2304.11411_ , 2023.
  * Hendrycks et al. (2021) Dan Hendrycks, Collin Burns, Anya Chen, and Spencer Ball.  Cuad: An expert-annotated nlp dataset for legal contract review.  _arXiv preprint arXiv:2103.06268_ , 2021.
  * Saxton et al. (2019) David Saxton, Edward Grefenstette, Felix Hill, and Pushmeet Kohli.  Analysing mathematical reasoning abilities of neural models.  In _International Conference on Learning Representations_ , 2019.
  * Zellers et al. (2019) Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi.  Defending against neural fake news.  _Advances in neural information processing systems_ , 32, 2019.
  * Feng et al. (2022b) Shangbin Feng, Zhaoxuan Tan, Herun Wan, Ningnan Wang, Zilong Chen, Binchi Zhang, Qinghua Zheng, Wenqian Zhang, Zhenyu Lei, Shujie Yang, et al.  Twibot-22: Towards graph-based twitter bot detection.  In _Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track_ , 2022b.
  * Hu et al. (2021) Linmei Hu, Tianchi Yang, Luhao Zhang, Wanjun Zhong, Duyu Tang, Chuan Shi, Nan Duan, and Ming Zhou.  Compare to the knowledge: Graph neural fake news detection with external knowledge.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 754–763, 2021.

##  Appendix A Discussion

#### Modularity at every turn.

All components in CooK are modular and easily substituted with future state-
of-the-art. 1) While Codex is the default LLM in the experiments, CooK also
works with text-davinci-003 and gpt-3.5-turbo (§5) and could be easily adapted
to future LLMs. 2) If better models for embedding space similarity,
abstractive summarization, and fact-checking are developed, the three
knowledge filters (§2.2) could be seamlessly updated. 3) When new knowledge,
information, and domains emerge, more specialized LMs could be trained and
uploaded to a model-sharing infrastructure [Wolf et al., 2020] by any member
of the machine learning community and adopted to improve general-purpose LLMs.

#### User-centric LLM adaptation.

When general-purpose large language models are released, everyone uses the
same LLM with the same API calls, while real-world users have heterogeneous
use cases and expectations that require personalization [Salemi et al., 2023].
For example, grade school students might expect LLMs to be absolutely factual
about knowledge and information in common textbooks, NLP researchers might
expect LLMs to have a basic understanding of current NLP research, cooking
amateurs might expect LLMs to understand the basic recipes and cuisines for
different occasions, and more. As a result, CooK presents a preliminary
approach by letting the user select and activate specialized LMs to empower
LLMs with different skill sets and domain expertise.

#### Compatible with diversified forms of knowledge.

By default, CooK employs language models trained on varying domains and
corpora as modular knowledge sources. In addition, CooK is also compatible
with 1) retrieval systems, where the retrieved text could similarly go through
the three knowledge filters and enrich LLM context, while retrieval corpora
are harder to share and use than modular language models; 2) knowledge graphs,
when combined with various proposals to construct natural language corpora out
of symbolic knowledge bases [Agarwal et al., 2021, Chen et al., 2020, Feng et
al., 2023a], which is already included in our prototype; 3) search engines,
where content on the web could also be integrated into the black-box LLMs
through CooK. Such flexibility and compatibility are possible since CooK
conducts knowledge integration through natural language. Compared to retrieval
models and search engines, using language models as knowledge sources enables
flexible information seeking (rather than rigid token exact match), searching
over knowledge domains, and employing private knowledge sources.

#### Specialized LM heterogeneity.

While existing modular LM proposals often require modular sub-models to be of
the same size and architecture for parameter averaging and model fusion [Li et
al., 2022], the specialized LMs in this work could be fully heterogeneous. 1)
Different specialized LMs could have different sizes. While OPT-1.3B is
adopted as the default architecture in this work, other sizes of OPT, from
125M all the way up to tens of billions, could all be used to initialize
different specialized LMs. In addition, 2) different specialized LMs could
have different model architectures. Since the integration of general-purpose
LLMs and modular knowledge cards happens at the natural-language level, any
autoregressive or language generation models could be adopted as specialized
LMs. These two levels of heterogeneity allow for flexibility in specialized LM
training: larger and more capable models could be trained on large corpora and
extensive knowledge domains by compute-rich individuals, while smaller
specialized LMs trained on small and dedicated domains by computationally
underprivileged researchers could also help improve black-box LLMs,
democratizing LLM research.

#### Specialized LM hierarchy.

We believe that specialized LMs could reflect the hierarchical nature of
knowledge. If CooK is adopted for general and encyclopedia-style question
answering, then a general biomedical specialized LM trained on an extensive
PubMed corpus would suffice. However, if CooK is adopted for more fine-grained
use cases, the “biomedical” domain could be further divided into, for example,
the three categories of basic, clinical, and translational research, thus one
specialized LM could be trained for each sub-domain. Similar divisions could
be applied to different sub-fields in NLP research, political news in
different countries, and more. In short, specialized LM domains could be
further divided to model the hierarchical granularity of knowledge and
information.

##  Appendix B Limitations

#### Specialized LMs are not perfect knowledge generators.

While specialized LMs could be of any size or model architecture, we used
OPT-1.3B, a relatively small language model to initialize specialized LMs
trained on different domains and sources. As a result, not all of the
generated knowledge documents are high-quality knowledge statements,
occasionally suffering from degeneration, topic deviation, and more. While the
three knowledge filters in part alleviate the impact of low-quality generated
knowledge documents, we hypothesize that improving the knowledge generation of
autoregressive language models is an important, yet orthogonal, research
question for future work. Two potential solutions include 1) increasing the
model size of specialized LMs and 2) using specialized training objectives for
specialized LMs, while both approaches require additional training and more
computational resources. In addition, Appendix A discussed CooK’s
compatibility with diverse knowledge sources, including retrieval, knowledge
graphs, and search engines, while these knowledge repositories have their
respective pros and cons. We leave it to future work on integrating multiple
types of external knowledge stores to extend CooK.

#### The factuality filter is biased towards information-rich domains and
existing knowledge.

To ensure the factuality of generated knowledge documents, we employed a
retrieval-augmented factuality filter based on both summarization factuality
metrics and fact-checking models while enabling flexibility through our
proposed _top-k factuality sampling_. However, domains with more Wikipedia
entries might be better supported by retrieved documents and might receive
higher factuality scores. In addition, new and emerging knowledge might be
well supported by existing retrieval corpora and receive low factuality
scores. We quantitatively evaluate this bias in Appendix D. Although top-k
factuality sampling enables flexibility to some extent, it remains an
important problem to design factuality evaluation measures that are
generalizable and adaptable to varying and emerging domains.

#### Prompting LLMs to seek help through yes/no questions is not perfect.

Inspired by the findings that LLMs do not need external knowledge for every
query [Zhao et al., 2022] and language models (mostly) know what they know
[Kadavath et al., 2022], we propose to ask yes/no questions to decide whether
to activate specialized LMs and encourage well-calibrated answers through in-
context learning. Our analysis (§5) shows that this strategy is effective but
far from perfect: LLMs are occasionally over-confident about their knowledge
capabilities. We leave it to future work on designing better strategies for
LLMs to abstain, acknowledge knowledge limitations, and seek help from
external information sources.

##  Appendix C Ethics Statement

We envision CooK as a community-driven and collaborative initiative to improve
general-purpose LLMs in knowledge-intensive tasks and contexts. An important
risk is the dual use and exploitation from malicious actors. Since modular
specialized LMs have the ability to change or update LLM knowledge, malicious
actors could advance their agenda by submitting malicious specialized LMs
trained on disinformation, hyperpartisan content, propaganda, and more, while
framing them as benign knowledge domains and deceive LLM users. We envision
two lines of approaches towards this ethical risk: on the _technical_ side,
research on adversarial manipulation of language models and corresponding
defense tactics [Bagdasaryan and Shmatikov, 2022, Perez et al., 2022] could be
integrated to alleviate the impact of malicious specialized LMs; on the
_social_ side, we could rely on and reinforce the existing rules for model
sharing on popular infrastructures [Wolf et al., 2020] to prevent such
malicious contribution from happening. We encourage the responsible use of
CooK, while we also call on users and researchers to be mindful of this dual-
use risk.

##  Appendix D Analysis (cont.)

#### Specialized LM Selection

In the top-down approach, we ask large language models to choose relevant
specialized LMs and obtain external knowledge. We illustrate the selection
results of the automatic selection strategy on the MMLU dataset separated into
the 57 sub-tasks. Figure 8 demonstrates that for most tasks knowledge
selection exhibits spike-like patterns on Wikipedia corpora and encyclopedic
knowledge graphs, suggesting that the majority of tasks have a few clearly
relevant specialized LMs. In addition, for other tasks (e.g. juris prudence
and high school U.S. history), it is not clear which specialized LMs would be
most helpful, thus the selection is more spread-out. These results suggest
that the selection patterns could also indicate whether a new and more in-
topic specialized LM is needed for any given task.

Figure 7: Yes/No questions in _top-down_ are mostly consistent across three
prompt templates, while there is space for improvement in future work.

#### Yes/No Template Sensitivity

In the top-down approach, we prompt LLMs with “ _Do you need more information?
(Yes/No)_ ” to identify if external knowledge is required and use in-context
learning to encourage well-calibrated responses. Since language models are
sensitive to minor changes in prompts, we devise two more questions: “ _Is
more information needed here?_ ” and “ _Would you like additional
information?_ ” and report the results on the 2-way misinformation task in
Figure 7. It is demonstrated that LLMs give _moderately_ consistent responses:
79.9% of cases received unanimous yes or no from the three prompts, while
20.1% examples received mixed results. This suggests that a potential
improvement to CooK is to employ multiple yes/no questions to probe knowledge
limitations and use an ensemble of answers to improve robustness.

LM Training | Inference Stage
---|---
Hyperparameter | Value | Hyperparameter | Value
learning rate |  $2e$-$5$ | $n_{1}$ | 3
weight decay |  $1e$-$5$ | $n_{2}$ | 5
max epochs | $10$ | $n_{3}$ | 3
batch size | $32$ | max iteration | 1
optimizer | Adam | temperature | 0.1
Adam epsilon |  $1e$-$6$ | default llm | codex
Adam beta |  $0.9$, $0.98$ |  |
warmup ratio | $0.06$ |  |

Table 6: Hyperparameter settings.

#### Factuality Scores of Specialized LMs

We use the MMLU datasets as queries to prompt different specialized LMs,
generate knowledge documents, and evaluate their factuality with the
factuality filter (§2.2). We illustrate the factuality score distributions of
different specialized LMs in Figure 9, which shows that different specialized
LMs have varying inherent factuality. We hypothesize that the distribution of
factuality scores given by the factuality filter could guide efforts to
evaluate the quality of community-contributed specialized LMs.

#### Working Examples

We present the specific prompts, generated knowledge documents, and prompts
for the bottom-up approach, and the top-down approach with automatic and
explicit selection in Tables 8, 9, and 10 respectively.

##  Appendix E Experiment Details

#### Algorithm Details

We present an algorithmic summary of the _bottom-up_ and _top-down_ approach
in Algorithm 1 and 2.

#### Specialized LM Domains

We train a total of 25 specialized LMs from the following corpora and domains:
one billion tokens [Chelba et al., 2013], ACL papers [Lo et al., 2020],
commonsense knowledge graph ATOMIC [West et al., 2022], book corpus [Zhu et
al., 2015], ConceptNet [Speer et al., 2017], biomedical knowledge graph UMLS
[Zhang et al., 2022a], Gutenberg [Rae et al., 2019], IMDB movie reviews [Wang
et al., 2023], political knowledge graph KGAP [Feng et al., 2021], legal
contracts [Hendrycks et al., 2021], math problems [Saxton et al., 2019],
midterm election news (ours), open subtitles
111https://github.com/sdtblck/Opensubtitles_dataset, political news corpora
POLITICS [Liu et al., 2022c], biomedical literature
222https://github.com/thoppe/The-Pile-PubMed, RealNews [Zellers et al., 2019],
Reddit [Feng et al., 2023b], Twitter [Feng et al., 2022b], Wikidata knowledge
graph [Vrandečić and Krötzsch, 2014], Wikipedia dump
333https://github.com/noanabeshima/wikipedia-downloader, YAGO [Pellissier
Tanon et al., 2020], and Yelp reviews 444https://www.yelp.com/dataset. For
knowledge graphs, we follow Feng et al. [2023a] to construct textual corpora
and use them as training data.

#### Hyperparameters

We present hyperparameter settings in Table 6.

#### Dataset Details

1) The MMLU dataset [Hendrycks et al., 2020] contains a total of 15,908 four-
choice questions further divided into 57 sub-tasks in four domains:
humanities, social sciences, STEM, and others. The official dataset also
provides a demonstration set, _i.e._ 5-shot examples in each sub-task to
enable few-shot in-context learning. We follow the official demonstration set
and test set in our experiments. 2) The LUN dataset [Rashkin et al., 2017] is
a misinformation detection dataset with two- or four-way classification
settings, either with true/false only or fine-grained categories of trusted,
hoax, propaganda, or satire. We use the official test set in [Hu et al., 2021]
with 2,999 examples throughout the experiments. 3) We curate _MidtermQA_ , a
QA dataset focusing on the 2022 U.S. midterm elections to evaluate CooK’s
ability for temporal knowledge update. Specifically, we first collect the
results of the 510 races in congressional, senate, or gubernatorial elections
in the 2022 midterms. We then construct three datasets: a) open-book, where we
ask LLMs to directly answer the name of the election winner for a given race,
b) two-way, where we ask LLMs to choose the winner from the two frontrunners,
and c) four-way, where we increase the difficulty by including two other
politicians contesting in the same state to create a distraction. We present
examples of the MidtermQA dataset in Table 7.

// _open-book setting_
---
Question: Who won the senate race of Oregon in the 2022 U.S. midterm
elections?
Answer: Ron Wyden
// _two-way setting_
Question: Who won the 24th congressional district of Texas in the 2022 U.S.
midterm elections?
A. Jan McDowell B. Beth Van Duyne
Answer: B
// _four-way setting_
Question: Who won the 6th congressional district of Pennsylvania in the 2022
U.S. midterm elections?
A. Christopher Hoeppner B. Doug Mastriano C. Chrissy Houlahan D. Guy
Ciarrocchi
Answer: C
Table 7: Examples of the MidtermQA dataset for the three settings.

#### Computation Resources Details

We used a GPU cluster with 16 NVIDIA A40 GPUs, 1988G memory, and 104 CPU cores
for the experiments. Training specialized LMs took from around 7 hours to 10
days depending on the training corpora size. For the black-box LLMs, we used
the OpenAI API to access code-davinci-002, text-davinci-003, and gpt-3.5-turbo
in the experiments.

#### Resource Availability

We will make the implementation of CooK, trained specialized LMs, and the
MidtermQA dataset publicly available.

<in-context examples with the same format>
---
…
Knowledge: … San Mateo is located in the northwest of California … Dianne
Feinstein, the senior senator from California, is rumored to retire … Tom
Brady returned to his hometown of San Mateo …
Question: Who is the senior senator from Tom Brady’s birth place?
Answer:
Table 8: Prompt example for the bottom-up approach. Different color boxes
indicate knowledge documents generated by different specialized LMs. <in-
context examples with the same format>
---
…
Question: Who is the senior senator from Tom Brady’s birth place?
Do you need more information? (Yes or No)
_Yes_
What kind of information do you need?
_The state Tom Brady is from._
Knowledge: Tom Brady returned to his hometown of San Mateo, CA …
Do you need more information? (Yes or No)
_No_
Answer:
Table 9: Prompt example for the top-down approach with automatic selection.
Italic texts indicate that this field is generated by black-box LLMs. <in-
context examples with the same format>
---
…
Question: Who is the senior senator from Tom Brady’s birth place?
Do you need more information? (Yes or No)
_Yes_
Choose an information source from the following: sports, biomedical
literature, NLP papers, book corpus.
_sports_
Knowledge: Tom Brady returned to his hometown of San Mateo, CA …
Do you need more information? (Yes or No)
_No_
Answer:
Table 10: Prompt example for the top-down approach with explicit selection.
Italic texts indicate that this field is generated by black-box LLMs.

Data: question $\boldsymbol{q}$; in-context examples prompt
$\boldsymbol{s}_{\textit{icl}}$; specialized LMs
$\mathcal{C}=\\{\mathrm{spec}_{1},\ldots,\mathrm{spec}_{n}\\}$; relevance,
pruning, and factuality filter $\phi_{\textit{rel}}$, $\phi_{\textit{prune}}$,
$\phi_{\textit{fact}}$

Result: answer string $\boldsymbol{s}_{\textit{ans}}$

prompt = $\boldsymbol{s}_{\textit{icl}}$

knowledge_list = []

for  _$\mathrm{spec}$ $\in$ $\mathcal{C}$_ do

knowledge_list.append($\mathrm{spec}$($\boldsymbol{q}$, $n_{1}$))

end for

knowledge_list = $\phi_{\textit{rel}}$($\boldsymbol{q}$, knowledge_list,
$n_{2}$)

knowledge_list = $\phi_{\textit{prune}}$(knowledge_list)

knowledge_list = $\phi_{\textit{fact}}$(knowledge_list, $n_{3}$)

prompt += “Knowledge: ”

for  _$\boldsymbol{s}$ $\in$ knowledge_list_ do

prompt += $\boldsymbol{s}$

end for

prompt += “Question: ” + $\boldsymbol{q}$ \+ “Answer:”

$\boldsymbol{s}_{\textit{ans}}$ = $\mathrm{LLM}$(prompt)

return $\boldsymbol{s}_{\textit{ans}}$

Algorithm 1 Bottom-Up Approach

Data: question $\boldsymbol{q}$; in-context examples prompt
$\boldsymbol{s}_{\textit{icl}}$; specialized LMs
$\mathcal{C}=\\{\mathrm{spec}_{1},\ldots,\mathrm{spec}_{n}\\}$; specialized LM
names $\mathcal{S}=\\{\boldsymbol{s}_{1},\ldots,\boldsymbol{s}_{n}\\}$; max
trial $k$; relevance and factuality filter $\phi_{\textit{rel}}$;
$\phi_{\textit{fact}}$; binary flags auto and exp

Result: answer string $\boldsymbol{s}_{\textit{ans}}$

prompt = $\boldsymbol{s}_{\textit{icl}}$

prompt += “Question: ” + $\boldsymbol{q}$

$i=0$

while  _$i\leq k$_ do

prompt += “Do you need more information? (Yes or No)”

response = $\mathrm{LLM}$(prompt)

if  _response == “Yes”_ then

if  _auto_ then

prompt += “What kind of information do you need?”

response = $\mathrm{LLM}$(prompt)

$\mathrm{spec}$ = $\phi_{\textit{rel}}$(response,
$\\{\boldsymbol{s}_{1},\ldots,\boldsymbol{s}_{n}\\}$, top-k = 1)

end if

if  _exp_ then

prompt += “Choose an information source from the following: ”

for  _$\boldsymbol{s}$ $\in$ $\mathcal{S}$_ do

prompt += $\boldsymbol{s}$

end for

$\mathrm{spec}$ = $\mathrm{LLM}$(prompt)

end if

knowledge = $\phi_{\textit{fact}}(\mathrm{spec}(\boldsymbol{q},n_{1}),1)$

prompt += “Knowledge: ” + knowledge

end if

if  _response == “No”_ then

break

end if

end while

prompt += “Answer:”

$\boldsymbol{s}_{\textit{ans}}$ = $\mathrm{LLM}$(prompt)

return $\boldsymbol{s}_{\textit{ans}}$

Algorithm 2 Top-Down Approach

Figure 8: Statistics of specialized LM selection in the top-down approach with
automatic selection across 57 sub-tasks in the MMLU benchmark. Encyclopedic
knowledge graph YAGO and Wikipedia are generally the most adopted specialized
LMs.

Figure 9: Factuality score distributions of the 25 specialized LMs when
prompted with questions in the MMLU benchmark. Different specialized LMs _do_
have varying factuality score distributions.