# Vector Quantization for Recommender Systems:
A Review and Outlook

Qijiong Liu  The HK PolyUHong Kong SARChina<EMAIL_ADDRESS>,  Xiaoyu Dong
The HK PolyUHong Kong SARChina<EMAIL_ADDRESS>,  Jiaren Xiao  The HK
PolyUHong Kong SARChina<EMAIL_ADDRESS>,  Nuo Chen  The HK PolyUHong
Kong SARChina<EMAIL_ADDRESS>,  Hengchang Hu  National
University of SingaporeSingapore<EMAIL_ADDRESS>,  Jieming Zhu  Huawei
Noah’s Ark LabShenzhenChina<EMAIL_ADDRESS>,  Chenxu Zhu  Huawei Noah’s
Ark LabShanghaiChina<EMAIL_ADDRESS>,  Tetsuya Sakai  Waseda
UniversityTokyoJapan<EMAIL_ADDRESS>and  Xiao-Ming Wu  The HK PolyUHong
Kong SARChina<EMAIL_ADDRESS>
(2018)

###### Abstract.

Vector quantization, renowned for its unparalleled feature compression
capabilities, has been a prominent topic in signal processing and machine
learning research for several decades and remains widely utilized today. With
the emergence of large models and generative AI, vector quantization has
gained popularity in recommender systems, establishing itself as a preferred
solution. This paper starts with a comprehensive review of vector quantization
techniques. It then explores systematic taxonomies of vector quantization
methods for recommender systems (VQ4Rec), examining their applications from
multiple perspectives. Further, it provides a thorough introduction to
research efforts in diverse recommendation scenarios, including efficiency-
oriented approaches and quality-oriented approaches. Finally, the survey
analyzes the remaining challenges and anticipates future trends in VQ4Rec,
including the challenges associated with the training of vector quantization,
the opportunities presented by large language models, and emerging trends in
multimodal recommender systems. We hope this survey can pave the way for
future researchers in the recommendation community and accelerate their
exploration in this promising field.

recommender system, vector quantization, survey

††copyright: acmlicensed††journalyear: 2018††doi: XXXXXXX.XXXXXXX††conference:
Make sure to enter the correct conference title from your rights confirmation
emai; June 03–05, 2018; Woodstock, NY††isbn: 978-1-4503-XXXX-X/18/06††ccs:
Information systems Recommender systems††ccs: General and reference Surveys
and overviews

##  1\. Introduction

Vector quantization (Buzo et al., 1980; Gray, 1984) (VQ), a cornerstone
technique in signal processing, was originally introduced by Gray and his team
(Buzo et al., 1980) in the 1980s to compress data representation while
preserving the fidelity of the original signal. The foundational standard VQ
technique aims to compress the entire representation space into a compact
codebook containing multiple codewords, typically using a single code to
approximate each vector. To improve the precision of quantization, advanced
methods such as product quantization (Sabin and Gray, 1984) and residual
quantization (Juang and Gray, 1982; Gray and Neuhoff, 1998; Martinez et al.,
2014) were introduced, representing parallel and sequential approaches,
respectively. These VQ techniques have proven to be highly effective in
domains including speech (Makhoul et al., 1985; Abe et al., 1990) and image
coding (Nasrabadi and King, 1988; Cosman et al., 1993).

Figure 1. Interest in VQ4Rec over time. denotes a milestone event or a
representative paper.

Despite its early development, it was not until the late 1990s that VQ found
application in the field of information retrieval, particularly in image
retrieval (Lu and Teng, 1999). The progress in applying VQ techniques was slow
until 2010 when Jegou and his team (Jegou et al., 2010) demonstrated the
effectiveness of parallel quantization for approximate nearest neighbor
search. This innovation enables fast similarity computations in high-
dimensional data spaces. In the same year, Chen and his team (Chen et al.,
2010) investigated the potential of sequential quantization for similar
applications.

Figure 2. Illustration of the three classical VQ techniques. indicates nearest
neighbor search.

Recommender systems, a prominent application in the field of artificial
intelligence and data science, typically build upon advancements in
information retrieval and machine learning. The integration of VQ into
recommender systems started in 2004, initially applied to music recommendation
(Huang and Jenor, 2004). However, a major turning point occurred 15 years
later, sparked by the introduction of VQ-VAE (Van Den Oord et al., 2017) for
image generation, which utilized VQ to discretize image representations. This
innovation led to the development of PQ-VAE (Van Balen and Levy, 2019), which
brought renewed attention to VQ within the recommendation community. The
success of VQ-VAE also catalyzed further advancements in residual
quantization, leading to the creation of RQ-VAE (Lee et al., 2022), which is
now at the heart of the burgeoning field of generative recommender systems
(Rajput et al., 2023). Furthermore, the emergence of large language models
(LLMs) (OpenAI, 2023; Touvron et al., 2023) has spurred new applications in
the recommendation domain (Liu et al., 2024a). However, due to their
substantial size and latency during inference, there’s a growing trend in
recommender systems to adopt VQ to enhance efficiency.

As shown in Figure 1, there has been a booming interest in vector quantization
for recommender systems (VQ4Rec) over recent years. This body of research can
be roughly categorized into efficiency-oriented and quality-oriented. The
former focuses on optimizing large-scale systems, tackling challenges
associated with large models, extensive datasets, and computational demands.
In this context, VQ proves to be highly effective, significantly improving
performance in crucial areas, including similarity search (Su et al., 2023),
space compression (Imran et al., 2023), and model acceleration (Wu et al.,
2021). The latter prioritizes recommendation accuracy, concentrating on the
refinement of feature usage. This involves optimizing features, fostering
interactions among various modalities, and aligning features to enhance
generative recommendation processes. It covers sub-scenarios such as feature
enhancement (Luo et al., 2024), modality alignment (Hou et al., 2023), and
discrete tokenization (Rajput et al., 2023). Moreover, VQ has shown promise in
integrating recommender systems with LLMs (Zheng et al., 2023) to improve
recommendation quality. This is achieved by using VQ to effectively tokenize
and structure recommendation-related data, such as information about items or
users. For instance, generative retrieval methods (Rajput et al., 2023)
leverage VQ to ensure that the recommendation data is well-aligned with LLMs.

Despite the growing interest in VQ4Rec amidst new challenges posed by large
language models, multimodal data, and generative AI, no work has yet
systematically surveyed the application of VQ in recommender systems. This
paper aims to bridge this gap through a comprehensive survey. We provide a
thorough analysis of VQ4Rec, exploring its uses, challenges, and future
directions in the field. The main contents and contributions of this paper are
summarized as follows:

  * •

We present an overview of both classical and modern VQ techniques,
encompassing standard VQ, parallel VQ, sequential VQ, and differentiable VQ.

  * •

We provide systematic taxonomies of VQ4Rec from various perspectives such as
training phase, application scenario, VQ techniques, and quantization target.

  * •

We conduct a thorough analysis of the strengths, weaknesses, and limitations
of existing VQ4Rec methods, focusing on addressing two main challenges in
recommender system: efficiency and quality.

  * •

We identify key challenges in VQ4Rec and present promising opportunities that
can serve as inspiration for future research in this burgeoning field.

Table 1. Comparison of the three classical VQ techniques. We use $\bar{K}=\frac{1}{M}\sum_{i}K_{i}$ to represent the arithmetic mean of $K_{i}$, and $\hat{K}=\sqrt[M]{\prod_{i}K_{i}}$ to represent their geometric mean, where $i\in\\{1,2,\ldots,M\\}$. Note that when $K_{i}=K$, $\bar{K}=\hat{K}=K$. | Input Dim | #Codebooks | #Codes per Book | Code Dim | Codebook Size | Feature Space
---|---|---|---|---|---|---
Standard VQ | $D$ | $1$ | $K$ | $D$ | $K\cdot D$ | $K$
Parallel VQ | $D$ | $M$ | $K_{i}$ | $D/M$ | $\bar{K}\cdot D$ | $\hat{K}^{M}$
Sequential VQ | $D$ | $M$ | $K_{i}$ | $D$ | $M\cdot\bar{K}\cdot D$ | $\hat{K}^{M}$

##  2\. Overview of VQ Techniques

VQ targets at grouping similar vectors into clusters by representing them with
a small set of prototype vectors (i.e., codes in the codebook). In this
section, we offer a comprehensive summary of classical VQ methods and the
modern differentiable VQ technique. The conventional VQ approaches include
standard VQ, which uses a single codebook, parallel VQ, which utilizes
multiple codebooks simultaneously to represent separate vector subspaces, and
sequential VQ, which involves using multiple codebooks in a sequence to refine
the quantization.

###  2.1. Standard Vector Quantization

The standard VQ (Buzo et al., 1980; Gray, 1984) serves as the atomic component
for the latter two VQ techniques. Formally, given a set of object vectors
$\mathbf{E}\in\mathbb{R}^{N\times D}$, a function $f$ (e.g., $k$-means) is
required to produce a codebook $\mathbf{C}\in\mathbb{R}^{K\times D}$ such that
the sum of distances between all vectors in $\mathbf{E}$ and their
corresponding nearest code vectors in $\mathbf{C}$ is minimized, as
illustrated in Figure 2(a). We can formally express this using the following
equations:

(1) $\displaystyle f$ $\displaystyle:\mathbf{E}\rightarrow\mathbf{C},$ (2)
$\displaystyle\textit{where }\mathbf{C}$
$\displaystyle=\underset{\mathbf{W}\in\mathbb{R}^{K\times
D}}{\operatorname{argmin}}\sum_{i=1}^{N}d(\mathbf{e}_{i},\mathbf{w}_{x}),$ (3)
$\displaystyle\textit{and }x$
$\displaystyle=\underset{j=1,\ldots,K}{\operatorname{argmin}}\,d\left(\mathbf{e}_{i},\mathbf{w}_{j}\right),$

where $N$ is the number of object vectors and $K$ is the number of code
vectors in the codebook (usually $N\gg K$), $\mathbf{e}_{i}$ is the $i$-th
object vector, $D$ is the embedding dimension, $d$ represents the distance
function (e.g., Euclidean distance or Manhattan distance), $\mathbf{W}$
denotes any codebook in the same space as $\mathbf{C}$, and $x$ is the index
of the code vector closest to $\mathbf{e}_{i}$. Therefore, we can use
$\mathbf{c}_{x}$, the $x$-th code in codebook $\mathbf{C}$, to approximate
$\mathbf{e}_{i}$:

(4) $\mathbf{e}_{i}\approx\mathbf{c}_{x}.$

###  2.2. Parallel Vector Quantization

As the embedding dimension $D$ increases, standard VQ methods face significant
challenges in terms of storage requirements, computational efficiency, and
quantization quality. In response to these challenges, approaches like product
quantization and optimized product quantization, representative of parallel
quantization techniques, emerge as effective solutions. These methods segment
high-dimensional vectors into multiple lower-dimensional sub-vectors and
perform quantization on each segment independently. As shown in Table 1, with
an increase in the number of segments ($M$), there is a corresponding
reduction in the dimensionality of each code, keeping the codebook storage
size unchanged. Yet, the representation space exhibits an exponential growth
compared to that of standard VQ.

####  2.2.1. Product Quantization (PQ) (Juang and Gray, 1982; Jegou et al.,
2010)

Product Quantization (PQ) represents an initial approach to parallel
quantization, where original high-dimensional vectors are segmented into
uniformly-sized sub-vectors. This process can be mathematically represented as
$\mathbf{E}=\left[\mathbf{E}^{(1)},\mathbf{E}^{(2)},\cdots,\mathbf{E}^{(M)}\right]$,
where $M$ denotes the number of the segments and the number of the codebooks,
and $\mathbf{E}^{(i)}\in\mathrm{R}^{N\times\frac{D}{M}}$. Each sub-vector is
then independently subjected to standard VQ, utilizing a distinct codebook for
each segment. Therefore, the $i$-th original vector can be approximated by
selecting and concatenating each single code vector $\mathbf{c}^{(j)}_{x_{j}}$
from each sub-codebook $\mathbf{C}^{(j)}$, which can be formulated as:

(5)
$\mathbf{e}_{i}=\left[\cdots,\mathbf{e}_{i}^{(j)},\cdots\right]\approx\left[\cdots,\mathbf{c}^{(j)}_{x_{j}},\cdots,\right]\quad\text{for
}j\in\\{1,2,\ldots,M\\},$

where $\mathbf{C}^{(j)}$ is the $j$-th codebook with size $K_{j}$, and $x_{j}$
is the index of the code vector in $\mathbf{C}^{(j)}$ closest to
$\mathbf{e}_{i}^{(j)}$. Due to its storage efficiency and capability for fast
approximate nearest neighbor searches, product quantization has become a
popular solution in the information retrieval domain, particularly for image
retrieval tasks, as evidenced by several studies (Cao et al., 2017; Jang and
Cho, 2021; Chen et al., 2022). Nonetheless, it overlooks the potential for
significant inter-correlations among sub-vectors, which may affect the
quantization performance and subsequent downstream tasks.

####  2.2.2. Optimized Product Quantization (OPQ) (Ge et al., 2013)

To eliminate the interdependence among multiple subspaces, optimized product
quantization is introduced and uses the learnable rotation matrix
$\mathbf{R}\in\mathbb{R}^{D\times D}$ for automatically selecting the most
effective orientation of the data in the high-dimensional space. Such rotation
minimizes the interdependence among different subspaces, allowing for a more
efficient and independent quantization process, which can be defined as:

(6) $\displaystyle\mathbf{E}^{\prime}$
$\displaystyle=\mathbf{E}\times\mathbf{R},$ (7) $\displaystyle\mathbf{I}$
$\displaystyle=\mathbf{R}^{T}\times\mathbf{R},$

where $\mathbf{E}^{\prime}$ is the rotated matrix, and $\mathbf{I}$ represents
the identity matrix. Next, $\mathbf{E}^{\prime}$ will be operated by product
quantization, as described in Sec 2.2.1. It is important to note that the
rotation matrix $\mathbf{R}$ is trained with the codebooks. Once trained, the
$i$-th original vector can be approximated by:

(8)
$\mathbf{e}_{i}\approx\left[\cdots,\mathbf{c}^{(j)}_{x_{j}},\cdots,\right]\times\mathbf{R}^{T}\quad\text{for
}j\in\\{1,2,\ldots,M\\}.$

###  2.3. Sequential Vector Quantization

Standard VQ and parallel VQ typically yield _rough_ approximations of vectors.
Specifically, each dimension of the original vector can only be approximated
by one single value from the corresponding code vector, leading to substantial
information loss. Taking standard VQ as an example, the difference between the
original vector $\mathbf{e}$ and its corresponding code $\mathbf{c}$, denoted
by $\mathbf{e}-\mathbf{c}$, reflects the unique characteristics that cannot be
represented by $\mathbf{c}$.

To achieve a more _precise_ quantization, approaches like residual
quantization (Juang and Gray, 1982; Martinez et al., 2014) and additive
quantization (Babenko and Lempitsky, 2014) have been developed, falling under
the umbrella of sequential quantization. This method employs multiple
codebooks, with each codebook approximates every dimension of the original
vectors. Essentially, every codebook offers a distinct approximation
perspective of the vectors, and the accuracy of these approximations improves
with an increase in the number of codebooks. As illustrated in Figure 2, using
the first layer codebook approximates 0.3 (the first dimension of the original
vector) as 0.5 (the first dimension of the code vector in the first codebook).
After applying the second codebook, it is more accurately approximated as 0.5
+ (-0.3) = 0.2 (the first dimension of the code vector in the second
codebook).

####  2.3.1. Residual Quantization (RQ) (Juang and Gray, 1982; Martinez et
al., 2014)

By designing $M$ individual codebooks where, as depicted in Table 1, code
vectors have the full same length of the input vector, residual quantization
aims to approximate the target vectors by compressing their information in a
coarse-to-fine manner. Specifically, the codebooks are learned iteratively
from the residual representations of the vectors. This process can be
formulated as:
$\mathbf{E}^{(j+1)}=\mathbf{E}^{(j)}-\mathbf{X}^{(j)}\mathbf{C}^{(j)}$, where
$\mathbf{E}^{1}=\mathbf{E}$, $\mathbf{C}^{(j)}$ is the $j$-th codebook with
size $K_{j}$, and $\mathbf{X}^{(j)}\in\mathrm{\\{0,1\\}}^{N}$ is a one-hot
mapper, where $\mathbf{X}^{(j)}_{i,k}=1$ only if the $k$-th code is the
nearest to the $i$-th vector of $\mathbf{E}^{(j)}$ in the codebook
$\mathbf{C}^{(j)}$. After iteratively residual approximation, the $i$-th
original vector can be represented by:

(9) $\displaystyle\mathbf{e}_{i}$
$\displaystyle\approx\sum_{j}^{M}\mathbf{c}^{(j)}_{x_{j}},$ (10)
$\displaystyle\textit{where }x_{j}$
$\displaystyle=\underset{k}{\operatorname{argmin}}\,\mathbf{X}^{(j)}_{i,k}.$

It is important to note that, as $M$ increases, the approximated
representation tends to be finer.

####  2.3.2. Additive Quantization (AQ) (Babenko and Lempitsky, 2014)

Similar to residual quantization, additive quantization aims to approximate
the target vectors by aggregating one selected code per codebook. However,
residual quantization employs a greedy approach by selecting only the
_nearest_ neighbor (i.e., $\mathbf{c}^{(j)}_{x_{j}}$) within the current
(i.e., $j$-th) layer, which does not guarantee the global optimum. Instead,
codebooks here are sequentially learned using beam search, where top candidate
code combinations (_not the only one_) from the first $j$ codebooks are
selected to infer the $(j+1)$-th codebook. Hence, the $i$-th original vector
can be approximated as in Equation 9.

###  2.4. Differentiable Vector Quantization

The technique of VQ fundamentally includes a non-differentiable procedure,
which entails identifying the nearest code in the codebook, consequently
making the calculation of gradients impractical. This lack of
differentiability presents a substantial hurdle in neural network training,
which relies heavily on gradient-based optimization methods. Consequently, in
the wake of the VQ-VAE (Van Den Oord et al., 2017), numerous research
initiatives (Kang et al., 2020; Rajput et al., 2023) have adopted the
Straight-Through Estimator (STE) (Bengio et al., 2013) as a leading solution
to this challenge.

The core idea of STE is relatively straightforward: during the forward pass of
a network, the non-differentiable operation (like quantization) is performed
as usual. However, during the backward pass, when gradients are propagated
back through the network, STE allows gradients to “pass through” the non-
differentiable operation as if it were differentiable. This is typically done
by approximating the derivative of the non-differentiable operation with a
constant value, often 1, which can be defined as:

(11)
$\frac{\partial\mathbf{c}_{x}}{\partial\mathbf{e}_{i}}\approx\frac{\partial\mathbf{e}_{i}}{\partial\mathbf{e}_{i}}=\mathbf{I},$

where $\mathbf{I}$ is the identity matrix.

However, training with straight-through estimator often encounters the
codebook collapse issue, wherein a significant portion of codes fails to map
onto corresponding vectors. Various strategies, such as employing exponential
moving average (EMA) (Łańcucki et al., 2020) during training or implementing
codebook reset (Zeghidour et al., 2021; Rajput et al., 2023) mechanisms, have
been developed to address this challenge.

In the above discussion, we have reviewed established vector quantization
techniques, but have not delved into recent innovations such as finite scalar
quantization (FSQ) (Donahue et al., 2019; Dieleman et al., 2021; Mentzer et
al., 2023). Drawing inspiration from model quantization (Shi et al., 2023;
Chen et al., 2023b; Yue et al., 2023), FSQ adopts a straightforward rounding
mechanism to approximate the value in each dimension of a vector. FSQ has
yielded competitive results comparable to those achieved by VQ-VAE (Mentzer et
al., 2023) in image generation. While FSQ has not yet been applied to
recommender systems, it presents a promising avenue for future exploration.

Figure 3. Integration of VQ techniques with the recommender system at
different training stages.

##  3\. Taxonomies of VQ4Rec

Table 2. A list of representative VQ4Rec methods and their features.
“Modality” denotes the type of feature utilized, and “Task” refers to the
specific training tasks employed in these methods. Note that all papers
pertaining to the post-processing stage are task-agnostic, hence the “-” in
the table for these entries. We use “CTR”, “NIP”, “CF”, and “Multi” to denote
“click-through rate prediction”, “next item prediction”, “collaborative
filtering” and “multiple” tasks, respectively.

Application | Paper | Venue | VQ Type | VQ Target | Modality | Stage | Task
---|---|---|---|---|---|---|---
Efficiency-Oriented
Space Compression | Liu et al. (2024c) | TheWebConf (2024) | Sequential / RQ | Item & User | ID & Text | Pre | CTR
Imran et al. (2023) | TOIS (2023) | Standard VQ | Item | ID | Pre | NIP
Kang et al. (2020) | TheWebConf (2020) | Parallel / PQ | Item | ID | In | Multi
Van Balen and Levy (2019) | RecSys (2019) | Parallel / PQ | User | ID | In | CF
Model Acc. | Wu et al. (2021) | TheWebConf (2021) | Standard VQ | Item | ID | In | NIP
Similarity Search | Su et al. (2023) | SIGIR (2023) | Parallel / PQ | User | ID | Post | -
Zhang et al. (2023a) | AAAI (2023) | Parallel / PQ | Item | ID & Text | Post | -
Lu et al. (2023) | TheWebConf (2023) | Parallel / OPQ | Item | ID | Post | -
Zhao et al. (2021) | KDD-IRS (2021) | Parallel / OPQ | Item | Text | Pre | CTR
Lian et al. (2020b) | TKDE (2020) | Parallel / OPQ | Item & User | ID | In | CF
Lian et al. (2020a) | TheWebConf (2020) | Sequential / RQ | Item | ID & Text | In | CF
Huang and Jenor (2004) | ICME (2004) | Standard VQ | Item | Music | Post | -
Quality-Oriented
Feature Enhancement | Liu et al. (2024b) | TheWebConf (2024) | Standard VQ | Item & User | ID | In | Multi
Luo et al. (2024) | arXiv (2024) | Standard VQ | Item | ID | In | NIP
Pan et al. (2021) | arXiv (2021) | Standard VQ | User | ID | In | CTR
Modality Alignment | Hu et al. (2024) | ECIR (2024) | Parallel / PQ | Item | Image & Text & ID | In | NIP
Hou et al. (2023) | TheWebConf (2023) | Parallel / OPQ | Item | Text | Pre | NIP
Discrete Tokenization | Zheng et al. (2023) | ICDE (2024) | Sequential / RQ | Item | Text | Pre | NIP
Liu et al. (2024d) | arXiv (2024) | Sequential / RQ | Item & User | Graph | Pre | CF
Jin et al. (2024) | arXiv (2024) | Sequential / RQ | Item | Text | Pre | NIP
Rajput et al. (2023) | NeurIPS (2023) | Sequential / RQ | Item | Text | Pre | NIP
Singh et al. (2023) | arXiv (2023) | Sequential / RQ | Item | Video | Pre | CTR
Jin et al. (2023) | arXiv (2023) | Standard VQ | Item | Text | Pre | NIP

To comprehensively understand the current advances in VQ4Rec, in this section,
we categorize previous studies from multiple viewpoints, such as training
phase or application scenario, to encapsulate the diverse methodologies and
applications in this field.

###  3.1. Classification by Training Phase

VQ techniques can be applied to recommender systems at different training
stages: pre-processing, in-processing, and post-processing, as depicted in
Figure 3.

  * •

Pre-processing: In this stage, VQ techniques are utilized to optimize or
compress input data, such as item features or user sequences, resulting in
static quantized inputs for recommender systems (Zhao et al., 2021; Imran et
al., 2023; Hou et al., 2023).

  * •

In-processing: Here, VQ is integrated to and trained together with the
recommender system, providing dynamically quantized features to enhance the
functionality of the system (Kang et al., 2020; Van Balen and Levy, 2019; Wu
et al., 2021).

  * •

Post-processing: This involves applying VQ to the embeddings generated by the
recommender systems, aiming to improve search speed or accuracy (Zhang et al.,
2023a; Lu et al., 2023; Huang and Jenor, 2004).

###  3.2. Classification by Application Scenario

The use of VQ in recommender systems can be broadly classified into two major
scenarios: one that prioritizes efficiency and another that emphasizes
quality. As depicted in Figure 4, each scenario addresses distinct challenges
and objectives inherent to the recommender system, leveraging the strengths of
VQ to enhance the overall performance and user experience.

Efficiency-oriented approaches primarily focus on enhancing the computational
and storage aspects of recommender systems. In this fast-evolving digital era,
where data volume and complexity are ever-increasing, these approaches play a
instrumental role in maintaining the scalability and responsiveness of
recommendation services. They are particularly pertinent in scenarios such as
similarity search (Zhang et al., 2023a; Lu et al., 2023; Zhao et al., 2021),
space compression (Imran et al., 2023; Kang et al., 2020; Van Balen and Levy,
2019), and time acceleration (Wu et al., 2021; Lian et al., 2020a, b).

Conversely, quality-oriented approaches aim to enhance the accuracy and
relevance of the recommendations. These methods leverage VQ to refine the data
and model representations, thereby improving the quality of the output
provided to the end-users. They are relevant in scenarios involving feature
enhancement (Liu et al., 2024b; Pan et al., 2021), modality correlation (Hou
et al., 2023), and item tokenization (Razavi et al., 2019; Singh et al., 2023;
Jin et al., 2023; Zheng et al., 2023).

###  3.3. Other Classification Frameworks

Here, we expand our perspective to explore additional classification
frameworks for VQ4Rec. This includes:

  * •

Classification by VQ Technique: As previously mentioned, existing studies
generally adopt three types of VQ techniques: Standard VQ, as seen in works
like (Huang and Jenor, 2004; Imran et al., 2023; Wu et al., 2021), Parallel
VQ, featured in studies (Zhang et al., 2023a; Lu et al., 2023; Zhao et al.,
2021), and Sequential VQ, highlighted in references (Lian et al., 2020a;
Rajput et al., 2023; Singh et al., 2023).

  * •

Classification by Quantization Target: The majority of existing research has
focused on Item Quantization (Huang and Jenor, 2004; Imran et al., 2023; Wu et
al., 2021). This is likely because item features are usually static, whereas
user preferences are dynamic. Additionally, the need to compress extensive
item datasets due to their large scale and rich content has been a driving
factor. Nonetheless, there is also some research on User Quantization (Van
Balen and Levy, 2019; Pan et al., 2021), as well as studies that investigate
both Item & User Quantization (Lian et al., 2020b; Liu et al., 2024b).

Figure 4.  Categorization of VQ4Rec methods based on application scenario. The
node colors denote different VQ techniques employed. The standard, parallel,
and sequential VQ techniques are denoted by green, blue, and red,
respectively. The overlap between nodes indicates that the application
scenarios they represent share certain similarities.

##  4\. Efficiency-oriented Approaches

Efficiency in machine learning is crucial for enhancing model speed and
optimizing resource use in environments with limited computational power
(Schwartz et al., 2020; Liu et al., 2024e). Advances in technology have led to
various solutions to improve model efficiency (Menghani, 2023), such as model
pruning (Beel and Brunel, 2019), model distillation (Tang and Wang, 2018), and
model quantization (Ko et al., 2021). Moreover, adopting efficient
architectures like parameter-efficient finetuning (Liu et al., 2024a) or
linear attention networks (Liu et al., 2023) optimizes training and inference
processes without increasing space requirements.

VQ enhances the efficiency of recommender systems with its superior clustering
capabilities, being widely used and verified in similarity search, space
compression, and model acceleration scenarios.

###  4.1. Space Compression

Recommender systems typically create a unique embedding vector for each user
or item, leading to high memory costs with large datasets. For example, 1
billion users would need 238 GB for 64-dimensional vectors in 32-bit floating
point (Chen et al., 2023a). To mitigate these costs, techniques like hashing
(Zhang et al., 2018) and low-rank factorization (Koren et al., 2009) have been
used. However, hashing can cause information loss due to hash collisions,
while low-rank factorization might overlook complex data patterns, reducing
model accuracy.

One line of research focuses on quantizing and condensing _sequential data_ ,
such as user behavior or item content, using a variational autoencoder
mechanism inspired by VQ-VAE (Van Den Oord et al., 2017) in image generation.
These methods integrate sequential knowledge into a unified representation,
subsequently compressed into discrete codes. For example, Van Balen and Levy
(2019) introduced PQ-VAE, employing product quantization to derive discrete
user representations from user-item interactions for quick prediction of
click-through rates. Similarly, ReFRS (Imran et al., 2023) uses a variational
autoencoder within a federated learning framework to learn user tokens for
decentralized recommendations. Recently, Liu et al. (2024c) introduces
residual quantization to condense both user history and item content into
short tokens. Compared with embedding-based models, caching these tokens would
achieve about 100x space compression rate. Another research approach directly
applies VQ to existing _embedding tables_ , as exemplified by MGQE (Kang et
al., 2020), which utilizes differentiable VQ for item embeddings.

These methods often also accelerate training and inference through more
streamlined model architectures. However, VQ techniques have yet to be
empirically tested for space compression in large-scale recommendation models,
where their feasibility may be challenged by high embedding dimensions.

###  4.2. Model Acceleration

Prior section has investigated methods for enhancing training and inference
efficiency through space compression and dimensionality reduction. Here, we
focus on summarizing research aimed at accelerating the model architecture.

Transformers and attention mechanisms (Vaswani et al., 2017), fundamental to
many influential models, exhibit inference efficiency that scales
quadratically with sequence length. Consequently, significant researches have
been directed toward developing attention modules that operate with linear
time complexity. Techniques such as low-rank matrix decomposition (used in
Linformer (Wang et al., 2020) and Performer (Choromanski et al., 2020)) and
hashing for matching attention values (used in EcoFormer (Liu et al., 2022a))
have been explored. Additionally, VQ, which applies clustering to condense the
attention matrix space, has demonstrated efficacy in fields like time series
forecasting and natural language processing. Notably, Wu et al. propose LISA
(Wu et al., 2021) which expedites inference for long-sequence recommender
systems. Compared with existing approaches which apply sparse attention
patterns where crucial information may be lost, LISA combines the
effectiveness of self-attention and the efficiency of sparse attention,
enabling full contextual attention through codeword histograms.

Currently, the application of VQ for model optimization and acceleration
remains limited. However, VQ-based linear attention modules are likely to gain
popularity with the increase in long sequence features and the emergence of
lifelong learning in the era of big data. Additionally, recent studies have
employed VQ for the identification and compression of graph structures,
followed by distillation of the compressed features into MLP format (Yang et
al., 2023). This approach enhances the processing of graph structural
information, offering potential benefits for graph-based recommender systems,
such as in social recommendation contexts.

###  4.3. Similarity Search

Similarity search, which relies on recommendation models for learning user and
item representations, enables the retrieval of similar users and items. In
2004, Huang and Jenor (2004) first highlighted the robust matching
capabilities of VQ for music recommendation, categorizing new music
representations into pre-existing groups using nearest neighbor search.
However, conducting exhaustive maximum inner product searches (MIPS) is often
costly and impractical with a large number of candidates. To mitigate these
issues, a substantial body of research has focused on approximate nearest
neighbor search (ANNs) and MIPS techniques, including hashing (Neyshabur and
Srebro, 2015), tree search (Feng et al., 2023), and graph search (Morozov and
Babenko, 2018).

In 2010, Jegou et al. (2010) pioneered a novel solution in the similarity
search domain by employing a divide-and-conquer strategy, which involved
subdividing vectors into sub-vectors followed by quantization. This product
quantization based method facilitates rapid estimation of approximate
distances between vectors represented by codes, through the pre-computation of
distance tables for each code. This efficient technique for approximate
nearest neighbors (ANNs) quickly became a mainstream approach in similarity
search, including _item-item search_ (Johnson et al., 2019; Lu et al., 2023;
Zhang et al., 2023a; Zhao et al., 2021) and _user-item search_ (Lian et al.,
2020b; Su et al., 2023). Beyond parallel quantization methods, Lian et al.
(2020a) have explored sequential quantization to discretize item embeddings,
thereby enhancing relevance score estimation and reducing memory requirements
in recommender systems.

Parallel and sequential quantization both aim to establish one-to-one mappings
between vectors and code combinations, expanding horizontally and vertically,
respectively, and have been validated in similarity search. However, there is
currently no method that combines these approaches to finely segment and
represent vectors. Additionally, similarity search techniques for weight
matrices and recent low-rank adaptation (LoRA) (Hu et al., 2021) methods share
similarities in achieving approximate effects through matrix compression. In
the future, these methods may also find application in parameter-efficient
finetuning (Liu et al., 2022b) for recommendations, offering potential new
directions for efficiency-oriented applications.

##  5\. Quality-oriented Approaches

Building high-quality recommender systems is imperative to effectively cater
to users’ increasing information demands. Both academia and industry have
explored various strategies to this end. These strategies include data
augmentation, as demonstrated by (Song and Suh, 2022), which entails
generating synthetic data from existing datasets through techniques like item
masking (Slokom et al., 2019). Additionally, hyperparameter tuning,
exemplified by (Wu et al., 2023), automates the optimization of model
settings, thereby mitigating the laborious process of grid search. Moreover,
feature engineering, as elucidated by (Schifferer et al., 2020), enhances
feature selection and data preprocessing.

VQ enhances recommender system quality by serving as a foundational step,
specifically in item indexing for generative retrieval, a process which is
further detailed in discrete tokenization applications. Furthermore, VQ aligns
diverse modalities with soft constraints, facilitating multimodal feature
learning.

###  5.1. Feature Enhancement

Presently, recommender systems face challenges in cold-start scenarios where
user interactions are sparse. By integrating features such as item combination
patterns and category information through VQ, these systems can be
significantly enhanced.

To effectively utilize the data of active users, Pan et al. (2021) apply VQ to
user interest clusters, facilitating cluster-level contrastive learning, which
balances the personalization of representations between inactive and active
users. Their auto-quantized approach captures cluster-level similarities
through VQ, in contrast to SimCLR proposed by Chen et al. (2020), which
focuses solely on instance-level similarities. To harness item combination
patterns, Luo et al. (2024) propose VQA, which combines neural attention
mechanism and VQ to determine the attention of candidate combination patterns.
To continuously generate and optimize the entity category trees over time,
another study, CAGE (Liu et al., 2024b) enables the simultaneous learning and
refinement of categorical code representations and entity embeddings in an
end-to-end manner for ID-based recommendation.

However, these efforts rely on ID-based approaches, which may not be optimal
in current diverse multimodal content landscape. Exploring methods to
effectively leverage VQ techniques to enhance information from text, images,
and other multimodal sources, and integrating it with recommendation features,
presents a promising avenue for research.

###  5.2. Modality Alignment

Another interesting branch of work aims to improve modality alignment in
recommender systems. Transferable recommender systems are becoming
increasingly important which can quickly adapt to new domains or scenarios.
However, ensuring the alignment of various modalities and preserving their
distinct patterns throughout downstream training models remains a challenge.

Under transferable scenario, VQ can be used for loosening the binding between
item _text_ and _ID_ representation, as a sparse representation technique. Hou
et al. (2023) introduce VQ to represent items in a compact form, capturing the
diverse characteristics of the products and addressing the transferability
issues in sequential recommender systems. In contrast, Hu et al. (2024) employ
product quantization to impose additional modality constraints, targeting the
mitigation of the modality forgetting issue in two-stage sequential
recommenders. This involves transforming dissected _text_ and _visual_
correlations into discrete codebook representations to enforce tighter
constraints.

Hence, VQ serves as a potent semantic bridge, particularly with the rise of
Large Language Models (LLMs), facilitating connectivity across diverse
modalities or domains. However, existing approaches primarily focus on
aligning two modalities. Addressing multimodal scenarios involving more than
three modalities necessitates novel solutions.

###  5.3. Discrete Tokenization

Tokenizing items and users in recommender systems has involved numerous
strategies. Traditional methods often use atomic item identifiers (IDs), which
can result in cold start problems. Later developments, inspired by document
retrieval techniques like DSI (Tay et al., 2022) and NCI (Wang et al., 2022),
introduced tree IDs using multi-layer K-Means (Krishna and Murty, 1999) to
achieve discrete yet partially shared item tokens, though semantic
discrepancies remained an issue.

To address this, one line of research applies _embedding-level reconstruction_
task. For example, Rajput et al. (2023) developed the TIGER model based on RQ-
VAE (Lee et al., 2022), consisting of three steps: extracting item embeddings
from content, discretizing these embeddings via residual quantization, and
applying the discretized item tokens for sequence recommendation. Due to the
inherent nature of residual quantization that can organise the tokens in a
hierarchical manner, such approach proved highly successful and foundational
for future research (Singh et al., 2023; Liu et al., 2024c, d; Jin et al.,
2024). Subsequent projects like LC-REC (Zheng et al., 2023) expanded on this
by integrating item tokens into large models, hinting at the development of
foundational recommendation models. Instead, some researchers (Jin et al.,
2023) optimize this process further at _text-level reconstruction_ by treating
item tokenization as a translation task within an encoder-decoder-decoder
framework, using standard VQ on the top of the first decoder outputs that also
achieves substantial performance.

However, the exploration of multimodal and multi-domain item tokenization
remains limited, and this area presents a promising opportunity for advancing
foundational recommender systems.

##  6\. Future Directions

In this section, we discuss the current challenges and emerging opportunities
for future research in VQ4Rec.

###  6.1. Codebook Collapse Problem

There are some limitations associated with the capability of VQ. For example,
the challenge of codebook collapse may arise when only a minor portion of the
codebook is effectively utilized. VQ-VAE (Van Den Oord et al., 2017) employs
STE (Bengio et al., 2013) to grant differentiability to VQ, consequently, many
entries in the codebook remain unused or underutilized, restricting the model
capacity to accurately represent and reconstruct input data. This core issue
extends its impact to subsequent developments in recommender systems employing
PQ-VAE (Van Balen and Levy, 2019) and RQ-VAE (Lee et al., 2022), which impairs
the recommender system’s ability to offer varied and personalized
recommendations to users as it fails to capture the diversity of the data. At
present, preliminary endeavors (Zhang et al., 2023b; Huh et al., 2023; Baykal
et al., 2023) have yielded encouraging results, with the scholarly community
being urged to continue their research efforts in this direction.

###  6.2. Item Discovery

In item tokenization scenarios, the codebook space significantly exceeds the
number of items in the dataset, suggesting that many potential code
combinations remain untapped (Rajput et al., 2023). Providing human-readable
description for these new code combinations, especially in generative
recommendation, represents a valuable direction. For instance, in product
recommendations, this can help merchants develop products tailored to user
demands; in video recommendations, it allows platforms to create personalized
content based on the description. Currently, code training mainly relies on
item embedding reconstruction tasks (Zheng et al., 2023; Rajput et al., 2023).
A viable alternative is an end-to-end reconstruction task based on item
content such as title and description, where new code combinations are
inputted into the decoder to generate the corresponding item content.

###  6.3. User Tokenization

Current VQ encoding schemes primarily focus on item discretization and have
shown success in generative recommendation scenarios. However, discretizing
user representation, i.e., user tokenization, also presents significant
opportunities for research. For instance, Liu et al. (2024c) has achieved
substantial storage efficiency by applying discretization to both user and
item in click through rate prediction. A pressing challenge is to enhance the
quality of user tokens, which could enable large models to offer personalized
responses through model personalization (Ning et al., 2024).

###  6.4. Multimodal Generative Recommendation

Item semantic tokenization is currently the leading method for indexing items
in generative recommender systems (Rajput et al., 2023; Singh et al., 2023;
Jin et al., 2023). However, current methods are mostly text-based, although
multimodal semantic tokenization has begun to emerge in tasks such as text-to-
image (Zheng et al., 2024) and video segmentation (Xia et al., 2024). In the
big data era, leveraging multimodal features offers a more comprehensive
representation of items. Therefore, the development and application of
multimodal tokenization techniques in recommender systems represents a
critical advancement.

###  6.5. RS–LLM Alignment

The significant success of large language models (OpenAI, 2023) has
established them as foundational elements across multiple fields. Current
efforts increasingly focus on aligning object features from diverse domains
with LLMs, enhancing their explainability and multimodal understanding (Ge et
al., 2023; Zhan et al., 2024). For example, LC-Rec (Zheng et al., 2023) has
successfully finetuned discretized item IDs obtained by RQ-VAE (Lee et al.,
2022) on the LLaMA model (Touvron et al., 2023), validating this strategy in
the recommendation domain. Future endeavors could involve integrating data
from various domains to develop a foundational recommendation model with
versatile skills.

###  6.6. Codebook Quality Evaluation

In some scenarios, the process of codebook generation and the recommendation
task are not executed through end-to-end training. For instance, in item
tokenization, item tokens are initially derived from item semantics before
being evaluated in applications like sequential recommendation. Evaluating
code quality through downstream tasks is both time-consuming and resource-
intensive, suggesting a need for optimization. Therefore, the exploration of
methodologies for assessing code quality through the comparison of generated
tokens against original inputs represents a significant and promising research
direction.

###  6.7. Efficient Large-scale Recommender Systems

As large-scale models proliferate, the demand for efficient model training and
inference is escalating within the recommendation community. VQ is emerging as
a promising tool for enhancing the efficiency of large recommender systems,
alongside other popular techniques like distillation and quantization. For
instance, Lingle (2023) and Wu et al. (2021) have demonstrated that optimizing
the attention mechanism through VQ can achieve linear time complexity in image
generation and recommendation task, respectively. However, these approaches
typically involve smaller models and embedding dimensions that can be
efficiently handled using a single codebook. In contrast, for larger models
like LLaMA (Touvron et al., 2023), which has embedding dimensions as large as
4096, the straightforward use of VQ may not be as effective. Exploring the
integration of parallel quantization techniques with linear attention could
potentially offer a viable solution.

##  7\. Conclusion

VQ has become a pivotal element in the development of innovative solutions
across various scenarios in recommender systems. With the advent of large
language models, there has been a notable shift towards generative
recommendation methods, where residual quantization has been widely adopted
for its inherent advantages. However, the research of VQ4Rec is still in its
early stage. This paper offers a comprehensive overview of current research in
VQ4Rec, highlighting both efficiency-oriented and quality-oriented approaches.
Additionally, we identify and discuss the open challenges and potential
avenues for advancement. We hope this survey will foster continued exploration
and innovation in VQ4Rec.

## Acknowledgement

Qijiong Liu is grateful to Prof. Min-Yen Kan from the National University of
Singapore for his valuable comments and advice on this work during Liu’s visit
to NUS.

## References

  * (1)
  * Abe et al. (1990) Masanobu Abe, Satoshi Nakamura, Kiyohiro Shikano, and Hisao Kuwabara. 1990.  Voice conversion through vector quantization.  _Journal of the Acoustical Society of Japan (E)_ 11, 2 (1990), 71–76.
  * Babenko and Lempitsky (2014) Artem Babenko and Victor Lempitsky. 2014.  Additive quantization for extreme vector compression. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. 931–938.
  * Baykal et al. (2023) Gulcin Baykal, Melih Kandemir, and Gozde Unal. 2023.  EdVAE: Mitigating Codebook Collapse with Evidential Discrete Variational Autoencoders.  _Available at SSRN 4671725_ (2023).
  * Beel and Brunel (2019) Joeran Beel and Victor Brunel. 2019.  Data pruning in recommender systems research: Best-practice or malpractice.  _ACM RecSys_ (2019).
  * Bengio et al. (2013) Yoshua Bengio, Nicholas Léonard, and Aaron Courville. 2013.  Estimating or propagating gradients through stochastic neurons for conditional computation.  _arXiv preprint arXiv:1308.3432_ (2013).
  * Buzo et al. (1980) Andrés Buzo, A Gray, R Gray, and John Markel. 1980.  Speech coding based upon vector quantization.  _IEEE Transactions on Acoustics, Speech, and Signal Processing_ 28, 5 (1980), 562–574.
  * Cao et al. (2017) Yue Cao, Mingsheng Long, Jianmin Wang, and Shichen Liu. 2017.  Deep visual-semantic quantization for efficient image retrieval. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. 1328–1337.
  * Chen et al. (2022) Bin Chen, Yan Feng, Tao Dai, Jiawang Bai, Yong Jiang, Shu-Tao Xia, and Xuan Wang. 2022.  Adversarial examples generation for deep product quantization networks on image retrieval.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ 45, 2 (2022), 1388–1404.
  * Chen et al. (2023b) Huiyuan Chen, Kaixiong Zhou, Kwei Herng Lai, Chin-Chia Michael Yeh, Yan Zheng, Xia Hu, and Hao Yang. 2023b.  Hessian-aware Quantized Node Embeddings for Recommendation. In _Proceedings of the 17th ACM Conference on Recommender Systems_. 757–762.
  * Chen et al. (2020) Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020.  A simple framework for contrastive learning of visual representations. In _International conference on machine learning_. PMLR, 1597–1607.
  * Chen et al. (2010) Yongjian Chen, Tao Guan, and Cheng Wang. 2010.  Approximate nearest neighbor search by residual vector quantization.  _Sensors_ 10, 12 (2010), 11259–11273.
  * Chen et al. (2023a) Yizhou Chen, Guangda Huzhang, Anxiang Zeng, Qingtao Yu, Hui Sun, Hengyi Li, Jingyi Li, Yabo Ni, Han Yu, and Zhiming Zhou. 2023a.  Clustered Embedding Learning for Recommender Systems.  _arXiv preprint arXiv:2302.01478_ (2023).
  * Choromanski et al. (2020) Krzysztof Choromanski, Valerii Likhosherstov, David Dohan, Xingyou Song, Andreea Gane, Tamas Sarlos, Peter Hawkins, Jared Davis, Afroz Mohiuddin, Lukasz Kaiser, et al. 2020\.  Rethinking attention with performers.  _arXiv preprint arXiv:2009.14794_ (2020).
  * Cosman et al. (1993) Pamela C Cosman, Karen L Oehler, Eve A Riskin, and Robert M Gray. 1993.  Using vector quantization for image processing.  _Proc. IEEE_ 81, 9 (1993), 1326–1341.
  * Dieleman et al. (2021) Sander Dieleman, Charlie Nash, Jesse Engel, and Karen Simonyan. 2021.  Variable-rate discrete representation learning.  _arXiv preprint arXiv:2103.06089_ (2021).
  * Donahue et al. (2019) Chris Donahue, Ian Simon, and Sander Dieleman. 2019.  Piano genie. In _Proceedings of the 24th International Conference on Intelligent User Interfaces_. 160–164.
  * Feng et al. (2023) Chao Feng, Defu Lian, Xiting Wang, Zheng Liu, Xing Xie, and Enhong Chen. 2023.  Reinforcement routing on proximity graph for efficient recommendation.  _ACM Transactions on Information Systems_ 41, 1 (2023), 1–27.
  * Ge et al. (2013) Tiezheng Ge, Kaiming He, Qifa Ke, and Jian Sun. 2013.  Optimized product quantization.  _IEEE transactions on pattern analysis and machine intelligence_ 36, 4 (2013), 744–755.
  * Ge et al. (2023) Yuying Ge, Yixiao Ge, Ziyun Zeng, Xintao Wang, and Ying Shan. 2023.  Planting a seed of vision in large language model.  _arXiv preprint arXiv:2307.08041_ (2023).
  * Gray (1984) Robert Gray. 1984.  Vector quantization.  _IEEE Assp Magazine_ 1, 2 (1984), 4–29.
  * Gray and Neuhoff (1998) Robert M. Gray and David L. Neuhoff. 1998.  Quantization.  _IEEE transactions on information theory_ 44, 6 (1998), 2325–2383.
  * Hou et al. (2023) Yupeng Hou, Zhankui He, Julian McAuley, and Wayne Xin Zhao. 2023.  Learning vector-quantized item representation for transferable sequential recommenders. In _Proceedings of the ACM Web Conference 2023_. 1162–1171.
  * Hu et al. (2021) Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021.  Lora: Low-rank adaptation of large language models.  _arXiv preprint arXiv:2106.09685_ (2021).
  * Hu et al. (2024) Hengchang Hu, Qijiong Liu, Chuang Li, and Min-Yen Kan. 2024.  Lightweight Modality Adaptation to Sequential Recommendation via Correlation Supervision. In _European Conference on Information Retrieval_. Springer International Publishing, Glasgow, Scotland, UK.
  * Huang and Jenor (2004) Yao-Chang Huang and Shyh-Kang Jenor. 2004.  An audio recommendation system based on audio signature description scheme in mpeg-7 audio. In _2004 IEEE International Conference on Multimedia and Expo (ICME)(IEEE Cat. No. 04TH8763)_ , Vol. 1. IEEE, 639–642.
  * Huh et al. (2023) Minyoung Huh, Brian Cheung, Pulkit Agrawal, and Phillip Isola. 2023.  Straightening out the straight-through estimator: Overcoming optimization challenges in vector quantized networks. In _International Conference on Machine Learning_. PMLR, 14096–14113.
  * Imran et al. (2023) Mubashir Imran, Hongzhi Yin, Tong Chen, Quoc Viet Hung Nguyen, Alexander Zhou, and Kai Zheng. 2023.  ReFRS: Resource-efficient federated recommender system for dynamic and diversified user preferences.  _ACM Transactions on Information Systems_ 41, 3 (2023), 1–30.
  * Jang and Cho (2021) Young Kyun Jang and Nam Ik Cho. 2021.  Self-supervised product quantization for deep unsupervised image retrieval. In _Proceedings of the IEEE/CVF international conference on computer vision_. 12085–12094.
  * Jegou et al. (2010) Herve Jegou, Matthijs Douze, and Cordelia Schmid. 2010.  Product quantization for nearest neighbor search.  _IEEE transactions on pattern analysis and machine intelligence_ 33, 1 (2010), 117–128.
  * Jin et al. (2023) Bowen Jin, Hansi Zeng, Guoyin Wang, Xiusi Chen, Tianxin Wei, Ruirui Li, Zhengyang Wang, Zheng Li, Yang Li, Hanqing Lu, et al. 2023\.  Language Models As Semantic Indexers.  _arXiv preprint arXiv:2310.07815_ (2023).
  * Jin et al. (2024) Mengqun Jin, Zexuan Qiu, Jieming Zhu, Zhenhua Dong, and Xiu Li. 2024.  Contrastive Quantization based Semantic Code for Generative Recommendation.  _arXiv preprint arXiv:2404.14774_ (2024).
  * Johnson et al. (2019) Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2019.  Billion-scale similarity search with gpus.  _IEEE Transactions on Big Data_ 7, 3 (2019), 535–547.
  * Juang and Gray (1982) Biing-Hwang Juang and A Gray. 1982.  Multiple stage vector quantization for speech coding. In _ICASSP’82. IEEE International Conference on Acoustics, Speech, and Signal Processing_ , Vol. 7. IEEE, 597–600.
  * Kang et al. (2020) Wang-Cheng Kang, Derek Zhiyuan Cheng, Ting Chen, Xinyang Yi, Dong Lin, Lichan Hong, and Ed H Chi. 2020.  Learning multi-granular quantized embeddings for large-vocab categorical features in recommender systems. In _Companion Proceedings of the Web Conference 2020_. 562–566.
  * Ko et al. (2021) Yunyong Ko, Jae-Seo Yu, Hong-Kyun Bae, Yongjun Park, Dongwon Lee, and Sang-Wook Kim. 2021.  MASCOT: A Quantization Framework for Efficient Matrix Factorization in Recommender Systems. In _2021 IEEE International Conference on Data Mining (ICDM)_. IEEE, 290–299.
  * Koren et al. (2009) Yehuda Koren, Robert Bell, and Chris Volinsky. 2009.  Matrix factorization techniques for recommender systems.  _Computer_ 42, 8 (2009), 30–37.
  * Krishna and Murty (1999) K Krishna and M Narasimha Murty. 1999.  Genetic K-means algorithm.  _IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics)_ 29, 3 (1999), 433–439.
  * Łańcucki et al. (2020) Adrian Łańcucki, Jan Chorowski, Guillaume Sanchez, Ricard Marxer, Nanxin Chen, Hans JGA Dolfing, Sameer Khurana, Tanel Alumäe, and Antoine Laurent. 2020.  Robust training of vector quantized bottleneck models. In _2020 International Joint Conference on Neural Networks (IJCNN)_. IEEE, 1–7.
  * Lee et al. (2022) Doyup Lee, Chiheon Kim, Saehoon Kim, Minsu Cho, and Wook-Shin Han. 2022.  Autoregressive image generation using residual quantization. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 11523–11532.
  * Lian et al. (2020a) Defu Lian, Haoyu Wang, Zheng Liu, Jianxun Lian, Enhong Chen, and Xing Xie. 2020a.  Lightrec: A memory and search-efficient recommender system. In _Proceedings of The Web Conference 2020_. 695–705.
  * Lian et al. (2020b) Defu Lian, Xing Xie, Enhong Chen, and Hui Xiong. 2020b.  Product quantized collaborative filtering.  _IEEE Transactions on Knowledge and Data Engineering_ 33, 9 (2020), 3284–3296.
  * Lingle (2023) Lucas D Lingle. 2023.  Transformer-vq: Linear-time transformers via vector quantization.  _arXiv preprint arXiv:2309.16354_ (2023).
  * Liu et al. (2022b) Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin A Raffel. 2022b.  Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning.  _Advances in Neural Information Processing Systems_ 35 (2022), 1950–1965.
  * Liu et al. (2024d) Han Liu, Yinwei Wei, Xuemeng Song, Weili Guan, Yuan-Fang Li, and Liqiang Nie. 2024d.  MMGRec: Multimodal Generative Recommendation with Transformer Model.  _arXiv preprint arXiv:2404.16555_ (2024).
  * Liu et al. (2022a) Jing Liu, Zizheng Pan, Haoyu He, Jianfei Cai, and Bohan Zhuang. 2022a.  Ecoformer: Energy-saving attention with linear complexity.  _Advances in Neural Information Processing Systems_ 35 (2022), 10295–10308.
  * Liu et al. (2023) Langming Liu, Liu Cai, Chi Zhang, Xiangyu Zhao, Jingtong Gao, Wanyu Wang, Yifu Lv, Wenqi Fan, Yiqi Wang, Ming He, et al. 2023\.  Linrec: Linear attention mechanism for long-term sequential recommender systems. In _Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval_. 289–299.
  * Liu et al. (2024a) Qijiong Liu, Nuo Chen, Tetsuya Sakai, and Xiao-Ming Wu. 2024a.  Once: Boosting content-based recommendation with both open-and closed-source large language models. In _Proceedings of the 17th ACM International Conference on Web Search and Data Mining_. 452–461.
  * Liu et al. (2024b) Qijiong Liu, Lu Fan, Jiaren Xiao, Jieming Zhu, and Xiao-Ming Wu. 2024b.  Learning Category Trees for ID-Based Recommendation: Exploring the Power of Differentiable Vector Quantization. In _Proceedings of the ACM Web Conference 2024_. Singapore.
  * Liu et al. (2024c) Qijiong Liu, Hengchang Hu, Jiahao Wu, Jieming Zhu, Min-Yen Kan, and Xiao-Ming Wu. 2024c.  Discrete Semantic Tokenization for Deep CTR Prediction.
  * Liu et al. (2024e) Qijiong Liu, Jieming Zhu, Quanyu Dai, and Xiao-Ming Wu. 2024e.  Benchmarking News Recommendation in the Era of Green AI.  _arXiv preprint arXiv:2403.04736_ (2024).
  * Lu and Teng (1999) Guojun Lu and Shyhwei Teng. 1999.  A novel image retrieval technique based on vector quantization. In _Proceedings of International Conference on Computational Intelligence for Modeling, Control and Automation_. Citeseer, 36–41.
  * Lu et al. (2023) Zepu Lu, Defu Lian, Jin Zhang, Zaixi Zhang, Chao Feng, Hao Wang, and Enhong Chen. 2023.  Differentiable Optimized Product Quantization and Beyond. In _Proceedings of the ACM Web Conference 2023_. 3353–3363.
  * Luo et al. (2024) Kai Luo, Tianshu Shen, Lan Yao, Ga Wu, Aaron Liblong, Istvan Fehervari, Ruijian An, Jawad Ahmed, Harshit Mishra, and Charu Pujari. 2024.  Within-basket Recommendation via Neural Pattern Associator.  _arXiv preprint arXiv:2401.16433_ (2024).
  * Makhoul et al. (1985) John Makhoul, Salim Roucos, and Herbert Gish. 1985.  Vector quantization in speech coding.  _Proc. IEEE_ 73, 11 (1985), 1551–1588.
  * Martinez et al. (2014) Julieta Martinez, Holger H Hoos, and James J Little. 2014.  Stacked quantizers for compositional vector compression.  _arXiv preprint arXiv:1411.2173_ (2014).
  * Menghani (2023) Gaurav Menghani. 2023.  Efficient deep learning: A survey on making deep learning models smaller, faster, and better.  _Comput. Surveys_ 55, 12 (2023), 1–37.
  * Mentzer et al. (2023) Fabian Mentzer, David Minnen, Eirikur Agustsson, and Michael Tschannen. 2023.  Finite scalar quantization: Vq-vae made simple.  _arXiv preprint arXiv:2309.15505_ (2023).
  * Morozov and Babenko (2018) Stanislav Morozov and Artem Babenko. 2018.  Non-metric similarity graphs for maximum inner product search.  _Advances in Neural Information Processing Systems_ 31 (2018).
  * Nasrabadi and King (1988) Nasser M Nasrabadi and Robert A King. 1988.  Image coding using vector quantization: A review.  _IEEE Transactions on communications_ 36, 8 (1988), 957–971.
  * Neyshabur and Srebro (2015) Behnam Neyshabur and Nathan Srebro. 2015.  On symmetric and asymmetric lshs for inner product search. In _International Conference on Machine Learning_. PMLR, 1926–1934.
  * Ning et al. (2024) Lin Ning, Luyang Liu, Jiaxing Wu, Neo Wu, Devora Berlowitz, Sushant Prakash, Bradley Green, Shawn O’Banion, and Jun Xie. 2024.  User-LLM: Efficient LLM Contextualization with User Embeddings.  _arXiv preprint arXiv:2402.13598_ (2024).
  * OpenAI (2023) R OpenAI. 2023.  Gpt-4 technical report. arxiv 2303.08774.  _View in Article_ 2, 5 (2023).
  * Pan et al. (2021) Yujie Pan, Jiangchao Yao, Bo Han, Kunyang Jia, Ya Zhang, and Hongxia Yang. 2021.  Click-through rate prediction with auto-quantized contrastive learning.  _arXiv preprint arXiv:2109.13921_ (2021).
  * Rajput et al. (2023) Shashank Rajput, Nikhil Mehta, Anima Singh, Raghunandan H Keshavan, Trung Vu, Lukasz Heldt, Lichan Hong, Yi Tay, Vinh Q Tran, Jonah Samost, et al. 2023\.  Recommender Systems with Generative Retrieval.  _arXiv preprint arXiv:2305.05065_ (2023).
  * Razavi et al. (2019) Ali Razavi, Aaron Van den Oord, and Oriol Vinyals. 2019.  Generating diverse high-fidelity images with vq-vae-2.  _Advances in neural information processing systems_ 32 (2019).
  * Sabin and Gray (1984) ML Sabin and R Gray. 1984.  Product code vector quantizers for waveform and voice coding.  _IEEE transactions on acoustics, speech, and signal processing_ 32, 3 (1984), 474–488.
  * Schifferer et al. (2020) Benedikt Schifferer, Gilberto Titericz, Chris Deotte, Christof Henkel, Kazuki Onodera, Jiwei Liu, Bojan Tunguz, Even Oldridge, Gabriel De Souza Pereira Moreira, and Ahmet Erdem. 2020.  GPU accelerated feature engineering and training for recommender systems.  In _Proceedings of the Recommender Systems Challenge 2020_. 16–23.
  * Schwartz et al. (2020) Roy Schwartz, Jesse Dodge, Noah A Smith, and Oren Etzioni. 2020.  Green ai.  _Commun. ACM_ 63, 12 (2020), 54–63.
  * Shi et al. (2023) Lingfeng Shi, Yuang Liu, Jun Wang, and Wei Zhang. 2023.  Quantize Sequential Recommenders Without Private Data. In _Proceedings of the ACM Web Conference 2023_. 1043–1052.
  * Singh et al. (2023) Anima Singh, Trung Vu, Raghunandan Keshavan, Nikhil Mehta, Xinyang Yi, Lichan Hong, Lukasz Heldt, Li Wei, Ed Chi, and Maheswaran Sathiamoorthy. 2023.  Better Generalization with Semantic IDs: A case study in Ranking for Recommendations.  _arXiv preprint arXiv:2306.08121_ (2023).
  * Slokom et al. (2019) Manel Slokom, Martha Larson, and Alan Hanjalic. 2019.  Data masking for recommender systems: prediction performance and rating hiding.  (2019).
  * Song and Suh (2022) Joo-yeong Song and Bongwon Suh. 2022.  Data Augmentation Strategies for Improving Sequential Recommender Systems.  _arXiv e-prints_ (2022), arXiv–2203.
  * Su et al. (2023) Liangcai Su, Fan Yan, Jieming Zhu, Xi Xiao, Haoyi Duan, Zhou Zhao, Zhenhua Dong, and Ruiming Tang. 2023.  Beyond Two-Tower Matching: Learning Sparse Retrievable Cross-Interactions for Recommendation. In _Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval_. 548–557.
  * Tang and Wang (2018) Jiaxi Tang and Ke Wang. 2018.  Ranking distillation: Learning compact ranking models with high performance for recommender system. In _Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining_. 2289–2298.
  * Tay et al. (2022) Yi Tay, Vinh Tran, Mostafa Dehghani, Jianmo Ni, Dara Bahri, Harsh Mehta, Zhen Qin, Kai Hui, Zhe Zhao, Jai Gupta, et al. 2022\.  Transformer memory as a differentiable search index.  _Advances in Neural Information Processing Systems_ 35 (2022), 21831–21843.
  * Touvron et al. (2023) Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023\.  Llama 2: Open foundation and fine-tuned chat models.  _arXiv preprint arXiv:2307.09288_ (2023).
  * Van Balen and Levy (2019) Jan Van Balen and Mark Levy. 2019.  PQ-VAE: Efficient Recommendation Using Quantized Embeddings.. In _RecSys (Late-Breaking Results)_. 46–50.
  * Van Den Oord et al. (2017) Aaron Van Den Oord, Oriol Vinyals, et al. 2017\.  Neural discrete representation learning.  _Advances in neural information processing systems_ 30 (2017).
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  _Advances in neural information processing systems_ 30 (2017).
  * Wang et al. (2020) Sinong Wang, Belinda Z Li, Madian Khabsa, Han Fang, and Hao Ma. 2020.  Linformer: Self-attention with linear complexity.  _arXiv preprint arXiv:2006.04768_ (2020).
  * Wang et al. (2022) Yujing Wang, Yingyan Hou, Haonan Wang, Ziming Miao, Shibin Wu, Qi Chen, Yuqing Xia, Chengmin Chi, Guoshuai Zhao, Zheng Liu, et al. 2022\.  A neural corpus indexer for document retrieval.  _Advances in Neural Information Processing Systems_ 35 (2022), 25600–25614.
  * Wu et al. (2023) Di Wu, Bo Sun, and Mingsheng Shang. 2023.  Hyperparameter learning for deep learning-based recommender systems.  _IEEE Transactions on Services Computing_ (2023).
  * Wu et al. (2021) Yongji Wu, Defu Lian, Neil Zhenqiang Gong, Lu Yin, Mingyang Yin, Jingren Zhou, and Hongxia Yang. 2021.  Linear-time self attention with codeword histogram for efficient recommendation. In _Proceedings of the Web Conference 2021_. 1262–1273.
  * Xia et al. (2024) Yan Xia, Hai Huang, Jieming Zhu, and Zhou Zhao. 2024.  Achieving Cross Modal Generalization with Multimodal Unified Representation.  _Advances in Neural Information Processing Systems_ 36 (2024).
  * Yang et al. (2023) Ling Yang, Ye Tian, Minkai Xu, Zhongyi Liu, Shenda Hong, Wei Qu, Wentao Zhang, Bin Cui, Muhan Zhang, and Jure Leskovec. 2023.  Vqgraph: Graph vector-quantization for bridging gnns and mlps.  _arXiv preprint arXiv:2308.02117_ (2023).
  * Yue et al. (2023) Zhenrui Yue, Sara Rabhi, Gabriel de Souza Pereira Moreira, Dong Wang, and Even Oldridge. 2023.  LlamaRec: Two-stage recommendation using large language models for ranking.  _arXiv preprint arXiv:2311.02089_ (2023).
  * Zeghidour et al. (2021) Neil Zeghidour, Alejandro Luebs, Ahmed Omran, Jan Skoglund, and Marco Tagliasacchi. 2021.  Soundstream: An end-to-end neural audio codec.  _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ 30 (2021), 495–507.
  * Zhan et al. (2024) Jun Zhan, Junqi Dai, Jiasheng Ye, Yunhua Zhou, Dong Zhang, Zhigeng Liu, Xin Zhang, Ruibin Yuan, Ge Zhang, Linyang Li, et al. 2024\.  AnyGPT: Unified Multimodal LLM with Discrete Sequence Modeling.  _arXiv preprint arXiv:2402.12226_ (2024).
  * Zhang et al. (2023a) Jin Zhang, Defu Lian, Haodi Zhang, Baoyun Wang, and Enhong Chen. 2023a.  Query-Aware Quantization for Maximum Inner Product Search. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 37. 4875–4883.
  * Zhang et al. (2023b) Jiahui Zhang, Fangneng Zhan, Christian Theobalt, and Shijian Lu. 2023b.  Regularized vector quantization for tokenized image synthesis. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 18467–18476.
  * Zhang et al. (2018) Kunpeng Zhang, Shaokun Fan, and Harry Jiannan Wang. 2018.  An efficient recommender system using locality sensitive hashing.  (2018).
  * Zhao et al. (2021) Jing Zhao, Jingya Wang, Madhav Sigdel, Bopeng Zhang, Phuong Hoang, Mengshu Liu, and Mohammed Korayem. 2021.  Embedding-based recommender system for job to candidate matching on scale.  _arXiv preprint arXiv:2107.00221_ (2021).
  * Zheng et al. (2023) Bowen Zheng, Yupeng Hou, Hongyu Lu, Yu Chen, Wayne Xin Zhao, and Ji-Rong Wen. 2023.  Adapting Large Language Models by Integrating Collaborative Semantics for Recommendation.  _arXiv preprint arXiv:2311.09049_ (2023).
  * Zheng et al. (2024) Sipeng Zheng, Bohan Zhou, Yicheng Feng, Ye Wang, and Zongqing Lu. 2024.  UniCode: Learning a Unified Codebook for Multimodal Large Language Models.  _arXiv preprint arXiv:2403.09072_ (2024).