# BitDelta: Your Fine-Tune May Only Be Worth One Bit

James Liu  Guangxuan Xiao  Kai Li  Jason D. Lee  Song Han  Tri Dao  Tianle Cai
James Liu  Guangxuan Xiao  Kai Li  Jason D. Lee  Song Han  Tri Dao  Tianle Cai

# BitDelta: Your Fine-Tune May Only Be Worth One Bit

James Liu  Guangxuan Xiao  Kai Li  Jason D. Lee  Song Han  Tri Dao  Tianle Cai
James Liu  Guangxuan Xiao  Kai Li  Jason D. Lee  Song Han  Tri Dao  Tianle Cai

###### Abstract

Large Language Models (LLMs) are typically trained in two phases: pre-training
on large internet-scale datasets, and fine-tuning for downstream tasks. Given
the higher computational demand of pre-training, it’s intuitive to assume that
fine-tuning adds less new information to the model, and is thus more
compressible. We explore this assumption by decomposing the weights of fine-
tuned models into their pre-trained components and an additional _delta_. We
introduce a simple method, BitDelta, which successfully quantizes this delta
down to 1 bit without compromising performance. This interesting finding not
only highlights the potential redundancy of information added during fine-
tuning, but also has significant implications for the multi-tenant serving and
multi-tenant storage of fine-tuned models. By enabling the use of a single
high-precision base model accompanied by multiple 1-bit deltas, BitDelta
dramatically reduces GPU memory requirements by more than 10$\times$, which
can also be translated to enhanced generation latency in multi-tenant
settings. We validate BitDelta through experiments across Llama-2 and Mistral
model families, and on models up to 70B parameters, showcasing minimal
performance degradation over all tested settings. Code is avaliable at
https://github.com/FasterDecoding/BitDelta.

Machine Learning, ICML

Machine Learning, ICML

.

##  1 Introduction

After large-scale pretraining, foundation models are typically fine-tuned for
specific downstream tasks (Devlin et al., 2019; Radford et al., 2018, 2019).
This pretrain-finetune paradigm has revolutionized machine learning; LLMs have
not only proven effective for critical tasks such as instruction following and
alignment (Ouyang et al., 2022), but are also performant on a wide array of
niche yet highly impactful applications (Xu et al., 2024; Qiu et al., 2023).
Through fine-tuning, LLMs are adeptly equipped to align with distinct user
preferences or specialized task requirements, showcasing an unprecedented
level of adaptability. Thus, the prospect of serving millions of uniquely
fine-tuned models, each tailored to individual tasks and user needs, presents
a promising vision for the future of machine learning.

Realizing this vision is challenging due to two key reasons: 1) Expensive
Storage. Each new fine-tuned model is large, even if we have relatively few
base models, making them expensive to store and challenging to manage on disk.
2) Expensive Serving. Distinct fine-tuned models each demand significant GPU
memory, making it difficult and expensive to concurrently serve such models
without noticeable downtime. To tackle these issues, we decompose the fine-
tuned model weights into the weights of the base pre-trained model and a
_delta_ induced by the fine-tuning process. By compressing this delta while
maintaining model performance, we aim to sidestep the prohibitive costs
associated with storage and GPU memory demands.

From the delta decomposition point of view, parameter-efficient fine-tuning
(PEFT) methods like LoRA (Hu et al., 2021; Houlsby et al., 2019a; Rebuffi et
al., 2017; Dettmers et al., 2023; Chen et al., 2023d) effectively enforce a
highly structured and compressed form of delta _during fine-tuning_ , a
powerful insight for model serving of PEFT-based fine-tunes. (Sheng et al.,
2023) and (Chen et al., 2023b) explore multi-tenant serving of LoRA-based
fine-tunes.

Figure 1: Overview of BitDelta. BitDelta applies 1-bit quantization to the
weight delta between fine-tuned and base models. For each weight matrix, we
quantize its delta as its sign bits and a trainable high-precision scale
factor. The scale factor is initialized to achieve the best approximation
error in $L_{2}$ norm and further refined with a few distillation steps.
BitDelta shows minimal degradation in model performance and reduces memory
consumption in multi-tenancy serving by representing multiple fine-tuned
models with a single high-precision base model and multiple 1-bit deltas.

Nevertheless, recent work has shown that PEFT methods may not yet match the
model quality of full parameter fine-tuning, especially on high resource tasks
(Chen et al., 2022), and are fairly sensitive to hyperparameter choice and
prompting methods (Niederfahrenhorst et al., 2023). As a result, we notice
that among the 2307 LLMs (as of time of writing) on the Open LLM Leaderboard
(Beeching et al., 2023) with a valid README file, only $<20\%$ indicate that
they exclusively use LoRA. Most models are full parameter fine-tunes, model
merges (Yu et al., 2023; Jin et al., 2023; Wortsman et al., 2022) of full
parameter fine-tunes, or model merges of LoRA based fine-tunes (which are
effectively high-rank).

It is also attractive to approximate general deltas with low-rank matrices
_post-training_. However, experimental results show that this is challenging
(Table 1), as deltas from full parameter fine-tunes tend to be fairly high-
rank (Figure 2).

Figure 2: CEV plot of a $4096\times 4096$ weight delta between Llama 2-7B and
Vicuna-7B v1.5. Deltas from full parameter fine-tuning are fairly high rank,
making low-rank approximations difficult.

Instead, we draw from the insight that motivates PEFT methods in general:
Given the higher computational demand of pre-training, it’s intuitive to
assume that fine-tuning adds less new information to the model, and is thus
_much_ more compressible. Indeed, we find that we can efficiently _quantize_
the delta to merely _1 bit_ with almost no performance drop. We propose
BitDelta, an efficient post-training quantization (PTQ) solution that acts on
the weight delta between a fine-tuned model and its underlying base model.

BitDelta has two stages: 1) We quantize the delta of each weight matrix into a
scaling factor multiplied by a binary matrix. Specifically, we take the sign
of the delta to form the binary matrix and initialize the scaling factor as
the average of the absolute values of the delta, which minimizes $L_{2}$ norm.
2) We further calibrate the scaling factors through model distillation over a
small calibration dataset while keeping the binary matrices frozen. Despite
the small number of trainable parameters and training steps, we find that this
distillation process is effective in further recovering model quality.
BitDelta is extremely efficient compared to other quantization methods; we are
able to compress 70B models in roughly 10 minutes, whereas methods like GPTQ
(Frantar et al., 2022) and AWQ (Lin et al., 2023) may take multiple GPU hours.
Our experiments over 17 popular fine-tuned models affirm that BitDelta can be
applied across various model types and sizes with minimal impact on
performance.

BitDelta opens up opportunities to efficiently serve multiple fine-tuned
models with shared servers: By only storing a single full-precision base
model, and (dynamically) loading and performing batched inference over
multiple 1-bit deltas, we can efficiently represent multiple fine-tuned
models. Compared to naively using full precision fine-tuned models, deltas
compressed by BitDelta are more than 10$\times$ smaller, consuming less GPU
memory, and can therefore be loaded faster. This addresses the storage
challenge. Moreover, since LLM inference is memory-bound (Leviathan et al.,
2022; Chen et al., 2023a; Cai et al., 2024), the latency of each decoding step
is proportional to the GPU memory consumption of the model weights. With an
efficient CUDA kernel implementation, we can translate this memory reduction
into a latency speedup, similar to other quantization methods (Frantar et al.,
2022; Lin et al., 2023). As a preliminary attempt, we implement a 1-bit matrix
multiplication kernel in Triton (Tillet et al., 2019) and show that it can
improve the multi-tenant serving latency by about 2$\times$, addressing the
serving challenge.

Finally, we study a few extensions of BitDelta, where we quantize the base
model, and where we iteratively apply BitDelta. Experimental results show that
our method is quite general and can be applied to various use cases.

Table 1: Comparison between BitDelta and a SVD based method ($r=16$), with Llama 2-7B and Vicuna-7B v1.5 as the base and fine-tuned models. BitDelta is performant across the board, whereas the SVD-based method fails to sufficiently capture the fine-tuned information. Model/Method | Train Loss | TruthfulQA | GSM8K | MT-Bench | Adjusted Average2.1 $\uparrow$
---|---|---|---|---|---
Llama 2-7B | – | 38.96 | 13.57 | – | 60.53
Vicuna-7B v1.5 | – | 50.36 | 19.03 | 6.04 | 60.51
BitDelta-Initial | 0.41 | 47.63 | 19.56 | 5.67 | 60.99
BitDelta | 0.052 | 49.97 | 20.17 | 5.99 | 60.68
SVD-Initial | 0.78 | 44.56 | 15.92 | 4.45 | 60.60
SVD | 0.34 | 44.55 | 14.18 | 4.98 | 60.20

##  2 Related Work

###  2.1 Full Model Compression

#### Quantization.

Quantization techniques are widely used to reduce memory consumption and
improve LLMs’ generation latency. Xiao et al. (2023) implement a technique
that rescales between activations and parameters, effectively mitigating
outlier activations to facilitate smoother quantization. Dettmers et al.
(2022) develop an approach that decomposes matrix multiplications into 8-bit
computations, with an additional 16-bit process for handling outliers.
Exploring further, Frantar et al. (2022) introduce a method that iteratively
rounds weight columns to 3-4 bits of precision. Similarly, Lin et al. (2023)
propose an activation-aware quantization scheme that selectively preserves
crucial weights while compressing the majority to 3-4 bits. In a different
vein, Kim et al. (2023) devise a sparse, low-precision pattern focusing on a
small yet significant set of weights. Lastly, Chee et al. (2023) utilize
incoherence processing to quantize model weights to as low as 2 bits with
minimal impact on performance.

#### Pruning.

Pruning also aims to reduce the memory consumption of neural networks. It
accomplishes this by pushing certain parameter values to zero, inducing
sparsity in the model (LeCun et al., 1989; Han et al., 2015, 2016; Zhu &
Gupta, 2017). However, these methods may fail to take advantage of modern
hardware like GPUs unless using certain structured sparsity patterns like 2:4
(50%) sparsity (Mishra et al., 2021). Frantar & Alistarh (2023) demonstrate a
pruning method on LLMs that successfully utilizes the 2:4 sparsity pattern and
achieves a 50% sparsity ratio. It is challenging to obtain higher sparsity
while being hardware-friendly.

22footnotetext: Adjusted Average is over ARC, BBH, HellaSwag, WinoGrande, and
excludes TruthfulQA, GSM8K, MT-Bench.

###  2.2 Delta Compression

#### Parameter-efficient fine-tuning.

Parameter-efficient fine-tuning (PEFT) techniques reduce the number of
trainable parameters during fine-tuning, reducing memory and compute demand
while achieving promising accuracy. Houlsby et al. (2019b) introduce adapter
layers, freezing the base model during fine-tuning. Hu et al. (2021) propose
Low-Rank Adaptation (LoRA), which enforces that weight updates during fine-
tuning are low rank.

By enforcing structure in the fine-tuning information, PEFT techniques can
viewed as compression-aware training. Similar to our method, such compression
induced by PEFT techniques can also enhance multi-tenant serving. Chen et al.
(2023b) and Sheng et al. (2023) explore this idea by developing scalable
multi-tenant systems for LoRA-based fine-tunes, based on customized CUDA
kernels that exploit the low-rank structure of the fine-tuned weight deltas.

#### Early work on post-training delta compression.

Most related to our work, a few studies explore the idea of post-training
delta compression by adopting existing compression techniques like GPTQ,
unstructured pruning (Han et al., 2016), or even classic lossless compression
algorithms. Isik et al. (2023) focus on reducing the delta size to save
storage. Yu et al. (2023) utilize pruning to improve model merging
applications. Yadav et al. (2023) reduces the size of PEFT modules to save
storage. Ryu et al. (2023) combines quantization with low-rank approximation
to reduce the delta size. The concurrent and independent work by Yao &
Klimovic (2023) also explores using delta compression to improve multi-tenant
serving, but focuses more on reducing the model loading time from disk to GPU.
Compared to existing work, we offer a much simpler and faster method,
BitDelta, achieving a more than 10$\times$ compression ratio while also being
friendly to modern accelerators.

##  3 BitDelta

###  3.1 Method

BitDelta consists of two stages: 1) We quantize each weight matrix into a
scalar multiplied by a binary matrix***In our experiments, we only quantize
the linear layers in the Transformer blocks as they contribute most of the
parameters and computation.. 2) We further calibrate the scalar factors with
distillation. We describe each stage in this section:

#### 1-bit quantization.

Let $W_{\text{base}},W_{\text{fine}}\in\mathbb{R}^{n\times m}$ be weight
matrices from the base model and fine-tuned model, respectively. We define the
weight delta as $\Delta=W_{\text{fine}}-W_{\text{base}}$, representing the
modification in weights post-fine-tuning. For efficient representation, we aim
to obtain a binarized estimator of this weight delta, denoted as
$\hat{\Delta}$, by encoding its sign bits:

$\hat{\Delta}=\alpha\odot\text{Sign}(\Delta),$ (1)

where

$\text{Sign}(W_{ij})=\begin{cases}+1,&\text{if }W_{ij}>0,\\\ -1,&\text{if
}W_{ij}\leq 0,\end{cases}$ (2)

and $\alpha$ is a high-precision scaling factor for the entire matrix. To
minimize the quantization error in $L_{2}$ norm:

$\left\|\Delta-\hat{\Delta}\right\|_{2}^{2}=\sum_{ij}\left(\left|W_{ij}\right|-\alpha\right)^{2},$
(3)

we take

$\alpha=\frac{1}{nm}\sum_{ij}|W_{ij}|.$ (4)

Surprisingly, we find that the above quantization approach already does quite
well and retains most of the fine-tuned models’ performance.

Table 2: BitDelta works on Llama-2 and Mistral families and on a wide range of model sizes ranging from 7B to 70B parameters. BitDelta works for many types of fine-tuned information, including SFT-based methods, RLHF-based methods, and context extension methods (RoPE scaling). Scale distillation is effective; it raises TruthfulQA/GSM8K scores to within 1-2 points of the baseline fine-tune, and MT-Bench scores to within 0.1-0.2 points. Model | Method | TruthfulQA | GSM8K | MT-Bench | Adjusted Average2.1 $\uparrow$
---|---|---|---|---|---
Llama 2-7B | – | 38.96 | 13.57 | – | 60.53
Llama 2-7B Chat | Baseline | 45.32 | 22.74 | 6.56 | 59.81
BitDelta-Initial | 41.10 | 18.27 | 6.31 | 60.7
BitDelta | 44.95 | 20.24 | 6.47 | 59.88
Vicuna-7B v1.5 16k | Baseline | 50.38 | 14.18 | 6.06 | 57.50
BitDelta-Initial | 45.58 | 13.95 | 5.69 | 58.51
BitDelta | 48.75 | 14.48 | 6.24 | 57.64
Llama 2-13B | – | 36.90 | 22.74 | – | 64.68
Llama 2-13B Chat | Baseline | 43.95 | 33.13 | 6.98 | 63.99
BitDelta-Initial | 41.70 | 33.36 | 7.06 | 64.25
BitDelta | 43.47 | 31.92 | 6.95 | 63.96
Vicuna-13B v1.5 16k | Baseline | 50.38 | 29.72 | 6.90 | 57.5
BitDelta-Initial | 41.7 | 26.76 | 6.60 | 64.25
BitDelta | 48.75 | 28.73 | 6.88 | 57.64
WizardLM-13B v1.2 | Baseline | 47.17 | 42.38 | 6.95 | 61.61
BitDelta-Initial | 44.89 | 42.08 | 6.73 | 61.91
BitDelta | 46.67 | 41.62 | 6.93 | 61.86
Llama 2-70B | – | 44.82 | 52.69 | – | 71.81
Llama 2-70B Chat | Baseline | 52.77 | 47.61 | 7.12 | 68.82
BitDelta-Initial | 41.63 | 42.38 | 6.85 | 66.01
BitDelta | 51.37 | 48.82 | 7.06 | 69.14
Solar-0-70B | Baseline | 62.03 | 56.18 | 7.07 | 73.77
BitDelta-Initial | 59.08 | 56.79 | 6.79 | 73.14
BitDelta | 62.03 | 56.63 | 6.82 | 73.57
Mistral-7B v0.1 | – | 42.60 | 37.76 | – | 65.98
Mistral-7B v0.1 Instruct | Baseline | 55.93 | 32.75 | 6.86 | 60.36
BitDelta-Initial | 51.27 | 38.82 | 6.54 | 63.83
BitDelta | 55.23 | 31.54 | 6.43 | 61.10
Zephyr-7B-$\beta$ | Baseline | 55.12 | 34.34 | 7.18 | 65.22
BitDelta-Initial | 54.53 | 40.26 | 6.70 | 66.12
BitDelta | 58.39 | 31.92 | 7.00 | 66.20
Dolphin 2.2.1 | Baseline | 54.02 | 54.28 | 7.36 | 67.31
BitDelta-Initial | 48.14 | 50.27 | 7.10 | 67.58
BitDelta | 54.91 | 52.84 | 7.20 | 66.97
Table 3: Comparison of model responses from Zephyr-7B-$\beta$ for Question 9
in MT-Bench, a concise advertisement task. BitDelta-Initial is unable to
follow the instructions, producing an advertisement that is overly formal and
makes no attempt to adhere to the word limit. With the addition of scale
distillation, BitDelta successfully produces a concise, catchy advertisement
slightly over the word limit. *Prompt slightly modified for clarity.

Prompt |  Please create a slogan about bioenergy that satisfies the following: 1. Make the tone sound casual 2. Embed an advertisement for a company called ”FlexPower” 3. Fewer than 10 words.
---|---
BitDelta |  ”FlexPower’s groundbreaking bioenergy solution: Produce fuel without compromising food production!” (10 words)

This headline is catchy, casual, and highlights the unique selling point of
the…
GPT-4 Score: 8
BitDelta-Initial |  ”FlexPower’s groundbreaking technology unlocks the key to harnessing the power of renewable bioenergy while minimizing ethical concerns”
GPT-4 Score: 4

#### Scale distillation.

Intuitively, the scaling factor $\alpha$ plays a more significant role in the
low-bit regime. Additionally, per-matrix $L_{2}$ weight error is not a perfect
measure of degradation in _overall_ model quality. Thus, we further optimize
these scales by performing model distillation to align the output logits of
the quantized model to that of the original fine-tuned model. More concretely,
we freeze the model weights and optimize for the following objective:

$\bm{\alpha}^{*}=\arg\min_{\bm{\alpha}}\mathbb{E}_{x\sim\mathbf{X}}\left[\left\|\mathbf{Z}_{\text{fine}}(x)-\mathbf{Z}_{\text{bin}}(x;\bm{\alpha})\right\|^{2}\right]$
(5)

where $\mathbf{X}$ is a calibration dataset, and $\mathbf{Z}(\cdot)$ are the
logits of the respective models. We find that scale distillation is fairly
insensitive to choice $\mathbf{X}$, as 1) the process is extremely parameter
efficient, and 2) the crucial aspect of the process is to logit match with the
fine-tuned model, regardless of the actual text content.

For our experiments, we distill on the C4 dataset (Raffel et al., 2023), which
is widely used for pre-training, using 800 samples of length 128, and use the
Adam optimizer (Kingma & Ba, 2017) with $lr=10^{-4}$, $\beta=(0.9,0.999)$,
$\epsilon=10^{-8}$. 1x80 GB A100 GPU is used to distill 7B and 13B models, and
6x80GB A100 GPUs are used to distill 70B models (2x for finetune, 4x for
binarized). Sale distillation is fast; we can compress 70B models in roughly
10 minutes, whereas methods like GPTQ (Frantar et al., 2022) and AWQ (Lin et
al., 2023) take multiple hours.

###  3.2 Implication

The ability to compress the delta to merely 1-bit opens up multiple
opportunities for improving efficiency, enabling more effective model storage
(Isik et al., 2023) – where a single base model can be maintained alongside
multiple compressed deltas – and facilitating model hot-swapping (Chen et al.,
2023b; Sheng et al., 2023). With hot-swapping, the base model remains in GPU
memory, and compressed deltas are dynamically loaded in accordance to incoming
requests. In both cases, the compression ratio can be directly translated into
reductions in storage needs and loading times.

Moreover, BitDelta enables the possibility of a multi-tenant serving system
like Punica (Chen et al., 2023b) or S-LoRA (Sheng et al., 2023) but for
general fine-tuned models instead of just LoRA models. Concretely, we consider
the scenario where multiple models fine-tuned from the same base model are
served with the same server. This setting greatly exploits the GPU resource
and saves each fine-tuned model’s inference cost when their traffic is low or
unbalanced. With BitDelta, we can keep one high-precision base model with
multiple compressed deltas in the GPU memory. Compared to directly serving
multiple fine-tuned models, this approach greatly saves memory consumption.

Since LLM inference follows the memory-bound computation pattern where the
generation latency is proportional to the GPU memory used by the model
weights, the lower memory consumption also suggests the opportunity to improve
the serving latency. For example, Punica and S-LoRA exploit LoRA’s structure
and memory saving by developing a CUDA kernel that can efficiently calculate
the batched delta-activation product for low rank deltas. Similarly, we
decompose the forward pass of each linear layer as follows:

$X^{\prime}_{i}=W_{\text{fine},i}X_{i}\approx
W_{\text{base}}X_{i}+\underbrace{\hat{\Delta}_{i}X_{i}}_{\text{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}{\text{Kernel}}}}$
(6)

where $X_{i}$ and $X_{i}^{\prime}$ represent input and output features to the
$i$-th fine-tuned model, and the base model weight and the delta are computed
separately. For a batch of requests, $W_{\text{base}}X_{i}$ can be computed
with the classic batched GEMM kernel. We implement a fused binary GEMM kernel
in Triton (Tillet et al., 2019) that allows us to calculate
$\hat{\Delta}_{i}X$ in a batched setting while keeping the 1-bit deltas
quantized until they are transferred to the GPU cache. This kernel fuses the
dequantization operation with the GEMM calculation, reducing the data moving
overhead by a large factor.

Table 4: We apply BitDelta to Llama 2-7B Chat, and find it holds up when the underlying base model is quantized at various levels. FP16 + $\Delta$ outperforms baseline GPTQ across the board, implying that in terms of model quality, we would rather store a single high-precision base model with many 1-bit deltas than store many quantized fine-tuned models. GPTQ + $\Delta$ with Llama 2-7B Chat as the base model also outperforms baseline GPTQ on many evaluations, because the delta diffuses 16-bit information through high precision scaling factors. Base Model | Method | TruthfulQA | GSM8K | MT-Bench | Adjusted Average2.1 $\uparrow$
---|---|---|---|---|---
Baseline | FP16 | 45.32 | 22.74 | 6.56 | 59.81
INT8 RTN | 45.02 | 22.29 | 6.28 | 59.63
GPTQ | 44.92 | 19.48 | 5.90 | 58.67
Llama 2-7B | FP16 + $\Delta$ | 44.95 | 20.24 | 6.47 | 59.88
INT8 RTN + $\Delta$ | 44.71 | 19.86 | 6.16 | 59.85
GPTQ + $\Delta$ | 42.52 | 19.94 | 6.02 | 59.22
Llama 2-7B Chat | GPTQ + $\Delta$ | 44.63 | 22.14 | 6.11 | 59.17

##  4 Experiments

###  4.1 Setup

#### Baselines.

Our primary baselines are the original fine-tuned models without compression.
We also compare with 8-bit RTN and 4-bit GPTQ (Frantar et al., 2022) on
evaluations where we run BitDelta on quantized base models.

#### Models and datasets.

We benchmark fine-tuned models based on the Llama-2 (Touvron et al., 2023) and
Mistral (Jiang et al., 2023) model families: Vicuna, Xwin-LM, Solar-70B,
Zephyr, OpenChat 3.5, Dolphin 2.2.1, and OpenOrca (Chiang et al., 2023; Team,
2023; Upstage, 2023; Tunstall et al., 2023; Wang et al., 2023; Hartford, 2023;
Mukherjee et al., 2023). We evaluate on eight tasks: MT-Bench, 25-shot ARC
Challenge, 5-shot BBH, 10-shot HellaSwag, zero-shot TruthfulQA, zero-shot
LAMBADA, zero-shot Winogrande, and 5-shot GSM8K (Zheng et al., 2023; Clark et
al., 2018; Suzgun et al., 2022; Zellers et al., 2019; Lin et al., 2022;
Paperno et al., 2016; Sakaguchi et al., 2019; Cobbe et al., 2021). We use
FastChat (Zheng et al., 2023) to evaluate on MT-Bench, and use lm-evaluation-
harness (Gao et al., 2023) to evaluate on the other tasks. We denote our
methodology before scale distillation is applied as BitDelta-Initial.

We primarily focus on high-margin metrics where fine-tuning is significantly
impactful and aggregate the other metrics. See Tables 6 to 9 in the Appendix
for full results. BitDelta performs quite well on the aggregated metrics, even
outperforming the baseline in many cases. However, it’s important to
contextualize these results with regard to the base model itself, which is
also performant on these metrics.

In such cases, it’s difficult to attribute performance to our methodology or
to the underlying base model. Because of this, we highlight TruthfulQA, GSM8K,
and MT-Bench, which base models tend to struggle on, to show that BitDelta
accurately preserves fine-tune information.

###  4.2 Accurate Quantization

#### SVD comparison.

We compare BitDelta to a low rank approx. of the weight delta on Vicuna-7B
v1.5. For the low rank approx., we decompose $\Delta=U\Sigma V$ and
approximate $\hat{\Delta}=AB$ where $A=U\sqrt{\hat{\Sigma}}$,
$B=\sqrt{\hat{\Sigma}}V$. During distillation, we treat all entries of the low
rank matrices as trainable parameters. We find that the low rank approx. fails
to fully capture the fine tune information, and underperforms across the board
(Table 1). Interestingly, we find that distillation is not as effective for
the low rank approx. compared to BitDelta, though we partly attribute this to
the excess of trainable parameters leading to training instability.

#### Main Results.

BitDelta is performant across various model families, across a wide range of
model sizes, and across many fine-tuning techniques. We benchmark on Llama-2
and Mistral families, and on models ranging from 7B to 70B parameters. Shown
in Table 2, we find that BitDelta is very general and can recover all types of
finetune information, including SFT-based methods (Radford et al., 2018) on
Mistral-7B v0.1 Instruct, RLHF-based methods (Christiano et al., 2023) on
Llama 2 Chat, and context extension methods (RoPE scaling) (Chen et al.,
2023c; Press et al., 2022) on Vicuna-7B v1.5 16k.

Figure 3: As the fidelity of $\Delta$ increases, the TruthfulQA scores of
Llama 2-7B \+ $\Delta$ approaches that of Vicuna-7B v1.5.

We note that GSM8K for BitDelta-Initial on Mistral-7B v0.1 Instruct and
Zephyr-7B-$\beta$ is abnormally high; we attribute this to how performant the
base model Mistral-7B v0.1 is on this task in comparison.

Scale distillation is effective. Introducing scale distillation raises
TruthfulQA and GSM8K scores to within 1-2 points of the baseline fine-tune,
and generally raises MT-Bench scores to within 0.1-0.2 points.

#### Case Study.

We present a sample response from Zephyr-7B-$\beta$ in Table 3, highlighting
the efficacy of scale distillation. BitDelta-Initial does not have a casual
tone, and makes no attempt to adhere to the word limit. With the introduction
of scale distillation, BitDelta exhibits greater instruction following
capabilities, producing a catchy response that slightly exceeds the word
limit.

#### Quantized base models.

Because 8-bit RTN and GPTQ work with 16-bit activations, we can keep the fine-
tune weights $W_{\text{fine}}$ and scaling factors $\alpha$ in high precision,
only quantizing the base weights $W_{\text{base}}$. As shown in Table 4, we
find that BitDelta is still performant when applied to quantized base models.

We observe that FP16 + $\Delta$ outperforms GPTQ across the board. In the
performance engineering context of multi-tenancy serving, as we add more
models, we would rather store a single high precision base model with many
1-bit deltas than store many quantized fine-tuned models. This interesting
result implies that the above also holds true in the _model quality_ context
of multi-tenancy serving.

We try using Llama 2-7B Chat as both the base model and fine-tune model,
quantizing the base model using GPTQ, and find that we’re able to outperform
baseline GPTQ on many evaluations. We hypothesize this is because we can
diffuse 16-bit information into the model through high precision scaling
factors, at the cost of including a 1-bit delta.

#### Ablation over fidelity of $\Delta$.

By successively applying BitDelta, treating the compressed model from the
previous iteration as our base model, we can vary the granularity over the
delta, associating it with multiple 1-bit masks. One advantage of doing this
is the ability to assign arbitrary scale factors to each 1-bit mask. In
contrast, when increasing the bit size, scale factors are implicitly fixed
with respect to each other. Figure 3 shows how the TruthfulQA of Llama 2-7B
plus an increasingly granular delta approaches that of Vicuna-7B v1.5. Full
results are in Table 8.

Table 5: BitDelta achieves over 10$\times$ compression. We can further
compress the embedding and LM head layers, but leave this to future work due
to inconsistencies in tokenizer vocabularies.

Base Model | Size |  $\Delta$ Size | Comp. Factor
---|---|---|---
Llama 2-7B | 13.48 GB | 1.24 GB | 10.87
Llama 2-13B | 26.03 GB | 2.09 GB | 12.45
Llama 2-70B | 137.95 GB | 8.95 GB | 15.41
Mistral-7B v0.1 | 14.48 GB | 1.30 GB | 11.14

###  4.3 Latency Improvement

To illustrate the idea of translating memory saving into improved latency, we
implement a simple Triton kernel for GEMM with a binary matrix and scaling
factor, as in BitDelta. We assume there are $B$ fine-tuned models served on
the same GPU. For simplicity, we consider the setting where the requests are
uniformly distributed, i.e., each model receives one request simultaneously.
Following the decomposition in Eq. (6), the kernel is used to compute the
batched matrix multiplication between $B$ binary matrices ($N\times M$) and
$B$ high-precision activations ($L\times N$) where $N,M$ are intermediate
dimensions and $L$ is the sequence length. We focus on decoding latency as
opposed to the prefill latency, as it consumes most of the time. Tokens are
generated one by one in decoding, meaning $L$ is always 1.

#### Kernel latency.

We benchmark the decoding latency of our kernel, a batched linear operation
over multiple deltas with a single base model, as in Eq. (6), corresponding to
a single linear layer, and compare against naively computing the forward pass
separately for each model. We ablate across the batch size and hidden size
dimensions and find that our kernel consistently achieves about 2$\times$
speedup.

Figure 4: Decoding latency of a linear layer with and without BitDelta. Blue:
Naive forward pass with $B$ distinct fine-tuned models. Yellow: Batched
forward pass with BitDelta, corresponding to one base model and $B$ 1-bit
deltas, utilizing a Triton kernel. Left: Ablation over hidden size, assuming
$N=M$ and $B=8$. Right: Ablation over batch size, assuming $N=M=8192$.

#### End-to-end latency.

We also benchmark the end-to-end decoding latency on Llama 2-7B variants with
an input length of 128 (we find the decoding latency is less sensitive to the
input length), ablated across the batch size. We find that BitDelta introduces
overhead when the batch size is low. However, BitDelta scales better and
successfully translates the saved GPU memory to improved decoding latency,
starting at $B=4$. This is exacerbated at higher batch sizes, where the naive
approach succumbs to out-of-memory issues and BitDelta is still performant.

Figure 5: End-to-end decoding latency of Llama 2-7B variants with and without
BitDelta. Blue: Naive forward pass with $B$ distinct fine-tuned models.
Orange: Projected values for the naive forward pass. Green: Batched forward
pass with BitDelta. The naive forward pass succumbs to GPU memory issues at
higher batch sizes, whereas BitDelta is still performant.

##  5 Conclusion and Discussion

We propose BitDelta, a simple yet effective approach for efficiently
quantizing the weight delta arising from fine-tuning in LLMs down to 1 bit.
BitDelta encodes the sign bits of the weight delta and a per-weight matrix
scaling factor, which is calibrated further through distillation. This allows
for representing multiple full-parameter fine-tuned models with one base model
and multiple 1-bit deltas, enhancing applications in multi-tenancy serving by
reducing GPU memory requirements and improving generation latency. BitDelta is
fast and accurate, showcasing minimal performance degradation, and opens new
avenues for efficient model deployment and resource utilization in machine
learning.

#### Future work.

We mainly focus on the qualitative evaluation of BitDelta, as quality is the
main challenge that most quantization methods struggle with. We show BitDelta
can maintain the quality of the original fine-tuned models and further
illustrate the idea of how to translate the memory saving into improved
latency with a fused kernel. Yet, there is significant room for improvement –
our kernel is fairly slow compared to the theoretical limit, considering the
small memory footprint of binary matrices. It may also be possible to achieve
better model quality by preserving salient weight deltas. Finally, the idea of
calibrating certain scale factors through distillation may be applied more
generally to PTQ methods. We hope our work motivates future refinement of
scale factors, making low-bit quantized LLMs more robust.

##  6 Impact Statement

#### Environmental Sustainability and Cost Reduction.

The reduction in GPU memory requirements through BitDelta translates to lower
energy consumption and a reduction in costs associated with serving multiple
fine-tuned models. By significantly lowering hardware requirements, BitDelta
contributes to making AI technologies more eco-friendly and economically
viable. This approach aligns with the growing need for sustainable and cost-
effective computing solutions in the deployment of large-scale AI
technologies.

#### Democratization of Fine-tuned Models.

By dramatically reducing the hardware requirements for serving fine-tuned
models, BitDelta enables smaller entities to deploy state-of-the-art models
more feasibly. This can accelerate innovation and application development
across various industries and academic fields, making fine-tuned models
accessible to a wider audience.

#### Dealignment Mitigation.

BitDelta is a lossy compression method on the fine-tune information in LLMs.
As such, crucial alignment information may be lost in the process of
compression. We believe this is an important consequence to highlight, as
BitDelta democratizes multi-tenant applications which may exacerbate this
dealignment concern. We encourage further work on evaluation techniques to
detect alignment loss in BitDelta, which can lead to the creation of robust
methods for its mitigation.

## Acknowledgments

We thank Together AI, MyShell AI, National Science Foundation (NSF), MIT-IBM
Watson AI Lab, and MIT Amazon Science Hub for supporting this research.

## References

  * Beeching et al. (2023) Beeching, E., Fourrier, C., Habib, N., Han, S., Lambert, N., Rajani, N., Sanseviero, O., Tunstall, L., and Wolf, T.  Open llm leaderboard.  https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard, 2023.
  * Cai et al. (2024) Cai, T., Li, Y., Geng, Z., Peng, H., Lee, J. D., Chen, D., and Dao, T.  Medusa: Simple llm inference acceleration framework with multiple decoding heads.  _arXiv preprint arXiv: 2401.10774_ , 2024.
  * Chee et al. (2023) Chee, J., Cai, Y., Kuleshov, V., and De Sa, C.  Quip: 2-bit quantization of large language models with guarantees.  _arXiv preprint arXiv:2307.13304_ , 2023.
  * Chen et al. (2023a) Chen, C., Borgeaud, S., Irving, G., Lespiau, J.-B., Sifre, L., and Jumper, J.  Accelerating large language model decoding with speculative sampling.  February 2023a.  doi: 10.48550/ARXIV.2302.01318.
  * Chen et al. (2022) Chen, G., Liu, F., Meng, Z., and Liang, S.  Revisiting parameter-efficient tuning: Are we really there yet?, 2022.
  * Chen et al. (2023b) Chen, L., Ye, Z., Wu, Y., Zhuo, D., Ceze, L., and Krishnamurthy, A.  Punica: Multi-tenant lora serving, 2023b.
  * Chen et al. (2023c) Chen, S., Wong, S., Chen, L., and Tian, Y.  Extending context window of large language models via positional interpolation, 2023c.
  * Chen et al. (2023d) Chen, Y., Qian, S., Tang, H., Lai, X., Liu, Z., Han, S., and Jia, J.  Longlora: Efficient fine-tuning of long-context large language models, 2023d.
  * Chiang et al. (2023) Chiang, W.-L., Li, Z., Lin, Z., Sheng, Y., Wu, Z., Zhang, H., Zheng, L., Zhuang, S., Zhuang, Y., Gonzalez, J. E., Stoica, I., and Xing, E. P.  Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023.  URL https://lmsys.org/blog/2023-03-30-vicuna/.
  * Christiano et al. (2023) Christiano, P., Leike, J., Brown, T. B., Martic, M., Legg, S., and Amodei, D.  Deep reinforcement learning from human preferences, 2023.
  * Clark et al. (2018) Clark, P., Cowhey, I., Etzioni, O., Khot, T., Sabharwal, A., Schoenick, C., and Tafjord, O.  Think you have solved question answering? try arc, the ai2 reasoning challenge, 2018.
  * Cobbe et al. (2021) Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., et al.  Training verifiers to solve math word problems.  _arXiv preprint arXiv:2110.14168_ , 2021.
  * Dettmers et al. (2022) Dettmers, T., Lewis, M., Belkada, Y., and Zettlemoyer, L.  Llm. int8 (): 8-bit matrix multiplication for transformers at scale.  _arXiv preprint arXiv:2208.07339_ , 2022.
  * Dettmers et al. (2023) Dettmers, T., Pagnoni, A., Holtzman, A., and Zettlemoyer, L.  Qlora: Efficient finetuning of quantized llms.  _arXiv preprint arXiv:2305.14314_ , 2023.
  * Devlin et al. (2019) Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K.  Bert: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pp. 4171–4186, 2019.
  * Frantar & Alistarh (2023) Frantar, E. and Alistarh, D.  Sparsegpt: Massive language models can be accurately pruned in one-shot, 2023.
  * Frantar et al. (2022) Frantar, E., Ashkboos, S., Hoefler, T., and Alistarh, D.  Gptq: Accurate post-training quantization for generative pre-trained transformers.  _arXiv preprint arXiv:2210.17323_ , 2022.
  * Gao et al. (2023) Gao, L., Tow, J., Abbasi, B., Biderman, S., Black, S., DiPofi, A., Foster, C., Golding, L., Hsu, J., Le Noac’h, A., Li, H., McDonell, K., Muennighoff, N., Ociepa, C., Phang, J., Reynolds, L., Schoelkopf, H., Skowron, A., Sutawika, L., Tang, E., Thite, A., Wang, B., Wang, K., and Zou, A.  A framework for few-shot language model evaluation, 12 2023.  URL https://zenodo.org/records/10256836.
  * Han et al. (2015) Han, S., Pool, J., Tran, J., and Dally, W. J.  Learning both weights and connections for efficient neural networks, 2015.
  * Han et al. (2016) Han, S., Mao, H., and Dally, W. J.  Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding, 2016.
  * Hartford (2023) Hartford, E.  Cognitivecomputations/dolphin-2.2.1-mistral-7b, hugging face, 2023.  URL https://huggingface.co/cognitivecomputations/dolphin-2.2.1-mistral-7b.
  * Houlsby et al. (2019a) Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., De Laroussilhe, Q., Gesmundo, A., Attariyan, M., and Gelly, S.  Parameter-efficient transfer learning for nlp.  In _International Conference on Machine Learning_ , pp. 2790–2799. PMLR, 2019a.
  * Houlsby et al. (2019b) Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., de Laroussilhe, Q., Gesmundo, A., Attariyan, M., and Gelly, S.  Parameter-efficient transfer learning for nlp, 2019b.
  * Hu et al. (2021) Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., and Chen, W.  Lora: Low-rank adaptation of large language models.  _ICLR_ , 2021.
  * Isik et al. (2023) Isik, B., Kumbong, H., Ning, W., Yao, X., Koyejo, S., and Zhang, C.  GPT-zip: Deep compression of finetuned large language models.  In _Workshop on Efficient Systems for Foundation Models @ ICML2023_ , 2023.  URL https://openreview.net/forum?id=hO0c2tG2xL.
  * Jiang et al. (2023) Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., de las Casas, D., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., Lavaud, L. R., Lachaux, M.-A., Stock, P., Scao, T. L., Lavril, T., Wang, T., Lacroix, T., and Sayed, W. E.  Mistral 7b, 2023.
  * Jin et al. (2023) Jin, X., Ren, X., Preotiuc-Pietro, D., and Cheng, P.  Dataless knowledge fusion by merging weights of language models, 2023.
  * Kim et al. (2023) Kim, S., Hooper, C., Gholami, A., Dong, Z., Li, X., Shen, S., Mahoney, M. W., and Keutzer, K.  Squeezellm: Dense-and-sparse quantization.  _arXiv preprint arXiv:2306.07629_ , 2023.
  * Kingma & Ba (2017) Kingma, D. P. and Ba, J.  Adam: A method for stochastic optimization, 2017.
  * LeCun et al. (1989) LeCun, Y., Denker, J., and Solla, S.  Optimal brain damage.  In Touretzky, D. (ed.), _Advances in Neural Information Processing Systems_ , volume 2. Morgan-Kaufmann, 1989.  URL https://proceedings.neurips.cc/paper_files/paper/1989/file/6c9882bbac1c7093bd25041881277658-Paper.pdf.
  * Leviathan et al. (2022) Leviathan, Y., Kalman, M., and Matias, Y.  Fast inference from transformers via speculative decoding.  November 2022.  doi: 10.48550/ARXIV.2211.17192.
  * Lin et al. (2023) Lin, J., Tang, J., Tang, H., Yang, S., Dang, X., and Han, S.  Awq: Activation-aware weight quantization for llm compression and acceleration.  _arXiv preprint arXiv:2306.00978_ , 2023.
  * Lin et al. (2022) Lin, S., Hilton, J., and Evans, O.  Truthfulqa: Measuring how models mimic human falsehoods, 2022.
  * Mishra et al. (2021) Mishra, A., Latorre, J. A., Pool, J., Stosic, D., Stosic, D., Venkatesh, G., Yu, C., and Micikevicius, P.  Accelerating sparse deep neural networks.  _arXiv preprint arXiv: 2104.08378_ , 2021.
  * Mukherjee et al. (2023) Mukherjee, S., Mitra, A., Jawahar, G., Agarwal, S., Palangi, H., and Awadallah, A.  Orca: Progressive learning from complex explanation traces of gpt-4, 2023.
  * Niederfahrenhorst et al. (2023) Niederfahrenhorst, A., Hakhamaneshi, K., and Ahmad, R.  Fine-tuning llms: In-depth analysis with llama-2, Sep 2023.  URL https://www.anyscale.com/blog/fine-tuning-llms-lora-or-full-parameter-an-in-depth-analysis-with-llama-2.
  * Ouyang et al. (2022) Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al.  Training language models to follow instructions with human feedback.  _arXiv preprint arXiv:2203.02155_ , 2022.
  * Paperno et al. (2016) Paperno, D., Kruszewski, G., Lazaridou, A., Pham, Q. N., Bernardi, R., Pezzelle, S., Baroni, M., Boleda, G., and Fernández, R.  The lambada dataset: Word prediction requiring a broad discourse context, 2016.
  * Press et al. (2022) Press, O., Smith, N. A., and Lewis, M.  Train short, test long: Attention with linear biases enables input length extrapolation, 2022.
  * Qiu et al. (2023) Qiu, J., Li, L., Sun, J., Peng, J., Shi, P., Zhang, R., Dong, Y., Lam, K., Lo, F. P.-W., Xiao, B., Yuan, W., Wang, N., Xu, D., and Lo, B.  Large ai models in health informatics: Applications, challenges, and the future.  _IEEE Journal of Biomedical and Health Informatics_ , 27(12):6074–6087, 2023.  doi: 10.1109/JBHI.2023.3316750.
  * Radford et al. (2018) Radford, A., Narasimhan, K., Salimans, T., and Sutskever, I.  Improving language understanding by generative pre-training.  2018\.
  * Radford et al. (2019) Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al.  Language models are unsupervised multitask learners.  _OpenAI blog_ , 1(8):9, 2019.
  * Raffel et al. (2023) Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J.  Exploring the limits of transfer learning with a unified text-to-text transformer, 2023.
  * Rebuffi et al. (2017) Rebuffi, S.-A., Bilen, H., and Vedaldi, A.  Learning multiple visual domains with residual adapters.  _Advances in neural information processing systems_ , 30, 2017.
  * Ryu et al. (2023) Ryu, S., Seo, S., and Yoo, J.  Efficient storage of fine-tuned models via low-rank approximation of weight residuals, 2023.
  * Sakaguchi et al. (2019) Sakaguchi, K., Bras, R. L., Bhagavatula, C., and Choi, Y.  Winogrande: An adversarial winograd schema challenge at scale, 2019.
  * Sheng et al. (2023) Sheng, Y., Cao, S., Li, D., Hooper, C., Lee, N., Yang, S., Chou, C., Zhu, B., Zheng, L., Keutzer, K., Gonzalez, J. E., and Stoica, I.  S-lora: Serving thousands of concurrent lora adapters.  _arXiv preprint arXiv:2311.03285_ , 2023.
  * Suzgun et al. (2022) Suzgun, M., Scales, N., Schärli, N., Gehrmann, S., Tay, Y., Chung, H. W., Chowdhery, A., Le, Q. V., Chi, E. H., Zhou, D., et al.  Challenging big-bench tasks and whether chain-of-thought can solve them.  _arXiv preprint arXiv:2210.09261_ , 2022.
  * Team (2023) Team, X.-L.  Xwin-lm, 9 2023.  URL https://github.com/Xwin-LM/Xwin-LM.
  * Tillet et al. (2019) Tillet, P., Kung, H.-T., and Cox, D. D.  Triton: an intermediate language and compiler for tiled neural network computations.  _Proceedings of the 3rd ACM SIGPLAN International Workshop on Machine Learning and Programming Languages_ , 2019.  URL https://api.semanticscholar.org/CorpusID:184488182.
  * Touvron et al. (2023) Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al.  Llama 2: Open foundation and fine-tuned chat models.  _arXiv preprint arXiv:2307.09288_ , 2023.
  * Tunstall et al. (2023) Tunstall, L., Beeching, E., Lambert, N., Rajani, N., Rasul, K., Belkada, Y., Huang, S., von Werra, L., Fourrier, C., Habib, N., Sarrazin, N., Sanseviero, O., Rush, A. M., and Wolf, T.  Zephyr: Direct distillation of lm alignment, 2023.
  * Upstage (2023) Upstage.  Upstage/solar-0-70b-16bit · hugging face, 2023.  URL https://huggingface.co/upstage/SOLAR-0-70b-16bit.
  * Wang et al. (2023) Wang, G., Cheng, S., Zhan, X., Li, X., Song, S., and Liu, Y.  Openchat: Advancing open-source language models with mixed-quality data, 2023.
  * Wortsman et al. (2022) Wortsman, M., Ilharco, G., Gadre, S. Y., Roelofs, R., Gontijo-Lopes, R., Morcos, A. S., Namkoong, H., Farhadi, A., Carmon, Y., Kornblith, S., and Schmidt, L.  Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time, 2022.
  * Xiao et al. (2023) Xiao, G., Lin, J., Seznec, M., Wu, H., Demouth, J., and Han, S.  Smoothquant: Accurate and efficient post-training quantization for large language models.  In _International Conference on Machine Learning_ , pp. 38087–38099. PMLR, 2023.
  * Xu et al. (2024) Xu, M., Du, H., Niyato, D., Kang, J., Xiong, Z., Mao, S., Han, Z., Jamalipour, A., Kim, D. I., Shen, X., Leung, V. C. M., and Poor, H. V.  Unleashing the power of edge-cloud generative ai in mobile networks: A survey of aigc services.  _IEEE Communications Surveys & Tutorials_, pp. 1–1, 2024.  doi: 10.1109/COMST.2024.3353265.
  * Yadav et al. (2023) Yadav, P., Choshen, L., Raffel, C., and Bansal, M.  Compeft: Compression for communicating parameter efficient updates via sparsification and quantization, 2023.
  * Yao & Klimovic (2023) Yao, X. and Klimovic, A.  Deltazip: Multi-tenant language model serving via delta compression.  _arXiv preprint arXiv:2312.05215_ , 2023.
  * Yu et al. (2023) Yu, L., Yu, B., Yu, H., Huang, F., and Li, Y.  Language models are super mario: Absorbing abilities from homologous models as a free lunch.  _arXiv preprint arXiv:2311.03099_ , 2023.
  * Zellers et al. (2019) Zellers, R., Holtzman, A., Bisk, Y., Farhadi, A., and Choi, Y.  Hellaswag: Can a machine really finish your sentence?, 2019.
  * Zheng et al. (2023) Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. P., Zhang, H., Gonzalez, J. E., and Stoica, I.  Judging llm-as-a-judge with mt-bench and chatbot arena, 2023.
  * Zhu & Gupta (2017) Zhu, M. and Gupta, S.  To prune, or not to prune: exploring the efficacy of pruning for model compression.  _International Conference on Learning Representations_ , 2017.

##  Appendix A Appendix

Table 6: Full results of the comparison of BitDelta to the SVD based method, corresponding to Table 1. Model/Method | Train Loss | ARC | BBH | HellaSwag | TruthfulQA | LAMBADA | WinoGrande | GSM8K | Average $\uparrow$ | MT-Bench
---|---|---|---|---|---|---|---|---|---|---
Llama 2-7B | – | 52.56 | 33.76 | 78.96 | 38.96 | 68.39 | 68.98 | 13.57 | 50.74 | –
Vicuna-7B v1.5 | – | 53.92 | 37.14 | 77.45 | 50.36 | 64.41 | 69.61 | 19.03 | 53.13 | 6.04
BitDelta-Initial | 0.41 | 54.69 | 36.74 | 78.47 | 47.63 | 66.31 | 68.75 | 19.56 | 53.16 | 5.67
BitDelta | 0.052 | 54.27 | 36.57 | 77.9 | 49.97 | 65.2 | 69.46 | 20.17 | 53.36 | 5.99
SVD-Initial | 0.78 | 54.61 | 33.96 | 78.61 | 44.56 | 67.77 | 68.03 | 15.92 | 51.92 | 4.45
SVD | 0.34 | 53.92 | 34.51 | 77.95 | 44.55 | 66.35 | 68.27 | 14.18 | 51.39 | 4.98
Table 7: Full results of the application of BitDelta to quantized base models, corresponding to Table 4. Base Model | Method | ARC | BBH | HellaSwag | TruthfulQA | LAMBADA | WinoGrande | GSM8K | Average $\uparrow$ | MT-Bench
---|---|---|---|---|---|---|---|---|---|---
Baseline | FP16 | 53.58 | 33.84 | 78.58 | 45.32 | 66.58 | 66.46 | 22.74 | 52.44 | 6.56
LLM.int8() | 53.24 | 33.71 | 78.62 | 45.02 | 66.5 | 66.06 | 22.29 | 52.21 | 6.28
GPTQ | 51.88 | 33.54 | 77.17 | 44.92 | 65.32 | 65.43 | 19.48 | 51.11 | 5.90
Llama 2-7B |  FP16 + $\Delta$ | 54.44 | 33.85 | 78.31 | 44.95 | 66.66 | 66.14 | 20.24 | 52.08 | 6.47
LLM.int8() \+ $\Delta$ | 53.67 | 33.48 | 78.57 | 44.71 | 66.7 | 66.85 | 19.86 | 51.98 | 6.16
GPTQ + $\Delta$ | 51.45 | 33.90 | 78.06 | 42.52 | 66.85 | 65.82 | 19.94 | 51.22 | 6.02
Llama 2-7B Chat |  GPTQ + $\Delta$ | 52.56 | 33.65 | 77.54 | 44.63 | 65.81 | 66.30 | 22.14 | 51.80 | 6.11
Table 8: Full results of the ablation over the fidelity of $\Delta$, corresponding to Figure 3. # bits in $\Delta$ | ARC | BBH | HellaSwag | TruthfulQA | LAMBADA | WinoGrande | GSM8K | Average $\uparrow$
---|---|---|---|---|---|---|---|---
Llama 2-7b | 52.56 | 33.76 | 78.96 | 38.96 | 68.39 | 68.98 | 13.57 | 50.74
1 bit | 54.27 | 36.57 | 77.90 | 49.97 | 65.20 | 69.46 | 20.17 | 53.36
2 bits | 54.44 | 36.78 | 77.71 | 49.69 | 65.26 | 69.22 | 20.62 | 53.39
3 bits | 54.27 | 36.94 | 77.58 | 49.90 | 65.11 | 70.09 | 19.48 | 53.34
4 bits | 54.18 | 36.94 | 77.54 | 49.80 | 64.95 | 69.53 | 19.18 | 53.16
5 bits | 53.67 | 36.78 | 77.63 | 50.15 | 65.22 | 69.69 | 18.57 | 53.10
6 bits | 53.67 | 36.85 | 77.64 | 50.20 | 65.07 | 69.69 | 18.80 | 53.13
7 bits | 53.74 | 37.01 | 77.56 | 50.29 | 65.15 | 69.38 | 18.50 | 53.09
8 bits | 53.84 | 36.94 | 77.51 | 50.15 | 64.95 | 70.17 | 18.80 | 53.19
Vicuna-7b v1.5 | 53.92 | 37.14 | 77.45 | 50.36 | 64.41 | 69.61 | 19.03 | 53.13
Table 9: Full results of BitDelta applied to fine-tuned models in the Llama-2 and Mistral families, corresponding to Table 2. Model | Method | ARC | BBH | HellaSwag | TruthfulQA | LAMBADA | WinoGrande | GSM8K | Average $\uparrow$ |  MT-Bench $\uparrow$
---|---|---|---|---|---|---|---|---|---|---
Llama 2-7B | – | 52.56 | 33.76 | 78.96 | 38.96 | 68.39 | 68.98 | 13.57 | 50.74 | –
Llama 2-7B Chat | Baseline | 53.58 | 33.84 | 78.58 | 45.32 | 66.58 | 66.46 | 22.74 | 52.44 | 6.56
BitDelta-Initial | 55.46 | 35.56 | 76.32 | 41.10 | 68.14 | 68.03 | 18.27 | 51.84 | 6.31
BitDelta | 54.44 | 33.85 | 78.31 | 44.95 | 66.66 | 66.14 | 20.24 | 52.08 | 6.47
Vicuna-7B v1.5 | Baseline | 53.92 | 37.14 | 77.45 | 50.36 | 64.41 | 69.61 | 19.03 | 53.13 | 6.04
BitDelta-Initial | 54.69 | 36.74 | 78.47 | 47.63 | 66.31 | 68.75 | 19.56 | 53.16 | 5.67
BitDelta | 54.27 | 36.57 | 77.9 | 49.97 | 65.2 | 69.46 | 20.17 | 53.36 | 5.99
Vicuna-7B v1.5 16k | Baseline | 54.86 | 35.63 | 77.06 | 50.38 | 52.32 | 67.64 | 14.18 | 50.30 | 6.06
BitDelta-Initial | 55.55 | 33.24 | 77.99 | 45.58 | 56.8 | 68.98 | 13.95 | 50.30 | 5.69
BitDelta | 54.61 | 34.68 | 77.14 | 48.75 | 53.89 | 67.88 | 14.48 | 50.20 | 6.24
Xwin LM-7B v0.1 | Baseline | 57.59 | 34.05 | 79.15 | 48.06 | 68.02 | 69.22 | 10.77 | 52.41 | 6.24
BitDelta-Initial | 56.40 | 33.90 | 80.26 | 44.56 | 69.86 | 69.14 | 16.68 | 52.97 | 5.79
BitDelta | 57.94 | 34.19 | 79.36 | 47.62 | 68.29 | 69.53 | 9.02 | 52.28 | 6.50
Llama 2-13B | – | 59.47 | 39.03 | 82.23 | 36.90 | 70.44 | 72.22 | 22.74 | 54.72 | –
Llama 2-13B Chat | Baseline | 60.32 | 37.89 | 82.15 | 43.95 | 68.62 | 70.96 | 33.13 | 56.72 | 6.98
BitDelta-Initial | 59.90 | 38.04 | 82.13 | 41.70 | 69.82 | 71.35 | 33.36 | 56.61 | 7.06
BitDelta | 59.98 | 38.03 | 81.92 | 43.47 | 68.46 | 71.43 | 31.92 | 56.46 | 6.95
Vicuna-13B v1.5 | Baseline | 57.34 | 39.47 | 81.14 | 50.86 | 68.48 | 71.67 | 29.72 | 56.95 | 6.48
BitDelta-Initial | 54.69 | 36.74 | 78.47 | 47.63 | 66.31 | 68.75 | 31.84 | 54.92 | 6.51
BitDelta | 57.42 | 39.20 | 81.33 | 50.39 | 68.81 | 71.51 | 30.48 | 57.02 | 6.81
Vicuna-13B v1.5 16k | Baseline | 54.86 | 35.63 | 77.06 | 50.38 | 52.32 | 67.64 | 29.72 | 52.52 | 6.90
BitDelta-Initial | 59.90 | 38.04 | 82.13 | 41.70 | 69.82 | 71.35 | 26.76 | 55.67 | 6.60
BitDelta | 54.61 | 34.68 | 77.14 | 48.75 | 53.89 | 67.88 | 28.73 | 52.24 | 6.88
WizardLM-13B v1.2 | Baseline | 60.15 | 40.82 | 82.58 | 47.17 | 69.26 | 71.90 | 42.38 | 59.18 | 6.95
BitDelta-Initial | 60.41 | 40.27 | 83.26 | 44.89 | 70.23 | 71.74 | 42.08 | 58.98 | 6.73
BitDelta | 60.92 | 41.30 | 82.55 | 46.67 | 68.97 | 71.51 | 41.62 | 59.08 | 6.93
Xwin LM-13B v0.1 | Baseline | 63.14 | 40.12 | 82.92 | 45.54 | 70.62 | 73.09 | 21.15 | 56.65 | 6.78
BitDelta-Initial | 63.4 | 40.33 | 83.71 | 43.6 | 71.26 | 73.09 | 26.76 | 57.45 | 6.70
BitDelta | 62.80 | 39.81 | 83.01 | 48.19 | 70.74 | 72.30 | 21.76 | 56.94 | 6.83
Llama 2-70B | – | 67.58 | 51.67 | 87.00 | 44.82 | 74.81 | 77.98 | 52.69 | 65.22 | –
Llama 2-70B Chat | Baseline | 65.44 | 43.93 | 85.91 | 52.77 | 73.90 | 74.90 | 47.61 | 63.49 | 7.12
BitDelta-Initial | 63.4 | 38.67 | 81.36 | 41.63 | 72.66 | 73.95 | 42.38 | 59.15 | 6.85
BitDelta | 65.87 | 44.97 | 85.65 | 51.37 | 74.29 | 74.90 | 48.82 | 63.70 | 7.06
Solar-0-70B | Baseline | 71.16 | 55.54 | 87.78 | 62.03 | 75.04 | 79.32 | 56.18 | 69.58 | 7.07
BitDelta-Initial | 69.54 | 54.52 | 87.57 | 59.08 | 75.37 | 78.69 | 56.79 | 68.79 | 6.79
BitDelta | 70.82 | 55.06 | 87.35 | 62.03 | 75.86 | 78.77 | 56.63 | 69.50 | 6.82
Xwin LM-70B v0.1 | Baseline | 70.65 | 52.40 | 87.15 | 60.06 | 75.04 | 78.06 | 40.33 | 66.24 | 7.45
BitDelta-Initial | 69.97 | 52.93 | 87.36 | 60.77 | 75.51 | 78.14 | 50.64 | 67.90 | 7.70
BitDelta | 70.22 | 52.22 | 86.97 | 58.57 | 75.49 | 77.58 | 40.18 | 65.89 | 7.34
Mistral-7B v0.1 | – | 61.35 | 41.18 | 83.46 | 42.60 | 70.10 | 73.80 | 37.76 | 58.61 | –
Mistral-7B v0.1 Instruct | Baseline | 55.03 | 38.66 | 75.52 | 55.93 | 63.28 | 69.30 | 32.75 | 55.78 | 6.86
BitDelta-Initial | 59.22 | 40.25 | 79.91 | 51.27 | 67.63 | 72.14 | 38.82 | 58.46 | 6.54
BitDelta | 55.38 | 37.95 | 75.62 | 55.23 | 66.06 | 70.48 | 31.54 | 56.04 | 6.43
Zephyr-7B-$\beta$ | Baseline | 63.82 | 39.04 | 84.33 | 55.12 | 66.23 | 72.69 | 34.34 | 59.37 | 7.18
BitDelta-Initial | 63.57 | 41.87 | 83.85 | 54.53 | 67.73 | 73.56 | 40.26 | 60.77 | 6.70
BitDelta | 65.02 | 41.64 | 84.05 | 58.39 | 66.33 | 73.95 | 31.92 | 60.19 | 7.00
OpenChat 3.5 | Baseline | 64.51 | 45.28 | 84.39 | 47.34 | 65.19 | 72.61 | 68.84 | 64.02 | 7.74
BitDelta-Initial | 64.16 | 45.23 | 84.13 | 43.34 | 68.62 | 77.43 | 57.77 | 62.95 | 5.71
BitDelta | 64.93 | 44.57 | 84.44 | 46.24 | 65.88 | 76.40 | 57.70 | 62.88 | 7.38
Dolphin 2.2.1 | Baseline | 64.16 | 44.49 | 83.30 | 54.02 | 69.36 | 75.22 | 54.28 | 63.55 | 7.36
BitDelta-Initial | 64.16 | 44.43 | 84.01 | 48.14 | 69.98 | 75.30 | 50.27 | 62.33 | 7.10
BitDelta | 64.59 | 43.08 | 83.44 | 54.91 | 68.39 | 75.37 | 52.84 | 63.23 | 7.20
OpenOrca-7B | Baseline | 62.80 | 44.45 | 83.58 | 52.30 | 66.10 | 73.24 | 50.11 | 61.80 | 6.70
BitDelta-Initial | 63.74 | 44.46 | 84.15 | 49.66 | 69.05 | 74.03 | 49.96 | 62.15 | 7.12
BitDelta | 63.65 | 43.46 | 83.49 | 51.67 | 66.12 | 74.27 | 49.58 | 61.75 | 7.05


$\displaystyle\frac{-2\pi}{\mathcal{T}g^{2}_{kn}(g^{2}_{kn}+\mathcal{T}^{2})}\left[\mathcal{T}g_{kn}\sin(\Omega
g_{kn})-\mathcal{T}^{2}\cos(\Omega g_{kn})+g^{2}_{kn}+\mathcal{T}^{2}\right]$
$\displaystyle+\frac{2\pi}{\mathcal{T}(g_{kn}+i\varepsilon)(g_{kn}-i\varepsilon)}$
(109)

Furthermore, we also calculate

$\displaystyle A_{k}(\mathcal{T},\Omega)+B_{k}(\mathcal{T},-\Omega)=$
$\displaystyle\frac{-2\pi}{\mathcal{T}g^{2}_{kn}(g^{2}_{kn}+\mathcal{T}^{2})}\left[\mathcal{T}g_{kn}\sin(\Omega
g_{kn})-\mathcal{T}^{2}\cos(\Omega
g_{kn})+g^{2}_{kn}+\mathcal{T}^{2}\right]\text{sign}(\Omega)$
$\displaystyle-\Bigg{[}\frac{4\pi
e^{\Omega\mathcal{T}}}{\mathcal{T}(g^{2}_{kn}+\mathcal{T}^{2})}\Bigg{]}\Theta(-\Omega),$
(110)

where we have introduced the sign function,
$\text{sign}(\Omega)=\Theta(\Omega)-\Theta(-\Omega)$.

Using (109) in (96) we can calculate $\mathcal{Y}_{0}$. In this way

$\displaystyle\mathcal{Y}_{0}(\omega,j,\mathcal{T},r)=$
$\displaystyle\frac{\mathcal{T}^{3}e^{-|\omega|\mathcal{T}}}{8}\sum^{n-1}_{k=0}\frac{\sin(jg_{kn})}{g^{2}_{kn}(g^{2}_{kn}+\mathcal{T}^{2})}\Big{[}g_{kn}\cos(|\omega|g_{kn})+\mathcal{T}\sin(|\omega|g_{kn})\Big{]}$
(111)

and if we analyze the expression for $g_{kn}=2r\sin(\pi k/n)$, we see that it
vanishes only when $k=0$, which corresponds to the first term of the sum. So
for this first term we can apply the limit $g_{kn}\rightarrow 0$, therefore

$\mathcal{Y}_{0}=\frac{1}{8}\mathcal{T}j(|\omega|\mathcal{T}+1)e^{-|\omega|\mathcal{T}}+\frac{\mathcal{T}^{3}e^{-|\omega|\mathcal{T}}}{8}\sum^{n-1}_{k=1}\frac{\sin(jg_{kn})}{g^{2}_{kn}(g^{2}_{kn}+\mathcal{T}^{2})}\Big{[}g_{kn}\cos(|\omega|g_{kn})+\mathcal{T}\sin(|\omega|g_{kn})\Big{]}.$
(112)

Lastly, we are going to calculate the function $\mathcal{Y}_{I}$ from (57). In
this way

$\mathcal{Y}_{I}(\omega,j,{\cal T},\Delta r,\Delta
z,\Delta\varphi)=\frac{-1}{4\pi^{2}}\int_{-{\cal T}/2}^{{\cal
T}/2}d\uptau\int_{-{\cal T}/2}^{{\cal T}/2}d\uptau^{\prime}\,e^{i\omega
T}\left(e^{ij\Delta\uptau}+e^{-ij\Delta\uptau}\right)\sum^{n-1}_{k=0}\frac{1}{(\Delta\uptau-i\epsilon)^{2}-g^{2}_{kn}},$
(113)

where $g_{kn}$ again becomes (80).

From the function of regularization (81). Therefore, we have to calculate

$\displaystyle\mathcal{Y}_{I}(\omega,j,{\cal T},\Delta r,\Delta
z,\Delta\varphi)=\frac{-\mathcal{T}^{4}}{8\pi^{2}}\sum^{n-1}_{k=0}$
$\displaystyle\int_{-\infty}^{\infty}d(\Delta\uptau)\,\frac{\left(e^{ij\Delta\uptau}+e^{-ij\Delta\uptau}\right)}{(\Delta\uptau-i\epsilon)^{2}-g^{2}_{kn}}$
$\displaystyle\times\int_{-\infty}^{\infty}dT\frac{e^{i\omega
T}}{\left[T^{2}-(\Delta\uptau+i\mathcal{T})^{2}\right]\left[T^{2}-(\Delta\uptau-i\mathcal{T})^{2}\right]}.$
(114)

Thus, from the result (95), we have

$\displaystyle\mathcal{Y}_{I}(\omega,j,\mathcal{T},\Delta r,\Delta
z,\Delta\varphi)=\frac{-\mathcal{T}^{3}e^{-|\omega|\mathcal{T}}}{32\pi}\sum^{n-1}_{k=0}\left[A_{k}(j+|\omega|)+B_{k}(-j-|\omega|)+A_{k}(-j+|\omega|)+B_{k}(j-|\omega|)\right].$
(115)

Using (110) we can calculate (115). In this way

$\displaystyle\mathcal{Y}_{I}(\omega,j,\mathcal{T},\Delta r,\Delta
z,\Delta\varphi)=$
$\displaystyle\frac{\mathcal{T}^{2}e^{-|\omega|\mathcal{T}}}{16}\sum^{n-1}_{k=0}\frac{1}{g^{2}_{kn}(g^{2}_{kn}+\mathcal{T}^{2})}\Bigg{\\{}\bigg{[}\mathcal{T}\cos(jg_{kn})\Big{(}g_{kn}\sin(|\omega|g_{kn})-\mathcal{T}\cos(|\omega|g_{kn})\Big{)}$
$\displaystyle+g^{2}_{kn}+\mathcal{T}^{2}\bigg{]}\gamma_{+}+\mathcal{T}\sin(jg_{kn})\Big{[}g_{kn}\cos(|\omega|g_{kn})+\mathcal{T}\sin(|\omega|g_{kn})\Big{]}\gamma_{-}\Bigg{\\}}$
$\displaystyle+\frac{\mathcal{T}^{2}}{8}\left(e^{j\mathcal{T}}\Theta(-j-|\omega|)+e^{-j\mathcal{T}}\Theta(j-|\omega|)\right)\sum^{n-1}_{k=0}\frac{1}{g^{2}_{kn}+\mathcal{T}^{2}},$
(116)

where we have define

$\displaystyle\gamma_{+}=\text{sign}(j+|\omega|)+\text{sign}(-j+|\omega|)$
(117)
$\displaystyle\gamma_{-}=\text{sign}(j+|\omega|)-\text{sign}(-j+|\omega|)$
(118)

and analyzing the expression for $g_{kn}$ in (80), we see that it vanishes
when $\Delta r^{2}=\Delta z^{2}=0$ and $\Delta\varphi/2+\pi k/n$ is multiple
of $\pi$ and if we take into account that $\varphi\in[0,2\pi/n]$ then $\pi
k/n+\Delta\varphi/2\in[\pi(k-1)/n,\pi(k+1)/n]$, so that three possible
conditions are obtained in which $g_{kn}$ vanishes. These are when $k=0$ then
$\Delta\varphi$ has to be $0$ or when $k=1$ then $\Delta\phi=-2\pi/n$ or also
when $k=n-1$ then $\Delta\varphi=2\pi/n$. Thus, making the limit
$g_{kn}\rightarrow 0$ for any of the terms in the sum that satisfies these 3
conditions, we have

$\displaystyle\frac{e^{-|\omega|\mathcal{T}}}{32}\bigg{[}\Big{(}\mathcal{T}^{2}j^{2}+(|\omega|\mathcal{T}+1)^{2}+1\Big{)}\gamma_{+}+2j\mathcal{T}(|\omega|\mathcal{T}+1)\gamma_{-}\bigg{]}+\frac{1}{8}\Big{(}e^{j\mathcal{T}}\Theta(-j-|\omega|)+e^{-j\mathcal{T}}\Theta(j-|\omega|)\Big{)}$
(119)

###### Acknowledgements.

W. I. would like to thank professor G. Krein for useful discussions and
conversations. The authors would like to thank Coordenação de Aperfeiçoamento
de Pessoal de Nível Superior CAPES (Brazilian agency) for financial support.
J. B. would like to thank Vicerrectorado de Investigación Universidad Nacional
de Ingeniería (VRI-UNI) for financial support Project number FC-PF-34-2021.

## References

  * (1) R. Horodecki, P. Horodecki, M. Horodecki and K. Horodecki, _Quantum entanglement_ , _Rev. Mod. Phys._ 81 (2009) 865.
  * (2) J. Preskill, _Quantum Computing in the NISQ era and beyond_ , _Quantum_ 2 (2018) 79.
  * (3) C.H. Bennett and G. Brassard, _Quantum cryptography: Public key distribution and coin tossing_ , _Theoretical Computer Science_ 560 (2014) 7.
  * (4) I.M. Georgescu, S. Ashhab and F. Nori, _Quantum simulation_ , _Rev. Mod. Phys._ 86 (2014) 153.
  * (5) C. Anastopoulos and B.-L. Hu, _Gravitational decoherence: A thematic overview_ , _AVS Quantum Science_ 4 (2022) 015602.
  * (6) S.N. Solodukhin, _Entanglement entropy of black holes_ , _Living Reviews in Relativity_ 14 (2011) .
  * (7) J. Maldacena and L. Susskind, _Cool horizons for entangled black holes_ , _Fortschritte der Physik_ 61 (2013) 781.
  * (8) E. Arias, J.G. Dueñas, G. Menezes and N.F. Svaiter, _Boundary effects on radiative processes of two entangled atoms_ , _Journal of High Energy Physics_ 2016 (2016) .
  * (9) S.J. Summers and R. Werner, _The vacuum violates bell’s inequalities_ , _Physics Letters A_ 110 (1985) 257.
  * (10) B. Reznik, _Entanglement from the vacuum_ , _Foundations of Physics_ 33 (2003) 167–176.
  * (11) E. Martín-Martínez and N.C. Menicucci, _Cosmological quantum entanglement_ , _Classical and Quantum Gravity_ 29 (2012) 224003.
  * (12) B. Reznik, A. Retzker and J. Silman, _Violating bell’s inequalities in vacuum_ , _Phys. Rev. A_ 71 (2005) 042104.
  * (13) J. León and C. Sabín, _Generation of atom-atom correlations inside and outside the mutual light cone_ , _Phys. Rev. A_ 79 (2009) 012304.
  * (14) G.V. Steeg and N.C. Menicucci, _Entangling power of an expanding universe_ , _Phys. Rev. D_ 79 (2009) 044027.
  * (15) E. Martín-Martínez, A.R. Smith and D.R. Terno, _Spacetime structure and vacuum entanglement_ , _Physical Review D_ 93 (2016) .
  * (16) E. Martín-Martínez and N.C. Menicucci, _Entanglement in curved spacetimes and cosmology_ , _Classical and Quantum Gravity_ 31 (2014) 214001.
  * (17) D. Braun, _Entanglement from thermal blackbody radiation_ , _Phys. Rev. A_ 72 (2005) 062324.
  * (18) S. Massar and P. Spindel, _Einstein-podolsky-rosen correlations between two uniformly accelerated oscillators_ , _Phys. Rev. D_ 74 (2006) 085031.
  * (19) H. Nielsen and P. Olesen, _Vortex-line models for dual strings_ , _Nuclear Physics B_ 61 (1973) 45.
  * (20) T.W.B. Kibble, _Topology of cosmic domains and strings_ , _Journal of Physics A: Mathematical and General_ 9 (1976) 1387.
  * (21) A. Smith, _on classical and quantum fields: Self-forces and vacuum fluctuations_ , _The Formation and Evolution of Cosmic Strings_ (1990) 263.
  * (22) A. Vilenkin and E.P.S. Shellard, _Cosmic strings and other topological defects_ , Cambridge monographs on mathematical physics, Cambridge University Press (1994).
  * (23) E.J. Copeland and T.W.B. Kibble, _Cosmic strings and superstrings_ , _Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences_ 466 (2010) 623.
  * (24) A. Vilenkin, _Gravitational field of vacuum domain walls and strings_ , _Phys. Rev. D_ 23 (1981) 852.
  * (25) W.A. Hiscock, _Exact gravitational field of a string_ , _Phys. Rev. D_ 31 (1985) 3288.
  * (26) M.R. Anderson, _The mathematical theory of cosmic strings: cosmic strings in the wire approximation_ , Series in high energy physics, cosmology and gravitation, Institute of Physics Pub, 1st ed. (2003).
  * (27) K. Życzkowski, P. Horodecki, A. Sanpera and M. Lewenstein, _Volume of the set of separable states_ , _Phys. Rev. A_ 58 (1998) 883.
  * (28) S. Hollands and R.M. Wald, _Quantum fields in curved spacetime_ , _Physics Reports_ 574 (2015) 1.
  * (29) S.A. Fulling, _Aspects of quantum field theory in curved space-time_ , London Mathematical Society student texts, 17, Cambridge University Press (1989).
  * (30) L. Parker and D. Toms, _Quantum Field Theory in Curved Spacetime: Quantized Fields and Gravity_ , Cambridge Monographs on Mathematical Physics, Cambridge University Press, 1 ed. (2009).
  * (31) N.D. Birrell and P.C.W. Davies, _Quantum Fields in Curved Space_ , Cambridge Monographs on Mathematical Physics, Cambridge University Press (1984).
  * (32) R.M. Wald, _Quantum field theory in curved spacetime and black hole thermodynamics_ , Chicago lectures in physics, University of Chicago Press, 1 ed. (1994).
  * (33) T.M. Helliwell and D.A. Konkowski, _Vacuum fluctuations outside cosmic strings_ , _Phys. Rev. D_ 34 (1986) 1918.
  * (34) A.A. Saharian and A.S. Kotanjyan, _Repulsive casimir–polder forces from cosmic strings_ , _The European Physical Journal C_ 71 (2011) .
  * (35) P. He, H. Yu and J. Hu, _Entanglement dynamics for static two-level atoms in cosmic string spacetime_ , _The European Physical Journal C_ 80 (2020) .
  * (36) Z. Huang and Z. He, _Quantum entanglement in the background of cosmic string spacetime_ , _Quantum Information Processing_ 19 (2020) .
  * (37) Z. Huang, _Quantum entanglement for atoms coupling to fluctuating electromagnetic field in the cosmic string spacetime_ , _Quantum Information Processing_ 20 (2021) .
  * (38) B.F. Svaiter and N.F. Svaiter, _Quantum processes: stimulated and spontaneous emission near cosmic strings_ , _Classical and Quantum Gravity_ 11 (1994) 347.
  * (39) A.H. Bilge, M. Hortaçsu and N. Özdemir, _Can an unruh detector feel a cosmic string?_ , _General Relativity and Gravitation_ 30 (1998) 861–876.
  * (40) L. Iliadakis, U. Jasper and J. Audretsch, _Quantum optics in static spacetimes: How to sense a cosmic string_ , _Phys. Rev. D_ 51 (1995) 2591.
  * (41) H. Cai, H. Yu and W. Zhou, _Spontaneous excitation of a static atom in a thermal bath in cosmic string spacetime_ , _Phys. Rev. D_ 92 (2015) 084062.
  * (42) W. Zhou and H. Yu, _Spontaneous excitation of a uniformly accelerated atom in the cosmic string spacetime_ , _Phys. Rev. D_ 93 (2016) 084028.
  * (43) W. Zhou and H. Yu, _Boundarylike behaviors of the resonance interatomic energy in a cosmic string spacetime_ , _Phys. Rev. D_ 97 (2018) 045007.
  * (44) H.F. Mota, E.R. Bezerra de Mello, C.H.G. Bessa and V.B. Bezerra, _Light-cone fluctuations in the cosmic string spacetime_ , _Phys. Rev. D_ 94 (2016) 024039.
  * (45) A. Aliev and D. Gal’tsov, _Gravitational aharonov-bohm radiation in string-generated conical space-time_ , _Annals of Physics_ 193 (1989) 142.
  * (46) X. Wang, _Entanglement in the quantum heisenberg $\mathrm{XY}$ model_, _Phys. Rev. A_ 64 (2001) 012313.
  * (47) S.R. Hedemann, _Evidence that All States Are Unitarily Equivalent to X States of the Same Entanglement_ , _arXiv e-prints_ (2013) arXiv:1310.7038 [1310.7038].
  * (48) A. Peres, _Separability criterion for density matrices_ , _Phys. Rev. Lett._ 77 (1996) 1413.
  * (49) G. Vidal and R.F. Werner, _Computable measure of entanglement_ , _Phys. Rev. A_ 65 (2002) 032314.
  * (50) E. Arias, T.R. de Oliveira and M.S. Sarandy, _The unruh quantum otto engine_ , _Journal of High Energy Physics_ 2018 (2018) .