marginparsep has been altered.
topmargin has been altered.
marginparpush has been altered.
The page layout violates the ICML style. Please do not change the page layout,
or include packages like geometry, savetrees, or fullpage, which change it for
you. We’re not able to reliably undo arbitrary changes to the style. Please
remove the offending package(s), or layout-changing commands and try again.

SLEB: Streamlining LLMs through Redundancy Verification and
Elimination of Transformer Blocks

Jiwon Song * 1  Kyungseok Oh * 1  Taesu Kim 2  Hyungjun Kim 2  Yulhwa Kim 3
Jae-Joon Kim 1

††footnotetext: *Equal contribution 1Seoul National University 2SqueezeBits
Inc. 3Sungkyunkwan University. Correspondence to: Yulhwa Kim
<EMAIL_ADDRESS>Jae-Joon Kim<EMAIL_ADDRESS>
###### Abstract

Large language models (LLMs) have proven to be highly effective across various
natural language processing tasks. However, their large number of parameters
poses significant challenges for practical deployment. Pruning, a technique
aimed at reducing the size and complexity of LLMs, offers a potential solution
by removing redundant components from the network. Despite the promise of
pruning, existing methods often struggle to achieve substantial end-to-end LLM
inference speedup. In this paper, we introduce SLEB, a novel approach designed
to streamline LLMs by eliminating redundant transformer blocks. We choose the
transformer block as the fundamental unit for pruning, because LLMs exhibit
block-level redundancy with high similarity between the outputs of neighboring
blocks. This choice allows us to effectively enhance the processing speed of
LLMs. Our experimental results demonstrate that SLEB successfully accelerates
LLM inference without compromising the linguistic capabilities of these
models, making it a promising technique for optimizing the efficiency of LLMs.
The code is available at: https://github.com/leapingjagg-dev/SLEB.

##  1 Introduction

Large language models (LLMs), such as GPT-3, OPT, and LLaMA demonstrate
exceptional proficiency in a variety of natural language processing (NLP)
tasks and have become key components in applications like chatbots and
question-answering systems Brown et al. (2020); Zhang et al. (2022); Chowdhery
et al. (2022); Touvron et al. (2023a; b). However, their substantial number of
parameters creates significant challenges in deploying these models for real-
world services, especially due to the increased memory consumption and
computational demands. This limitation restricts their widespread use.
Consequently, it is critical to develop techniques that improve the
compactness and processing efficiency of LLMs while preserving their
linguistic prowess.

Network pruning is a technique aimed at reducing the size and complexity of
neural networks by eliminating redundant weight parameters LeCun et al.
(1989); Hassibi et al. (1993); Han et al. (2015). Its application in LLMs has
been somewhat limited, primarily due to challenges that arise in managing
sparse matrices Frantar & Alistarh (2022); Sun et al. (2023); Frantar &
Alistarh (2023); Zhang et al. (2024). This complexity becomes particularly
evident when using modern GPU hardware, as these systems are typically
optimized for operations involving dense matrices Wang (2020); Shi et al.
(2020).

Figure 1: Typical LLM architecture

In the realm of LLMs, a significant similarity in output is observed among
successive transformer blocks Din et al. (2023); Liu et al. (2023). This
similarity arises because each transformer block incrementally contributes to
the residual path spanning the entire LLM. Figure 1 depicts the typical
architecture of conventional LLMs, characterized by a continuous stack of
transformer blocks. A key aspect of LLM computation is the residual path that
traverses the entire network, a feature introduced to stabilize the
backpropagation process during training. Consequently, each transformer block
contributes its computational outputs, derived from both the attention
mechanisms and feed-forward layers, to this residual path. This design feature
results in a considerable degree of output similarity between consecutive
transformer blocks, leading to redundancy within the LLMs.

Figure 2: Overview of previous pruning methods, 2:4 pruning and channel-wise
(a.k.a. row/column-wise) pruning, and proposed SLEB

In this paper, we propose SLEB, a novel approach designed to streamline LLMs
by identifying and eliminating redundant transformer blocks. Figure 2 compares
the proposed approach with previous pruning methods. SLEB is tailored to
refine LLMs through the strategic removal of redundant transformer blocks,
thereby effectively aligning speedup with the pruning ratio. The fundamental
principle behind SLEB is that the careful elimination of redundant transformer
blocks can be achieved without affecting the text generation capabilities of
LLMs. By targeting these specific redundant elements within the LLM’s
architecture, SLEB seeks to provide a more efficient pruning approach. It aims
to overcome the challenges typically associated with traditional network
pruning, particularly in enhancing the acceleration of LLMs.

##  2 Motivation

###  2.1 Pruning

Standard LLMs are designed as shown in Figure 1, with a series of identically
structured transformer blocks arranged sequentially. Each transformer block in
these models consists of attention and feed-forward layers. The operations in
both the linear and feed-forward layers primarily involve matrix
multiplication between the weight parameters and input activations. Given this
structure, there are two main strategies for developing compact and fast LLMs:
enhancing the efficiency of each individual transformer block or reducing the
overall number of blocks. To improve both the compactness and processing speed
of individual blocks, pruning is frequently utilized LeCun et al. (1989);
Hassibi et al. (1993); Han et al. (2015). This technique involves the
strategic removal of superfluous weight parameters, thereby reducing the
computational load associated with matrix multiplication in each linear and
feed-forward layer. However, the application of pruning techniques to LLMs
encounters a notable challenge.

Challenge 1) Limitation in Achieving LLM Inference Speedup: Pruning can be
divided into two main types: unstructured and structured Frantar & Alistarh
(2022); Sun et al. (2023); Frantar & Alistarh (2023); Zhang et al. (2024).
Unstructured pruning targets the removal of individual weights, leading to
sparse weight matrices within the model. This sparsity can create complex data
access patterns, complicating the management of sparse data and potentially
hindering the model’s acceleration. Particularly with NVIDIA GPUs, achieving a
speedup through unstructured pruning typically requires reaching a high level
of sparsity, often more than 90% Wang (2020); Shi et al. (2020). However, for
LLMs, achieving pruning ratios above 50% is often difficult without
substantially compromising their linguistic prowess. Structured pruning, on
the other hand, involves the elimination of predefined units of weights to
create more hardware-friendly patterns, such as 2:4 pruning Sun et al. (2023);
Frantar & Alistarh (2023); Zhang et al. (2024) or channel-wise (a.k.a.
row/column-wise) pruning Ashkboos et al. (2024); Ma et al. (2023). The goal of
structured pruning is to form dense matrices that are more efficiently
processed by hardware, thereby enhancing efficiency. Nonetheless, the
anticipated speedup from structured pruning, ideally proportional to the
pruning ratio, frequently fails to be realized in practical settings.

Figure 3: The speedup achieved through 2:4 pruning on matrix multiplication
between $b\times h$ matrix and $h\times h$ ($b$: batch size, $h$: hidden
size). The test is conducted on an NVIDIA RTX A6000 GPU. The dashed grey line
represents the peak speedup attainable with 2:4 pruning.

Recent NVIDIA GPUs support the acceleration of 2:4 fine-grained structured
sparsity using their Tensor Cores Mishra et al. (2021). This form of sparsity
involves having two zero values within each contiguous block of four values,
resulting in a natural sparsity rate of 50%. The peak performance of sparse
tensor cores is twice that of dense tensor cores. Thus, previous research on
pruning LLMs has extensively examined 2:4 pruning techniques Sun et al.
(2023); Frantar & Alistarh (2023); Zhang et al. (2024). However, it is
challenging to achieve the desired speedup with 2:4 pruning. The speedup
achieved with 2:4 pruning heavily depends on the size of the input matrix.
Figure 3 provides a comparative analysis of matrix multiplication latency on
an NVIDIA RTX A6000 GPU, using both NVIDIA cuBLAS and CUTLASS libraries. The
matrix multiplication involves batched inputs with dimensions of $b\times h$,
along with a weight matrix size of $h\times h$, where $b$ represents the batch
size and $h$ represents the hidden size. The measurement results indicate that
within a realistic batch size range of 1 to 128, 2:4 pruning does not result
in a speedup. However, as the batch size is increased beyond this range, 2:4
pruning begins to show a speedup, eventually reaching approximately
1.5$\times$ the speed of dense matrix multiplication. To achieve significant
speedups, it is essential to have a sufficiently large matrix size, which can
lead to high arithmetic intensity Mishra et al. (2021). In summary, achieving
faster LLM inference with conventional pruning methods has two main
limitations. Firstly, it requires dedicated hardware, such as the 2:4 sparse
matrix multiplication accelerator adapted in recent NVIDIA GPUs, to enhance
inference speed. Secondly, even with dedicated hardware support, the extent of
speedup achievable through 2:4 pruning is influenced by factors like batch
size and model size.

Figure 4: Perplexity comparison on WikiText-2 for OPT-6.7B (left) and OPT-13B
(right) after removing consecutive transformer blocks. “Early Exit” refers to
removing the very last blocks from the target model, while “Chunk Best”
represents the best perplexity results achieved by testing all possible
removable points of consecutive blocks.

As an alternative to traditional pruning methods, approaches like LLM-Pruner
Ma et al. (2023) and SliceGPT Ashkboos et al. (2024) have been developed,
focusing on the elimination of redundant neurons and the weight parameters
connected to them. These methods involve the elimination of the entire channel
(row/column) from weight matrices, allowing pruned weights to remain in a
dense format. This retention of dense matrices enables direct speedup in
matrix multiplication. However, it is challenging to maintain linguistic
capabilities of LLMs with these channel-wise pruning approaches without the
help of extensive fine-tuning Ma et al. (2023). Additionally, speedup in
matrix multiplication does not always lead to corresponding improvement in the
end-to-end inference speed of LLMs Ashkboos et al. (2024). This limitation
arises because transformer blocks in LLMs encompass various operations beyond
matrix multiplication that involve weight parameters. These operations
include, but are not limited to, layer normalization and attention mechanisms
(Figure 1).

Recently, Deja Vu Liu et al. (2023) has introduced a new pruning approach
termed contextual sparsity. This method dynamically assesses whether to bypass
certain segments of a layer operation, based on the context of the input. It
has shown effectiveness in accelerating inference, particularly in single-
batch scenarios. However, its dynamic nature introduces a set of challenges
akin to those encountered with early exit strategies. These challenges will be
further examined and discussed in the following subsection.

###  2.2 Early Exit

Figure 5: Percentage of token prediction alignment with the final predictions
of LLMs for each transformer block.

Reducing the total number of transformer blocks in the LLM can directly lead
to increased processing speed, proportional to the reduction in the number of
blocks. A notable technique in this context is the early exit Schuster et al.
(2022); Din et al. (2023); Varshney et al. (2023); Chen et al. (2023). This
method dynamically assesses whether to process subsequent transformer blocks.
When the model reaches a certain level of confidence in its early outputs, it
can halt further processing, effectively reducing computational cost. Another
recent approach Del Corro et al. (2023) proposes an alternative method for
bypassing transformer blocks. This strategy suggests that removing blocks,
particularly those at the beginning of LLMs, might be more feasible. However,
all of these strategies, which involve skipping transformer blocks, typically
require either dynamic decision-making or extensive training to be effective.
To assess the impact of such removals without dynamic decision-making and
training, we analyze the perplexity scores of OPT-6.7B/13B on the WikiText-2
dataset Merity et al. (2016) after removing various consecutive blocks, as
illustrated in Figure 4. The analysis results show that the “Early Exit”
approach leads to a noticeable decline in the linguistic prowess of the LLMs,
as evidenced by a significant increase in their perplexity scores. Moreover,
“Chunk Best”, which represents the best perplexity results achieved by testing
all possible removable points of consecutive blocks, also results in a
significant increase in perplexity as the number of removed blocks increases.
Therefore, the concept of removing consecutive blocks from LLMs is not
effective without dynamic decision-making and training. In addition, applying
early exit to LLMs, particularly in real-world implementation, faces the
following challenges.

Challenge 2) Limitation in Acceleration in Multi-batch Settings: In practical
LLM applications, multi-batch scenarios are common and pose significant
challenges. Processing individual tokens can lead to diverse dynamic skipping
decisions. When these tokens are batched for parallel processing, each token
may require different processing paths. This variation complicates the
implementation and diminishes the efficiency benefits in multi-batch contexts.

Challenge 3) Inability to Reduce Memory Requirements: Dynamic methods such as
early exit face limitations in reducing memory consumption, as they require
maintaining the entire set of model parameters. This limitation is
particularly impactful for models with a large number of parameters, where
memory efficiency is crucial.

Challenge 4) Resource-Intensive Training: As shown in Figure 5, transformer
blocks that produce the same prediction results as the final token prediction
of the LLM are typically positioned near the end of the model. Even with the
use of an ideal skipping predictor for accurately determining an early exit
point, it is estimated that around 90% of the transformer blocks would still
require processing. Hence, implementing early exit strategies demands
extensive training. This process is not limited to training the skipping
predictor but also involves adapting the LLMs to effectively utilize early
exit mechanisms. Such a training process can be resource-heavy and
complicated. Owing to these challenges, the implementation of early exit
strategies in state-of-the-art LLMs, particularly those with more than 13
billion parameters, has not been extensively pursued. This situation
emphasizes the significant difficulties in adapting these methods for use in
extremely large-scale models like LLaMA-2-70B.

In the upcoming section, we introduce SLEB, a novel approach designed to
streamline LLMs by identifying and eliminating redundant transformer blocks.
SLEB is developed to address challenges faced by previous works in the
following ways:

Solution 1) LLM Inference Speedup: SLEB adopts transformer block as the
primary unit for elimination, aiming to achieve a speedup that is directly
proportional to the number of blocks removed.

Solution 2) Acceleration in Multi-batch Settings: Unlike the early exit
strategy, SLEB employs a static approach to eliminate transformer blocks
following an in-depth verification of redundancy, aligning with traditional
pruning techniques. This allows SLEB to enhance processing speed in multi-
batch scenarios effectively.

Solution 3) Reduction in Memory Requirements: By completely removing redundant
blocks, SLEB can significantly decrease memory usage, contributing to more
efficient memory management in LLMs.

Solution 4) Training-free Compression: Relying on a training-free redundancy
analysis, SLEB eliminates the need for intensive training processes. This
makes it an especially suitable solution for large-scale models, where the
demands and constraints of extensive training are often prohibitive.

##  3 Proposed SLEB

###  3.1 Output Similarity across Transformer Blocks

Figure 6: Cosine similarity between the outputs of two transformer blocks,
Block $i$ and Block $j$.

During the inference process of LLMs, there is a notable similarity in the
outputs of consecutive transformer blocks. This similarity arises because each
block incrementally adds to the residual path that runs throughout the entire
LLM. The computation results of each transformer block, considering the
residual path, can be simplified using the following equation:

$x_{i+1}=T_{i+1}(x_{i})+x_{i}$ (1)

Here, $x_{i}$ denotes the output of the $i$-th transformer block, and $T_{i}$
denotes the operation results of the $i$-th transformer block. To
comprehensively analyze and quantify the output similarity within the
transformer blocks, we analyze cosine similarity between outputs of these
blocks. This analysis is based on the distance measurement method outlined in
Din et al. (2023). It involves feeding a single random token into each LLM and
measuring the cosine similarity between the output of the $i$-th transformer
block and that of the $j$-th transformer block is defined as follows:

$similarity(x_{i},x_{j})=\frac{x_{i}\cdot x_{j}}{||x_{i}||||x_{j}||}$ (2)

The measurement results for OPT-6.7B and LLaMA-2-7B models are displayed in
Figure 6.

These results reveal that, although the similarity between transformer blocks
that are further apart varies across the LLM architecture, there is a
consistently high degree of similarity between neighboring transformer blocks.
This insight underscores the potential redundancy within these models and
highlights areas where efficiency could be enhanced through strategic block
elimination.

Indeed, this observation emphasizes a fundamental misalignment between
existing early exit methods and the nature of LLMs. The primary limitation of
early exit methods is their tendency to skip a continuous sequence of
transformer blocks. This approach does not take into account the high degree
of output similarity between adjacent transformer blocks, as indicated by our
cosine similarity measurements. Consequently, such strategies risk bypassing
blocks that are essential for preserving the linguistic integrity of the LLMs.

###  3.2 Redundancy Verification of Transformer Blocks

Figure 7: Perplexity comparison on WikiText-2 for OPT-6.7B (top) and OPT-13B
(bottom) when transformer blocks are removed using different metrics.

To streamline LLMs without affecting their linguistic prowess, it is essential
to identify redundant transformer blocks first. We design metrics to measure
the significance of a block to eliminate blocks with lower metric values. One
potential metric is to assess the distance between the input and output of
each block. A small change after passing through the block might indicate a
minor impact on the overall LLM inference. In this context, we adopt Eq. 2 as
the metric to measure the significance of transformer blocks. Given the input
and output of the $j$-th transformer block as $A_{j}$ and $B_{j}$, this metric
can be defined as follows:

$Metric_{j}^{1}=1-similarity(A_{j},B_{j})$ (3)

$Metric^{1}$ is tested on removing transformer blocks from the OPT-6.7B and
OPT-13B model, and the results are reported in Figure 7. However, using this
metric led to a significant increase in perplexity. The reason for this
increase is that even though a transformer block may have a minor impact on
the residual path at its stage, as the LLM inference progresses, these minor
changes in that block can be amplified, especially if the block lies in the
early stage of the LLM, leading to a more substantial impact on the overall
results.

To evaluate the significance of transformer blocks within the overall LLM
inference process, we design a new metric to assess the impact of each block
on the token prediction results of the target model. We denote the target LLM
model as $M$ and the LLM model after removing $j$-th transformer block as
$M_{j}$. Using a tokenized sequence for the calibration as
$w_{1},w_{2},...,w_{K}$, the second metric for measuring the significance of
$j$-th transformer block can be defined as follows:

$\vspace{-2mm}Metric_{j}^{2}=-\frac{1}{K}\sum_{k=0}^{K}{logp_{M_{j}}(w_{k}|w_{<k})}$
(4)

$Metric^{2}$ shows significant improvements in perplexity on the block-removed
LLMs compared to those removed using $Metric^{1}$ (Figure 7). However, as the
number of removed blocks increases, $Metric^{2}$ also exhibits significant
degradation in token prediction results, particularly for OPT-6.7B models.
This behavior can be explained by the observation such that the significance
of a block keeps changing as other blocks are removed. For example, when using
$Metric^{2}$, it selects blocks with indices 3, 4, 5, 6, 7, 8, and 10 for
removal from OPT-6.7B. While these blocks individually have a minor impact on
LLM inference results, the removal of a continuous sequence of blocks can
significantly impact the overall LLM inference result.

To accurately assess the redundancy of transformer blocks and determine the
next block for removal, we utilize the LLM with previously removed redundant
blocks in an iterative removal process. In this revised metric, we first
evaluate the token prediction results of the original LLM to identify the most
redundant block for removal. In the next stage, we assess the significance of
each block with a block-removed LLM to select the next block for removal. This
iterative process guides the removal of transformer blocks based on the
current state of the LLM. Denoting the target LLM model with previously
removed redundant blocks as $M^{\prime}$, the metric for identifying the next
block for removal is as follows:

$\vspace{-2mm}Metric_{j}^{3}(M^{\prime})=-\frac{1}{K}\sum_{k=0}^{K}{logp_{M_{j}^{{}^{\prime}}}(w_{k}|w_{<k})}$
(5)

When utilizing $Metric^{3}$ to assess the significance of each block and
remove a total 7 of blocks from OPT-6.7B, blocks with the following indices
are removed at each block removal step: 6, 7, 3, 24, 18, 30, 11. It avoids the
removal of a continuous sequence of blocks and successfully considers the
changing significance of each block. Consequently, as shown in Figure 7, the
LLMs streamlined using this metric maintain their perplexity scores remarkably
well regardless of the target models.

###  3.3 Proposed SLEB Algorithm

Algorithm 1 SLEB algorithm. We remove the transformer blocks until the number
of removed blocks reaches the target number.

$M\longleftarrow original$ $model$

$C\longleftarrow calibration$ $dataset$

$N\longleftarrow\\#$ $blocks$ $of$ $M$

$n\longleftarrow\\#$ $blocks$ $to$ $remove$

for $i=0$ to $n-1$ do

for $j=0$ to $N-i-1$ do

$S\longleftarrow Metric_{j}^{3}(M,C)$

if $S<min\\_S$ then

$min\\_S\longleftarrow S$

$min\\_S\\_idx\longleftarrow j$

end if

end for

$M\longleftarrow remove\\_block(M,min\\_S\\_idx)$

end for

The proposed SLEB verifies and eliminates redundant transformer blocks with
proposed $Metric^{3}$. The overall SLEB algorithm is summarized in Algorithm
1. SLEB can be seamlessly integrated into the forward pass of the LLM model,
similar to recent methods for pruning LLMs. It leverages calibration data to
estimate the redundancy of transformer blocks and uses $Metric^{3}$ to
pinpoint and remove the most redundant block. To determine the next block for
removal, the algorithm calculates the redundancy of transformer blocks within
the LLM that has been streamlined so far, allowing it to iteratively identify
and prune redundant blocks. This approach streamlines the model without the
need for an additional training process.

##  4 Experiments

###  4.1 Experimental Setup

We implement SLEB in PyTorch Paszke et al. (2019), using the HuggingFace
Transformers library Wolf et al. (2020). The experiments on redundancy
verification and elimination of transformer blocks are executed on NVIDIA A100
GPUs equipped with 80GB of memory. SLEB requires 2 GPUs for pruning OPT-66B
and LLaMA-70B, and 1 GPU for pruning smaller models. Runtime of SLEB for
pruning LLMs falls within the range of runtime observed with other pruning
methods (See Appendix A.1). For example, SLEB can fully compress LLaMA-2-70B,
within approximately 1.5 hours. SLEB is processed based on the inference-only
approach without any fine-tuning, aligning with contemporary practices in
post-training pruning of LLMs. We use 128 samples randomly selected from
WikiText-2 training dataset as calibration data, following the approach used
in previous works Ashkboos et al. (2024).

Our evaluation encompasses models from the OPT and LLaMA-2 families. We assess
SLEB under two target sparsity levels: 10% and 20%. In case that the product
of the total number of transformer blocks in a model and the target sparsity
is not an integer, we round up the product to determine the number of
transformer blocks to remove. The proposed SLEB is compared to previous works
that are widely recognized for their potential to accelerate LLM inference.
These previous works include SparseGPT Frantar & Alistarh (2023), Wanda Sun et
al. (2023), and DSnoT Zhang et al. (2024), which utilize 2:4 structured
pruning methods, and LLM-Pruner and SliceGPT, which employ channel-wise
pruning.

###  4.2 Elimination of Transformer Blocks using SLEB

We assess which transformer blocks are chosen to be eliminated with SLEB in
Figure 8. The locations of removed transformer blocks vary significantly
across the target models. In the case of OPT-13B, earlier transformer blocks
are selected more frequently, while middle and later blocks are chosen for
LLaMA-2-13B. This demonstrates that the sensitivity to the removal of
transformer blocks varies depending on the model. Therefore, it is crucial to
select a metric like the proposed $Metric^{3}$ that accurately reflects the
characteristics of each model to identify the optimal combinations of
transformer blocks for removal from various models. The removed block indices
for all evaluated LLMs are provided in Appendix A.2.

Figure 8: Locations of removed transformer blocks after applying SLEB with
target sparsity of 20%

###  4.3 Language Modeling

Table 1: OPT and LLaMA-2 perplexity results on C4 dataset |  Pruning |  | OPT | LLaMA-2
---|---|---|---|---
Method |  Unit | Sparsity | 6.7B | 13B | 30B | 66B | 7B | 13B | 70B
Dense |  - | 0% | 12.71 | 12.06 | 11.44 | 10.99 | 7.26 | 6.73 | 5.71
SparseGPT |  2:4 | 50% | 16.42 | 14.85 | 13.17 | 12.25 | 13.54 | 11.39 | 8.16
Wanda |  2:4 | 50% | 19.03 | 16.18 | 16.18 | 8414.05 | 15.57 | 12.47 | 8.10
DSnoT |  2:4 | 50% | 18.41 | 16.51 | 14.71 | 8360.81 | 15.56 | 12.22 | 8.15
LLM-Pruner |  Channel | 20% | - | - | - | - | 12.25 | 10.43 | -
SliceGPT |  Channel | 25% | 27.35 | 19.43 | 14.46 | 12.29 | 32.74 | 29.86 | 20.03
SliceGPT |  Channel | 30% | 33.43 | 22.58 | 15.89 | 13.08 | 41.69 | 38.43 | 25.79
SLEB |  T. Block | 10% | 13.84 | 12.43 | 11.65 | 11.25 | 9.34 | 7.80 | 6.32
SLEB |  T. Block | 20% | 15.99 | 13.81 | 12.74 | 12.54 | 12.32 | 9.42 | 7.31

We evaluate the linguistic capabilities of pruned LLMs on language modeling
tasks. Table 1 showcases the perplexity results on C4 validation dataset
Raffel et al. (2020). It is important to note that though the sparsity of SLEB
is 10% to 20% whereas previous works show a sparsity ratio between 20% to 50%,
as we will discuss in Section 4.6, SLEB outperforms in LLM inference speedup.
Though Wanda and DSnoT completely fail on OPT-66B, SLEB sufficiently preserves
perplexity scores for all evaluated cases. Moreover, SLEB outperforms previous
pruning methods in the majority of cases by a significant margin, though it
uses a coarser granularity of pruning unit. These results once again emphasize
a point discussed in Section 3.1 \- LLMs exhibit robustness at the level of
transformer blocks. By effectively identifying and removing transformer
blocks, SLEB can have a minimal impact on perplexity scores. The perplexity
results on WikiText-2 test dataset can be found in Appendix B.1.

###  4.4 Dependency on Calibration Dataset

We analyze the influence of the calibration data on the perplexity results of
pruned LLMs. During the pruning process of the target LLMs, 128 sequences are
randomly sampled either from WikiText-2 or C4 training datasets to serve as
the calibration data. Subsequently, the pruned LLMs are evaluated using both
WikiText-2 and C4 datasets. We compare pruning results of SparseGPT, Wanda,
SliceGPT with 25% sparsity, and SLEB with 20% sparsity.

(a) Perplexity results on WikiText-2

(b) Perplexity results on C4

Figure 9: Perplexity results of pruned LLMs depend on the calibration dataset. The colored bar graphs depict the results when there is a difference between the calibration dataset and the evaluation dataset.  Table 2: Mean accuracies (%) of OPT and LLaMA-2 on zero-shot tasks (T.Block: Transformer Block) |  Pruning |  | OPT | LLaMA-2
---|---|---|---|---
Method |  Unit | Sparsity | 6.7B | 13B | 30B | 66B | 7B | 13B | 70B
Dense |  - | 0% | 60.70 | 61.79 | 64.40 | 66.16 | 69.00 | 71.76 | 76.57
SparseGPT |  2:4 | 50% | 54.94 | 56.76 | 59.96 | 62.33 | 58.23 | 63.06 | 71.87
Wanda |  2:4 | 50% | 53.14 | 55.12 | 58.89 | 35.93 | 55.59 | 61.23 | 72.34
SliceGPT |  Channel | 25% | 54.28 | 59.27 | 62.11 | 65.17 | 55.49 | 58.90 | 69.75
SliceGPT |  Channel | 30% | 53.00 | 57.42 | 61.27 | 64.24 | 51.50 | 55.16 | 66.11
SLEB |  T. Block | 10% | 60.00 | 62.07 | 64.48 | 65.38 | 62.24 | 66.77 | 73.14
SLEB |  T. Block | 20% | 57.61 | 60.08 | 62.86 | 62.53 | 56.80 | 62.96 | 70.81

As shown in Figure 9, SLEB demonstrates the least dependency on the choice of
the calibration dataset. SliceGPT struggles to preserve perplexity scores when
the type of evaluation data differs from the type of calibration data.
SliceGPT relies on the principal component analysis of activation matrices to
determine which neurons to prune for channel-wise pruning, making its
performance more reliant on the calibration dataset. SparseGPT and Wanda take
a different approach by measuring the significance of weights in the matrix
multiplication results of each linear layer. These methods leverage the
information contained in the pre-trained weight matrices of LLMs and show less
dependency on the dataset. While previous methods measure redundancy at the
layer level, SLEB assesses the redundancy of each transformer block at the
entire network level using $Metric^{3}$. This approach fully exploits the
information present in the pre-trained LLMs and demonstrates less dependency
on the dataset.

###  4.5 Zero-shot Tasks

Following the evaluation methods presented in previous works Ashkboos et al.
(2024), we evaluate SLEB on following zero-shot tasks: PIQA Bisk et al.
(2020), WinoGrande Sakaguchi et al. (2019), HellaSwag Zellers et al. (2019),
ARC-easy and ARC-challenge Clark et al. (2018). LM Evaluation Harness Gao et
al. (2023) with its default parameters is used for the evaluation.

Table 2 shows the average accuracies of pruned LLMs on zero-shot tasks. SLEB
with 10% sparsity achieves superior accuracies compared to previous pruning
methods, preserving the original accuracies to a significant extent. Even when
the sparsity of SLEB is increased to 20%, it continues to outperform previous
approaches in most cases. The task-wise accuracies are provided in Appendix
B.2.

###  4.6 Speedup

Table 3: LLaMA-2-70B latency for prompt processing on 2 NVIDIA A100 GPUs. The sparsity is 25% for channel pruning and 20% for SLEB. |  Dense | 2:4 | Channel |  SLEB
---|---|---|---|---
| Pruning | Pruning
Latency (ms) | 1718.4 | 1555.5 | 1440.7 | 1364.1
Speedup | - | 1.10$\times$ | 1.19$\times$ | 1.26$\times$
Table 4: LLaMA-2-70B throughput for token generation on 2 NVIDIA A100 GPUs. The sparsity is 25% for channel pruning and 20% for SLEB. |  Dense | 2:4 | Channel |  SLEB
---|---|---|---|---
| Pruning | Pruning
Tokens/s | 299 | 293 | 331 | 381
Improvement | - | 0.98$\times$ | 1.10$\times$ | 1.27$\times$

We evaluate the LLM inference speedup achieved by SLEB in comparison to
previous pruning methods - 2:4 pruning and 25% channel-wise pruning. The
inference speed is assessed using the LLM implementation provided by
HuggingFace Transformers libraries for each pruning method. The language
processing of LLMs can be categorized into two stages: prompt processing and
token generation. In the prompt processing stage, LLMs assess the given prompt
and generate the KV cache. In the token generation stage, LLMs generate new
tokens in an auto-regressive manner. Because these two stages have distinct
characteristics in terms of inference speed, with the prompt processing stage
tending to be compute-bound and the token generation stage tending to be
memory-bound, we analyze the speedup in each of them separately.

Table 3 and Table 4 present latency and throughput results for LLaMA-2-70B
with 2 NVIDIA A100 GPUs during prompt processing and token generation,
respectively. For prompt processing, we measure the latency when processing an
input sequence with 2048 tokens. For token generation, the test scenario
consists of generating sentences with a length of 128 tokens and a batch size
of 64. In both scenarios, SLEB demonstrates improvements in latency and
throughput that are proportional to the pruning ratio. This performance
enhancement is attributed to the adoption of transformer block, which is a
fundamental building block of LLMs, as the basic unit of pruning. In contrast,
previous approaches fail to achieve significant improvements, particularly in
the case of token generation. For more detailed analysis results, please refer
to Appendix B.3. SLEB consistently provides speedup benefits across various
serving scenarios, while the speedup achieved by other methods is
significantly influenced by the specific serving scenarios.

###  4.7 Compatibility with Post-Training Quantization

Figure 10: Perplexity comparison between LLMs pruned with SLEB (target
sparsity: 20%) and those further compressed using AWQ, a 4-bit weight
quantization

Post-training quantization (PTQ) is a well-established technique for
compressing and accelerating LLMs Frantar et al. (2023); Lee et al. (2024);
Lin et al. (2023). If PTQ can be applied to compress LLMs alongside SLEB, we
can expect further improvements in memory efficiency and LLM inference speed.
In Figure 10, we present a comparison of perplexity scores for LLMs that have
undergone SLEB with target sparsity of 20% and those that have been further
compressed with AWQ Lin et al. (2023), a state-of-the-art PTQ algorithm that
quantizes weights to 4-bit integer values. The experimental results indicate
that applying PTQ to streamlined models has no discernible impact on their
perplexity. More analysis results on compatibility with PTQ are provided in
Appendix B.4.

##  5 Conclusion

This paper introduces SLEB, a novel technique for streamlining LLMs by
removing redundant transformer blocks. SLEB carefully identifies and
eliminates redundant blocks, ensuring that it effectively preserves the
language capabilities of LLMs without the need for any resource-intensive
training. Our experimental results demonstrate that SLEB can successfully
prune up to 20% of blocks while maintaining the linguistic performance of LLMs
across language modeling and zero-shot tasks. One of the significant
advantages of SLEB is its ability to remove entire transformer blocks,
resulting in substantial speed improvements in end-to-end LLM inference. This
speedup is applicable across various implementation scenarios, making SLEB a
practical solution for real-world LLM serving scenarios.

## Impact Statements

This paper presents work whose goal is to advance the field of Machine
Learning. There are many potential societal consequences of our work, none
which we feel must be specifically highlighted here.

## References

  * Ashkboos et al. (2024) Ashkboos, S., Croci, M. L., do Nascimento, M. G., Hoefler, T., and Hensman, J.  SliceGPT: Compress large language models by deleting rows and columns.  In _The Twelfth International Conference on Learning Representations_ , 2024.
  * Bisk et al. (2020) Bisk, Y., Zellers, R., Le bras, R., Gao, J., and Choi, Y.  Piqa: Reasoning about physical commonsense in natural language.  _Proceedings of the AAAI Conference on Artificial Intelligence_ , 34(05):7432–7439, Apr. 2020.  doi: 10.1609/aaai.v34i05.6239.  URL https://ojs.aaai.org/index.php/AAAI/article/view/6239.
  * Brown et al. (2020) Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al.  Language models are few-shot learners.  _Advances in neural information processing systems_ , 33:1877–1901, 2020.
  * Chen et al. (2023) Chen, Y., Pan, X., Li, Y., Ding, B., and Zhou, J.  Ee-llm: Large-scale training and inference of early-exit large language models with 3d parallelism.  _arXiv preprint arXiv:2312.04916_ , 2023.
  * Chowdhery et al. (2022) Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., Schuh, P., Shi, K., Tsvyashchenko, S., Maynez, J., Rao, A., Barnes, P., Tay, Y., Shazeer, N., Prabhakaran, V., Reif, E., Du, N., Hutchinson, B., Pope, R., Bradbury, J., Austin, J., Isard, M., Gur-Ari, G., Yin, P., Duke, T., Levskaya, A., Ghemawat, S., Dev, S., Michalewski, H., Garcia, X., Misra, V., Robinson, K., Fedus, L., Zhou, D., Ippolito, D., Luan, D., Lim, H., Zoph, B., Spiridonov, A., Sepassi, R., Dohan, D., Agrawal, S., Omernick, M., Dai, A. M., Pillai, T. S., Pellat, M., Lewkowycz, A., Moreira, E., Child, R., Polozov, O., Lee, K., Zhou, Z., Wang, X., Saeta, B., Diaz, M., Firat, O., Catasta, M., Wei, J., Meier-Hellstern, K., Eck, D., Dean, J., Petrov, S., and Fiedel, N.  Palm: Scaling language modeling with pathways, 2022.
  * Clark et al. (2018) Clark, P., Cowhey, I., Etzioni, O., Khot, T., Sabharwal, A., Schoenick, C., and Tafjord, O.  Think you have solved question answering? try arc, the ai2 reasoning challenge, 2018.
  * Del Corro et al. (2023) Del Corro, L., Del Giorno, A., Agarwal, S., Yu, B., Awadallah, A., and Mukherjee, S.  Skipdecode: Autoregressive skip decoding with batching and caching for efficient llm inference.  _arXiv preprint arXiv:2307.02628_ , 2023.
  * Din et al. (2023) Din, A. Y., Karidi, T., Choshen, L., and Geva, M.  Jump to conclusions: Short-cutting transformers with linear transformations.  _arXiv preprint arXiv:2303.09435_ , 2023.
  * Frantar & Alistarh (2022) Frantar, E. and Alistarh, D.  Optimal brain compression: A framework for accurate post-training quantization and pruning.  _Advances in Neural Information Processing Systems_ , 35:4475–4488, 2022.
  * Frantar & Alistarh (2023) Frantar, E. and Alistarh, D.  Sparsegpt: Massive language models can be accurately pruned in one-shot.  In _International Conference on Machine Learning_ , pp. 10323–10337. PMLR, 2023.
  * Frantar et al. (2023) Frantar, E., Ashkboos, S., Hoefler, T., and Alistarh, D.  Gptq: Accurate post-training quantization for generative pre-trained transformers.  _International Conference on Learning Representations_ , 2023.
  * Gao et al. (2023) Gao, L., Tow, J., Abbasi, B., Biderman, S., Black, S., DiPofi, A., Foster, C., Golding, L., Hsu, J., Le Noac’h, A., Li, H., McDonell, K., Muennighoff, N., Ociepa, C., Phang, J., Reynolds, L., Schoelkopf, H., Skowron, A., Sutawika, L., Tang, E., Thite, A., Wang, B., Wang, K., and Zou, A.  A framework for few-shot language model evaluation, 12 2023.  URL https://zenodo.org/records/10256836.
  * Han et al. (2015) Han, S., Pool, J., Tran, J., and Dally, W.  Learning both weights and connections for efficient neural network.  _Advances in neural information processing systems_ , 28, 2015.
  * Hassibi et al. (1993) Hassibi, B., Stork, D. G., and Wolff, G. J.  Optimal brain surgeon and general network pruning.  In _IEEE international conference on neural networks_ , pp. 293–299. IEEE, 1993.
  * LeCun et al. (1989) LeCun, Y., Denker, J., and Solla, S.  Optimal brain damage.  _Advances in neural information processing systems_ , 2, 1989.
  * Lee et al. (2024) Lee, C., Jin, J., Kim, T., Kim, H., and Park, E.  Owq: Lessons learned from activation outliers for weight quantization in large language models.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , 2024.
  * Lin et al. (2023) Lin, J., Tang, J., Tang, H., Yang, S., Dang, X., and Han, S.  Awq: Activation-aware weight quantization for llm compression and acceleration.  _arXiv preprint arXiv:2306.00978_ , 2023.
  * Liu et al. (2023) Liu, Z., Wang, J., Dao, T., Zhou, T., Yuan, B., Song, Z., Shrivastava, A., Zhang, C., Tian, Y., Re, C., et al.  Deja vu: Contextual sparsity for efficient llms at inference time.  In _International Conference on Machine Learning_ , pp. 22137–22176. PMLR, 2023.
  * Ma et al. (2023) Ma, X., Fang, G., and Wang, X.  Llm-pruner: On the structural pruning of large language models.  In _Thirty-seventh Conference on Neural Information Processing Systems_ , 2023.
  * Merity et al. (2016) Merity, S., Xiong, C., Bradbury, J., and Socher, R.  Pointer sentinel mixture models.  _arXiv preprint arXiv:1609.07843_ , 2016.
  * Mishra et al. (2021) Mishra, A., Latorre, J. A., Pool, J., Stosic, D., Stosic, D., Venkatesh, G., Yu, C., and Micikevicius, P.  Accelerating sparse deep neural networks.  _arXiv preprint arXiv:2104.08378_ , 2021.
  * Paszke et al. (2019) Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al.  Pytorch: An imperative style, high-performance deep learning library.  _Advances in neural information processing systems_ , 32, 2019.
  * Raffel et al. (2020) Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _Journal of Machine Learning Research_ , 21(140):1–67, 2020.  URL http://jmlr.org/papers/v21/20-074.html.
  * Sakaguchi et al. (2019) Sakaguchi, K., Bras, R. L., Bhagavatula, C., and Choi, Y.  Winogrande: An adversarial winograd schema challenge at scale.  _arXiv preprint arXiv:1907.10641_ , 2019.
  * Schuster et al. (2022) Schuster, T., Fisch, A., Gupta, J., Dehghani, M., Bahri, D., Tran, V., Tay, Y., and Metzler, D.  Confident adaptive language modeling.  _Advances in Neural Information Processing Systems_ , 35:17456–17472, 2022.
  * Shi et al. (2020) Shi, S., Wang, Q., and Chu, X.  Efficient sparse-dense matrix-matrix multiplication on gpus using the customized sparse storage format.  In _2020 IEEE 26th International Conference on Parallel and Distributed Systems (ICPADS)_ , pp. 19–26. IEEE, 2020.
  * Sun et al. (2023) Sun, M., Liu, Z., Bair, A., and Kolter, J. Z.  A simple and effective pruning approach for large language models.  _arXiv preprint arXiv:2306.11695_ , 2023.
  * Touvron et al. (2023a) Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., Rodriguez, A., Joulin, A., Grave, E., and Lample, G.  Llama: Open and efficient foundation language models, 2023a.
  * Touvron et al. (2023b) Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al.  Llama 2: Open foundation and fine-tuned chat models.  _arXiv preprint arXiv:2307.09288_ , 2023b.
  * Varshney et al. (2023) Varshney, N., Chatterjee, A., Parmar, M., and Baral, C.  Accelerating llama inference by enabling intermediate layer decoding via instruction tuning with lite.  _arXiv e-prints_ , pp. arXiv–2310, 2023.
  * Wang (2020) Wang, Z.  Sparsert: Accelerating unstructured sparsity on gpus for deep learning inference.  In _Proceedings of the ACM international conference on parallel architectures and compilation techniques_ , pp. 31–42, 2020.
  * Wolf et al. (2020) Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., Davison, J., Shleifer, S., von Platen, P., Ma, C., Jernite, Y., Plu, J., Xu, C., Scao, T. L., Gugger, S., Drame, M., Lhoest, Q., and Rush, A. M.  Transformers: State-of-the-art natural language processing.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_ , pp. 38–45, Online, October 2020. Association for Computational Linguistics.  URL https://www.aclweb.org/anthology/2020.emnlp-demos.6.
  * Zellers et al. (2019) Zellers, R., Holtzman, A., Bisk, Y., Farhadi, A., and Choi, Y.  Hellaswag: Can a machine really finish your sentence?  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , 2019.
  * Zhang et al. (2022) Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M., Li, X., Lin, X. V., Mihaylov, T., Ott, M., Shleifer, S., Shuster, K., Simig, D., Koura, P. S., Sridhar, A., Wang, T., and Zettlemoyer, L.  Opt: Open pre-trained transformer language models, 2022.
  * Zhang et al. (2024) Zhang, Y., Zhao, L., Lin, M., Sun, Y., Yao, Y., Han, X., Tanner, J., Liu, S., and Ji, R.  Dynamic sparse no training: Training-free fine-tuning for sparse llms.  In _The Twelfth International Conference on Learning Representations_ , 2024.

##  Appendix A SLEB details

###  A.1 Runtime

We compare runtime of SLEB and previous works for pruning LLaMA-2 models. The
official implementation of SparseGPT 111https://github.com/IST-
DASLab/sparsegpt offloads the entire LLMs to the CPU and loads the target
transformer block to GPU one by one for pruning, resulting in long runtime.
Hence, for both SparseGPT and Wanda, we utilize the official implementation of
Wanda 222https://github.com/locuslab/wanda, which supports distributing LLMs
across multiple GPUs. For SliceGPT, we use its official implementation
333https://github.com/microsoft/TransformerCompression/tree/main. As shown in
Table 5, runtime of SLEB is generally within the range of runtime of other
pruning methods.

Table 5: Comparison of runtime for pruning LLaMA-2 using NVIDIA A100 GPU | 7B | 13B | 70B
---|---|---|---
Method | #GPU | Time (s) | #GPU | Time (s) | #GPU | Time (s)
SparseGPT | 1 | 528 | 1 | 920 | 3 | 5040
Wanda | 1 | 64 | 1 | 104 | 2 | 480
SliceGPT | 1 | 452 | 1 | 633 | 1 | 3010
SLEB 20% | 1 | 113 | 1 | 279 | 2 | 5420

###  A.2 Selected Transformer Blocks

We summarize the number and indices of transformer blocks selected for removal
using SLEB in Table 6. The order of indices presented in Table 6 corresponds
to the order in which they are removed.

Table 6: Number and indices of removed blocks when applying SLEB on different LLM models. In 20% sparsity scenarios, it initially removes blocks selected for 10% sparsity and then selects additional blocks for removal to reach the desired sparsity level. Model | 10% Sparsity | 20% Sparsity
---|---|---
Type | Size | # Blocks | # Removed | Removed Indices | # Removed | Removed Indices
OPT | 6.7B | 32 | 4 | 6, 7, 3, 24 | 7 | \+ 18, 30, 11
13B | 40 | 4 | 5, 4, 9, 2 | 8 | \+ 14, 25, 34, 10
30B | 48 | 5 | 4, 17, 7, 12, 14 | 10 | \+ 40, 23, 2, 13, 43
66B | 64 | 7 | 18, 12, 21, 46, 8, 16, 9 | 13 | \+ 65, 13, 27, 61, 35, 42
LLaMA-2 | 7B | 32 | 4 | 14, 23, 11, 24 | 7 | \+ 10, 27, 15
13B | 40 | 4 | 31, 27, 13, 12 | 8 | \+ 33, 29, 14, 2
70B | 80 | 8 | 33, 29, 61, 11, 55, 30, 59, 28 | 16 | \+ 50, 27, 68, 32, 31, 15, 60, 4

##  Appendix B More Evaluation Results

###  B.1 Language Modeling

We use sequence length 2048 for all models to evaluate perplexity. Table 7
provides perplexity results of pruned models on WikiText-2 dataset. Notably,
SliceGPT demonstrates good perplexity in this case because it uses calibration
data sampled from the WikiText-2 training dataset. As discussed in Section
4.4, SliceGPT tends to perform well in terms of perplexity when the types of
calibration and evaluation datasets align. However, it may yield poor results
when the datasets do not align. This behavior is attributed to SliceGPT
relying solely on information from activation matrices to determine the target
channels for pruning. Hence, when SliceGPT is evaluated on C4 dataset, it
shows poor perplexity results (Table 1).

Table 7: OPT and LLaMA-2 perplexity results on WikiText-2 dataset (T.Block: Transformer Block) |  Pruning |  | OPT | LLaMA-2
---|---|---|---|---
Method |  Unit |  Sparsity | 6.7B | 13B | 30B | 66B | 7B | 13B | 70B
Dense |  - |  0% | 10.86 | 10.13 | 9.56 | 9.34 | 5.47 | 4.88 | 3.32
SparseGPT |  2:4 |  50% | 14.16 | 12.98 | 10.92 | 10.09 | 10.79 | 8.75 | 5.70
Wanda |  2:4 |  50% | 15.93 | 15.57 | 13.38 | 12122.15 | 12.09 | 8.99 | 5.48
DSnoT |  2:4 |  50% | 16.21 | 15.16 | 12.39 | 11037.35 | 11.97 | 8.87 | 5.49
LLM-Pruner |  Channel |  20% | - | - | - | - | 10.58 | 8.56 | -
SliceGPT |  Channel |  25% | 12.10 | 11.04 | 10.13 | 9.75 | 7.55 | 6.63 | 4.89
SliceGPT |  Channel |  30% | 12.73 | 11.49 | 10.39 | 9.94 | 8.59 | 7.44 | 5.44
SLEB |  T. Block |  10% | 11.22 | 10.16 | 9.57 | 9.33 | 6.95 | 5.63 | 3.98
SLEB |  T. Block |  20% | 12.94 | 11.40 | 10.73 | 10.18 | 9.14 | 6.80 | 4.88

###  B.2 Zero-shot tasks

Table 8: Accuracies (%) on zero-shot tasks Model | Method | Sparsity | PIQA | WinoGrande | HellaSwag | ARC-e | ARC-c | Avg.
---|---|---|---|---|---|---|---|---
| Dense | - | 76.39 | 65.19 | 67.16 | 60.14 | 34.64 | 60.70
| SpareGPT | 2:4 (50%) | 74.21 | 60.77 | 57.25 | 53.03 | 29.44 | 54.94
| Wanda | 2:4 (50%) | 71.76 | 60.22 | 54.20 | 51.18 | 28.33 | 53.14
OPT-6.7B | SliceGPT | 25% | 70.35 | 60.62 | 58.15 | 52.78 | 29.52 | 54.28
| SliceGPT | 30% | 68.61 | 60.69 | 54.56 | 52.15 | 29.01 | 53.00
| SLEB | 10% | 76.61 | 64.72 | 66.36 | 58.67 | 33.62 | 60.00
| SLEB | 20% | 74.92 | 61.33 | 62.13 | 57.07 | 32.59 | 57.61
| Dense | - | 76.82 | 64.80 | 69.81 | 61.87 | 35.67 | 61.79
| SpareGPT | 2:4 (50%) | 74.16 | 62.43 | 59.18 | 56.27 | 31.74 | 56.76
| Wanda | 2:4 (50%) | 72.25 | 61.72 | 57.97 | 53.96 | 29.69 | 55.12
OPT-13B | SliceGPT | 25% | 73.67 | 64.25 | 63.28 | 60.52 | 34.64 | 59.27
| SliceGPT | 30% | 71.82 | 62.90 | 60.66 | 58.80 | 32.94 | 57.42
| SLEB | 10% | 76.12 | 65.51 | 69.81 | 61.78 | 37.12 | 62.07
| SLEB | 20% | 75.90 | 63.46 | 66.87 | 59.60 | 34.56 | 60.08
| Dense | - | 78.07 | 68.19 | 72.27 | 65.24 | 38.23 | 64.40
| SpareGPT | 2:4 (50%) | 75.24 | 65.67 | 65.10 | 59.76 | 34.04 | 59.96
| Wanda | 2:4 (50%) | 75.46 | 63.54 | 63.41 | 60.14 | 31.91 | 58.89
OPT-30B | SliceGPT | 25% | 75.30 | 66.61 | 69.42 | 63.55 | 35.67 | 62.11
| SliceGPT | 30% | 74.97 | 65.04 | 68.15 | 63.55 | 34.64 | 61.27
| SLEB | 10% | 77.64 | 68.75 | 72.32 | 65.12 | 38.57 | 64.48
| SLEB | 20% | 76.93 | 67.40 | 70.62 | 61.99 | 37.37 | 62.86
| Dense | - | 79.82 | 68.90 | 74.85 | 67.21 | 40.02 | 66.16
| SpareGPT | 2:4 (50%) | 77.75 | 66.22 | 68.59 | 63.34 | 35.75 | 62.33
| Wanda | 2:4 (50%) | 50.65 | 50.51 | 25.97 | 25.29 | 27.22 | 35.93
OPT-66B | SliceGPT | 25% | 78.40 | 67.09 | 73.33 | 67.89 | 39.16 | 65.17
| SliceGPT | 30% | 77.42 | 66.30 | 72.62 | 66.90 | 37.97 | 64.24
| SLEB | 10% | 78.94 | 68.90 | 74.18 | 65.95 | 38.91 | 65.38
| SLEB | 20% | 76.99 | 66.22 | 70.77 | 63.34 | 35.32 | 62.53
| Dense | - | 79.11 | 69.06 | 75.99 | 74.58 | 46.25 | 69.00
| SpareGPT | 2:4 (50%) | 72.14 | 64.96 | 58.93 | 60.90 | 34.22 | 58.23
| Wanda | 2:4 (50%) | 70.84 | 62.27 | 55.33 | 57.58 | 31.91 | 55.59
LLaMA-7B | SliceGPT | 25% | 66.87 | 63.38 | 54.16 | 58.46 | 34.56 | 55.49
| SliceGPT | 30% | 63.55 | 61.33 | 49.62 | 51.77 | 31.23 | 51.50
| SLEB | 10% | 76.44 | 63.14 | 70.23 | 63.68 | 37.71 | 62.24
| SLEB | 20% | 73.07 | 58.96 | 62.47 | 56.48 | 33.02 | 56.80
| Dense | - | 80.47 | 72.22 | 79.39 | 77.48 | 49.23 | 71.76
| SpareGPT | 2:4 (50%) | 75.46 | 68.51 | 65.52 | 66.04 | 39.76 | 63.06
| Wanda | 2:4 (50%) | 73.94 | 67.01 | 63.09 | 64.31 | 37.80 | 61.23
LLaMA-13B | SliceGPT | 25% | 68.55 | 67.48 | 58.10 | 62.50 | 37.88 | 58.90
| SliceGPT | 30% | 66.10 | 65.11 | 52.69 | 56.82 | 35.07 | 55.16
| SLEB | 10% | 79.16 | 66.93 | 74.36 | 71.84 | 41.55 | 66.77
| SLEB | 20% | 76.61 | 64.96 | 70.55 | 64.35 | 38.31 | 62.96
| Dense | - | 82.70 | 77.98 | 83.84 | 80.98 | 57.34 | 76.57
| SpareGPT | 2:4 (50%) | 80.03 | 76.56 | 76.09 | 76.94 | 49.74 | 71.87
| Wanda | 2:4 (50%) | 80.30 | 74.66 | 79.22 | 76.35 | 51.19 | 72.34
LLaMA-70B | SliceGPT | 25% | 74.92 | 75.37 | 68.84 | 77.90 | 51.71 | 69.75
| SliceGPT | 30% | 72.31 | 73.56 | 63.69 | 73.40 | 47.61 | 66.11
| SLEB | 10% | 81.56 | 75.06 | 80.02 | 76.77 | 52.30 | 73.14
| SLEB | 20% | 80.14 | 72.93 | 77.21 | 75.38 | 48.38 | 70.81

###  B.3 Speedup

We provide a comprehensive analysis of LLM inference speed across a range of
LLM models using both NVIDIA A100 GPUs and A6000 GPUs. For prompt processing,
we conduct a single dummy test and then measure the latencies of 50 additional
tests. The average latency is calculated from these measurements. Similarly,
for token generation, we conduct a single dummy test and then measure the
throughputs of 10 additional tests. The average throughput is calculated from
these measurements.

As presented in Table 9 and Table 10, SLEB consistently enhances the latency
and throughput of LLM inference across various serving scenarios. However, the
impact of other pruning methods that prune weight values is significantly
influenced by the specific serving scenarios. As discussed in Section 2.1,
there are two reasons behind this. First, the impact of weight pruning on the
speed of matrix multiplication varies depending on the original matrix size,
even when the pruning is conducted in a structured manner. Second, LLMs have
various types of operations besides linear operations with matrix
multiplication, and these operations can also affect the overall inference
speed.

Table 9: LLaMA-2 latency for prompt processing |  |  | 7B | 13B | 70B
---|---|---|---|---|---
| Method |  Sparsity |  #GPU |  Latency |  Speedup |  #GPU |  Latency |  Speedup |  #GPU |  Latency |  Speedup
| Dense |  - |  1 |  240.0 |  - |  1 |  397.3 |  - |  2 |  1718.4 |  -
| 2:4 Pruning |  50% |  1 |  218.2 |  1.10$\times$ |  1 |  372.2 |  1.07$\times$ |  2 |  1555.5 |  1.10$\times$
A100 | Ch. Pruning |  25% |  1 |  213.3 |  1.13$\times$ |  1 |  349.9 |  1.14$\times$ |  2 |  1440.7 |  1.19$\times$
| SLEB |  10% |  1 |  209.3 |  1.15$\times$ |  1 |  355.1 |  1.12$\times$ |  2 |  1529.1 |  1.12$\times$
| SLEB |  20% |  1 |  187.3 |  1.28$\times$ |  1 |  316.0 |  1.26$\times$ |  2 |  1364.1 |  1.26$\times$
| Dense |  - |  1 |  234.5 |  - |  1 |  400.8 |  - |  4 |  1806.8 |  -
| 2:4 Pruning |  50% |  1 |  216.7 |  1.08$\times$ |  1 |  371.7 |  1.08$\times$ |  4 |  1707.2 |  1.06$\times$
A6000 | Ch. Pruning |  25% |  1 |  201.3 |  1.16$\times$ |  1 |  339.7 |  1.18$\times$ |  4 |  1475.6 |  1.22$\times$
| SLEB |  10% |  1 |  205.8 |  1.14$\times$ |  1 |  360.2 |  1.11$\times$ |  4 |  1616.2 |  1.12$\times$
| SLEB |  20% |  1 |  184.4 |  1.27$\times$ |  1 |  321.0 |  1.25$\times$ |  4 |  1434.3 |  1.26$\times$
Table 10: LLaMA-2 throughput for token generation |  |  | 7B | 13B | 70B
---|---|---|---|---|---
| Method |  Sparsity |  #GPU |  Tokens/s |  Improve. |  #GPU |  Tokens/s |  Improve. |  #GPU |  Tokens/s |  Improve.
| Dense |  - |  1 |  1649 |  - |  1 |  1078 |  - |  2 |  299 |  -
| 2:4 Pruning |  50% |  1 |  1306 |  0.79$\times$ |  1 |  952 |  0.88$\times$ |  2 |  293 |  0.98$\times$
A100 | Ch. Pruning |  25% |  1 |  1685 |  1.02$\times$ |  1 |  1109 |  1.03$\times$ |  2 |  331 |  1.11$\times$
| SLEB |  10% |  1 |  1856 |  1.13$\times$ |  1 |  1189 |  1.10$\times$ |  2 |  336 |  1.12$\times$
| SLEB |  20% |  1 |  2060 |  1.25$\times$ |  1 |  1337 |  1.24$\times$ |  2 |  381 |  1.27$\times$
| Dense |  - |  1 |  1200 |  - |  1 |  712 |  - |  4 |  104 |  -
| 2:4 Pruning |  50% |  1 |  1247 |  1.04$\times$ |  1 |  795 |  1.12$\times$ |  4 |  129 |  1.24$\times$
A6000 | Ch. Pruning |  25% |  1 |  1335 |  1.11$\times$ |  1 |  827 |  1.16$\times$ |  4 |  135 |  1.29$\times$
| SLEB |  10% |  1 |  1362 |  1.14$\times$ |  1 |  783 |  1.10$\times$ |  4 |  117 |  1.13$\times$
| SLEB |  20% |  1 |  1515 |  1.26$\times$ |  1 |  879 |  1.24$\times$ |  4 |  135 |  1.30$\times$

###  B.4 Compatibility with Post-Training Quantization

We demonstrate the compatibility of SLEB with PTQ. When further compressing
LLMs that have been pruned with SLEB using AWQ, we observe a negligible impact
on perplexity results for both C4 and WikiText-2 datasets. Additionally, it is
notable that the impact of quantization is less pronounced on larger models
with more than 13 billion parameters.

(a) 10% SLEB & 4-bit AWQ

(b) 20% SLEB & 4-bit AWQ

Figure 11: Perplexity comparison between LLMs pruned with SLEB and those
further compressed using AWQ, a 4-bit weight quantization (Evaluation dataset:
C4)

(a) 10% SLEB & 4-bit AWQ

(b) 20% SLEB & 4-bit AWQ

Figure 12: Perplexity comparison between LLMs pruned with SLEB and those
further compressed using AWQ, a 4-bit weight quantization (Evaluation dataset:
Wikitext-2)


# Space-Time Adaptive Processing in Connected and
Automated Vehicular Radar Platoons

###### Abstract

In this study, we develop a holistic framework for space-time adaptive
processing (STAP) in connected and automated vehicle (CAV) radar systems. We
investigate a CAV system consisting of multiple vehicles that transmit
frequency-modulated continuous-waveforms (FMCW), thereby functioning as a
multistatic radar. Direct application of STAP in a network of radar systems
such as in a CAV may lead to excess interference. We exploit time division
multiplexing (TDM) to perform transmitter scheduling over FMCW pulses to
achieve high detection performance. The TDM design problem is formulated as a
quadratic assignment problem which is tackled by power method-like iterations
and applying the Hungarian algorithm for linear assignment in each iteration.
Numerical experiments confirm that the optimized TDM is successful in
enhancing the target detection performance.

##  1 Introduction

Recent innovations in connected and automated vehicle (CAV) systems provide
new opportunities to enable better sensing of the environment. CAVs offer
advanced collision avoidance through onboard sensor technologies like radar,
cameras, and lidar [1]. Platooning in the CAV systems is when vehicles travel
in groups with very short inter-vehicle spacing. The connectivity among the
vehicles in a platoon enables a CAV to receive information from surrounding
vehicles and infrastructure [2]. In this regard, a network of automotive radar
systems assisting each other in sensing the environment is enabled. Whilst a
single vehicle radar may suffer from obstruction, fading, or lack of radial
velocity component of the target with respect to the radar, it is unlikely
that this will be the case with multi-vehicle systems with different
transmitter-target-receiver paths, also known as networked radar in the
literature [3, 4].

In this study, we aim to leverage the cooperative capabilities of
transportation systems—specifically, vehicle-to-vehicle (V2V) and vehicle-to-
infrastructure (V2I) communications—to develop a distributed space-time
adaptive processing (STAP) scheme for CAVs. Each vehicle is assumed to travel
at a speed and in a direction that can be different from those of the other
vehicles. However, the properties of the environment, positioning and velocity
information of the vehicles in the platoon are assumed to be accessible at
each vehicle through V2V communication, therefore making a cooperative STAP
scheme feasible. We formulate the cooperative STAP as a decentralized
multistatic target detection problem [5].

Direct application of STAP can reduce target detection performance because of
high sidelobes and susceptibility to interference [6, 7, 8, 9, 10, 11, 12]. On
the transmitter side, the interference can be addressed by transmitting well-
designed radar signals that are nearly orthogonal to each other in the
spectral or temporal domains [13]. In order to enhance robustness against
interference, we propose a time division multiplexing(TDM) framework in which
among all the transmitters in the CAV, only one is scheduled to transmit
during each pulse. In other words, we assume that frequency-modulated
continuous-waveforms (FMCW) signals with similar properties but with active or
silent chirps are generated. We consider an extended version of the
transmitter scheduling framework proposed by [7] and demonstrate that the
transmitter scheduling problem is a quadratic assignment problem (QAP) and
address it by means of power method-like iterations. At each iteration the
problem is boiled down to a linear assignment problem which is efficiently
solved by the Hungarian algorithm [14].

The rest of this paper is organized as follows. In the next section, we
introduce the system model for the cooperative STAP in CAVs. In Section 3, we
formulate the design problem for obtaining the optimal TDM in our system.
Section 4 presents our approach based on the Hungarian algorithm to optimize
the TDM matrix. We evaluate our methods via numerical experiments in Section 5
and conclude the paper in Section 6.

_Notation:_ Throughout this paper, we use bold lowercase and bold uppercase
letters for vectors and matrices, respectively. The $(m,n)$-th element of the
matrix $\mathbf{A}$ is $\mathbf{A}_{mn}$. The sets of complex and real numbers
are $\mathbb{C}$ and $\mathbb{R}$, respectively; $(\cdot)^{\top}$,
$(\cdot)^{\ast}$ and $(\cdot)^{\mathrm{H}}$ are the vector/matrix transpose,
conjugate, and Hermitian transpose, respectively. The trace of a matrix is
$\operatorname{Tr}(.)$; the function $\textrm{diag}(.)$ returns the diagonal
elements of the input matrix; and $\textrm{Diag}(.)$ and
$\text{Blkdiag}\left(\cdot\right)$ produce a diagonal/block-diagonal matrix
with the same diagonal entries/blocks as their vector/matrices argument. The
Hadamard (element-wise) and Kronecker products are $\odot$ and $\otimes$,
respectively. $l_{2}$-norm of $\mathbf{a}$ and Frobenius norm of $\mathbf{A}$
is denoted by $\|\mathbf{a}\|_{{}_{2}}$ and $\|\mathbf{A}\|_{{}_{\text{F}}}$,
respectively. $\mathrm{vec}_{{}_{M,N}}^{-1}\left(\mathbf{a}\right)$ reshapes
the input vector $\mathbf{a}\in\mathbb{C}^{MN\times 1}$ into a matrix
$\mathbf{A}\in\mathbb{C}^{M\times N}$ such that
$\mathrm{vec}\left(\mathbf{A}\right)=\mathbf{a}$.

##  2 System Model

We consider a network of $K$ cooperative vehicles, each equipped with radars
that have $N$ transmit antennas and $M$ receive antennas arranged as a uniform
linear array (ULA). Each vehicle transmits $L$ FMCW chirps during the CPI time
of $T$, with similar bandwidth $B$ and chirp time $T_{c}$. The transmit
waveform at $l-$th pulse of one transmitter antenna is

$s(t,l)=\text{rect}\left(\frac{t-lT_{c}}{T}\right)e^{\mathrm{j}2\pi[f_{c}+\frac{B}{T}(t-lT_{c})](t-lT_{c})}.$
(1)

We denote the position of the $n-$th transmitter on vehicle $k$ by
$\mathbf{p}_{{}_{T,kn}}\in\mathds{R}^{2\times 1}$ and the position of $m-$th
receiver on vehicle $i$ by $\mathbf{p}_{{}_{R,im}}\in\mathds{R}^{2\times 1}$.
We further assume that the target at position
$\mathbf{p}_{{}_{t}}\in\mathds{R}^{2\times 1}$ is moving with a velocity
$\mathbf{v}_{t}\in\mathds{R}^{2\times 1}$. The Doppler velocity of the target
with respect to vehicle $k$ is

$\nu_{{}_{k}}=\mathbf{v}_{t}^{\top}\mathbf{p}_{{}_{tk}},$ (2)

where
$\mathbf{p}_{{}_{tk}}=[\sin{\theta_{{}_{tk}}},\cos{\theta_{{}_{tk}}}]^{\top}$
is the direction vector of the target with DoA $\theta_{{}_{tk}}$ with respect
to vehicle $k$.

The range from the $n$-th transmitter on vehicle $k$ to the target is

$R_{{}_{k}}(t,l,n)=\|\mathbf{p}_{{}_{t}}-\mathbf{p}_{{}_{R,im}}\|_{{}_{2}}+\nu_{{}_{k}}(t+(n-1+(l-1)N)T_{c}),$
(3)

and range from target to the m-th Rx on vehicle $i$ is

$R_{{}_{i}}(t,l,m)=\|\mathbf{p}_{{}_{t}}-\mathbf{p}_{{}_{R,im}}\|_{{}_{2}}+\nu_{{}_{i}}(t+(m-1+(l-1)M)T_{c}).$
(4)

Consequently, the delay introduced into the signal is

$\tau_{{}_{ki}}=\frac{R_{{}_{k}}(t,l,n)+R_{{}_{i}}(t,l,m)}{c},$ (5)

where $c$ is the speed of light.

We assume that each pair of vehicles $i,k\in\\{1,\ldots,K\\}$ is connected via
V2V communication links. In automotive radar, the signal processing flow
sequentially comprises of sampling, range estimation, Doppler processing, and
DoA estimation. In STAP, after range processing the Doppler and DoA are
processed simultaneously by means of 2D adaptive matched filters [16, 5].
Therefore, after sampling the signal backscattered from the target i.e.
$s(t-\tau_{{}_{ki}},l)$ and estimating the range, the received signal at the
designated range bin at the $m$-th Rx on vehicle $i$ from the $n$-th Tx on
vehicle $k$ is

$\displaystyle s_{{}_{ki}}(l,n,m)=$
$\displaystyle\alpha_{k}e^{-\mathrm{j}\frac{2\pi
f_{c}}{c}\nu_{k}\left((l-1)N+(n-1)\right)T_{c}}$ $\displaystyle
e^{-\mathrm{j}\frac{2\pi f_{c}}{c}\nu_{i}\left((l-1)M+(m-1)\right)T_{c}}$
$\displaystyle e^{\mathrm{j}2\pi
f_{c}\left(\mathbf{p}_{{}_{T,kn}}^{\top}\mathbf{p}_{{}_{tk}}\right)}e^{\mathrm{j}2\pi
f_{c}\left(\mathbf{p}_{{}_{R,im}}^{\top}\mathbf{p}_{{}_{ti}}\right)},$ (6)

where the delay in (5) is expanded with respect to the first element of ULA in
a similar manner as in [7] and $\alpha_{k}$ is the complex target reflection
factor. We introduce the Doppler steering vector as

$\mathbf{a}_{{}_{d,N}}(\nu)=\left[1,e^{-\mathrm{j}\frac{2\pi
f_{c}\nu}{c}NT_{c}},\ldots,e^{-\mathrm{j}\frac{2\pi
f_{c}\nu}{c}(L-1)NT_{c}}\right]^{\top},$ (7)

and the auxiliary Doppler steering vector as

$\mathbf{a}_{{}_{D,M}}(\nu)=\left[1,e^{-\mathrm{j}\frac{2\pi
f_{c}\nu}{c}T_{c}},\ldots,e^{-\mathrm{j}\frac{2\pi
f_{c}\nu}{c}(M-1)T_{c}}\right]^{\top}.$ (8)

The transmit array steering vector at vehicle $k$ is

$\mathbf{a}_{{}_{T,k}}(\theta)=\left[e^{\mathrm{j}2\pi
f_{c}\left(\mathbf{p}_{{}_{T,k1}}^{\top}\mathbf{p}_{{}_{tk}}\right)},\ldots,e^{\mathrm{j}2\pi
f_{c}\left(\mathbf{p}_{{}_{T,kN}}^{\top}\mathbf{p}_{{}_{tk}}\right)}\right]^{\top},$
(9)

and the receive array steering vector at vehicle $i$ is

$\mathbf{a}_{{}_{R,i}}(\theta)=\left[e^{\mathrm{j}2\pi
f_{c}\left(\mathbf{p}_{{}_{R,i1}}^{\top}\mathbf{p}_{{}_{ti}}\right)},\ldots,e^{\mathrm{j}2\pi
f_{c}\left(\mathbf{p}_{{}_{R,iM}}^{\top}\mathbf{p}_{{}_{ti}}\right)}\right]^{\top}.$
(10)

The snapshot signal received at vehicle $i$ from $n$-th Tx on vehicle $k$ is

$\displaystyle\mathbf{s}_{{}_{ki}}(n)=$
$\displaystyle\left[\mathbf{a}_{{}_{d,N}}(\nu_{k})\odot\mathbf{a}_{{}_{d,M}}(\nu_{i})\right]$
$\displaystyle\otimes\left[\mathbf{a}_{{}_{R,i}}(\theta)\odot\mathbf{a}_{{}_{D,M}}(\nu_{i})\right]\in\mathds{C}^{LM\times
1}.$ (11)

By stacking the echoes from all N Tx on vehicle $k$, we obtain

$\displaystyle\mathbf{s}_{{}_{ki}}$
$\displaystyle=\begin{bmatrix}\mathbf{s}_{{}_{ki}}(1)\\\ \vdots\\\
\mathbf{s}_{{}_{ki}}(N)\\\ \end{bmatrix}$
$\displaystyle=\left(\mathbf{a}_{{}_{T,k}}(\theta)\odot\mathbf{a}_{{}_{D,N}}(\nu_{k})\right)\otimes\Big{(}\left(\mathbf{a}_{{}_{d,N}}(\nu_{k})\odot\mathbf{a}_{{}_{d,M}}(\nu_{i})\right)$
$\displaystyle\;\;\otimes\left(\mathbf{a}_{{}_{R,i}}(\theta)\odot\mathbf{a}_{{}_{D,M}}(\nu_{i})\right)\Big{)}\in\mathds{C}^{NLM\times
1}.$ (12)

As illustrated in Fig. 1, we focus on a scenario wherein vehicle $i$, serving
as the lead in the platoon, receives assistance from all other vehicles for
its sensing task. The target is in the field of view (FoV) of all vehicles and
therefore the whole CAV can perform as a multistatic radar to sense it. To be
succinct, hereafter we remove the subscript $i$ i.e. without loss of
generality $\mathbf{s}_{ki}$ will be replaced by $\mathbf{s}_{k}$.

RX/TX 1TX 3TX 2

Fig. 1: A simplified illustration of a CAV radar platoon consisting of three
vehicles, sensing a target in the FoV of all three vehicles. The radar on
vehicle 1, denoted by RX/TX1, leads the platoon and is assisted by two other
radars, denoted by TX2 and TX3.

In order to decide whether a target is present in a particular known range-
cell, we perform binary hypothesis testing between $\mathcal{H}_{0}$ (target-
free hypothesis) and $\mathcal{H}_{1}$ (target-present hypothesis), i.e.,

$\displaystyle\mathcal{H}_{0}$
$\displaystyle:\quad\mathbf{y}_{k}=\mathbf{n}_{k}$
$\displaystyle\mathcal{H}_{1}$
$\displaystyle:\quad\mathbf{y}_{k}=\alpha_{k}\mathbf{s}_{k}+\mathbf{n}_{k},$
(13)

where $\alpha_{k}$ is the complex target reflectivity factor and
$\mathbf{n}_{k}$ is the noise and interference with covariance
$\mathbf{R}_{k}$ [17]. The log-likelihood ratio test statistic is given by
[5],

$\zeta=\sum_{k=1}^{K}\frac{|\mathbf{s}_{k}^{\mathrm{H}}\mathbf{R}_{k}^{-1}\mathbf{y}_{k}|^{2}}{\frac{|\alpha_{k}|^{2}}{2}+\mathbf{s}_{k}^{\mathrm{H}}\mathbf{R}_{k}^{-1}\mathbf{s}_{k}}\underset{\mathcal{H}_{0}}{\overset{\mathcal{H}_{1}}{\gtrless}}\gamma.$
(14)

Since $|\mathbf{s}_{k}^{\mathrm{H}}\mathbf{R}_{k}^{-1}\mathbf{y}_{k}|$ is the
magnitude of a complex Gaussian random variable, the test statistic in (14) is
a sum of $K$ exponential random variables and therefore follows a hypo-
exponential distribution [18].

Under $\mathcal{H}_{1}$, we have

$\mathds{E}\left\\{{\zeta|\mathcal{H}_{1}}\right\\}=\sum_{k=1}^{K}\frac{\mathbf{C}_{k}+2|\alpha_{k}|^{2}\mathbf{C}_{k}^{2}}{\frac{|\alpha_{k}|^{2}}{2}+\mathbf{C}_{k}},$
(15)

where
$\mathbf{C}_{k}=\mathbf{s}_{k}^{\mathrm{H}}\mathbf{R}_{k}^{-1}\mathbf{s}_{k}$.
The probability of detection and false alarm are obtained respectively as

$\displaystyle\text{P}_{{}_{\text{D}}}$
$\displaystyle=\mathds{P}\mathrm{r}\left\\{{\zeta>\gamma|\mathcal{H}_{1}}\right\\}=1-\mathds{P}\mathrm{r}\left\\{{\zeta\leq\gamma|\mathcal{H}_{1}}\right\\}=1-F_{\zeta|\mathcal{H}_{1}}(\gamma|\mathcal{H}_{1}),$
$\displaystyle\text{P}_{{}_{\text{FA}}}$
$\displaystyle=\mathds{P}\mathrm{r}\left\\{{\zeta>\gamma|\mathcal{H}_{0}}\right\\}=1-\mathds{P}\mathrm{r}\left\\{{\zeta\leq\gamma|\mathcal{H}_{0}}\right\\}=1-F_{\zeta|\mathcal{H}_{0}}(\gamma|\mathcal{H}_{0}),$
(16)

where $F_{\zeta|\mathcal{H}}(.)$ is the cumulative distribution function of
the test statistic with hypo-exponential distribution. By carefully observing
(14), one can verify that the weighting
$(\frac{|\alpha_{k}|^{2}}{2}+\mathbf{s}_{k}^{\mathrm{H}}\mathbf{R}_{k}^{-1}\mathbf{s}_{k})^{-1}$
accounts for the contribution of vehicle $k$ in the test statistic. If
$\mathbf{s}_{k}^{\mathrm{H}}\mathbf{R}_{k}^{-1}\mathbf{s}_{k}\ll\frac{|\alpha_{k}|^{2}}{2}$
then we can interpret it as the signal propagated from vehicle $k$ not
reaching the receiver. Therefore, for the sake of interpretation here we
assume
$\mathbf{s}_{k}^{\mathrm{H}}\mathbf{R}_{k}^{-1}\mathbf{s}_{k}\gg\frac{|\alpha_{k}|^{2}}{2}$
for all vehicles i.e. $k\in\\{1,\ldots,K\\}$. Under this condition, (15) is

$\mathds{E}\left\\{{\zeta|\mathcal{H}_{1}}\right\\}=\sum_{k=1}^{K}1+2|\alpha_{k}|^{2}\mathbf{C}_{k}$
(17)

The signal received at vehicle $i$ from all vehicles, i.e.,
$k\in\\{1,\ldots,K\\}$, is thus given by

$\displaystyle\mathbf{s}$ $\displaystyle=\begin{bmatrix}\mathbf{s}_{{}_{1}}\\\
\vdots\\\ \mathbf{s}_{{}_{K}}\\\ \end{bmatrix}\in\mathds{C}^{KNLM},$ (18)

where $\mathbf{s}$ is also equivalent to the 2D space-time steering vector of
the target as observed at vehicle $i$.

##  3 TDM design

Since multiple signals share the same communication channel, the transmitted
signals need to be orthogonal to be distinguishable at the receiver. For this
matter in a CPI the antennas need to take turns to transmit. We propose to
incorporate TDM by designing a transmitter scheduling matrix for the platoon
of vehicles. In this particular TDM scheme, at most one antenna within the
platoon is allowed to transmit during each pulse. We reformulate the steering
vectors under the TDM scheme. If $\mathbf{e}_{{}_{kn}}$ is a one-hot vector of
size $L\times 1$ with only one element as unity and remaining elements zero,
then under TDM, (2) is

$\mathbf{s}_{{}_{ki}}(n)=\left[\mathbf{e}_{{}_{kn}}\odot\mathbf{a}_{{}_{d,N}}(\nu_{k})\odot\mathbf{a}_{{}_{d,M}}(\nu_{i})\right]\otimes\left[\mathbf{a}_{{}_{R,i}}(\theta)\odot\mathbf{a}_{{}_{D,M}}(\nu_{i})\right],$
(19)

where the unity element in $\mathbf{e}_{kn}$, indicates the pulse at which the
transmitter antenna $n$ on vehicle $k$ transmits. Let
$\mathbf{J}_{k}=\left[\mathbf{e}_{{}_{k1}}\bigm{\lvert}\ldots\bigm{\lvert}\mathbf{e}_{{}_{kN}}\right]\in\\{0,1\\}^{L\times
N}$. Following the same procedure leading to (2), we can write

$\displaystyle\bar{\mathbf{s}}_{{}_{k}}$
$\displaystyle=\Big{(}\vec{\mathbf{J}_{k}}\otimes\boldsymbol{1}\Big{)}\odot\Big{(}\left(\mathbf{a}_{{}_{T,k}}(\theta)\odot\mathbf{a}_{{}_{D,N}}(\nu_{k})\right)\otimes$
$\displaystyle\left(\mathbf{a}_{{}_{d,N}}(\nu_{k})\odot\mathbf{a}_{{}_{d,M}}(\nu_{i})\right)\otimes\left(\mathbf{a}_{{}_{R,i}}(\theta)\odot\mathbf{a}_{{}_{D,M}}(\nu_{i})\right)\Big{)}$
$\displaystyle=\Big{(}\vec{\mathbf{J}_{k}}\otimes\boldsymbol{1}\Big{)}\odot\mathbf{s}_{k}$
(20)

and (18) after performing the TDM is

$\displaystyle\bar{\mathbf{s}}$
$\displaystyle=\begin{bmatrix}\Big{(}\vec{\mathbf{J}_{1}}\otimes\boldsymbol{1}\Big{)}\odot\mathbf{s}_{{}_{1}}\\\
\vdots\\\
\Big{(}\vec{\mathbf{J}_{k}}\otimes\boldsymbol{1}\Big{)}\odot\mathbf{s}_{{}_{K}}\\\
\end{bmatrix}=\Big{(}\mathrm{vec}\left([\mathbf{J}_{1}\bigm{\lvert}\ldots\bigm{\lvert}\mathbf{J}_{K}]\right)\otimes\boldsymbol{1}_{M}\Big{)}\odot\mathbf{s}$
$\displaystyle=\Big{(}\mathrm{vec}\left(\mathbf{J}\right)\otimes\boldsymbol{1}_{M}\Big{)}\odot\mathbf{s}\in\mathds{C}^{KNLM\times
1},$ (21)

where
$\mathbf{J}=\left[\mathbf{J}_{1}\bigm{\lvert}\ldots\bigm{\lvert}\mathbf{J}_{K}\right]\in\\{0,1\\}^{L\times
KN}$ is the waveform selection matrix. Without loss of generality, we design a
TDM scheme in which the antennas in the whole platoon take turns over the
pulses to transmit. Under this scheme for a CPI of length $L=KN$, each antenna
selected to transmit at pulse $l$, excludes all other antennas to be
transmitting and as a consequence the waveform selection matrix $\mathbf{J}$
is a permutation matrix of size $L\times L$. We intend to maximize the target
detection performance. We use the mean of the test statistic as the design
criteria. Consequently the TDM design problem is

$\displaystyle\mathcal{P}_{1}:\;\underset{\mathbf{J}}{\text{minimize}}$
$\displaystyle\quad\mathds{E}\left\\{{\zeta|\mathcal{H}_{1}}\right\\}$ subject
to $\displaystyle\quad\sum_{p}\mathbf{J}_{pn}=1,\quad\quad
p,n\in\\{1,\ldots,L\\};$ $\displaystyle\quad\sum_{n}\mathbf{J}_{pn}=1;$
$\displaystyle\quad\leavevmode\nobreak\ \leavevmode\nobreak\
\mathbf{J}_{pn}\in\\{0,1\\}.$ (22)

##  4 solution methodology

In this section, we first demonstrate that $\mathcal{P}_{1}$ is a QAP [19]
which is a combinatorial optimization problem that is NP-hard in general form.
Consequently, we introduce a computationally efficient procedure to obtain a
local optimum of QAP. Our proposed method takes advantage of the power method-
like iterations introduced in [20], which resembles the well-known power
method for computing the dominant eigenvalue and vector pairs of matrices.

We accumulate the interference covariance matrices in $\mathbf{R}$ such that
$\mathbf{R}=\text{Blkdiag}\left(\mathbf{R}_{1},\ldots,\mathbf{R}_{K}\right)$.
By substituting (3) in (17) we obtain

$\displaystyle\mathds{E}\left\\{{\zeta|\mathcal{H}_{1}}\right\\}=\bar{\mathbf{s}}^{\mathrm{H}}\mathbf{R}^{-1}\bar{\mathbf{s}}$
$\displaystyle=\left(\Big{(}\mathrm{vec}\left(\mathbf{J}\right)\otimes\boldsymbol{1}_{M}\Big{)}\odot\mathbf{s}\right)^{\mathrm{H}}\mathbf{R}^{-1}\left(\Big{(}\mathrm{vec}\left(\mathbf{J}\right)\otimes\boldsymbol{1}_{M}\Big{)}\odot\mathbf{s}\right)$
$\displaystyle=\Big{(}\mathrm{vec}\left(\mathbf{J}\right)\otimes\boldsymbol{1}_{M}\Big{)}^{\mathrm{H}}\mathrm{Diag}\left(\mathbf{s}\right)^{\mathrm{H}}\mathbf{R}^{-1}\mathrm{Diag}\left(\mathbf{s}\right)\Big{(}\mathrm{vec}\left(\mathbf{J}\right)\otimes\boldsymbol{1}_{M}\Big{)}$
$\displaystyle=\mathrm{Tr}\left(\Big{(}\mathrm{vec}\left(\mathbf{J}\right)\otimes\boldsymbol{1}_{M}\Big{)}^{\mathrm{H}}\mathbf{Q}\Big{(}\mathrm{vec}\left(\mathbf{J}\right)\otimes\boldsymbol{1}_{M}\Big{)}\right)$
$\displaystyle=\Big{(}\mathrm{vec}\left(\mathbf{J}\right)\otimes\boldsymbol{1}_{M}\Big{)}^{\mathrm{H}}\mathrm{vec}\left(\mathbf{Q}\Big{(}\mathrm{vec}\left(\mathbf{J}\right)\otimes\boldsymbol{1}_{M}\Big{)}\right)$
$\displaystyle=\mathrm{vec}\left(\mathbf{J}\right)^{\mathrm{H}}\mathbf{G}^{\mathrm{H}}\mathbf{Q}\;\mathbf{G}\;\mathrm{vec}\left(\mathbf{J}\right)$
(23)

where

$\displaystyle\mathbf{Q}$
$\displaystyle=\mathrm{Diag}\left(\mathbf{s}\right)^{\mathrm{H}}\mathbf{R}^{-1}\mathrm{Diag}\left(\mathbf{s}\right)$
(24) $\displaystyle\mathbf{G}$
$\displaystyle=(\mathbf{K}_{{}_{1,KNL}}\otimes\mathbf{I}_{M})(\mathbf{I}_{KNL}\otimes\boldsymbol{1}_{M})$
(25)

and $\mathbf{K}_{{}_{1,KNL}}$ is the commutation matrix satisfying

$\displaystyle\mathbf{K}_{{}_{1,KNL}}\mathrm{vec}\left(\mathbf{J}\right)=\mathrm{vec}\left(\mathbf{J}^{\top}\right).$
(26)

The above algebraic manipulations cast $\mathcal{P}_{1}$ equivalent to a QAP
as

$\displaystyle\mathcal{P}_{2}:\leavevmode\nobreak\
\underset{\mathbf{J}\in\boldsymbol{\Omega}}{\textrm{minimize}}$
$\displaystyle\quad\mathrm{vec}\left(\mathbf{J}\right)^{\mathrm{H}}\mathbf{S}\;\mathrm{vec}\left(\mathbf{J}\right),$
(27)

where $\mathbf{S}=\mathbf{G}^{\mathrm{H}}\mathbf{Q}\mathbf{G}$ and
$\boldsymbol{\Omega}$ is the set of permutation matrices i.e.

$\displaystyle\boldsymbol{\Omega}=\Bigg{\\{}\mathbf{J}\;\biggm{\lvert}\;\sum_{p}\mathbf{J}_{pn}=1,$
$\displaystyle\quad\sum_{n}\mathbf{J}_{pn}=1,\quad$ (28)
$\displaystyle\mathbf{J}_{pn}\in\\{0,1\\},\quad
p,n\in\\{1,\ldots,L\\}\Bigg{\\}}.$

By performing diagonal loading, i.e., substituting $\mathbf{S}$ with positive
semi-definite matrix $\bar{\mathbf{S}}=\lambda_{m}\mathbf{I}-\mathbf{S}$, with
$\lambda_{m}$ being the maximum eigenvalue of $\mathbf{S}$, $\mathcal{P}_{2}$
will be turned into the equivalent maximization problem:

$\displaystyle\mathcal{P}_{3}:\leavevmode\nobreak\
\underset{\mathbf{J}\in\boldsymbol{\Omega}}{\textrm{maximize}}$
$\displaystyle\quad\mathrm{vec}\left(\mathbf{J}\right)^{\mathrm{H}}\bar{\mathbf{S}}\;\mathrm{vec}\left(\mathbf{J}\right).$
(29)

One can locally optimize $\mathcal{P}_{3}$ by resorting to power method-like
iterations of the form [20, 21]:

$\displaystyle\mathcal{P}_{4}$ $\displaystyle:\leavevmode\nobreak\
\underset{\mathbf{J}^{(s+1)}\in\boldsymbol{\Omega}}{\textrm{minimize}}\quad\|\mathrm{vec}\left(\mathbf{J}^{(s+1)}\right)-\bar{\mathbf{S}}\;\mathrm{vec}\left(\mathbf{J}^{(s)}\right)\|_{{}_{2}}$
$\displaystyle\equiv$
$\displaystyle\quad\underset{\mathbf{J}^{(s+1)}\in\boldsymbol{\Omega}}{\textrm{minimize}}\quad\|\mathbf{J}^{(s+1)}-\mathrm{vec}_{{}_{L,L}}^{-1}\left(\bar{\mathbf{S}}\;\mathrm{vec}\left(\mathbf{J}^{(s)}\right)\right)\|_{{}_{\text{F}}}.$
(30)

We define the matrix
$\mathbf{C}^{(s)}=-\mathrm{vec}_{{}_{L,L}}^{-1}\left(\bar{\mathbf{S}}\;\mathrm{vec}\left(\mathbf{J}^{(s)}\right)\right)$.
It is straightforward to see

$\displaystyle\|\mathbf{J}^{(s+1)}+\mathbf{C}^{(s)}\|^{2}_{{}_{\text{F}}}=\mathrm{Tr}\left((\mathbf{J}^{(s+1)}+\mathbf{C}^{(s)})^{\mathrm{H}}(\mathbf{J}^{(s+1)}+\mathbf{C}^{(s)})\right)$
$\displaystyle=\mathrm{Tr}\left(\mathbf{I}+\mathbf{C}^{(s)\mathrm{H}}\mathbf{C}^{(s)}\right)+\mathrm{Tr}\left(\mathbf{C}^{(s)\mathrm{H}}\mathbf{J}^{(s+1)}+\mathbf{J}^{(s+1)\mathrm{H}}\mathbf{C}^{(s)}\right)$
$\displaystyle=\mathrm{Tr}\left(\mathbf{I}+\mathbf{C}^{(s)\mathrm{H}}\mathbf{C}^{(s)}\right)+2\mathrm{Tr}\left(\mathbf{J}^{(s+1)}\mathbf{C}^{(s)\mathrm{H}}\right),$
(31)

where we used the orthogonality property of permutation matrices i.e.
$\mathbf{J}^{(s+1)\mathrm{H}}\mathbf{J}^{(s+1)}=\mathbf{I}$ in the first
equality. Consequently, $\mathcal{P}_{4}$ is equivalent to

$\displaystyle\mathcal{P}_{5}:$
$\displaystyle\quad\underset{\mathbf{J}^{(s+1)}\in\boldsymbol{\Omega}}{\textrm{minimize}}\quad\mathrm{Tr}\left(\mathbf{J}^{(s+1)}\mathbf{C}^{(s)\mathrm{H}}\right).$
(32)

Note that the above problem is in fact a linear assignment problem with cost
matrix $\mathbf{C}^{(s)\mathrm{H}}$ that can be solved efficiently using the
Hungarian algorithm also known as Munkres assignment algorithm, with
computational complexity of $\mathcal{O}(L^{2})$ [14]. Our final proposed
algorithm for transmitter scheduling in CAVs based on power method-like
iterations is presented in Algorithm 1. As shown in [20], the objective
$f(\mathbf{J})=\mathrm{vec}\left(\mathbf{J}\right)^{\mathrm{H}}\bar{\mathbf{S}}\;\mathrm{vec}\left(\mathbf{J}\right)$
is increasing through the power method-like iterations and convergent in the
sense of the objective value. Consequently, we set the stopping criteria
$\bigm{|}\bigm{[}f(\mathbf{J}^{(s+1)})-f(\mathbf{J}^{(s)})\bigm{]}/f(\mathbf{J}^{(s)})\bigm{|}<\epsilon$
for the algorithm.

Algorithm 1 Power method-like iterations for transmitter scheduling in CAVs.

1:Input The overall steering vector of the CAV $\bar{\mathbf{s}}$

2:Initialization $\mathbf{J}^{(0)}\in\boldsymbol{\Omega}$, $s=0$

3:$\bar{\mathbf{S}}=\lambda_{m}\mathbf{I}-\mathbf{S}$

4:While
$\bigm{|}\bigm{[}f(\mathbf{J}^{(s+1)})-f(\mathbf{J}^{(s)})\bigm{]}/f(\mathbf{J}^{(s)})\bigm{|}\geq\epsilon$
do

5:
$\mathbf{C}^{(s)}=-\mathrm{vec}_{{}_{L,L}}^{-1}\left(\bar{\mathbf{S}}\;\mathrm{vec}\left(\mathbf{J}^{(s)}\right)\right)$

6: $\mathbf{J}^{(s+1)}\leftarrow\text{Hungarian}(\mathbf{C}^{(s)\mathrm{H}})$

7: $s\leftarrow s+1$

8:$\mathbf{J}_{{}_{\text{opt}}}\leftarrow\mathbf{J}^{(s)}$

9:Output $\mathbf{J}_{{}_{\text{opt}}}$

##  5 Numerical investigation

We carried out numerical experimentation to evaluate the performance of the
proposed algorithm for TDM in CAVs. We considered a platoon consisting of
$K=3$ vehicles, each equipped with $M=N=8$ TX and Rx antennas arranged as ULA.
All vehicles are assumed to have FMCW radar systems operating at carrier
frequency $f_{c}=77$ GHz, bandwidth $B=150$ MHz and chirp time
$T_{c}=8\;\mu$s. The vehicles are moving with 2D velocities
$\mathbf{v}_{1}=[20,20]$ m/s, $\mathbf{v}_{2}=[-10,-20]$ m/s and
$\mathbf{v}_{3}=[30,15]$ m/s. Algorithm 1 is performed to design the TDM for a
CPI of length $L=24$ pulses. The receiver operating characteristics (RoC)
associated with the described CAV when one, two or three of the vehicles are
cooperating and transmitting with optimized TDM is illustrated in Fig. 2. The
results demonstrate significant improvement in comparison with sequential
transmission where the transmitters on each vehicle are activated with in
order of their indexes $\\{1,2,\ldots,N\\}$ i.e. the TDM matrix is identity.

Fig. 2: RoC of detection for a CAV of FMCW radars. The optimized TDM is
compared with uniform transmission where the antennas are activated uniformly
in a sequence.

##  6 summary

Vehicle-to-vehicle communication is a fundamental capability that enables CAVs
to perform distributed STAP. A cooperative STAP scheme for the vehicles in a
CAV platoon was within the purview of this paper. Furthermore, we introduced a
TDM scheme for orthogonal transmission of the antennas in the CAV platoon. The
transmitter scheduling was formulated as a quadratic assignment problem. By
means of the well-known power method-like iterations we showed that the TDM
design problem can be reduced to a linear assignment problem in each iteration
and consequently the efficient Hungarian algorithm can effectively address it.
Through numerical simulations, we confirmed the efficacy of the suggested TDM
approach in enhancing target detection performance.

## References

  * [1] A. Sarker, H. Shen, M. Rahman, M. Chowdhury, K. Dey, F. Li, Y. Wang, and H. S. Narman, “A review of sensing and communication, human factors, and controller aspects for information-aware connected and automated vehicles,” _IEEE Transactions on Intelligent Transportation Systems_ , vol. 21, no. 1, pp. 7–29, 2020.
  * [2] Y. Wang, R. Zhang, N. Masoud, and H. X. Liu, “Anomaly detection and string stability analysis in connected automated vehicular platoons,” _Transportation research part C: emerging technologies_ , vol. 151, p. 104114, 2023.
  * [3] H. Griffiths, “Multistatic, MIMO and networked radar: The future of radar sensors?” pp. 81–84, 2010.
  * [4] H. Zhang, W. Liu, J. Shi, T. Fei, and B. Zong, “Joint detection threshold optimization and illumination time allocation strategy for cognitive tracking in a networked radar system,” _IEEE Transactions on Signal Processing_ , vol. 70, pp. 5833–5847, 2022.
  * [5] N. A. Goodman and D. Bruyere, “Optimum and decentralized detection for multistatic airborne radar,” _IEEE Transactions on Aerospace and Electronic Systems_ , vol. 43, no. 2, pp. 806–813, 2007.
  * [6] M. Alaee-Kerahroodi, M. Soltanalian, P. Babu, and M. R. B. Shankar, _Signal Design for Modern Radar Systems_. Artech House, 2022.
  * [7] G. Wang, K. V. Mishra _et al._ , “STAP in automotive MIMO radar with transmitter scheduling,” in _IEEE Radar Conference_ , 2020, pp. 1–6.
  * [8] A. Bose, B. Tang, M. Soltanalian, and J. Li, “Mutual interference mitigation for multiple connected automotive radar systems,” _IEEE Transactions on Vehicular Technology_ , vol. 70, no. 10, pp. 11 062–11 066, 2021.
  * [9] B. Tang and J. Tang, “Joint design of transmit waveforms and receive filters for MIMO radar space-time adaptive processing,” _IEEE Transactions on Signal Processing_ , vol. 64, no. 18, pp. 4707–4722, 2016.
  * [10] B. Tang, J. Tuck, and P. Stoica, “Polyphase waveform design for MIMO radar space time adaptive processing,” _IEEE Transactions on Signal Processing_ , vol. 68, pp. 2143–2154, 2020.
  * [11] A. Bose, B. Tang, W. Huang, M. Soltanalian, and J. Li, “Waveform design for mutual interference mitigation in automotive radar,” _arXiv preprint arXiv:2208.04398_ , 2022.
  * [12] Z. Esmaeilbeig, A. Bose, and M. Soltanalian, “Mutual interference mitigation in PMCW automotive radar,” 2023.
  * [13] L. Xu, S. Sun, K. V. Mishra, and Y. D. Zhang, “Automotive FMCW radar with difference co-chirps,” _IEEE Transactions on Aerospace and Electronic Systems_ , pp. 1–19, 2023.
  * [14] H. W. Kuhn, “The hungarian method for the assignment problem,” _Naval research logistics quarterly_ , vol. 2, no. 1-2, pp. 83–97, 1955.
  * [15] A. Bose and M. Soltanalian, “Non-convex shredded signal reconstruction via sparsity enhancement,” in _IEEE International Conference on Acoustics, Speech and Signal Processing_ , 2017, pp. 4691–4695.
  * [16] D. P. Bruyere and N. A. Goodman, “Adaptive detection and diversity order in multistatic radar,” _IEEE Transactions on Aerospace and Electronic Systems_ , vol. 44, no. 4, pp. 1615–1623, 2008.
  * [17] Z. Esmaeilbeig, A. Eamaz, K. V. Mishra, and M. Soltanalian, “Moving target detection via multi-IRS-aided OFDM radar,” in _IEEE Radar Conference_. IEEE, 2023, pp. 1–6.
  * [18] T. Rouffet, P. Vallet, E. Grivel, C. Enderli, B. Joseph, and S. Kemkemiant, “Analysis of a GLRT for the detection of an extended target,” in _IEEE Radar Conference_. IEEE, 2016, pp. 1–5.
  * [19] R. E. Burkard, E. Cela, P. M. Pardalos, and L. S. Pitsoulis, _The quadratic assignment problem_. Springer, 1998.
  * [20] M. Soltanalian and P. Stoica, “Designing unimodular codes via quadratic optimization,” _IEEE Transactions on Signal Processing_ , vol. 62, no. 5, pp. 1221–1234, 2014.
  * [21] M. Soltanalian, B. Tang, J. Li, and P. Stoica, “Joint design of the receive filter and transmit sequence for active sensing,” _IEEE Signal Processing Letters_ , vol. 20, no. 5, pp. 423–426, 2013.