11institutetext:  Northeastern University, Boston MA 02115, USA
22institutetext: Carnegie Mellon University, Pittsburgh PA 15213, USA
33institutetext: University of Georgia Chapel, Herty Dr, Athens, GA 30602
Corresponding Authors

# Efficient Pruning of Large Language Model with Adaptive Estimation Fusion

Jun Liu 1122 Chao Wu 11 Changdi Yang 11 Hao Tang 2*2* Haoye Dong 22 Zhenglun
Kong 11 Geng Yuan 33 Wei Niu 33 Dong Huang 2*2* Yanzhi Wang 1*1*

###### Abstract

Large language models (LLMs) have become crucial for many generative
downstream tasks, leading to an inevitable trend and significant challenge to
deploy them efficiently on resource-constrained devices. Structured pruning is
a widely used method to address this challenge. However, when dealing with the
complex structure of the multiple decoder layers, general methods often employ
common estimation approaches for pruning. These approaches lead to a decline
in accuracy for specific downstream tasks. In this paper, we introduce a
simple yet efficient method that adaptively models the importance of each sub-
structure. Meanwhile, it can adaptively fuse coarse-grained and fine-grained
estimations based on the results from complex and multilayer structures. All
aspects of our design seamlessly integrate into the end-to-end pruning
framework. Our experimental results, compared with state-of-the-art methods on
mainstream datasets, demonstrate average accuracy improvements of 1.1%, 1.02%,
2.0%, and 1.2% for LLaMa-7B, Vicuna-7B, Baichuan-7B, and Bloom-7b1,
respectively.

###### Keywords:

Adaptive Fusion Efficient LLM Structured Pruning

##  1 Introduction

Large Language Models (LLMs) have gained significant popularity, but their
scale and complexity have reached unprecedented levels. The parameter sizes re
all above 1 billion, such as Phi [18], Llama [29], Bloom [32], PaLM [5], etc.,
This has posed unforeseen challenges in deploying them.

Figure 1:  A task-agnostic LLM pruning framework. Our framework divides the
pruning task into three stages: discovery, importance assessment, and
recovery. The framework supports various large language models. In the
discovery stage, it explores dependencies among different structures of large
models. Subsequently, using Taylor coarse-grained and Taylor fine-grained
estimation methods with input tokens, it obtains the gradient changes for
various structures in large models. The framework then adaptively adapts
pruning by fusing the gradient changes from both methods. In the recovery
stage, after fine-tuning, the model can be deployed on different hardware and
can perform various tasks such as translation, Q&A, text generation, and chat.

A promising solution to address these challenges is to leverage model
compression techniques, popular techniques for compressing large models
include pruning, quantization, low-rank decomposition, etc. These methods have
been demonstrated to effectively reduce computational burden without
significantly impacting performance. This paper primarily focuses on the
introduction of large model pruning techniques. LLM-Pruner [22] follows a
method of importance estimation, pruning, fine-tuning, allowing compression of
large language models under limited resources. It employs a structured pruning
method to group the sub-structures of LLaMA. This method makes use of gradient
information and a limited quantity of data to estimate the importance of
weights, parameters, and groups. While LLM-Pruner adopts a generic estimation
method to assess each decoder layer. This could be problematic because each
decoder layer in LLM contains different types of operators, such as Multilayer
Perceptron (MLP) and Attention. meanwhile, the chosen number of samples by
LLM-Pruner is fixed, but the quantity of samples has a certain impact on the
final gradients, thereby influencing the structure and performance of the
pruned model. SparseGPT [10] works by simplifying the pruning problem into a
large-scale sparse regression instance. Wanda [28] consists of two simple yet
essential components - the pruning metric and the pruning granularity. The
pruning metric is used to assess the importance of weights, followed by
pruning based on the chosen granularity. This approach employs a
straightforward magnitude pruning technique without conducting external data-
driven backpropagation computations. It cannot dynamically assess the
importance of substructures in LLMs based on gradient variations.

Our work introduces a simple yet efficient method. The overview of our
framework can be divided into three stages in Table 1: discovery, importance
estimation, and recovery. In the discovery stage, dependencies among various
structures within the LLM are explored. During the importance estimation
stage, each sub-structure in the LLM undergoes importance assessment, followed
by pruning. In the recovery stage, fine-tuning with Low-Rank Adaptation(LoRA)
is applied to restore the accuracy of the model. Subsequently, it is deployed
on hardware, enabling it to downstream tasks. Especially in the estimation
stage, we adaptively fuse coarse-grained and fine-grained estimation to
adaptively predict the effects of various samples on sub-structures.
Throughout the end-to-end pruning, the estimation stage enhances the
algorithm’s robust estimation of sub-structures. The main of our work can be
summarized as follows:

  * •

Firstly, We observe that the drawback of current structural pruning methods
for LLMs is rooted in the complex multi-layer decoder structure, comprising
different types of operators (such as MLP and Attention) in each layer. In
such cases, employing a generic estimation method for each decoder layer can
be problematic.

  * •

Secondly, we propose a data-driven fusion approach designed to enhance the
accuracy and importance estimation of internal sub-structures based on
external data.

  * •

Lastly, after validation on multiple mainstream datasets, our approach has
achieved state-of-the-art performance. The average accuracy improvements for
LLaMa-7B, Vicuna-7B, Baichuan-7B, and Bloom-7b1, compared to the current
state-of-the-art methods, are 1.1%, 1.02%, 2.0%, and 1.2%, respectively.

##  2 Related Work

###  2.1 Structured Pruning

SPLLMs [30] utilizing the low-rank factorization of each weight matrix to
parameterize it, and during training, adaptively eliminating rank-1
components. NASH [15] provide the straightforward and efficient NASH
framework, which shortens and narrows the encoder networks of encoder-decoder
models. X-Pruner [34] considers the explainability of the pruning criterion,
measures each prunable unit’s contribution to predicting each target class,
preserves the most informative units and learn the layer-wise pruning rate,
search the layer-wise threshold that differentiates between unpruned and
pruned units based on their explainability aware mask values. SPNSH [14] offer
a thorough examination of current Neural architecture search tactics and
contrasts various search space definitions.

###  2.2 Unstructured Pruning

CAP [12] is an unstructured pruning framework that introduces a theoretically
justified pruner, enabling efficient and accurate handling of complex weight
correlations during the pruning process. Combined with an efficient fine-
tuning procedure for post-compression recovery. SparseGPT [10] is the
inaugural one-shot pruning method that operates efficiently on a model scale
with parameters ranging from 10 to over 100 billion. Wanda [28] used the
assess the importance of weights, followed by pruning based on the chosen
granularity. The method achieves pruning on large in just a few seconds while
achieving results comparable to SparseGPT. The unstructured pruning method and
the pruned weights are not evenly distributed, making the generated model
unsuitable for hardware acceleration.

###  2.3 Efficient Large Model

SpQR [9] employed a method involving the identification and isolation of
outlier weights. These weights, when compressed, could result in significant
errors and diminish the model’s accuracy. LLM-FP4 [20] suggests FP4 as a post-
training method for quantizing weights and activations in large language
models (LLMs) down to 4-bit floating-point values. However, it requires
specific hardware support for low-bit quantization. PuMer [3] addresses the
computational and memory-intensive nature of Large-scale vision language
models by employing text-informed pruning and modality-aware merging
strategies. OPTQ [11], a one-shot weight quantization method based on
approximate second-order information, effectively reducing the bitwidth of GPT
model parameters to 3 or 4 bits per weight. EES [13] introduces the concept of
”essential sparsity,” characterized by a distinct tipping point beyond which
the performance degrades rapidly. discover that self-supervised learning (SSL)
objectives trigger stronger emergent sparsification properties than supervised
learning (SL). LLM in a flash [1] proposing a method that stores model
parameters in flash memory and brings them to DRAM on demand. QLORA [8]
introduces methods to save memory without sacrificing performance 4-bit
NormalFloat (NF4) is information-theoretically optimal for normally
distributed weights. Less [19] proposes Task-aware layEr-wise Distillation
(TED) as a solution to the challenges of layer-wise distillation effectively
reducing the knowledge gap between teacher and student models. Localized
Symbolic Knowledge Distillation for Visual Commonsense [25] Models address the
limitation of existing instruction following vision-language models by
introducing a Localized Visual Commonsense model.

##  3 Method

This part comprises three sections, Table 3.1 introduces the method employed
for structural pruning of large model architectures, Table 3.2 presents the
motivation and algorithm for sub-adaptive fusion, and Table 3.3 details the
utilization of weight fine-tuning for post-training the pruned model.

###  3.1 Structured Pruning of Large Model

Struct Pruning of a Large Model is comprised of three steps: Discovery Stage,
where the objective is to identify interdependent structure groups in LLMs.
Estimation Stage involves identifying which group should be pruned. Recover
Stage entails quick post-training to mitigate any potential performance loss
brought on by the removal of structures.

Discovery stage

Figure 2:  Coupled structure diagram in LLaMA. Neurons in each layer are
simplified to expose the dependent group. The trigger neuron, depicted as a
red star, initiates weight pruning in dependent neurons represented by dashed
lines. This pruning can extend to connected neurons, indicated by dashed
circles and emphasized with red dashed lines.

To prune, the first step involves building dependencies for LLMs. Assuming
$N_{i}$ and $N_{j}$ are two neurons in the model, where In($N_{i}$) represents
all the neurons pointing towards $N_{i}$, and Out($N_{i}$) represents all the
neurons pointing from $N_{i}$. The dependency between structures can be
defined as:

$\texttt{Dependency}(N_{i},N_{j})=\frac{|\texttt{In}(N_{i})\cap\texttt{Out}(N_{j})|}{|\texttt{In}(N_{i})|+|\texttt{Out}(N_{j})|}.$
(1)

This formula calculates the dependency between neurons $N_{i}$ and $N_{j}$
within the subgraph, which can be obtained and located through the defined
dependency relationships. As shown in Table 2, the neurons in each layer are
simplified to clarify the dependent groups. Weights with pruned dependencies,
which are represented by dashed lines, indicate that these weights are
generated by the trigger neuron, depicted by a circle with a red star. It has
the potential to propagate to connected neurons, as depicted by dashed circles
along with red dashed lines. The neurons connected by dashed lines are
referred to as coupled structures.

Estimation stage After grouping all subgraphs in the model, prune weights
within the same group that minimally impact predictions. However, this
introduces challenges, including increased parameter size and misaligned
intermediate representations. Hence, it is crucial to evaluate the overall
importance of the group. The significance of individual weights within a sub-
struct can be assessed by expressing the change in loss based on the Taylor
series approximation of the loss deviation function [17] E in a dataset D:

$\Delta{E}(\mathcal{D})=\sum_{i=1}^{N}\frac{\partial{E}(\mathcal{D})}{\partial
w_{i}}\ w_{i}+\frac{1}{2}\sum_{{i,j=1}\atop{i\neq
j}}^{N}\frac{\partial^{2}{E}(\mathcal{D})}{\partial w_{i}\partial w_{j}}\
w_{i}^{2}+O(||\bar{\ell}^{*}||^{3}){,}$ (2)

where N is the number of samples, $\ w_{i}$ is the weight of each structure,
and $\frac{\partial{E}(\mathcal{D})}{\partial w_{i}}$ represent the components
of the gradient of E concerning each structure’s weights,
$\frac{\partial^{2}{E}(\mathcal{D})}{\partial w_{i}\partial w_{j}}$ are the
elements of the Hessian matrix [17] H. The term
$\mathcal{O}(||\bar{\ell}^{*}||^{3})$ represents the remainder term that can
be ignored in the calculation. The $\Delta{E}(\mathcal{D})$ can be expressed
as:

$\Delta{E}(\mathcal{D})=\sum_{i=1}^{N}\frac{\partial{E}(\mathcal{D})}{\partial
w_{i}}\ w_{i}+\frac{1}{2}\sum_{{i,j=1}\atop{i\neq j}}^{N}\
{w_{i}}^{\top}Hw_{i}+O(||\bar{\ell}^{*}||^{3}){.}$ (3)

Recover stage To accelerate the model recovery process and enhance efficiency
with constrained data, low-rank approximation is used to post-train the pruned
model.The update of $m_{0}$ is constrained by expressing it through a low-rank
decomposition $m_{0}+\Delta m=m_{0}+\Gamma\beta$, where
$\Gamma\in\mathbb{R}^{d\times r}$, $\beta\in\mathbb{R}^{r\times k}$.
Throughout the training, $m_{0}$ remains fixed and does not receive gradient
updates, while $\Gamma$ and $\beta$ contain trainable parameters. the forward
pass is given by:

$\mathcal{R}(x)=m_{0}x+\Delta mx=(m_{0}+\Gamma\beta)x.$ (4)

###  3.2 Adaptive Fusion

Structured pruning is employed in the model architecture. Given that the LLM
model comprises sophisticated multi-layer decoders, with diverse operators
like MLP and Attention in each layer, using a generic estimation approach for
all decoder layers becomes challenging.
Motivation.

In LLMs, front-layer decoders, specific parameters play a crucial role in
capturing detailed features of the input token, making fine-grained estimation
suitable. last layer decoders focus more on understanding semantics and
context, where a set of parameters becomes pivotal for understanding abstract
semantics and long-distance dependency relationships. for grasping abstract
semantics and long-range dependencies, making coarse-grained estimation more
appropriate. LLM-Pruner assesses the coupled structure in LLM using general
methods but does not consider the relationship between assessment criteria and
the structures. This leads to challenges in assessing the relationships
between different layers of the structure. Meanwhile, it fixes the number of
samples at 10, but the selected number of samples also has an impact on the
final gradients, thereby influencing the structure and performance of the
pruned model.
Problem Definition.

Coarse-grained refers to a set of coupled structures that can be defined as
$G$ is defined as $\\{W_{i}\\}_{i=1}^{T}$, where T represents the total number
of coupled structures in a group, and $W_{i}$ denotes the weight assigned to
each coupled structure.

Fine-grained refers to the $k$-th parameter $W_{i}^{k}$ in the $i$-th coupled
structure $W_{i}$ ($k=1$ to $N$, where $N$ is the total number of parameters
in $W_{i}$).

Algorithm. The pruning process aims to eliminate the group with the minimal
impact on the model’s predictions. Based on the previous definition in Section
3.1, the variation in loss can be expressed as:

$L_{W_{i}}=\Delta{E}(\mathcal{D})=\
{\frac{\partial{E}^{\top}(\mathcal{D})}{\partial
W_{i}}W_{i}}+\frac{1}{2}{W_{i}}^{\top}HW_{i}+\mathcal{O}\left(\|W_{i}\|^{3}\right){.}$
(5)

In this way, we obtain the assessment criterion for vector-wise importance in
our framework.

During the pruning process, the goal is to remove the parameters in $W_{i}$
that have the least impact on the model’s predictions. Similarly, the
variation in loss can be expressed as:

$L_{W_{i}^{k}}=\Delta{E}(\mathcal{D})\approx\frac{\partial{E}(\mathcal{D})}{\partial
W_{i}^{k}}W_{i}^{k}+\frac{1}{2}\sum_{i=1}^{N}\left(\frac{\partial{E}(\mathcal{D}_{i})}{\partial
W_{i}^{k}}W_{i}^{k}\right)^{2}+\mathcal{O}\left(\|W_{i}^{k}\|^{3}\right){.}$
(6)

By doing so, we obtain the assessment criterion for element-wise importance in
our framework. Throughout the pruning process, the criteria used to evaluate
the importance of coupling structures have a strong impact on the output of
the discovery stage in Section 3.1.

Our work proposes a simple yet efficient method, which adaptively fuses two
assessment criteria. The vector-wise estimation criterion serves as a coarse-
grained metric, while the element-wise estimation criterion is employed as a
fine-grained metric. Simultaneously, the impact of different samples on the
ultimate performance is determined through the loss calculation during the
discovery stage. The above description represents an adaptive process where we
employ an adaptive modeling approach to adaptive fuse coarse-grained and fine-
grained estimation criteria, learning the impact of different samples on
coupled structures. We add Coarse-graining criteria and fine-grained criteria
to obtain the following:

$L_{W_{i},W_{i}^{k}}=L_{W_{i}}+L_{W_{i}^{k}}{,}$ (7)

in this process, coarse-grained $L_{W_{i}}$ and fine-grained criteria
$L_{W_{i}^{k}}$ are introduced respectively, and by fusing these two,
$L_{W_{i},W_{i}^{k}}$ is derived.
The next step involves calculating the first-order of the Taylor series T,
where the first-order derivatives with respect to $L_{W_{i}},L_{W_{i}^{k}}$.
This computation is grounded in the first-order derivatives of the joint loss
E concerning the dataset $D$.

$T=({\frac{\partial{E}^{\top}(\mathcal{D})}{\partial
W_{i}}W_{i}}+\frac{\partial{E}(\mathcal{D})}{\partial W_{i}^{k}}W_{i}^{k}){,}$
(8)

where the variable $T$ in the formula represents the first term of the Taylor
series, utilizing the first-order derivatives of the loss function with
respect to parameters $L_{W_{i},W_{i}^{k}}$. This can be regarded as a linear
approximation, taking into account the first-order information in the vicinity
of the current point.
The Hessian matrix ($H$) can be approximated using the Fisher information
matrix ($F$) [16], as it is not possible to calculate the precise Hessian
matrix explicitly.

$\ {H}\approx\
{F}=\frac{1}{|\mathcal{D}|}\sum_{{j=1}\atop{(w_{i})\in\mathcal{D}}}^{N}\left(\frac{\partial{E}(\mathcal{D})}{\partial
w_{i}}\right)\left(\frac{\partial{E}(\mathcal{D})}{\partial
w_{i}}\right)^{\mathsf{T}}{,}$ (9)

here, H is approximated by F, and F is computed as the average outer product
of the gradients with respect to the parameters ${W_{i}}$ over the dataset D.
Subsequently, an approximation for Second-order S is obtained through the
expansion of the Taylor series.

$S=\frac{1}{2}{W_{i}}^{\top}FW_{i}+\frac{1}{2}\sum_{i=1}^{N}\left(\frac{\partial{E}(\mathcal{D}_{j})}{\partial
W_{i}^{k}}W_{i}^{k}\right)^{2}=\frac{1}{2}F({W_{i}}^{\top}W_{i}+{\ W_{i}^{k}\
W_{i}^{k}}){,}$ (10)

this formula represents the second-order term, considering the Fisher
information matrix F and the partial derivatives of the loss function with
respect to coarse-grained importance $L_{W_{i}}$ and fine-grained importance
$L_{W_{i}^{k}}$.
Additionally, there is a remainder term P expressed as:

$P=\mathcal{O}\left(\|W_{i}\|^{3}\right)+\mathcal{O}\left(\|W_{i}^{k}\|^{3}\right){,}$
(11)

This term accounts for the higher-order terms in the Taylor series expansion,
involving cubic terms of both $L_{W_{i}}$ and $L_{W_{i}^{k}}$.
Finally, the combined expression for $L_{W_{i}},L_{W_{i}^{k}}$ is given by:

$\begin{split}L_{W_{i},W_{i}^{k}}=({\frac{\partial{E}(\mathcal{D})^{\top}}{\partial
W_{i}}W_{i}}+\frac{\partial{E}(\mathcal{D})}{\partial
W_{i}^{k}}W_{i}^{k})+\frac{1}{2}H({W_{i}}^{\top}W_{i}+{\ W_{i}^{k}\
W_{i}^{k}})\\\
+\mathcal{O}\left(\|W_{i}\|^{3}\right)+\mathcal{O}\left(\|W_{i}^{k}\|^{3}\right){.}\end{split}$
(12)

This equation represents the final expression for $L_{W_{i},W_{i}^{k}}$, which
includes the first-order term, the second-order term approximated by H using
F, and the remainder terms accounting for higher-order contributions.

Input: Fusion (model, samples)

Output: importance

model.zero_grad() // set model gradient to 0

coarse_importance $\leftarrow$ estimation_module(samples)

for  _$i=0$ to number of samples_ do

// Pick the next sample

input $\leftarrow$ samples[i] fine_importance $\leftarrow$
estimation_module(input)

loss $\leftarrow$ model.forward(input) // calculate loss

loss.backward() // model loss backward

for  _parameters in model.parameters()_ do

gradient $\leftarrow$ taylor_gradient gradient_accumulate $\leftarrow$
gradient + fine_importance + coarse_importance

model.zero_grad() // set model gradient to 0

del loss.grad // delete loss gradient

loss $\leftarrow$ model.forward(samples) // calculate loss

loss.backward() // model loss backward

importance $\leftarrow$ gradient_accumulate

return importance

Algorithm 1 The proposed Adaptive Fusion.

Table 1 has implemented $L_{W_{i},W_{i}^{k}}$, where fine importance is
denoted as $L_{W_{i}}$, and coarse importance is represented as
$L_{W_{i}^{k}}$. These two metrics respectively signify coarse-grained and
fine-grained estimation rules, showcasing gradient changes in the form of
Taylor series after input samples. This algorithm achieves an adaptive fusion
of coarse-grained and fine-grained importance assessment of coupled structures
by optimizing the loss of a large model.

###  3.3 Weight-Efficient Fine-Tuning

To accelerate the model recovery process and enhance efficiency with limited
data, it is essential to minimize the number of parameters requiring
optimization during the recovery stage. To facilitate this, we utilize weight
fine-tuning for post-training the pruned model. In this stage, we loosen this
constraint. Layer-wise Reconstruction through Linear Least Squares involves
adjusting the weight variables to minimize the reconstruction error for each
layer. From the initial to the final layer, we reconstruct the output
activation of the original model using the remaining heads/filters in the
pruned model. The formula is as follows::

$\underset{\mathbf{m}_{l}}{\mathrm{argmin}}\left\|\mathbf{x}+\text{layer}(\mathbf{x};\mathbf{m}_{l})-\left(\mathbf{x}^{\prime}+\text{layer}(\mathbf{x}^{\prime};{\mathbf{m}_{0}})\right)\right\|_{2}^{2}.$
(13)

The layer mentioned can be either the Multi-Head Attention or Feedforward
Neural Network. The variables x and $\text{x}^{\prime}$ represent the inputs
to the layer, corresponding to the pruned model and the original model,
respectively. In this comparison, we analyze the activations following the
residual connection. Table 13 can be simplified into a linear least squares
problem of
${\mathrm{argmin}}_{\text{m}_{l}}||\text{A}\text{m}_{l}-\text{b}||_{2}^{2}$,
seeking to minimize the norm of the difference between the matrix
${\text{A}\text{w}_{l}}$, which representing head/filter-wise output
activations of the model pruned by the weight and the vector b, which
indicates the disparity in output activations between the two models. Taking
into account the extensive dimensions of matrix A, directly solving the least-
squares problem may result in numerically unstable outcomes. In response to
this challenge, we incorporate a regularization hyperparameter for enhanced
stability. In practical terms, we re-parameterize the least squares problem as
${\mathrm{argmin}_{r_{l}}}\left\|Ar_{l}+A\cdot{m}_{0}-b\right\|_{2}^{2}$ where
$m_{l}={m}_{0}+r_{l}$. If the solver identifies a layer weight beyond this
specified range, we discard the weight for that layer and terminate the weight
tuning process.

##  4 Experiments

###  4.1 Experimental Setup

Our experiment is implemented in PyTorch 2.1.2 [26], CUDA 11.6 and the
HuggingFace 4.29.1 [31], LLaMA-7B [29], Vicuna-7B [4], BLOOM-7b1 [32],
Baichuan-7B [33] model. All pruning experiments are performed on a single
NVIDIA A6000 GPU with 48GB of memory.

###  4.2 Benchmark & Metric

The model was evaluated on the following datasets which cover a range of
challenges in natural language understanding and reasoning, including common
sense reasoning, physical interaction understanding, and coreference
resolution.: BoolQ [6] assesses the model’s accuracy in providing correct
answers to questions. PIQA [2] evaluates the model’s performance using
accuracy related to question answering. HellaSwag [35] assess the model’s
ability to correctly predict endings. WinoGrande [27] assesses the model’s
understanding of gender-related information, potentially using accuracy and
other indicators. Arc Easy [7] and Arc Challenge [7] evaluate the model’s
performance in answering common-sense reasoning questions. WikiText2 [24]
focuses on predicting the next word in a sequence, PTB [23] is on syntactic
parsing and understanding grammatical relationships within sentences.
Perplexity (PPL) is used to measure the predictive capability of a language
model on a given text sequence.Parameters(Params), MACs, Memory, and Latency
are common metrics in the estimation of deep learning models, and they are all
related to the model’s size, computational requirements, and memory needs.

###  4.3 Main Results

Baseline and configurations. The baseline provides unpruned test results,
demonstrating the performance metrics of LLaMA-7B across various estimation
methods with 50 samples, pruning from the 4th layer to the 29th layer.
Specific performance metrics include estimation scores for BoolQ, PIQA,
HellaSwag, WinoGrande, ARC-e, ARC-c, OBQA tasks, as well as the average
accuracy, and Perplexity (PPL) for WikiText2 and PTB. Perplexity is used to
measure the predictive capability of a language model on a given text
sequence. The baseline also presents statistical data for the LLaMA-7B model
used in the experiment, including parameter count, MACs, memory requirements,
and latency.

Table 1: Zero-shot Performance of the compressed LLaMA-7B with a pruning
rate=20%, , number of samples=50. The Accuracy Average is computed from BoolQ,
PIQA, HellaSwag, WinoGrande, ARC-e, ARC-c, OBQA datasets. The vector [22]
method is the coarse-grained method, $\text{Element}^{1}$ [22] and
$\text{Element}^{2}$ [22] are fine-grained methods. PPL metrics for WikiText2
and PTB, lower PPL is considered better.

Method | WikiText2$\color[rgb]{0,.5,.5}\downarrow$ | PTB$\color[rgb]{0,.5,.5}\downarrow$ | BoolQ | PIQA | HellaSwag | WinoGrande | ARC-e | ARC-c | OBQA | Accuracy Average | #Params | #MACs | Memory | Latency
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
LLaMA-7B[29] | 12.62 | 22.15 | 73.21 | 77.48 | 72.99 | 67.09 | 67.38 | 41.38 | 42.40 | 63.12 | 6.74B | 425.12G | 12892.65MiB | 77.16s
Vector | 18.84 | 33.05 | 65.75 | 74.70 | 64.52 | 59.35 | 60.65 | 36.26 | 39.40 | 57.25 | 5.71B | 358.82G | 10926.80MiB | 47.56s
$\text{Element}^{1}$ | 16.78 | 29.68 | 65.57 | 77.37 | 70.24 | 63.14 | 53.24 | 38.74 | 41.8 | 58.59 | 5.71B | 358.82G | 10926.80MiB | 43.62s
$\text{Element}^{2}$ | 16.81 | 30.45 | 67.40 | 76.77 | 70.61 | 65.19 | 53.54 | 39.08 | 41.20 | 59.11 | 5.71B | 358.82G | 10926.80MiB | 43.23s
Ours | 16.68 | 30.17 | 70.13 | 77.42 | 70.56 | 65.22 | 55.04 | 41.20 | 41.80 | 60.20 | 5.71B | 358.82G | 10926.80MiB | 42.41s

Accuracy. Our code is implemented based on the LLM-Pruner framework [21]. We
selected LLaMa-7B as a representative case for analysis. In the scenario with
a pruning rate of 20%, Table. 1 presents the comparison results between our
method and the coarse-grained estimation method, as well as the fine-grained
estimation methods. In terms of Accuracy Average for BoolQ, PIQA, HellaSwag,
WinoGrande, ARC-e, ARC-C, and OBQA, Our results stand out as the highest among
all methods. We outperform the Vector method by 3.04%, the
$\text{Element}^{1}$ method by nearly 1.61%, and the $\text{Element}^{2}$
method by 1.1%. For OBQA, our accuracy is 1.4% higher than the Vector method,
and 0.6% higher than the $\text{Element}^{2}$ method. In terms of PIQA
accuracy, we surpass the Vector method by 2.83%, the $\text{Element}^{1}$
method by 0.16%, and the $\text{Element}^{2}$ method by 0.76%. Our PPL metric
for WikiText2 is the lowest among all methods, surpassing the Vector method by
2.16%, the $\text{Element}^{1}$ method by 0.10%, and the $\text{Element}^{2}$
method by 0.13%. For PTB, our PPL is 1.88% lower than the Vector. A lower PPL
is considered better.
Efficiency.

Figure 3:  The pruned network structure diagram of LLaMa-7B. The left side of
the figure displays the model structure and parameters before pruning, while
the right side showcases the model structure and parameters after pruning with
our method. The values for the pruned decoder layers, parameter count, and
MACs are highlighted in red.

In terms of performance, as shown in Table 3, at each layer from 4 to 29, the
self-attention block experienced a reduction in parameters by 18.75%,
resulting in a corresponding decrease in MACs (Multiply-Accumulates) of
18.74%. Similarly, the parameters in the MLP block were reduced by 20%,
leading to a concurrent reduction in MACs by 20%. Table 1 presents the results
of our Latency is approximately 1.82 times faster than the baseline, 1.12
times faster than the Vector method, 1.04 times faster than the
$\text{Element}^{2}$ method, and 1.06 times faster than the
$\text{Element}^{1}$ method. Overall, our method achieves the best
performance.

We employ the same methodology to evaluate Vicuna-7B with a pruning rate of
20%, Baichuan-7B, and Bloom-7b1 with a pruning rate of 25%. As shown in Tables
2, 3, and 4, the metrics consistently demonstrate the best performance.

Table 2: Zero-shot Performance of the compressed Vicuna-7B with a pruning
rate=20%, number of samples=50.

Method | WikiText2$\color[rgb]{0,.5,.5}\downarrow$ | PTB$\color[rgb]{0,.5,.5}\downarrow$ | BoolQ | PIQA | HellaSwag | WinoGrande | ARC-e | ARC-c | OBQA | Accuracy Average | #Params | #MACs | Memory | Latency
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
Vicuna-7B [4] | 16.23 | 58.19 | 75.66 | 77.80 | 71.05 | 67.64 | 65.02 | 39.93 | 42.20 | 62.76 | 6.73B | 425.12G | 12924.65MiB | 72.54s
Vector | 21.85 | 75.80 | 72.08 | 76.12 | 68.96 | 65.27 | 54.42 | 38.48 | 30.60 | 57.99 | 5.71B | 358.82G | 10958.80MiB | 44.68s
$\text{Element}^{1}$ | 21.85 | 75.80 | 71.83 | 75.95 | 69.02 | 65.67 | 53.91 | 38.82 | 41.40 | 59.51 | 5.71B | 358.82G | 10958.80MiB | 45.92s
$\text{Element}^{2}$ | 21.68 | 72.89 | 70.46 | 76.22 | 68.47 | 66.14 | 53.24 | 38.23 | 41.60 | 59.19 | 5.71B | 358.82G | 10958.80MiB | 44.81s
Ours | 21.63 | 72.61 | 70.85 | 77.19 | 70.12 | 67.16 | 53.97 | 40.13 | 42.06 | 60.21 | 5.71B | 358.82G | 10958.80MiB | 43.11s

Table 3: Zero-shot Performance of the compressed Bloom-7b1 with a pruning
rate=25%, number of samples=50.

Method | WikiText2$\color[rgb]{0,.5,.5}\downarrow$ | PTB$\color[rgb]{0,.5,.5}\downarrow$ | BoolQ | PIQA | HellaSwag | WinoGrande | ARC-e | ARC-c | OBQA | Accuracy Average | #Params | #MACs | Memory | Latency
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
Bloom-7b1 [32] | - | - | 62.91 | 73.56 | 59.67 | 64.40 | 57.28 | 33.53 | 36.00 | 55.34 | 7.00B | 452.91G | 13491.20MiB | 66.89s
Vector | 101.20 | 319.13 | 61.19 | 71.16 | 47.65 | 55.56 | 50.38 | 30.89 | 32.8 | 49.95 | 5.68B | 369.03G | 10994.85MiB | 50.49s
$\text{Element}^{1}$ | 167.52 | 518.01 | 61.83 | 70.57 | 48.59 | 55.49 | 50.88 | 29.18 | 33.2 | 49.96 | 5.68B | 369.03G | 10994.85MiB | 51.74s
$\text{Element}^{2}$ | 101.20 | 319.13 | 61.62 | 70.40 | 48.28 | 56.12 | 50.42 | 30.12 | 34.4 | 50.19 | 5.68B | 369.03G | 10994.85MiB | 53.41s
Ours | 197.38 | 586.98 | 62.33 | 71.16 | 49.49 | 57.73 | 52.10 | 31.14 | 35.8 | 51.39 | 5.68B | 368.98G | 10994.85MiB | 51.79s

Table 4: Zero-shot Performance of the compressed Baichuan-7B with a pruning
rate=25%, number of samples=50.

Method | WikiText2$\color[rgb]{0,.5,.5}\downarrow$ | PTB$\color[rgb]{0,.5,.5}\downarrow$ | BoolQ | PIQA | HellaSwag | WinoGrande | ARC-e | ARC-c | OBQA | Accuracy Averagee | #Params | #MACs | Memory | Latency
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
Baichuan-7b [33] | - | - | 68.35 | 76.39 | 67.18 | 62.98 | 56.05 | 38.14 | 42.8 | 58.93 | 7.00B | 433.51G | 13491.20MiB | 74.8
Vector | 25.74 | 90.72 | 61.47 | 74.59 | 61.54 | 61.40 | 50.00 | 33.87 | 38.40 | 54.47 | 5.69B | 347.95G | 10915.65MiB | 42.42s
$\text{Element}^{1}$ | 20.76 | 81.32 | 63.00 | 75.03 | 62.81 | 57.46 | 50.84 | 34.98 | 39.00 | 54.73 | 5.69B | 347.95G | 10915.65MiB | 42.45s
$\text{Element}^{2}$ | 20.96 | 81.96 | 61.62 | 73.07 | 59.87 | 54.70 | 49.92 | 33.45 | 37.80 | 52.92 | 5.69B | 347.95G | 10915.65MiB | 44.93s
Ours | 20.68 | 81.00 | 63.05 | 75.38 | 62.93 | 57.83 | 51.04 | 34.76 | 39.6 | 54.94 | 5.69B | 358.82G | 10915.65MiB | 41.46s

###  4.4 Ablation Study

We conducted separate ablation experiments to analyze the impact of varying
sample sizes and pruning rates on the model. This allowed us to systematically
investigate the performance under different configurations and assess the
robustness of our approach.

Ablation Study on Sample numbers. We experiment with sample numbers 10, 20,
30, 40, and 50 to provide input to the model and compare the impact on
accuracy. We investigate whether the sample numbers affect various aspects of
training and model performance.

From Tables 5 and 6, the first row of each section represents the experimental
results for $\text{Element}^{2}$, while the second row displays our
experimental results. It is evident that the average accuracy of BoolQ, PIQA,
HellaSwag, WinoGrande, ARC-e, ARC-c, and OBQA exhibits an increasing trend
with the lengthening of example prompts. Concurrently, the perplexity (PPL) of
WikiText2 and PTB decreases with the increase in sample number. Our model
consistently demonstrates higher accuracy compared to $\text{Element}^{2}$
methods.
Ablation Study on Prune Ratio. The choice of pruning rate directly affects the
pruning effect and performance of the model. In our experiments, we tried
pruning rates of 5%, 10%, 15%, and 20% to compare the accuracy of the models,
studying whether different pruning rates impact the model’s performance.

In Tables 7 and 8, divided into four sections based on pruning rates of 5%,
10%, 15%, and 20%. In each section’s first row represents the experimental
results for $\text{Element}^{2}$, and the second row represents our
experimental results. From Tables 7 and 8, it can be observed that our
experimental results, overall, outperform the $\text{Element}^{2}$ method.
With increasing pruning rates, parameters, MACs, Memory, and Latency
consistently decrease. Both our method and $\text{Element}^{2}$ method
experienced a slight loss in accuracy and PPL with the increase in pruning
ratio.

Table 5: Ablation Study on sample numbers LLama-7B with a pruning rate=20%

Length | WikiText2$\color[rgb]{0,.5,.5}\downarrow$ | PTB$\color[rgb]{0,.5,.5}\downarrow$ | BoolQ | PIQA | HellaSwag | WinoGrande | ARC-e | ARC-c | OBQA | Average Accuracy | #Params | #MACs | Memory | Latency
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
10 | 17.30 | 30.74 | 66.7 | 76.99 | 70.24 | 63.30 | 52.02 | 39.42 | 40.40 | 58.43 | 5.71B | 358.82G | 10958.80MiB | 39.87s
| 17.38 | 31.16 | 69.14 | 77.64 | 69.70 | 63.46 | 51.22 | 40.1 | 40.80 | 58.86 | 5.71B | 358.82G | 10958.80MiB | 39.08s
20 | 17.28 | 31.41 | 62.81 | 77.15 | 69.94 | 64.80 | 52.78 | 38.91 | 40.40 | 58.11 | 5.71B | 358.82G | 10958.80MiB | 44.68s
| 17.89 | 33.83 | 67.83 | 77.15 | 69.81 | 65.04 | 52.02 | 38.74 | 41.4 | 58.86 | 5.71B | 358.82G | 10958.80MiB | 40.41s
30 | 17.25 | 31.41 | 65.57 | 77.37 | 70.24 | 63.14 | 53.24 | 38.74 | 41.80 | 58.59 | 5.71B | 358.82G | 10958.80MiB | 45.92s
| 17.22 | 30.93 | 67.55 | 77.08 | 70.15 | 65.02 | 51.47 | 40.27 | 41.60 | 59.02 | 5.71B | 358.82G | 10958.80MiB | 41.23s
40 | 17.17 | 30.68 | 68.56 | 76.99 | 70.23 | 65.27 | 52.61 | 39.33 | 40.6 | 59.08 | 5.71B | 358.82G | 10958.80MiB | 43.20s
| 17.15 | 30.66 | 69.14 | 77.53 | 70.29 | 63.85 | 53.07 | 40.10 | 40.6 | 59.22 | 5.71B | 358.82G | 10958.80MiB | 42.22s
50 | 16.81 | 30.45 | 67.40 | 76.77 | 70.61 | 65.19 | 53.54 | 39.08 | 41.20 | 59.11 | 5.71B | 358.82G | 10958.80MiB | 43.23s
| 16.68 | 30.17 | 70.13 | 77.42 | 70.56 | 65.22 | 55.04 | 41.20 | 41.80 | 60.20 | 5.71B | 358.82G | 10926.80MiB | 42.41s

Table 6: Ablation Study on sample numbers for Vicuna-7B with a pruning
rate=20%.

Length | WikiText2$\color[rgb]{0,.5,.5}\downarrow$ | PTB$\color[rgb]{0,.5,.5}\downarrow$ | BoolQ | PIQA | HellaSwag | WinoGrande | ARC-e | ARC-c | OBQA | Average Accuracy | #Params | #MACs | Memory | Latency
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
10 | 21.55 | 73.47 | 53.79 | 77.37 | 68.44 | 64.72 | 54.17 | 38.48 | 41.20 | 56.88 | 5.71B | 358.82G | 10958.80MiB | 72.54s
| 21.85 | 74.33 | 60.53 | 76.12 | 68.42 | 63.93 | 57.06 | 38.74 | 40.60 | 57.91 | 5.71B | 358.82G | 10958.80MiB | 72.54s
20 | 21.85 | 75.80 | 72.20 | 76.17 | 68.92 | 65.19 | 54.63 | 38.57 | 40.80 | 59.49 | 5.71B | 358.82G | 10958.80MiB | 44.68s
| 21.88 | 74.63 | 72.78 | 76.33 | 68.74 | 64.56 | 64.29 | 38.14 | 42.00 | 60.98 | 5.71B | 358.82G | 10958.80MiB | 40.41s
30 | 22.15 | 74.33 | 69.42 | 76.50 | 68.42 | 66.77 | 54.42 | 37.97 | 40.80 | 59.19 | 5.71B | 358.82G | 10958.80MiB | 45.92s
| 22.05 | 73.46 | 71.74 | 76.82 | 68.33 | 65.51 | 54.00 | 37.80 | 42.00 | 59.45 | 5.71B | 358.82G | 10958.80MiB | 41.23s
40 | 22.41 | 74.05 | 71.87 | 76.61 | 68.62 | 65.51 | 54.88 | 39.08 | 40.40 | 59.56 | 5.71B | 358.82G | 10958.80MiB | 43.11s
| 22.62 | 74.63 | 72.20 | 76.28 | 68.58 | 66.38 | 53.32 | 38.65 | 41.00 | 59.49 | 5.71B | 358.82G | 10958.80MiB | 42.22s
50 | 21.68 | 72.89 | 70.46 | 76.22 | 68.47 | 66.14 | 53.24 | 38.23 | 41.60 | 59.19 | 5.71B | 358.82G | 10958.80MiB | 43.11s
| 21.63 | 72.61 | 70.85 | 77.19 | 70.12 | 67.16 | 53.97 | 40.13 | 42.06 | 60.21 | 5.71B | 358.82G | 10958.80MiB | 43.11s

Table 7: Ablation Study on Prune Ratio: LLama-7B with number of samples=50.

Ratio | WikiText2$\color[rgb]{0,.5,.5}\downarrow$ | PTB$\color[rgb]{0,.5,.5}\downarrow$ | BoolQ | PIQA | HellaSwag | WinoGrande | ARC-e | ARC-c | OBQA | Average Accuracy | #Params | #MACs | Memory | Latency
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
5% | 17.11 | 30.74 | 72.6 | 78.35 | 73.54 | 67.01 | 53.24 | 42.15 | 42.20 | 61.30 | 6.50B | 410.31G | 12924.65MiB | 58.55s
| 17.38 | 31.16 | 74.98 | 78.78 | 73.53 | 66.61 | 55.3 | 42.06 | 42.60 | 61.98 | 6.50B | 410.31G | 12924.65MiB | 53.37s
10% | 17.28 | 31.41 | 72.54 | 78.02 | 72.12 | 65.43 | 57.03 | 43.52 | 40.20 | 61.26 | 6.22B | 391.97G | 12060.64MiB | 58.08s
| 17.89 | 33.83 | 73.76 | 78.62 | 72.87 | 66.14 | 54.55 | 42.15 | 41.80 | 61.41 | 6.22B | 391.97G | 12060.648MiB | 40.41s
15% | 17.25 | 31.41 | 70.7 | 77.09 | 71.3 | 64.25 | 54.34 | 40.87 | 40.60 | 59.87 | 5.92B | 377.16G | 11540.64MiB | 55.28s
| 17.22 | 30.93 | 70.98 | 77.97 | 72.00 | 66.85 | 53.87 | 41.98 | 40.80 | 60.63 | 5.92B | 377.16G | 11540.64MiB | 41.23s
20% | 16.81 | 30.45 | 67.40 | 76.77 | 70.61 | 65.19 | 53.54 | 39.08 | 41.20 | 59.11 | 5.71B | 358.82G | 10958.80MiB | 43.23s
| 16.68 | 30.17 | 70.13 | 77.42 | 71.56 | 65.22 | 55.04 | 41.20 | 41.80 | 60.20 | 5.71B | 358.82G | 10926.80MiB | 42.41s

Table 8: Ablation Study on Prune Ratio for Vicuna-7B with number of
sampless=50.

Ratio | WikiText2$\color[rgb]{0,.5,.5}\downarrow$ | PTB$\color[rgb]{0,.5,.5}\downarrow$ | BoolQ | PIQA | HellaSwag | WinoGrande | ARC-e | ARC-c | OBQA | Accuracy Average | #Params | #MACs | Memory | Latency
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
5% | 16.91 | 58.73 | 77.98 | 77.58 | 71.01 | 67.25 | 53.83 | 39.85 | 41.60 | 61.03 | 6.50B | 425.12G | 12924.65MiB | 72.54s
| 16.75 | 58.80 | 72.97 | 77.80 | 71.56 | 68.75 | 57.74 | 41.21 | 42.60 | 61.80 | 6.50B | 425.12G | 12924.65MiB | 72.54s
10% | 18.25 | 63.83 | 75.99 | 76.12 | 69.96 | 67.40 | 54.00 | 39.85 | 40.40 | 60.53 | 6.22B | 358.82G | 10958.80MiB | 44.68s
| 17.69 | 61.38 | 74.77 | 76.71 | 76.71 | 68.27 | 54.50 | 38.57 | 42.40 | 61.70 | 6.22B | 358.82G | 10958.80MiB | 44.68s
15% | 19.17 | 66.12 | 71.31 | 76.99 | 70.10 | 67.48 | 55.18 | 40.02 | 40.4 | 60.21 | 5.92B | 358.82G | 10958.80MiB | 45.92s
| 19.35 | 65.86 | 72.32 | 76.66 | 70.00 | 67.56 | 56.65 | 39.76 | 41.8 | 60.68 | 5.92B | 358.82G | 10958.80MiB | 45.92s
20% | 21.68 | 72.89 | 70.46 | 76.22 | 68.47 | 66.14 | 53.24 | 38.23 | 41.60 | 59.19 | 5.71B | 358.82G | 10958.80MiB | 43.11s
| 21.63 | 72.61 | 70.85 | 77.19 | 70.12 | 67.16 | 53.97 | 40.13 | 42.06 | 60.21 | 5.71B | 358.82G | 10958.80MiB | 43.11s

###  4.5 Discussion

Adopting adaptive fusion of coarse-grained estimation methods, as opposed to
solely relying on a generic coarse-grained estimation method or fine-grained
importance estimation methods, offer several advantages:

  * •

Adaptability and Flexibility: Adaptive fusion allows for flexible integration
of coarse-grained and fine-grained estimation methods based on specific
contexts and task requirements. This adaptability makes the pruning process
more targeted, better aligning with the model’s unique structure and task
demands.

  * •

Combining Global and Local Insights: Adaptive fusion permits the simultaneous
consideration of global and local information during the pruning process. A
generic coarse-grained estimation method overlooks differences in various
parts of the model, while fine-grained methods might be too localized.
Adaptive fusion strikes a balance between global and local considerations,
comprehensively addressing contributions from both the overall model and its
individual components.

  * •

Performance Optimization: Adaptive fusion optimizes pruning performance by
adaptively adjusting the balance between coarse-grained and fine-grained
estimations. This optimization enhances the benefits of pruning while
maintaining performance, avoiding potential pitfalls such as over-pruning or
conservative pruning in different scenarios.

  * •

Task Awareness: Adaptive fusion methods consider the model’s performance
across diverse tasks, making the pruning process more task-aware. This
enhances the general applicability of pruning, enabling the pruned model to
perform well across multiple tasks.

In summary, adaptive fusion of coarse-grained estimation methods provides a
more comprehensive and flexible approach to the pruning process, resulting in
superior pruning effectiveness. However, the specific outcomes depend on the
model’s structure, task characteristics, and the design of the adaptive fusion
strategy.

##  5 Conclusion

This paper observes that the limitation of existing structural pruning
techniques for LLMs arises from the complex multi-layer decoder structure,
which includes various operators (e.g., MLP and Attention) in each layer. This
paper introduces a simple yet effective method that can adaptively model the
importance of each coupled structure. it adaptively integrates coarse-grained
and fine-grained estimation in the end-to-end pruning. We validate the
proposed method on the popular LLM benchmarks, achieving average accuracy
improvements of 1.1%, 1.02%, 2.0%, and 1.2% for LLaMa-7B, Vicuna-7B,
Baichuan-7B, and Bloom-7b1, respectively, over state-of-the-art methods.

In the future, we will explore unstructured pruning as well as leverage
specialized hardware resources for further efficient computation. Parameter-
efficient fine-tuning is also a popular research area that can be integrated
into our end-to-end pruning framework.

## References

  * [1] Alizadeh, K., Mirzadeh, I., Belenko, D., Khatamifard, K., Cho, M., Del Mundo, C.C., Rastegari, M., Farajtabar, M.: Llm in a flash: Efficient large language model inference with limited memory. arXiv preprint arXiv:2312.11514 (2023)
  * [2] Bisk, Y., Zellers, R., Bras, R.L., Gao, J., Choi, Y.: Piqa: Reasoning about physical commonsense in natural language. In: Thirty-Fourth AAAI Conference on Artificial Intelligence (2020)
  * [3] Cao, Q., Paranjape, B., Hajishirzi, H.: Pumer: Pruning and merging tokens for efficient vision language models. In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, Toronto, Canada, July 9-14, 2023. pp. 12890–12903. Association for Computational Linguistics (2023), https://doi.org/10.18653/v1/2023.acl-long.721
  * [4] Chiang, W.L., Li, Z., Lin, Z., Sheng, Y., Wu, Z., Zhang, H., Zheng, L., Zhuang, S., Zhuang, Y., Gonzalez, J.E., Stoica, I., Xing, E.P.: Vicuna: An open-source chatbot impressing gpt-4 with 90% chatgpt quality (March 2023), https://lmsys.org/blog/2023-03-30-vicuna/
  * [5] Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H.W., Sutton, C., Gehrmann, S., et al.: Palm: Scaling language modeling with pathways. Journal of Machine Learning Research 24(240), 1–113 (2023)
  * [6] Clark, C., Lee, K., Chang, M.W., Kwiatkowski, T., Collins, M., Toutanova, K.: BoolQ: Exploring the surprising difficulty of natural yes/no questions. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). pp. 2924–2936. Association for Computational Linguistics, Minneapolis, Minnesota (Jun 2019). https://doi.org/10.18653/v1/N19-1300, https://aclanthology.org/N19-1300
  * [7] Clark, P., Cowhey, I., Etzioni, O., Khot, T., Sabharwal, A., Schoenick, C., Tafjord, O.: Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv:1803.05457v1 (2018)
  * [8] Dettmers, T., Pagnoni, A., Holtzman, A., Zettlemoyer, L.: Qlora: Efficient finetuning of quantized llms. arXiv preprint arXiv:2305.14314 (2023)
  * [9] Dettmers, T., Svirschevski, R., Egiazarian, V., Kuznedelev, D., Frantar, E., Ashkboos, S., Borzunov, A., Hoefler, T., Alistarh, D.: Spqr: A sparse-quantized representation for near-lossless llm weight compression. In: The Twelfth International Conference on Learning Representations (2023), https://openreview.net/pdf?id=Q1u25ahSuy
  * [10] Frantar, E., Alistarh, D.: Sparsegpt: Massive language models can be accurately pruned in one-shot. In: International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA. pp. 10323–10337. PMLR (2023), https://proceedings.mlr.press/v202/frantar23a.html
  * [11] Frantar, E., Ashkboos, S., Hoefler, T., Alistarh, D.: Optq: Accurate quantization for generative pre-trained transformers. In: The Eleventh International Conference on Learning Representations (2022), https://openreview.net/forum?id=tcbBPnfwxS
  * [12] He, W., Wu, M., Liang, M., Lam, S.K.: Cap: Context-aware pruning for semantic segmentation. In: Proceedings of the IEEE/CVF winter conference on applications of computer vision. pp. 960–969. IEEE Computer Society (2021), https://doi.ieeecomputersociety.org/10.1109/WACV48630.2021.00100
  * [13] Jaiswal, A., Liu, S., Chen, T., Wang, Z.: The emergence of essential sparsity in large pre-trained models: The weights that matter. In: Thirty-seventh Conference on Neural Information Processing Systems (2023), https://openreview.net/forum?id=bU9hwbsVcy
  * [14] Klein, A., Golebiowski, J., Ma, X., Perrone, V., Archambeau, C.: Structural pruning of large language models via neural architecture search. In: AutoML Conference 2023 (Workshop) (2023), https://openreview.net/forum?id=SHlZcInS6C
  * [15] Ko, J., Park, S., Kim, Y., Ahn, S., Chang, D.S., Ahn, E., Yun, S.Y.: NASH: A simple unified framework of structured pruning for accelerating encoder-decoder language models. In: The 2023 Conference on Empirical Methods in Natural Language Processing (2023), https://openreview.net/forum?id=dwGKBFXiy2
  * [16] Kwon, W., Kim, S., Mahoney, M.W., Hassoun, J., Keutzer, K., Gholami, A.: A fast post-training pruning framework for transformers. In: Oh, A.H., Agarwal, A., Belgrave, D., Cho, K. (eds.) Advances in Neural Information Processing Systems (2022), https://openreview.net/forum?id=0GRBKLBjJE
  * [17] LeCun, Y., Denker, J., Solla, S.: Optimal brain damage. In: Advances in Neural Information Processing Systems (1989), https://proceedings.neurips.cc/paper_files/paper/1989/file/6c9882bbac1c7093bd25041881277658-Paper.pdf
  * [18] Li, Y., Bubeck, S., Eldan, R., Del Giorno, A., Gunasekar, S., Lee, Y.T.: Textbooks are all you need ii: phi-1.5 technical report. arXiv preprint arXiv:2309.05463 (2023)
  * [19] Liang, C., Zuo, S., Zhang, Q., He, P., Chen, W., Zhao, T.: Less is more: Task-aware layer-wise distillation for language model compression. In: International Conference on Machine Learning. pp. 20852–20867. PMLR (2023), https://openreview.net/forum?id=-Ov808Vm7dw
  * [20] yang Liu, S., Liu, Z., Huang, X., Dong, P., Cheng, K.T.: LLM-FP4: 4-bit floating-point quantized transformers. In: The 2023 Conference on Empirical Methods in Natural Language Processing (2023), https://openreview.net/forum?id=wiI8ycNfgJ
  * [21] Ma, X., Fang, G., Ikko E, A.: LLM-Pruner: On the Structural Pruning of Large Language Models. https://github.com/horseee/LLM-Pruner (2023)
  * [22] Ma, X., Fang, G., Wang, X.: LLM-pruner: On the structural pruning of large language models. In: Thirty-seventh Conference on Neural Information Processing Systems (2023), https://openreview.net/forum?id=J8Ajf9WfXP
  * [23] Marcus, M.P., Santorini, B., Marcinkiewicz, M.A.: Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics 19(2), 313–330 (1993), https://www.aclweb.org/anthology/J93-2004
  * [24] Merity, S., Xiong, C., Bradbury, J., Socher, R.: Pointer sentinel mixture models (2016)
  * [25] Park, J.S., Hessel, J., Chandu, K., Liang, P.P., Lu, X., West, P., Yu, Y., Huang, Q., Gao, J., Farhadi, A., Choi, Y.: Localized symbolic knowledge distillation for visual commonsense models. In: Thirty-seventh Conference on Neural Information Processing Systems (2023), https://openreview.net/forum?id=V5eG47pyVl
  * [26] Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al.: Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems 32 (2019)
  * [27] Sakaguchi, K., Bras, R.L., Bhagavatula, C., Choi, Y.: Winogrande: An adversarial winograd schema challenge at scale (2019)
  * [28] Sun, M., Liu, Z., Bair, A., Kolter, J.Z.: A simple and effective pruning approach for large language models. In: The Twelfth International Conference on Learning Representations (2024), https://openreview.net/forum?id=PxoFut3dWW
  * [29] Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al.: Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023)
  * [30] Wang, Z., Wohlwend, J., Lei, T.: Structured pruning of large language models. In: EMNLP. pp. 6151–6162 (2020)
  * [31] Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al.: Huggingface’s transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771 (2019)
  * [32] Workshop, B., Scao, T.L., Fan, A., Akiki, C., Pavlick, E., Ilić, S., Hesslow, D., Castagné, R., Luccioni, A.S., Yvon, F., et al.: Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint arXiv:2211.05100 (2022)
  * [33] Yang, A., Xiao, B., Wang, B., Zhang, B., Bian, C., Yin, C., Lv, C., Pan, D., Wang, D., Yan, D., et al.: Baichuan 2: Open large-scale language models. arXiv preprint arXiv:2309.10305 (2023)
  * [34] Yu, L., Xiang, W.: X-pruner: explainable pruning for vision transformers. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 24355–24363. IEEE Computer Society (2023), https://doi.org/10.1109/CVPR52729.2023.02333
  * [35] Zellers, R., Holtzman, A., Bisk, Y., Farhadi, A., Choi, Y.: Hellaswag: Can a machine really finish your sentence? In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (2019)


# Decompositions of the Automorphism Group of a Locally Compact Abelian Group

Iian B. Smythe<EMAIL_ADDRESS>Department of Mathematics, Cornell
University, Ithaca, NY 14853-4201

(Date: October 9, 2011.)

###### Abstract.

It is well known that every locally compact abelian group $L$ can be
decomposed as $L_{1}\oplus\mathbb{R}^{n}$, where $L_{1}$ contains a compact-
open subgroup. In this paper, we use this decomposition to study the
topological group $\operatorname{Aut}(L)$ of automorphisms of $L$, equipped
with the $g$-topology. We show that $\operatorname{Aut}(L)$ is topologically
isomorphic to a matrix group with entries from $\operatorname{Aut}(L_{1})$,
$\operatorname{Hom}(L_{1},\mathbb{R}^{n})$,
$\operatorname{Hom}(\mathbb{R}^{n},L_{1})$, and
$\operatorname{GL}_{n}(\mathbb{R})$, respectively. It is also shown that the
algebraic portion of the decomposition is not specific to locally compact
abelian groups, but is also true for objects with a well-behaved decomposition
in an additive category with kernels.

###### 2010 Mathematics Subject Classification:

Primary: 22D45, 22B05. Secondary: 54H11, 18E05, 20K30.

I would like to thank the Natural Sciences and Engineering Research Council of
Canada, the University of Manitoba, and Cornell University for financial
support which has enabled this research.

##  1\. Introduction

Given a collection of mathematical objects with a notion of isomorphism, it is
often of interest to study the self-isomorphisms, or automorphisms, of those
objects. In particular, the set of all such automorphisms is a group under
composition, and there is an interplay between the structure of this group of
automorphisms and the underlying object. Classical examples include
permutation groups of sets, which encompasses the whole of group theory,
automorphism groups of fields in the context of Galois theory, and groups of
diffeomorphisms of smooth manifolds. In the setting of topological spaces,
where automorphisms are self-homeomorphisms of a space $X$, it is natural to
consider endowing this automorphism group $\operatorname{Homeo}(X)$ with a
topology related to that of $X$. If $X$ is locally compact, then
$\operatorname{Homeo}(X)$ and its subgroups can be made into topological
groups, via the so-called _$g$ -topology_, or _Birkhoff topology_ , generated
by the subbasis consisting of sets of the form

$(C,U)=\\{f\in\operatorname{Homeo}(X):f(C)\subseteq U\text{ and
}f^{-1}(C)\subseteq U\\},$

where $C$ is a compact subset of $X$, and $U$ an open subset of $X$ [1]. This
is the coarsest refinement of the compact-open topology wherein both
composition and inversion are continuous.

When $L$ is a Hausdorff locally compact group, denote by
$\operatorname{Aut}(L)$ the group of topological automorphisms of $L$, a
closed subgroup of $\operatorname{Homeo}(L)$, endowed with the $g$-topology.
In general, $\operatorname{Aut}(L)$ is not locally compact, even in the case
where $L$ is a compact abelian group [9, 26.18 (k)], which has led many to
study conditions under which local compactness holds. For example, if $L$ is
compact, totally disconnected, and nilpotent, then local compactness, and in
fact, compactness, of $\operatorname{Aut}(L)$ are equivalent to all Sylow
subgroups having finitely many topological generators [16]. Recent work of
Caprace and Monod has shown that if $L$ is totally disconnected, compactly
generated and locally finitely generated, then $\operatorname{Aut}(L)$ is
locally compact [5, I.6]. It is also known that $\operatorname{Aut}(L)$ is a
Lie group provided $L$ is connected and finite dimensional [12]. It has been
shown that automorphism groups of compact abelian groups are universal for the
class of non-archimedean groups in the sense that every non-archimedean group
embeds as a topological subgroup of $\operatorname{Aut}(K)$, for some compact
abelian $K$; see [15] and [14].

In the case where $L$ is a locally compact abelian (LCA) group, Levin [10]
gave criterion for local compactness of $\operatorname{Aut}(L)$, provided $L$
contained a discrete subgroup such that the quotient was compact. Levin’s
analysis utilizes the additional structure of LCA groups afforded to us by
their duality theory, and in particular, the following canonical decomposition
of such groups.

######  Theorem 1.1. ([9, 24.30], [3, Cor. 1])

If $L$ is an LCA group, then $L\cong L_{1}\oplus\mathbb{R}^{n}$, where $L_{1}$
is an LCA group containing a compact-open subgroup. Further, $n$ is uniquely
determined, and $L_{1}$ is determined up to isomorphism.

The main result of this paper is a structural decomposition of the
automorphism group of an LCA group, using the decomposition in Theorem 1.1:

######  Theorem A.

Let $L$ be an LCA group with $L=L_{1}\oplus\mathbb{R}^{n}$, where $L_{1}$
contains a compact-open subgroup. Then, as topological groups,

$\operatorname{Aut}(L)\cong\begin{pmatrix}\operatorname{Aut}(L_{1})&\operatorname{Hom}(\mathbb{R}^{n},L_{1})\\\
\operatorname{Hom}(L_{1},\mathbb{R}^{n})&\operatorname{GL}_{n}(\mathbb{R})\end{pmatrix},$

where the latter is equipped with the product topology.

The algebraic portion of Theorem A can be extracted and established in a more
general setting.

######  Theorem B.

Let $\mathcal{C}$ be an additive category with kernels, and $A=B\oplus C$ an
object in $\mathcal{C}$ such that:

  1. (I)

$\delta\in\operatorname{End}(C)$ is an automorphism of $C$ if and only if the
zero morphism $\mathbf{0}$ is a kernel of $\delta$; and

  2. (II)

For every pair of morphisms $\gamma:B\to C$ and $\beta:C\to B$, one has that
$\gamma\beta=\mathbf{0}$.

Then, as groups,

$\operatorname{Aut}(A)\cong\begin{pmatrix}\operatorname{Aut}(B)&\mathcal{C}(C,B)\\\
\mathcal{C}(B,C)&\operatorname{Aut}(C)\end{pmatrix}.$

The paper is structured as follows: In §2, we provide topological
preliminaries regarding the compact-open and $g$-topologies. §3 is a
discussion of an abstract categoral setting wherein we prove Theorem B. In §4,
we present the proof of Theorem A.

##  2\. Preliminaries

Throughout this paper, all spaces are assumed to be Hausdorff, and in
particular, all topological groups are Tychonoff [11, 1.21]. Recall that if
$X$ and $Y$ are topological spaces and $\mathcal{F}$ a collection of
continuous functions from $X$ to $Y$, the _compact-open topology_ on
$\mathcal{F}$ is the topology generated by the subbasis consisting of sets of
the form

$[C,U]=\\{f\in\mathcal{F}:f(C)\subseteq U\\},$

where $C$ is a compact subset of $X$, and $U$ an open subset of $Y$ (see [7],
[18, §43]). For locally compact $X$, composition of maps is continuous in
$\operatorname{Homeo}(X)$ when endowed with the compact-open topology, a
consequence of the following property:

######  Theorem 2.1. ([6, 3.4.2])

If $X$, $Y$ and $Z$ are topological spaces, with $Y$ locally compact, then the
composition map $C(Y,Z)\times C(X,Y)\to C(X,Z)$ is continuous with respect to
the compact-open topology.

However, inversion may fail to be continuous in $\operatorname{Homeo}(X)$ with
respect to the compact-open topology [4, p. 57-58]; this shortcoming is
remedied by the $g$-topology. The two topologies coincide when $X$ is compact,
discrete, or locally connected, but not in general [1]. One can characterize
convergence in the $g$-topology in terms of the compact-open topology as in
the following proposition.

######  Proposition 2.2. ([1, 5. (ii)])

Let $X$ be a locally compact space. A net $(f_{\lambda})$ in
$\operatorname{Homeo}(X)$ converges to $f\in\operatorname{Homeo}(X)$ in the
g-topology, written $f_{\lambda}\xrightarrow{g}f$, if and only if
$(f_{\lambda})$ converges to $f$ and $(f_{\lambda}^{-1})$ converges to
$f^{-1}$ in the compact-open topology, written
$f_{\lambda}\xrightarrow{c.o.}f$ and
$f_{\lambda}^{-1}\xrightarrow{c.o.}f^{-1}$.

Given an LCA group $L$, $\operatorname{Aut}(L)$ is a closed subgroup of
$\operatorname{Homeo}(L)$, endowed with the $g$-topology. Theorem 1.1 implies
a decomposition of $\operatorname{End}(L)$, the (additive) group of
topological endomorphisms of $L$, endowed with the compact-open topology, into
a topological ring of $2\times 2$ matrices. In particular, every element of
$\operatorname{Aut}(L)$ can be algebraically represented in this way, but we
caution that since $\operatorname{Aut}(L)$ carries the $g$-topology, it is
_not_ a subspace of $\operatorname{End}(L)$. We note for future reference that
if $L=\mathbb{R}^{n}$, then its ring of endomorphisms and group of
automorphisms are familiar objects:

######  Remark 2.3. ([9, 26.18 (i)])

  1. (a)

Taken with the compact-open topology,
$\operatorname{End}(\mathbb{R}^{n})=M_{n}(\mathbb{R})$, where the latter
carries its standard topology as a subspace of $\mathbb{R}^{n^{2}}$.

  2. (b)

Taken with the $g$-topology,
$\operatorname{Aut}(R^{n})=\operatorname{GL}_{n}(\mathbb{R})$, where the
latter carries its standard topology. In particular, the compact-open and
$g$-topologies on $\operatorname{Aut}(\mathbb{R}^{n})$ coincide.

##  3\. A Categorical Setting

In this section, we prove Theorem B. First, we recall the following
terminology from category theory:

######  Definition 3.1 ([13, VIII.2]).

Let $\mathcal{C}$ be a category.

  1. (a)

An object $\mathbf{0}$ in $\mathcal{C}$ is a _zero object_ if for every object
$A$ of $\mathcal{C}$, there are unique morphisms $\mathbf{0}\to A$ and
$A\to\mathbf{0}$.

  2. (b)

If $\mathcal{C}$ has a zero object and $A$ and $B$ are objects in
$\mathcal{C}$, then the _zero morphism_ $\mathbf{0}:A\to B$ is the composite
of the morphism $A\to\mathbf{0}$ and $\mathbf{0}\to B$.

  3. (c)

A _kernel_ of a morphism $f:A\to B$ is a morphism $k:K\to A$ such that:

    1. (i)

$fk=\mathbf{0}$; and

    2. (ii)

every morphism $h\colon C\to A$ such that $fh=\mathbf{0}$ factors uniquely
through $k$, that is, there is a unique morphism $k^{\prime}\colon C\to A$
making the following diagram commutative:

$\textstyle{K\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{k}$$\scriptstyle{\mathbf{0}}$$\textstyle{A\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{f}$$\textstyle{B}$$\textstyle{C\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\exists!k^{\prime}}$$\scriptstyle{h}$$\scriptstyle{\mathbf{0}}$

  4. (d)

$\mathcal{C}$ is _preadditive_ if for every pair of objects $A$ and $B$ in
$\mathcal{C}$, the set $\mathcal{C}(A,B)$ of morphisms from $A$ to $B$ is an
abelian group, and the composition
$\circ\colon\mathcal{C}(B,C)\times\mathcal{C}(A,B)\to\mathcal{C}(A,C)$ is
bilinear for every $A$, $B$, and $C$.

  5. (e)

$\mathcal{C}$ is _additive_ if it has a zero object, and every two objects in
$\mathcal{C}$ have a biproduct.

######  Examples 3.2.

  1. (a)

The category $\mathsf{Ab}$ of abelian groups and their homomorphisms is
additive, with the zero object being the trivial group, and biproducts being
direct sums.

  2. (b)

The category $\mathsf{LCA}$ of locally compact abelian groups and their
continuous homomorphisms is additive, with the zero object being the trivial
group, and biproducts being direct products with the product topology.

In an additive category $\mathcal{C}$, the abelian group
$\operatorname{End}(A):=\mathcal{C}(A,A)$ is a ring with respect to
composition for every object $A$ in $\mathcal{C}$.

######  Proposition 3.3. ([13, p. 192])

Let $\mathcal{C}$ be an additive category and $A=A_{1}\oplus
A_{2}\oplus\cdots\oplus A_{n}$ an object of $\mathcal{C}$. Then,

$\operatorname{End}(A)\cong\begin{pmatrix}\operatorname{End}(A_{1})&\mathcal{C}(A_{2},A_{1})&\cdots&\mathcal{C}(A_{n},A_{1})\\\
\mathcal{C}(A_{1},A_{2})&\operatorname{End}(A_{2})&\cdots&\mathcal{C}(A_{n},A_{2})\\\
\vdots&\vdots&&\vdots\\\
\mathcal{C}(A_{1},A_{n})&\mathcal{C}(A_{2},A_{n})&\cdots&\operatorname{End}(A_{n})\\\
\end{pmatrix}$

as rings, where composition is given by matrix multiplication.

For an object $A$ in $\mathcal{C}$, we denote the set of all automorphisms
(self-isomorphisms) of $A$ by $\operatorname{Aut}(A)$; it is a group under
composition.

######  Remark 3.4.

Let $\mathcal{C}$ be an additive category, and suppose that $A=B\oplus C$ is
an object of $\mathcal{C}$ such that $\mathcal{C}(B,C)=\\{\mathbf{0}\\}$.
Then,

$\operatorname{Aut}(A)\cong\begin{pmatrix}\operatorname{Aut}(B)&\mathcal{C}(C,B)\\\
\mathbf{0}&\operatorname{Aut}(C)\end{pmatrix}$

as groups.

Theorem B is an analogue of the aforementioned decomposition of
$\operatorname{Aut}(A)$ when $\mathcal{C}(B,C)$ is not necessarily trivial. To
this end, for the remainder of this section, we fix an additive category
$\mathcal{C}$ such that every morphism has a kernel, and an object $A$ in
$\mathcal{C}$ such that $A=B\oplus C$, with $B$ and $C$ objects of
$\mathcal{C}$ satisfying the following conditions:

  1. (I)

$\delta\in\operatorname{End}(C)$ is an automorphism of $C$ if and only if the
zero morphism $\mathbf{0}$ is a kernel of $\delta$.

  2. (II)

For every pair of morphisms $\gamma:B\to C$ and $\beta:C\to B$, one has that
$\gamma\beta=\mathbf{0}$.

Theorem B is a consequence of Proposition 3.3, and the equivalence of (i) and
(iii) in Theorem 3.5 below.

######  Theorem 3.5.

Let $\varphi\in\operatorname{End}(A)$, where

$\varphi=\begin{pmatrix}\alpha&\beta\\\
\gamma&\delta\end{pmatrix}\in\begin{pmatrix}\operatorname{End}(B)&\mathcal{C}(C,B)\\\
\mathcal{C}(B,C)&\operatorname{End}(C)\end{pmatrix}.$

Then, the following statements are equivalent:

  1. (i)

$\varphi$ is an automorphism of $A$;

  2. (ii)

$\delta$ is an automorphism of $C$, and the _quasi-determinant_
$\det(\varphi):=\alpha-\beta\delta^{-1}\gamma$ is an automorphism of $B$;

  3. (iii)

$\delta$ is an automorphism of $C$, and $\alpha$ is an automorphism of $B$.

We rely on the following elementary fact from ring theory in the proof of
Theorem 3.5.

###### Remark 3.6.

Let $R$ be a (unital) ring, and $n\in R$ a nilpotent element such that
$n^{2}=0$. Then, $(\mathbf{1}+n)^{-1}=(\mathbf{1}-n)$, and in particular,
$(\mathbf{1}+n)$ is invertible.

###### Proof.

(i)$\Longrightarrow$(ii): In order to show that $\delta$ is automorphism, let
$k:K\to C$ be a kernel of $\delta$. Denote the canonical projections
$\pi_{1}:B\oplus C\to B$ and $\pi_{2}:B\oplus C\to C$, $\iota_{K}\colon K\to
B\oplus K$ and $\iota_{B}\colon B\to B\oplus C$ canonical injections, and set
$\psi=\mathbf{0}\oplus k:B\oplus K\to B\oplus C$. Then, one has
$\psi\iota_{K}=(0,k)$ written componentwise as a morphism into $B\oplus C$,
and so $\varphi\psi\iota_{K}=(\beta k,0)$. Thus,

$\displaystyle\pi_{1}\varphi\psi\iota_{K}$ $\displaystyle=\beta k.$ Put
$g:=\pi_{2}\varphi^{-1}\iota_{B}:B\to C$. Since $\displaystyle\iota_{B}\beta
k$ $\displaystyle=(\beta k,0)=\varphi\psi\iota_{K},$ one obtains that
$\displaystyle g\beta k=\pi_{2}\varphi^{-1}\iota_{B}\beta k$
$\displaystyle=\pi_{2}\varphi^{-1}\varphi\psi\iota_{K}=\pi_{2}\psi\iota_{K}=k.$

However, $g\colon B\to C$ and $\beta\colon C\to B$, and so by condition (II),
$g\beta=\mathbf{0}$. Therefore, $k=\mathbf{0}$, and it follows from condition
(I) that $\delta$ is an automorphism.

To establish the second condition, observe that $\varphi$ can be expressed as
follows:

$\displaystyle\varphi=\begin{pmatrix}\mathbf{1}_{B}&\mathbf{0}\\\
\mathbf{0}&\delta\end{pmatrix}\begin{pmatrix}\mathbf{1}_{B}&\beta\\\
\mathbf{0}&\mathbf{1}_{C}\end{pmatrix}\begin{pmatrix}\det(\varphi)&\mathbf{0}\\\
\mathbf{0}&\mathbf{1}_{C}\end{pmatrix}\begin{pmatrix}\mathbf{1}_{B}&\mathbf{0}\\\
\delta^{-1}\gamma&\mathbf{1}_{C}\end{pmatrix}.$ (1)

Since the matrices $\begin{pmatrix}\mathbf{1}_{B}&\mathbf{0}\\\
\mathbf{0}&\delta\end{pmatrix}$, $\begin{pmatrix}\mathbf{1}_{B}&\beta\\\
\mathbf{0}&\mathbf{1}_{C}\end{pmatrix}$ and
$\begin{pmatrix}\mathbf{1}_{B}&\mathbf{0}\\\
\delta^{-1}\gamma&\mathbf{1}_{C}\end{pmatrix}$ are invertible, the remaining
matrix is also invertible. The latter occurs if and only if
$\det(\varphi)\in\operatorname{Aut}(B)$.

(ii)$\Longrightarrow$(i) is an immediate consequence of (1).

(ii)$\Longrightarrow$(iii): It is given that $\delta$ is an automorphism of
$C$. It follows from the definition of $\det(\varphi)$ that
$\alpha=\det(\varphi)+\beta\delta^{-1}\gamma$. By multiplying both sides by
$\det(\varphi)^{-1}$, one obtains

$\alpha\det(\varphi)^{-1}=\mathbf{1}_{B}+\beta\delta^{-1}\gamma\det(\varphi)^{-1}.$

By condition (II), $\gamma\det(\varphi)^{-1}\beta=\mathbf{0}$, because
$\det(\varphi)^{-1}\beta\in\mathcal{C}(C,B)$, and thus
$(\beta\delta^{-1}\gamma\det(\varphi)^{-1})^{2}=\mathbf{0}$. Therefore, by
Remark 3.6, $\alpha\det(\varphi)^{-1}$ is invertible, and its inverse is
$\mathbf{1}_{B}-\beta\delta^{-1}\gamma\det(\varphi)^{-1}$. Hence, $\alpha$ is
invertible, and

$\displaystyle\alpha^{-1}=\det(\varphi)^{-1}(\mathbf{1}_{B}-\beta\delta^{-1}\gamma\det(\varphi)^{-1}).$
(2)

(iii)$\Longrightarrow$(ii): It is given that $\delta$ is an automorphism of
$C$. One can express $\det(\varphi)\alpha^{-1}$ as

$\det(\varphi)\alpha^{-1}=\mathbf{1}_{B}-\beta\delta^{-1}\gamma\alpha^{-1}.$

By condition (II), $\gamma\alpha^{-1}\beta=\mathbf{0}$, because
$\alpha^{-1}\beta\in\mathcal{C}(C,B)$, and thus
$(-\beta\delta^{-1}\gamma\alpha^{-1})^{2}=0$. Therefore, by Remark 3.6,
$\det(\varphi)\alpha^{-1}$ is invertible, and its inverse is
$\mathbf{1}_{B}+\beta\delta^{-1}\gamma\alpha^{-1}$. Hence, $\det(\varphi)$ is
invertible, and

$\displaystyle\det(\varphi)^{-1}=\alpha^{-1}(\mathbf{1}_{B}+\beta\delta^{-1}\gamma\alpha^{-1}).$
(3)

This completes the proof. ∎

The proof of Theorem 3.5 also enables us to provide an explicit formula for
the inverse of an element in $\operatorname{Aut}(A)$.

######  Corollary 3.7.

If $\varphi\in\operatorname{Aut}(A)$ with
$\varphi=\begin{pmatrix}\alpha&\beta\\\ \delta&\gamma\end{pmatrix}$, then

$\varphi^{-1}=\begin{pmatrix}(\det(\varphi))^{-1}&-(\det(\varphi))^{-1}(\beta\delta^{-1})\\\
-\delta^{-1}\gamma(\det(\varphi))^{-1}&\delta^{-1}\end{pmatrix}.$

###### Proof.

The inverse $\varphi^{-1}$ can be obtained by expressing $\varphi$ in the form
provided in (1), and calculating the inverse of each of the factors as
follows:

$\varphi^{-1}=\begin{pmatrix}\mathbf{1}&\mathbf{0}\\\
-\delta^{-1}\gamma&\mathbf{1}_{C}\end{pmatrix}\begin{pmatrix}\det(\varphi)^{-1}&\mathbf{0}\\\
\mathbf{0}&\mathbf{1}_{C}\end{pmatrix}\begin{pmatrix}\mathbf{1}_{B}&-\beta\\\
\mathbf{0}&\mathbf{1}_{C}\end{pmatrix}\begin{pmatrix}\mathbf{1}_{B}&\mathbf{0}\\\
\mathbf{0}&\delta^{-1}\end{pmatrix}.$

Therefore,

$\varphi^{-1}=\begin{pmatrix}\det(\varphi)^{-1}&-(\det(\varphi))^{-1}(\beta\delta^{-1})\\\
-\delta^{-1}\gamma(\det(\varphi))^{-1}&\delta^{-1}\gamma(\det(\varphi))^{-1}\beta\delta^{-1}+\delta^{-1}\end{pmatrix}.$

However, $(\det(\varphi))^{-1}\beta\in\mathcal{C}(C,B)$, so
$\gamma(\det(\varphi))^{-1}\beta=\mathbf{0}$ by condition (II), and
$\delta^{-1}\gamma(\det(\varphi))^{-1}\beta\delta^{-1}=\mathbf{0}$. ∎

We now apply these general results to the category $\mathsf{LCA}$.

######  Proposition 3.8.

$\mathsf{LCA}$ is an additive category with kernels, and the decomposition of
an LCA group given in Theorem 1.1 satisfies conditions (I) and (II). That is,
given $L=L_{1}\oplus\mathbb{R}^{n}$, where $L_{1}$ contains a compact-open
subgroup, then:

  1. (a)

$\delta\in\operatorname{End}(\mathbb{R}^{n})$ is an automorphism if and only
if it has trivial kernel;

  2. (b)

for every pair of continuous homomorphisms $\gamma:L_{1}\to\mathbb{R}^{n}$ and
$\beta:\mathbb{R}^{n}\to L_{1}$, one has $\gamma\beta=\mathbf{0}$.

###### Proof.

By Example 3.2(b), $\mathsf{LCA}$ is an additive category. If $f\colon L\to H$
is a continuous homomorphism of LCA groups, then the inclusion map
$k\colon\ker f\to L$ is a kernel of $f$ in the sense of Definition 3.1 (c).

(a) follows from Proposition 2.3.

(b) Since $\mathbb{R}^{n}$ is connected, $\beta(\mathbb{R}^{n})$ is contained
in the connected component $c(L_{1})$ of $L_{1}$, which is compact. One has
$\gamma(c(L_{1}))=\\{0\\}$, because the only compact subgroup of
$\mathbb{R}^{n}$ is the trivial one. Hence, $\gamma\beta=\mathbf{0}$. ∎

######  Corollary 3.9.

Let $L=L_{1}\oplus\mathbb{R}^{n}$ be an LCA group, where $L_{1}$ contains a
compact-open subgroup, and $\varphi\in\operatorname{End}(L)$, with

$\varphi=\begin{pmatrix}\alpha&\beta\\\
\gamma&\delta\end{pmatrix}\in\begin{pmatrix}\operatorname{End}(L_{1})&\operatorname{Hom}(\mathbb{R}^{n},L_{1})\\\
\operatorname{Hom}(L_{1},\mathbb{R}^{n})&M_{n}(\mathbb{R})\end{pmatrix}.$

Then, the following statements are equivalent:

  1. (i)

$\varphi$ is an automorphism of $L$;

  2. (ii)

$\delta$ is an automorphism of $\mathbb{R}^{n}$ (i.e., in
$\operatorname{GL}_{n}(\mathbb{R})$), and the quasi-determinant of $\varphi$
is an automorphism of $L_{1}$;

  3. (iii)

$\delta$ is an automorphism of $\mathbb{R}^{n}$, and $\alpha$ is an
automorphism of $L_{1}$. ∎

##  4\. $\operatorname{Aut}(L)$ and Decompositions of $L$

In this section, whenever $L$ and $H$ are LCA groups, the group
$\operatorname{Hom}(L,H)$ of continuous homomorphisms from $L$ to $H$, and the
ring $\operatorname{End}(L)$ of continuous endomorphisms of $L$, are endowed
with the compact-open topology, while the group $\operatorname{Aut}(L)$ of
topological automorphisms, will have the $g$-topology. We show that the
results of §3 remain true for LCA groups with a topological enrichment in the
sense that the algebraic isomorphisms from §3 become topological isomorphisms
in the presence of the aforementioned topologies. The culmination of this work
is Theorem A, a topological enrichment of Theorem B. We begin with the
following enrichment of Proposition 3.3:

######  Proposition 4.1.

Let $L=L_{1}\oplus L_{2}\oplus\cdots\oplus L_{n}$, where each $L_{i}$ is an
LCA group. Then,

$\operatorname{End}(L)\cong\begin{pmatrix}\operatorname{End}(L_{1})&\operatorname{Hom}(L_{2},L_{1})&\cdots&\operatorname{Hom}(L_{n},L_{1})\\\
\operatorname{Hom}(L_{1},L_{2})&\operatorname{End}(L_{2})&\cdots&\operatorname{Hom}(L_{n},L_{2})\\\
\vdots&\vdots&&\vdots\\\
\operatorname{Hom}(L_{1},L_{n})&\operatorname{Hom}(L_{2},L_{n})&\cdots&\operatorname{End}(L_{n})\\\
\end{pmatrix}$

as topological rings, where the latter is equipped with the product topology.

###### Proof.

Let $[A_{(i,j)}]$ denote the matrix ring on the right-hand side, where
$A_{(i,j)}=\operatorname{End}(L_{i})$ if $i=j$, and
$A_{(i,j)}=\operatorname{Hom}(L_{j},L_{i})$ otherwise. We define the map
$F:\operatorname{End}(L)\to[A_{(i,j)}]$ by

$F(\varphi)=[\pi_{i}\varphi\iota_{j}],$

where $\pi_{i}$ is the canonical projection of $L$ onto $L_{i}$, and
$\iota_{j}$ the inclusion of $L_{j}$ into $L$. By Proposition 3.3, $F$ is a
ring homomorphism from $\operatorname{End}(L)$ onto $[A_{(i,j)}]$. This map is
continuous, since all of the spaces involved are given the compact-open
topology, and the map $\varphi\mapsto\pi_{i}\varphi\iota_{j}$ is continuous by
Proposition 2.1. The inverse of $F$ is given by

$F^{-1}([\alpha_{i,j}])=\sum_{(i,j)}{\iota_{i}\alpha_{i,j}\pi_{j}},$

which is continuous by Proposition 2.1 and the continuity of addition in
$\operatorname{End}(L)$. ∎

From now on, if $L$ is a direct sum of (finitely many) LCA groups, we identify
$\operatorname{End}(L)$ with the aforesaid matrix decomposition. Recall that
$\operatorname{Aut}(L)$ is equipped with the $g$-topology, which need not
coincide with the topology inherited from $\operatorname{End}(L)$. Therefore,
decomposition results concerning $\operatorname{End}(L)$ do not automatically
give rise to those for $\operatorname{Aut}(L)$. Nevertheless, in the simplest
case, a topological enrichment of Remark 3.4 holds.

######  Proposition 4.2.

Suppose that $L=A\oplus B$, where $A$ and $B$ are LCA groups with
$\operatorname{Hom}(A,B)=\mathbf{0}$. Then,

$\operatorname{Aut}(L)\cong\begin{pmatrix}\operatorname{Aut}(A)&\operatorname{Hom}(B,A)\\\
\mathbf{0}&\operatorname{Aut}(B)\end{pmatrix},$

as topological groups, where the right-hand side is equipped with the product
topology.

###### Proof.

Define

$F\colon\operatorname{Aut}(L)\to\begin{pmatrix}\operatorname{Aut}(A)&\operatorname{Hom}(B,A)\\\
\mathbf{0}&\operatorname{Aut}(B)\end{pmatrix},\text{ by
}F(\varphi)=\begin{pmatrix}\alpha&\beta\\\ 0&\delta\end{pmatrix}.$

By Remark 3.4, $F$ is well-defined and it is a group isomomorphism, and in
particular, $\alpha$ and $\delta$ are automorphisms of $A$ and $B$,
respectively. Thus, it remains to be seen that $F$ is also a homeomorphism.
Let $(\varphi_{\lambda})$ be a net converging (in the $g$-topology) to
$\varphi\in\operatorname{Aut}(L)$, where

$F(\varphi_{\lambda})=\begin{pmatrix}\alpha_{\lambda}&\beta_{\lambda}\\\
0&\delta_{\lambda}\end{pmatrix},\text{ and
}F(\varphi)=\begin{pmatrix}\alpha&\beta\\\ 0&\delta\end{pmatrix}.$

One may show by direct computation that

$\begin{pmatrix}\alpha_{\lambda}&\beta_{\lambda}\\\
0&\delta_{\lambda}\end{pmatrix}^{-1}=\begin{pmatrix}\alpha_{\lambda}^{-1}&-\beta_{\lambda}\\\
0&\delta_{\lambda}^{-1}\end{pmatrix},\text{ and
}\begin{pmatrix}\alpha&\beta\\\
0&\delta\end{pmatrix}^{-1}=\begin{pmatrix}\alpha^{-1}&-\beta\\\
0&\delta^{-1}\end{pmatrix}.$

Since $\varphi_{\lambda}\xrightarrow{g}\varphi$, by Proposition 2.2,

$\begin{pmatrix}\alpha_{\lambda}&\beta_{\lambda}\\\
0&\delta_{\lambda}\end{pmatrix}\xrightarrow{c.o.}\begin{pmatrix}\alpha&\beta\\\
0&\delta\end{pmatrix},\text{ and
}\begin{pmatrix}\alpha_{\lambda}^{-1}&-\beta_{\lambda}\\\
0&\delta_{\lambda}^{-1}\end{pmatrix}\xrightarrow{c.o.}\begin{pmatrix}\alpha^{-1}&-\beta\\\
0&\delta^{-1}\end{pmatrix}.$

In particular, $\alpha_{\lambda}\xrightarrow{c.o.}\alpha$ and
$\alpha_{\lambda}^{-1}\xrightarrow{c.o.}\alpha^{-1}$, and by Proposition 2.2
applied to $\operatorname{Aut}(A)$, we have that
$\alpha_{\lambda}\xrightarrow{g}\alpha$. Similarly,
$\delta_{\lambda}\xrightarrow{g}\delta$ in $\operatorname{Aut}(C)$. It is
clear that $\beta_{\lambda}\to\beta$ in $\operatorname{Hom}(B,A)$, and thus,
$F(\varphi_{\lambda})\to F(\varphi)$. Therefore $F$ is continuous.

To see that $F^{-1}$ is continuous, suppose that

$\begin{pmatrix}\alpha_{\lambda}&\beta_{\lambda}\\\
0&\delta_{\lambda}\end{pmatrix}\to\begin{pmatrix}\alpha&\beta\\\
0&\delta\end{pmatrix}\text{ in
}\begin{pmatrix}\operatorname{Aut}(A)&\operatorname{Hom}(B,A)\\\
\mathbf{0}&\operatorname{Aut}(B)\end{pmatrix}.$

One has that $\beta_{\lambda}\to\beta$ in $\operatorname{Hom}(B,A)$,
$\alpha_{\lambda}\xrightarrow{g}\alpha$ in $\operatorname{Aut}(A)$, and
$\delta_{\lambda}\xrightarrow{g}\delta$ in $\operatorname{Aut}(B)$. By
Proposition 2.2, $\alpha_{\lambda}\xrightarrow{c.o.}\alpha$ and
$\alpha_{\lambda}^{-1}\xrightarrow{c.o.}\alpha$, and
$\delta_{\lambda}\xrightarrow{c.o.}\delta$ and
$\delta_{\lambda}^{-1}\xrightarrow{c.o.}\delta^{-1}$. The compact-open
topology on $\operatorname{End}(L)$ coincides with the product topology where
each of the component spaces have the compact-open topology, as given in
Proposition 4.1. Therefore,

$\begin{pmatrix}\alpha_{\lambda}&\beta_{\lambda}\\\
0&\delta_{\lambda}\end{pmatrix}\xrightarrow{c.o.}\begin{pmatrix}\alpha&\beta\\\
0&\delta\end{pmatrix},\text{ and
}\begin{pmatrix}\alpha_{\lambda}^{-1}&-\beta_{\lambda}\\\
0&\delta_{\lambda}^{-1}\end{pmatrix}\xrightarrow{c.o.}\begin{pmatrix}\alpha^{-1}&-\beta\\\
0&\delta^{-1}\end{pmatrix}.$

Hence, $F$ is a topological isomorphism. ∎

If $L$ is a compactly generated LCA group, then $L\cong
K\oplus\mathbb{R}^{n}\oplus\mathbb{Z}^{m}$, where $K$ is the maximal compact
subgroup of $L$ [9, 9.8], while if $L$ is a connected LCA group, then $L\cong
K\oplus\mathbb{R}^{n}$ where $K$ is the maximal compact connected subgroup of
$L$ [9, 9.14]. Combining these facts with Proposition 4.2, we have the
following:

######  Corollary 4.3.

  1. (a)

Let $L\cong K\oplus\mathbb{R}^{n}\oplus\mathbb{Z}^{m}$ be a compactly
generated LCA group, where $K$ is the maximal compact subgroup of $L$. Then,

$\operatorname{End}(L)\cong\begin{pmatrix}\operatorname{End}(K)&\operatorname{Hom}(\mathbb{R}^{n},K)&K^{m}\\\
\mathbf{0}&M_{n}(\mathbb{R})&\mathbb{R}^{mn}\\\
\mathbf{0}&\mathbf{0}&M_{m}(\mathbb{Z})\\\ \end{pmatrix}\text{ and
}\operatorname{Aut}(L)\cong\begin{pmatrix}\operatorname{Aut}(K)&\operatorname{Hom}(\mathbb{R}^{n},K)&K^{m}\\\
\mathbf{0}&\operatorname{GL}_{n}(\mathbb{R})&\mathbb{R}^{mn}\\\
\mathbf{0}&\mathbf{0}&\operatorname{GL}_{m}(\mathbb{Z})\\\ \end{pmatrix}.$

  2. (b)

Let $L\cong K\oplus\mathbb{R}^{n}$ be a connected LCA group, where $K$ is the
maximal compact connected subgroup of $L$. Then,

$\operatorname{End}(L)\cong\begin{pmatrix}\operatorname{End}(K)&\operatorname{Hom}(\mathbb{R}^{n},K)\\\
\mathbf{0}&M_{n}(\mathbb{R})\end{pmatrix},\text{ and
}\operatorname{Aut}(L)\cong\begin{pmatrix}\operatorname{Aut}(K)&\operatorname{Hom}(\mathbb{R}^{n},K)\\\
\mathbf{0}&\operatorname{GL}_{n}(\mathbb{R})\end{pmatrix}.$

###### Proof.

(a) The only compact subgroup of $\mathbb{R}^{n}\oplus\mathbb{Z}^{m}$ is the
trivial one, and so
$\operatorname{Hom}(K,\mathbb{R}^{n}\oplus\mathbb{Z}^{m})=\mathbf{0}$. Thus,
by Proposition 4.2, we have that

$\operatorname{Aut}(L)\cong\begin{pmatrix}\operatorname{Aut}(K)&\operatorname{Hom}(\mathbb{R}^{n}\oplus\mathbb{Z}^{m},K)\\\
\mathbf{0}&\operatorname{Aut}(\mathbb{R}^{n}\oplus\mathbb{Z}^{m})\end{pmatrix}.$

The only connected subgroup of $\mathbb{Z}^{m}$ is trivial, so
$\operatorname{Hom}(\mathbb{R}^{n},\mathbb{Z}^{m})=\mathbf{0}$, and so

$\operatorname{Aut}(\mathbb{R}^{n}\oplus\mathbb{Z}^{m})=\begin{pmatrix}\operatorname{Aut}(\mathbb{R}^{n})&\operatorname{Hom}(\mathbb{Z}^{m},\mathbb{R}^{n})\\\
\mathbf{0}&\operatorname{Aut}(\mathbb{Z}^{m})\end{pmatrix}.$

One can easily show that
$\operatorname{Hom}(\mathbb{R}^{n}\oplus\mathbb{Z}^{m},K)\cong\operatorname{Hom}(\mathbb{R}^{n},K)\times\operatorname{Hom}(\mathbb{Z}^{m},K)$.
$\operatorname{Aut}(\mathbb{R}^{n})=\operatorname{GL}_{n}(\mathbb{R})$ by
Remark 2.3,
$\operatorname{Aut}(\mathbb{Z}^{m})=\operatorname{GL}_{m}(\mathbb{Z})$ by [9,
26.18(g)], and it is elementary that
$\operatorname{Hom}(\mathbb{Z}^{m},G)\cong G^{m}$ for any topological group
$G$. Hence,

$\operatorname{Aut}(L)\cong\begin{pmatrix}\operatorname{Aut}(K)&\operatorname{Hom}(\mathbb{R}^{n},K)&\operatorname{Hom}(\mathbb{Z}^{m},K)\\\
\mathbf{0}&\operatorname{Aut}(\mathbb{R}^{n})&\operatorname{Hom}(\mathbb{Z}^{m},\mathbb{R}^{n})\\\
\mathbf{0}&\mathbf{0}&\operatorname{Aut}(\mathbb{Z})\\\
\end{pmatrix}\cong\begin{pmatrix}\operatorname{Aut}(K)&\operatorname{Hom}(\mathbb{R}^{n},K)&K^{m}\\\
\mathbf{0}&\operatorname{GL}_{n}(\mathbb{R})&\mathbb{R}^{mn}\\\
\mathbf{0}&\mathbf{0}&\operatorname{GL}_{m}(\mathbb{Z})\\\ \end{pmatrix}.$

(b) The only compact subgroup of $\mathbb{R}^{n}$ is the trivial one, and so
$\operatorname{Hom}(K,\mathbb{R}^{n})=\mathbf{0}$. The remainder of the result
follows from Proposition 4.2. ∎

A few additional results of this flavour are found in [17, §25].

Fix an LCA group $L=L_{1}\oplus\mathbb{R}^{n}$, where $L_{1}$ contains a
compact-open subgroup. Corollaries 3.7 and 3.9 imply that

$\displaystyle\operatorname{Aut}(L)\cong\begin{pmatrix}\operatorname{Aut}(L_{1})&\operatorname{Hom}(\mathbb{R}^{n},L_{1})\\\
\operatorname{Hom}(L_{1},\mathbb{R}^{n})&\operatorname{GL}_{n}(\mathbb{R})\end{pmatrix}$
(4)

as (abstract) groups. Theorem A is established once we show that this
isomorphism is topological, a result that follows from the equivalence of (i)
and (iv) in Theorem 4.4 below.

######  Theorem 4.4.

Let $(\varphi_{\lambda})$ be a net in $\operatorname{Aut}(L)$, and
$\varphi\in\operatorname{Aut}(L)$. The following statements are equivalent:

  1. (i)

$\varphi_{\lambda}\xrightarrow{g}\varphi$ in $\operatorname{Aut}(L)$;

  2. (ii)

$\varphi_{\lambda}\xrightarrow{c.o.}\varphi$ and
$\det(\varphi_{\lambda})\xrightarrow{g}\det(\varphi)$;

  3. (iii)

$\varphi_{\lambda}\xrightarrow{c.o.}\varphi$ and
$(\det(\varphi_{\lambda}))^{-1}\xrightarrow{c.o.}(\det(\varphi))^{-1}$;

  4. (iv)

$\varphi_{\lambda}\xrightarrow{c.o.}\varphi$ and
$\alpha_{\lambda}\xrightarrow{g}\alpha$;

  5. (v)

$\varphi_{\lambda}\xrightarrow{c.o.}\varphi$ and
$\alpha_{\lambda}^{-1}\xrightarrow{c.o.}\alpha^{-1}$.

###### Proof.

Throughout the proof, we identify automorphisms in $\operatorname{Aut}(L)$
with their matrix representations as provided in (4), and use the following
convention to denote components:
$\varphi_{\lambda}=\begin{pmatrix}\alpha_{\lambda}&\beta_{\lambda}\\\
\gamma_{\lambda}&\delta_{\lambda}\end{pmatrix}$ and
$\varphi=\begin{pmatrix}\alpha&\beta\\\ \gamma&\delta\end{pmatrix}$.
Furthermore, by Remark 2.3, the compact-open and $g$-topologies coincide on
$\operatorname{Aut}(\mathbb{R}^{n})$. So $\delta_{\lambda}\to\delta$ if and
only if $\delta_{\lambda}^{-1}\to\delta^{-1}$, and given one, we need not
verify the other.

(i)$\Longrightarrow$(ii): Since the $g$-topology is finer than the compact-
open one, it follows that $\varphi_{\lambda}\xrightarrow{c.o.}\varphi$. By
Proposition 2.2 applied to $\operatorname{Aut}(L_{1})$, it suffices to show
that $\det(\varphi_{\lambda})\xrightarrow{c.o.}\det(A)$ and
$\det(\varphi_{\lambda})^{-1}\xrightarrow{c.o.}\det(\varphi)^{-1}$. Since
$\alpha_{\lambda}\xrightarrow{c.o.}\alpha$, $\beta_{\lambda}\to\beta$,
$\gamma_{\lambda}\to\gamma$ and $\delta_{\lambda}\to\delta$, where each of the
spaces involved carries the compact-open topology, it follows by Proposition
2.1 that

$\det(\varphi_{\lambda})=\alpha_{\lambda}-\beta_{\lambda}\delta_{\lambda}^{-1}\gamma_{\lambda}\xrightarrow{c.o.}\alpha-\beta\delta^{-1}\gamma=\det(\varphi).$

By Corollary 3.7,

$\displaystyle\varphi_{\lambda}^{-1}$
$\displaystyle=\begin{pmatrix}(\det(\varphi_{\lambda}))^{-1}&-(\det(\varphi_{\lambda}))^{-1}(\beta_{\lambda}\delta_{\lambda}^{-1})\\\
-\delta_{\lambda}^{-1}\gamma_{\lambda}(\det(\varphi_{\lambda}))^{-1}&\delta_{\lambda}^{-1}\end{pmatrix},\text{
and }$ $\displaystyle\varphi^{-1}$
$\displaystyle=\begin{pmatrix}(\det(\varphi))^{-1}&-(\det(\varphi))^{-1}(\beta\delta^{-1})\\\
-\delta^{-1}\gamma(\det(\varphi))^{-1}&\delta^{-1}\end{pmatrix}.$

Since $\varphi_{\lambda}^{-1}\xrightarrow{c.o.}\varphi^{-1}$, in particular,
the $(1,1)$-entry of $\varphi_{\lambda}^{-1}$ converges to the $(1,1)$-entry
of $\varphi^{-1}$. Hence,
$(\det(\varphi_{\lambda}))^{-1}\xrightarrow{c.o.}(\det(\varphi))^{-1}$.

(ii)$\Longrightarrow$(iii) follows from Proposition 2.2 applied to
$\operatorname{Aut}(L_{1})$.

(iii)$\Longrightarrow$(i): Since $\varphi_{\lambda}\xrightarrow{c.o.}\varphi$,
by Proposition 2.2, it suffices to show that
$\varphi_{\lambda}^{-1}\xrightarrow{c.o.}\varphi^{-1}$. We know that
$(\det(\varphi_{\lambda}))^{-1}\xrightarrow{c.o.}(\det(\varphi))^{-1}$, so by
Proposition 2.1,

$\displaystyle-(\det(\varphi_{\lambda}))^{-1}(\beta_{\lambda}\delta_{\lambda}^{-1})$
$\displaystyle\to-(\det(\varphi))^{-1}(\beta\delta^{-1})\text{ in
$\operatorname{Hom}(\mathbb{R}^{n},L_{1})$, and}$
$\displaystyle-\delta_{\lambda}^{-1}\gamma_{\lambda}(\det(\varphi_{\lambda}))^{-1}$
$\displaystyle\to-\delta^{-1}\gamma(\det(\varphi))^{-1}\text{ in
$\operatorname{Hom}(L_{1},R^{n})$.}$

Thus, one has

$\begin{pmatrix}(\det(\varphi_{\lambda}))^{-1}&-(\det(\varphi_{\lambda}))^{-1}(\beta_{\lambda}\delta_{\lambda}^{-1})\\\
-\delta_{\lambda}^{-1}\gamma_{\lambda}(\det(\varphi_{\lambda}))^{-1}&\delta_{\lambda}^{-1}\end{pmatrix}\xrightarrow{c.o.}\begin{pmatrix}(\det(\varphi))^{-1}&-(\det(\varphi))^{-1}(\beta\delta^{-1})\\\
-\delta^{-1}\gamma(\det(\varphi))^{-1}&\delta^{-1}\end{pmatrix}.$

That is, $\varphi_{\lambda}^{-1}\xrightarrow{c.o.}\varphi^{-1}$, and hence,
$\varphi_{\lambda}\xrightarrow{g}\varphi$.

(iii)$\Longrightarrow$(iv): Since
$\varphi_{\lambda}\xrightarrow{c.o.}\varphi$, one has
$\alpha_{\lambda}\xrightarrow{c.o.}\alpha$. Thus, by Proposition 2.2, it
suffices to show that $\alpha_{\lambda}^{-1}\to\alpha^{-1}$. By (2),

$\displaystyle\alpha_{\lambda}^{-1}$
$\displaystyle=(\det(\varphi_{\lambda}))^{-1}(\mathbf{1}_{B}-\beta_{\lambda}\delta_{\lambda}^{-1}\gamma_{\lambda}(\det(\varphi_{\lambda}))^{-1})$
$\displaystyle\alpha^{-1}$
$\displaystyle=(\det(\varphi))^{-1}(\mathbf{1}_{B}-\beta\delta^{-1}\gamma(\det(\varphi))^{-1}).$

Therefore, by Proposition 2.1,

$\alpha_{\lambda}^{-1}=(\det(\varphi_{\lambda}))^{-1}(\mathbf{1}_{B}-\beta_{\lambda}\delta_{\lambda}^{-1}\gamma_{\lambda}(\det(\varphi_{\lambda}))^{-1})\xrightarrow{c.o.}(\det(\varphi))^{-1}(\mathbf{1}_{B}-\beta\delta^{-1}\gamma(\det(\varphi))^{-1})=\alpha^{-1}.$

(iv)$\Longrightarrow$(v) follows from Proposition 2.2 applied to
$\operatorname{Aut}(L_{1})$.

(v)$\Longrightarrow$(iii): By (3),

$\displaystyle(\det(\varphi_{\lambda}))^{-1}$
$\displaystyle=\alpha_{\lambda}^{-1}(\mathbf{1}_{B}+\beta_{\lambda}\delta_{\lambda}^{-1}\gamma_{\lambda}\alpha_{\lambda}^{-1})$
$\displaystyle(\det(\varphi))^{-1}$
$\displaystyle=\alpha^{-1}(\mathbf{1}_{B}+\beta\delta^{-1}\gamma\alpha^{-1})$

Therefore, by Proposition 2.1,

$(\det(\varphi_{\lambda}))^{-1}=\alpha_{\lambda}^{-1}(\mathbf{1}_{B}+\beta_{\lambda}\delta_{\lambda}^{-1}\gamma_{\lambda}\alpha_{\lambda}^{-1})\xrightarrow{c.o.}\alpha^{-1}(\mathbf{1}_{B}+\beta\delta^{-1}\gamma\alpha^{-1})=(\det(\varphi))^{-1}.$

This establishes the remaining equivalence. ∎

We remark that the previous theorem has a striking similarity to the purely
algebraic Theorem 3.5. In both cases, we have reduced a question regarding
elements of $\operatorname{Aut}(L)$ to a question regarding only its diagonal
components, utilizing the quasi-determinant as an intermediate step. Also,
observe that (i)$\Longrightarrow$(ii) in Theorem 4.4 implies that the quasi-
determinant $\det:\operatorname{Aut}(L)\to\operatorname{Aut}(L_{1})$ is
continuous.

## Acknowledgments

This research was conducted as part of an NSERC Undergraduate Summer Research
Award under the supervision of Gábor Lukács. I thank Dr. Lukács for his
wisdom, guidance, attention to detail, and understanding; without him, this
work would simply have not been possible.

I would also like to thank Karen Kipper for carefully proof-reading this paper
for grammar and spelling.

## References

  * [1] Arens, Richard. Topologies for homeomorphism groups. Amer. J. Math. 68 (4) (1946), 593-610.
  * [2] Armacost, David L. _The structure of locally compact abelian groups_ , Marcel Dekker Inc., New York, 1981.
  * [3] Armacost D. L. and Armacost W. L. Uniqueness in structure theorems for lca groups. Can. J. Math. 30 (3) (1978), 593-599.
  * [4] Braconnier, J. Sur les groupes topologiques localement compacts, J. Math. Pures Appl. 27 (9) (1948), 1-85.
  * [5] Caprace, P. and Monod, N. Decomposing locally compact groups into simple pieces. Math. Proc. Cambridge Philos. Soc. 150 (2011), 97-128.
  * [6] Engelking, Ryszard. _General topology_ , Sigma Series in Pure Math., 6, Heldermann Verlag, Berlin, 2e, 1989. Translated from Polish by the author.
  * [7] Fox, Ralph H. On topologies for function spaces. Bull. Amer. Math. Soc. 51 (1945), 429-432.
  * [8] Gleason, A. M. and Palais, R. S. On a class of transformation groups. Amer. J. Math. 79 (3) (1957), 631-648.
  * [9] Hewitt, E. and Ross, K. A. _Abstract harmonic analysis Vol. I_. Springer, Berlin, 1963.
  * [10] Levin, Martin D. The automorphism group of a locally compact abelian group. Acta. Math. 127 (1971), 259-278.
  * [11] Lukács, Gábor. _Compact-like topological groups_. Research and Exposition in Math., 31, Heldermann Verlag, Berlin, 2009.
  * [12] Lee, D. H. and Wu, T.-S. The group of automorphisms of a finite-dimensional topological group. 15 (3) (1968), 321-324.
  * [13] Mac Lean, Saunders. _Categories for the working mathematician_. Graduate Texts in Mathematics. Springer, New York, 2e, 1998.
  * [14] Megrelishvili, M. and Shlossberg, M. Notes on non-archimedean topological groups. Topology Appl. To appear. (2011)
  * [15] Mel’nikov, O. V. Compactness conditions for groups of automorphisms of topological groups. Matematicheskie Zametki. 19 (5) (1976) 735-743.
  * [16] Moskalenko, Z. I. Automorphism groups of compact, totally disconnected, nilpotent groups. Ukrainskii Matematicheskii Zhurnal. 32 (1) (1980), 46 52.
  * [17] Stroppel, Markus. _Locally compact groups_. EMS Textbooks in Mathematics. European Mathematical Society, Zurich, 2006.
  * [18] Willard, Stephen. _General topology_. Addison-Wesley, Reading, Mass., 1970. (reprinted by Dover)