# Efficient Representation of the Activation Space in Deep Neural Networks

Tanya Akumu
IBM Research Africa
Nairobi, Kenya
&Celia Cintas
IBM Research Africa
Nairobi, Kenya
Girmaw Abebe Tadesse
Microsoft AI for Good Research Lab **footnotemark: *
Nairobi, Kenya
&Adebayo Oshingbesan
IBM Research Africa
Nairobi, Kenya
&Skyler Speakman
IBM Research Africa
Nairobi, Kenya
&Edward McFowland III
Harvard Business Institute
Masachusetts, USA

###### Abstract

The representations of the activation space of deep neural networks (DNNs) are
widely utilized for tasks like natural language processing, anomaly detection
and speech recognition. Due to the diverse nature of these tasks and the large
size of DNNs, an efficient and task-independent representation of activations
becomes crucial. Empirical $p$-values have been used to quantify the relative
strength of an observed node activation compared to activations created by
already-known inputs. Nonetheless, keeping raw data for these calculations
increases memory resource consumption and raises privacy concerns. To this
end, we propose a model-agnostic framework for creating representations of
activations in DNNs using node-specific histograms to compute $p$-values of
observed activations without retaining already-known inputs. Our proposed
approach demonstrates promising potential when validated with multiple network
architectures across various downstream tasks and compared with the kernel
density estimates and brute-force empirical baselines. In addition, the
framework reduces memory usage by 30% with up to 4 times faster $p$-value
computing time while maintaining state-of-the-art detection power in
downstream tasks such as the detection of adversarial attacks and synthesized
content. Moreover, as we do not persist raw data at inference time, we could
potentially reduce susceptibility to attacks and privacy issues.
11footnotetext: This work was done while he was at IBM Research Africa.

## Introduction

The ubiquity of applications that utilize deep neural networks (DNNs) has
brought the emergence of a variety of architectures and improvements in the
accuracy of deep learning tasks such as natural language processing, signal,
and image processing [1; 2; 3]. These DNNs learn representations, which are
abstractions of the data in the shape of feature vectors, that capture the
most relevant characteristics for the task at hand [4]. Investigating the
representations learned by DNNs is crucial for enhancing algorithms to improve
performance on downstream tasks, model interpretability, fairness, and robust
explanation [5; 6; 7; 8; 9]. However, working directly on the activations
derived from previous inputs pose a significant challenge in terms of memory
resource consumption and privacy concerns. Thus, efficient representations of
activations, without substantial memory or runtime overhead, is critical to
enable practical deployment in secure low-resource settings.

Existing works employed different strategies to derive these efficient
representations by considering them as data-generating systems applying
pattern detection techniques in the activation space of hidden layers. These
patterns provide insights into the behavior of the DNN, and its performance on
different tasks such as DNNs testing and interpretability [10; 11], fake
content detection [12], and anomalous pattern detection [13]. Some of these
approaches use empirical $p$-values to quantify the strength of observed node
activations compared to activations created by already-known background for
anomalous patterndetection [14; 15; 16]. This strategy cannot distinguish
between tied likelihoods and lower likelihoods, introducing a bias towards
higher $p$-values. Other approaches build representations with Bayesian-based
uncertainty estimation was also employed, followed with a kernel density
estimation in the activation space for adversarial attack detection [17].

Although these strategies have proven to be useful, we observe common
challenges across the existing approaches. For instance, they require
significant memory resources to handle high-dimensional inputs, and their
applicability is often limited to specific tasks, choice of network
architectures, and layers hindering their generalizability. Furthermore,
retaining all the information of the activation space even at inference time
poses privacy concerns and vulnerability to adversarial attacks [18; 19].
Consequently, characterizing the activation space of DNNs in a more efficient
and task-independent manner is imperative.

This work presents a framework for creating simple but effective
representations of activations in DNNs, which applies to any off-the-shelf
pre-trained neural network in a way that optimizes memory usage and run-time.
Further, it is designed to keep the minimal information about the activation
space as possible by using node-specific histograms. In summary, this paper
makes the following contributions.

  * •

We propose a task-independent and model-agnostic technique for obtaining
representation of activations in DNNs.

  * •

We employ node-specific histograms that minimizes the amount of information
needed to represent the activations, thereby achieving effective use of memory
usage and potentially reducing vulnerability of privacy and adversarial
concerns.

  * •

We compare our approach with two task-independent representation
characterization baselines and show empirically that we can obtain up to 30%
memory savings while maintaining state-of-the-art performances in various
downstream tasks [14; 16; 20].

  * •

We reduce the run-time of computing $p$-value ranges by up to 4 times compared
to existing methods. By employing $p$-value ranges instead of traditional
empirical $p$-values for downstream tasks, we may enhance the robustness of
the representations.

  * •

We validate our proposed approach using several pre-trained models of
different architectures (Generative Adversarial Networks, Graph models, and
Convolutional Neural Networks), and multiple datasets (WikiArt [21], ZINC [22;
23], MNIST [24]).

## Related Work

In the field of deep learning, the activation space of neural networks plays a
crucial role in determining their performance on various tasks. To better
understand this space, researchers have developed several methods for
characterizing the activation patterns of DNNs. In this section, we discuss
two categories of these methods: task-dependent characterization and task-
agnostic characterization of the activation space of DNNs. A summary of the
comparison of the key characteristics of these techniques is given in Table 1.

### Task-dependent characterization methods

[17] utilized the Bayesian-encoded uncertainty in dropout neural networks and
KDE on the subspace learned by the model for adversarial attack detection. In
another stream of work, testing and verification of DNNs were performed with
node coverage criteria. For example, DeepXplore [11] utilizes a pre-defined
threshold $\tau$ and computes the percentage of node activations greater than
$\tau$. DeepGauge [10] extends this criterion to consider the output value of
each neuron by using k-multi-section node coverage, which divides the output
range of each node activations into $k$ chunks. Similarly, FakeSpotter [12]
monitors the node behavior of a DNN using a threshold calculated from the
average values of activations in each layer, driven from training samples.
These studies mainly focus on creating representations that apply only to
their specific downstream task and model architectures, rendering their
approach non-generalizable.

Table 1: Summarization of existing methods in the state-of-the-art
representation characterization in deep neural networks compared to our
proposed approach.

Approach | Description | Activation representation | Compressed inputs | Model-agnostic | Multi-task
---|---|---|---|---|---
[12] | Average-based thresholding | ✓ |  |  |
[17] | Density estimates | ✓ |  | ✓ | ✓
[13] | Empirical $p$-values | ✓ |  | ✓ | ✓
Proposed | Node-specific histograms | ✓ | ✓ | ✓ | ✓

### Task-agnostic characterization methods

To overcome the limitations of task-dependent representation characterization,
most recent works [13; 14; 15; 16; 25] have employed empirical $p$-value based
strategies to characterize the representations of node activations in a DNN.
These works utilize subset scanning [26], which identifies subsets of neurons
with activation patterns that deviate from the expected distribution. The
empirical $p$-values are calculated to determine the likelihood of observing
such deviations by chance. These studies use empirical $p$-value based
representations to understand the features detected by the network and their
relationships without being limited to specific downstream tasks.

Although these techniques can be task-independent, they still require all the
information about the activation space to be known. This makes them memory
inefficient, slower to use at inference time, and more susceptible to privacy
concerns and adversarial attacks.

In this work, we propose a representation method that is agnostic to
downstream tasks and network architectures. Furthermore, our method compresses
the input required, therefore reducing the memory used to perform a downstream
task. This makes it faster and possibly less susceptible to adversarial and
privacy issues.

## Preliminaries

In characterizing node activations of a neural network, subset scanning [26]
uses $p$-values to assess the statistical significance of activation pattern
deviations in intermediate layers and to identify subsets of deviating
neurons. This section outlines current $p$-value calculation methods.

#### Empirical $p$-values estimation

To describe the estimation of empirical $p$-values, let $A^{\mathcal{B}}_{lj}$
be the matrix of activations in a node $j\in O_{l}$ of $l\in\mathcal{L}$ layer
of a DNN. These latent vectors are drawn from $\mathcal{B}$ background samples
containing $\mathcal{N_{B}}$ samples, at each of $O_{l}$ nodes in a pre-
trained neural network. Let $A^{\mathcal{T}_{k}}_{lj}$ be the matrix of
activations induced by $\mathcal{L}$ latent vectors drawn from the test set,
${\mathcal{T}_{k}}$. We compute an empirical $p$-value for each
$A^{\mathcal{T}_{k}}_{lj}$, as a measurement for how anomalous the activation
value of each $k^{th}$ sample in the test set, $\mathcal{T}_{k}$ , is at node
$j\in O_{l}$, following [27; 14]. This $p$-value, $p^{k}_{lj}$, is the
proportion of activations from the $\mathcal{B}$ background samples,
$A^{\mathcal{B}}_{lj}$, that are larger or equal to the activation from an
evaluation sample $A^{\mathcal{T}_{k}}_{lj}$ at node $j\in O_{l}$. The
empirical $p$-values are calculated as in Equation 1. This method treats tied
likelihoods the same as lower likelihood values, thus introducing a bias
towards higher $p$-values when ties are present. Our approach extends these
traditional empirical $p$-values to $p$-value ranges which accounts for ties
in the likelihoods, plausible making more robust representations.

$p^{k}_{lj}=\frac{1+\sum_{l=1}^{|\mathcal{L}|}I(A^{\mathcal{B}}_{lj}\geq
A^{\mathcal{T}_{k}}_{lj})}{|\mathcal{N_{B}}|+1}$ (1)

where $I(\cdot)$ is the indicator function. A shift is added to the numerator
and denominator so that a test activation that is larger than all activations
from the background at that node is given a non-zero $p$-value.

#### Kernel Density Estimate $p$-values

Another technique for computing $p$-value based representation is using kernel
density estimates (KDE) [17; 28]. This technique estimates the unknown
probability density function (PDF) of a given sample [29] and computes
$p$-values for each $k^{th}$ sample in the test set, $\mathcal{T}_{k}$, as
shown in Equation 2.

$p^{k}_{lj}=\begin{cases}\int_{A^{\mathcal{T}_{k}}_{lj}}^{\Omega_{j}}\phi_{lj}\,dk&\text{for
}A^{\mathcal{T}_{k}}_{lj}<\Omega_{j},\\\
\vspace{0.25pt}\frac{1}{\mathcal{N_{B}}+1},&\text{for
}A^{\mathcal{T}_{k}}_{lj}\geq\Omega_{j}\\\ \end{cases}$ (2)

where $\Omega_{j}$ is the maximum observed activation in the background data
at node $j\in O_{l}$, $\phi_{lj}$ is the PDF and $\mathcal{N_{B}}$ is the
total number of samples in the background data.

## Proposed Approach

In this section, we describe the problem formulation, and the details of the
proposed framework (see Fig. 1). The core elements of our method are Node-
specific histograms, $p$-values Ranges Computation, and Significance testing.

Given a pre-trained DNN $M$, consisting of a set of intermediate layers,
$\mathcal{L}$, with each layer $l\in\mathcal{L}$ consisting of $O_{l}$ nodes,
we aim to characterize the activation space of $M$ efficiently and in a
generalizable way. We propose a method that employs node-specific histograms,
$\Theta_{lj}$, to represent the activation space $A^{\mathcal{B}}$ of the pre-
trained model $M$ via efficiently driven $p$-values.

There are three steps to generate the new $p$-value representation, $p^{k}$,
of the activations drawn from the nodes of the layers of model $M$. First, $M$
is trained on a set of background samples, $\mathcal{B}$, containing
$\mathcal{N_{B}}$ samples. The corresponding activations,
$A^{\mathcal{B}}_{lj}$, are drawn at each $j\in O_{l}$ node of the
$l\in\mathcal{L}$ layer in $M$. We create node-specific histograms,
$\Theta_{lj}$ in the form of bin edges and counts, for each node in the
background activations, $A^{\mathcal{B}}_{lj}$. Second, given a set of test
samples, $\mathcal{T}$, containing $N_{\mathcal{T}}$ samples, we extract test
activations, $A^{\mathcal{T}_{k}}_{lj}$, from the pre-trained model passed
with the test samples. This is done for each $k^{th}$ sample in $\mathcal{T}$,
that will be utilized to compute a $p$-values $p^{k}_{lj}$, using the
histogram bins edges and counts in $\Theta_{lj}$ and the test activations
$A^{\mathcal{T}_{k}}_{lj}$. Finally, these $p$-values are utilized for
downstream tasks.

(a)

(b)

Figure 1: Overview of our proposed framework for activation representation of
DNNs. (a) First, we extract background activations from the hidden layers of a
pre-trained model for three downstream tasks [16; 20; 14]. (b) Second, we
create node-specific histograms consisting of bins and counts for each node in
the network and use them to compute the $p$-values ranges of activations to be
evaluated. Lastly, we use the $KS$-test to evaluate the $p$-value
distributions.

##### Node-specific histograms

Once the activations drawn at each $j\in O_{l}$ node of the $l\in\mathcal{L}$
layer in $M$ are obtained, we create node-specific histograms for each node.
We implement a histogram,$\Theta_{lj}$, for a node $j\in O_{l}$ described by
the number of bins, $b$, and the height of the bins. This height represents
the count of activations that fall into a specific bin. Since modal
activations can cause spikes that will affect the number of bins, we only
build histograms for activations outside the mode and assign a unique bin to
represent modal activations. For activations outside the modal, we assign the
number of bins as a maximum of the Freedman Diaconis estimator [30], and the
Sturges estimator [31] which computes the optimal bin width and consequently,
the number of bins per node. We choose these estimators because they take into
account the data size and variability in the data. The Freedman Diaconis bin-
width estimation is given by:

$h=2\frac{IQR}{\sqrt[3]{n}}$ (3)

where IQR is the interquartile range, and $n$ is the size of the data.

The number of bins estimated by Sturges method is given by:

$n_{h}=\log_{2}(n)+1$ (4)

As a result, the bin-width of the histograms is non-uniform. We describe these
histograms in the form of bin edges which tell us the number of bins as well
as the boundaries for each bin and the height/count of each bin. For a test
activation $A^{\mathcal{T}_{k}}_{lj}$, we find the bin to which it belongs by
searching the bin edges of the node’s histogram. Then we use the height of the
obtained bin to compute its $p$-value as described in the next paragraph.

#####  $\mathbf{p}$-values ranges computation

In the Preliminaries section, we saw that traditional empirical $p$-values do
not take into account ties in the likelihood leading to a bias for larger
$p$-values. Our approach defines the $p$-value as a range between a maximum
and a minimum to account for these ties. These ranges are made wider by using
histogram summaries of the background distributions, likely making the
representations more robust. We compute $p$-values for each activation
$A^{\mathcal{T}_{k}}_{lj}$, as a measure of how anomalous the activation value
at a node $j\in O_{l}$ of $l\in\mathcal{L}$ layer is. This $p$-value,
$p^{k}_{lj}$, is the proportion of activations from the background input,
$A^{\mathcal{B}}_{lj}$, that are greater than or equal to the observed
activation, $A^{\mathcal{T}_{k}}_{lj}$. Given a histogram distribution,
$\Theta_{lj}$ for a given node, we obtain the $p$-values using the height of
the bins from the histogram (counts). This means that the $p$-values will
correspond to their respective bins, and thus test activations falling into
the same bin will have the same $p$-value. Formally, our $p-$value ranges
computation is defined as follows:

$\displaystyle p^{k}_{lj}=[P_{min}(p^{k}_{lj}),P_{max}(p^{k}_{lj})]$ (5)

where:

$P_{min}(p^{k}_{lj})=\frac{\mathcal{N_{B}}-\sum_{i=b_{kj+1}}^{b_{nj}}{C_{ij}}}{\mathcal{N_{B}}+1}$
(6)

and,

$P_{max}(p^{k}_{lj})=\frac{\mathcal{N_{B}}-\sum_{i=b_{kj}}^{b_{nj}}{C_{ij}+1}}{\mathcal{N_{B}}+1}$
(7)

where $b_{kj}$ represents the bin of the $k^{th}$ test activation of node
$j\in O_{l}$, $b_{nj}$ is the $n^{th}$ bin of the histogram of node $j\in
O_{l}$ with $n$ bins and $C_{ij}$ is the count/height of the $i^{th}$ bin.

##### Comparing learned representations

We compare the distribution of node-specific histogram $p$-values
representation with the empirical $p$-values representation using goodness-of-
fit statistics. Specifically, we use the two-sample Kolmogorov-Smirnov test
[32] to ascertain that the distributions of the empirical and node-specific
histograms $p$-value representations are similar. To compare the $p$-value
distributions of the baselines, we draw a $p$-value uniformly at random from
each range $[P_{min}(p^{k}_{lj}),P_{max}(p^{k}_{lj})]$ and apply the test to
the drawn values.

Model Architecture (Task) | Detection Power | Subset Scores Distribution
---|---|---
ArtGAN (Creative artifacts) |  |
GCPN (Invalid graph molecules) |  |
CNN (Adversarial attacks) |  |
Figure 2: Comparison of Detection Power for three different downstream tasks [16; 20; 14] using our proposed node-specific histogram $p$-value representation and the Empirical $p$-values baseline [26; 33; 34].  Table 2: Memory usage measured in megabytes (MB) and learning representation run-time measured in seconds (s) of the different baselines for the ArtGAN using the ArtWiki Dataset [21]. $(^{\ast})$Note that the empirical baseline does not learn any representation, but rather does a single sort of background activations when comparing the incoming test activations.  | Memory Usage (MB) | Learning representation run-time (s) $\pm$std
---|---|---
# Nodes ($n$) | Empirical | Proposed | KDE | Empirical$(^{\ast})$ | Proposed | KDE
1000 | 668.9 | 484.0 | 705.3 | N/A | 1.04s $\pm$ 1.12 ms | 88s $\pm$344ms
20000 | 767.7 | 535.4 | 1100 | N/A | 20.7s $\pm$ 41.4 ms | 2067s $\pm$78s
$>$30000 | 834.1 | 569.8 | 1400 | N/A | 33.8s $\pm$ 24ms | 2371s $\pm$35s
Table 3: Execution run-time measured in seconds (s) $\pm$ standard deviation (std) for the $p$-value computation by the different baselines for the ArtGAN DNN using the ArtWiki Dataset [21]. The sample size is reported as a % of the total samples in the background data.  |  $p$-value computation run-time (s) $\pm$ std
---|---
% Background samples | Empirical | Proposed | KDE
20 | 0.024s $\pm$115$\mu$s | 0.009s $\pm$10.4$\mu$s | 32.9s $\pm$122ms
50 | 0.029s $\pm$49$\mu$s | 0.009s $\pm$67.7$\mu$s | 38.4s $\pm$65.7ms
100 | 0.033s $\pm$112$\mu$s | 0.009s $\pm$13$\mu$s | 44.2s $\pm$94.6ms

Model Architecture | Empirical | Proposed
---|---|---
ArtGAN |  |
|  |  $\overline{KS}=0.06$, $p-value=0.97$
GCPN |  |
|  |  $\overline{KS}=0.05$ , $p-value=0.51$
CNN |  |
|  |  $\overline{KS}=0.04$ , $p-value=0.7$

Figure 3: Ridgeplots comparing the empircal baselines with our proposed node-
specific histogram $p$-values representation for the different DNNs and
downstream tasks of creative [16] (ArtGAN), invalid graph [20] (GCPN) and
adversarial detection [14] tasks (CNN). We can observe a visual similarity of
distribution plots of the $p$-value representations of the node-specific
histograms compared with the empirical baseline. This observed similarity is
supported by the lower average $\overline{KS}$ statistic and the average
$\overline{KS}$ $p$-value noted under each plot for the node-specific
histograms.

## Experimental Setup

This section provides a detailed description of the experimental setup
including the datasets used, the tasks performed, the histogram hyperparameter
settings, and the evaluation metrics employed to assess the performance of our
proposed approach. We evaluated the performance of our proposed method under
different downstream tasks and compared it against the empirical [26; 27; 34]
and KDE [17; 28] baselines.

##### Datasets & tasks

To demonstrate the generalizable and efficiency properties of the proposed
framework, we evaluated the node-specific histogram $p$-value representations
under multiple tasks and datasets that we will define below.

First, we look at the task of detecting creative artifacts [16], where a
Creative Generator variant of ArtGAN was used [35] for the WikiArt dataset
[21]. The dataset contains paintings from $195$ different artists, and the
dataset has $42129$ RGB images for training and $10628$ images for testing.
Second, we analyze the task of detecting invalid graph representation for
molecule generation, using a Graph Convolutional Policy Network (GCPN) [20] as
a base model and the ZINC dataset [22; 23]. This dataset contains $727842$
SMILES molecules. Lastly, we test our representation approach for detecting
adversarial attacks in classical Convolutional Neural Networks (CNN) [14] and
Fast Gradient Signal Method (FGSM) attacks [36] on the MNIST dataset [24].
MNIST contains $60000$ images, and $10000$ images were extracted for this
task. We selected the layer that was used in each of the cited papers as the
more discriminative layer for a given task.

##### Extracting activations from DNNs

Our approach requires constructing the histogram, $\Theta_{lj}$, of the
distribution of expected activations at each node of the layer, $\mathcal{L}$,
of the pre-trained DNN, $M$. We extract the expected activation distribution
by passing forward the training data through $\mathcal{L}$ of the model $M$
and record the activations at each node. We create three sets of activation
distributions. Namely, the background $A^{\mathcal{B}}_{lj}$ which is obtained
by training $M$ on the background samples,$\mathcal{B}$, and clean and
anomalous activations,$A^{\mathcal{T}_{k}}_{lj}$, which are obtained by
training $M$ on test samples,$\mathcal{T}$. We assume that clean activations
are drawn from the same distribution as the background samples, whereas the
anomalous activations are from a noised sample of the training data passed
through $M$. For the creative artifacts and adversarial attack detection
tasks, we used $1000$ nodes in the activation space, whereas the invalid graph
representation detection task had $256$ nodes.

##### Histograms hyper-parameter setup

Once we obtain the background activation distributions $A^{\mathcal{B}}_{lj}$,
we create histograms for each node $j\in O_{l}$ of the selected discriminative
layer. We remove modal activations that are greater than 10% of the data, and
manually add a unique bin for these groups of activations. We create
histograms for activations outside the modal value with automatic binning,
setting the maximum bin size to $10$.

##### Performance evaluation

The evaluation metrics used is the area under the receiver operating
characteristic curve (AUROC) for detection power. To measure the anomalousness
of a subset of node activations in a test sample, we use subset scanning [37;
33]. This technique searches for the most anomalous subset of nodes within the
inner layers of a DNN and calculates an anomalousness score for this subset,
referred to as the ’subset score’, using a scoring function. For significance
testing, we used the two-sample Kolmogorov-Smirnov-test, $\overline{KS}$, [32]
for estimating the distribution similarity of the $p$-value representations
obtained by all baseline methods. Specifically, we average the KS-Statistic
$\overline{KS}$ and $p$-value obtained from the test across all the nodes.

We also conducted benchmarks to profile the memory usage and run-time of our
proposed approach and compared it to the other baseline methods. We profile
the execution run-time it takes to learn representations of the activation
space offline, and the run-time for computing the $p$-value after obtaining
these representations. These profiling tests were performed in a desktop
machine (2.6 GHz 6-Core Intel Core i7, 32 GB 2400 MHz DDR4) under Linux
4.15.0-139-generic operating system.

We conducted experiments to analyze the memory and execution time of the
learning representation, we passed $500$ background samples through the
pretrained ArtGAN while varying the number of nodes to $1000$, $20000$, and
$32768$ (the total number of nodes in the selected layer) to obtain memory
performance profiles.

## Results

##### Memory profile & run-time benchmark

In Table 2, we can see the memory profile of the baseline methods, and run-
time for learning representations for the KDE [17; 28] baseline and node-
specific histograms approach for creating $p$-value representations of the
ArtGAN. As observed, our method shows the best memory optimization when
compared to the other methods. Our proposed approach reduces the memory used
by about $30\%$, with a lower rise in memory usage as the number of nodes
increases as compared to the other baselines. In terms of the learning
representations run-time, the proposed method has a run-time of 33.8s, which
is a huge improvement from the KDE baseline run-time of 2371s. Furthermore, in
Table 3, we see the proposed framework is not only four times as fast as the
empirical method in computing the $p$-value representations, but also constant
regardless of the number of samples in the background activations.

##### Downstream task performance

Figure 2 reports the detection power through the AUROC score for the
downstream tasks using the node-specific histogram $p$-values representation
and the empirical $p$-values representation. We note that the detection power
of our method’s node-specific histograms $p$-values maintains the state-of-
the-art results obtained by using the empirical $p$-values representation. The
subset scores obtained from the scanning also show that the two baselines are
comparable, indicating that the node-specific histogram approach is as good as
the empirical baseline.

Lastly, in Figure 3, we show a visual of the $p$-values distribution created
by the empirical [26; 27; 34], KDE [17; 28], and our proposed node-specific
histogram method in the form of ridge plots for ten nodes of three DNNs for
each downstream task. We observe that for the downstream tasks of creative
artifacts [16], adversarial [14] and invalid graph representation detection
[20], the distributions of $p$-value representations created by the node-
specific histogram approach are visually similar to those created by the
empirical baseline. This is further supported by the low average
$\overline{KS}$-test statistic of $0.06$, $0.05$ , $0.04$ for the DNNs in the
proposed baseline. This $\overline{KS}$-test statistic indicates the maximum
absolute difference between the distribution functions of the samples under
test. Moreover, assuming a confidence level of 95%, we see that the average
$p$-value obtained from the $\overline{KS}$-test of $0.97$, $0.51$, and $0.7$
for the ArtGAN, GCPN, and CNN respectively, are not below the threshold of
$0.05$. Therefore, we cannot reject the null hypothesis that the empirical
$p$-value representations distribution and the node-specific histogram
$p$-value representations distribution are similar.

## Conclusion & Future Work

We proposed a framework for creating representations of activations in DNNs
based on node-specific histograms. We evaluate this approach by comparing it
with two other baseline methods, namely, the empirical $p$-value
representation [26; 27; 34]and the Kernel Density Estimate $p$-value
representation [17; 28], and validate it across different downstream tasks and
datasets from multiple domains (chemistry, computational creativity, and
adversarial machine learning). The results show that our proposed approach is
memory efficient and reduces the $p$-value computing time, while maintaining
state-of-the-art performance in downstream tasks. Moreover, our approach
extends the empirical $p$-values to $p$-value ranges which account for ties in
likelihoods, thereby eliminating biases for higher $p$-values which could lead
to more robust representations. Additionally, the use of representations of
the activation space with node-specific histograms poses a potential advantage
in privacy since the original data and full activations do not need to be
shared. Although encouraging results in efficiency, the proposed approach
faces some limitations. Currently, the proposed framework creates node-
specific histograms for each node. This means that the run-time at inference
increases linearly with the number of nodes. Future work would explore
building histograms across subgroups of nodes across layers of DNN instead of
individual nodes and using them to compute the $p$-value representations as
well as other possible optimization strategies to further improve efficiency.

## References

  * Touvron et al. [2023] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al.  Llama 2: Open foundation and fine-tuned chat models.  _arXiv e-prints_ , pages arXiv–2307, 2023.
  * Huang et al. [2022] R Huang, MWY Lam, J Wang, D Su, D Yu, Y Ren, and Z Zhao.  Fastdiff: A fast conditional diffusion model for high-quality speech synthesis.  In _IJCAI International Joint Conference on Artificial Intelligence_ , pages 4157–4163. IJCAI: International Joint Conferences on Artificial Intelligence Organization, 2022.
  * Ramesh et al. [2022] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen.  Hierarchical text-conditional image generation with clip latents.  _arXiv preprint arXiv:2204.06125_ , 2022.
  * Bengio et al. [2013] Yoshua Bengio, Aaron Courville, and Pascal Vincent.  Representation learning: A review and new perspectives.  _IEEE transactions on pattern analysis and machine intelligence_ , 35(8):1798–1828, 2013.
  * Samek et al. [2016] Wojciech Samek, Alexander Binder, Grégoire Montavon, Sebastian Lapuschkin, and Klaus-Robert Müller.  Evaluating the visualization of what a deep neural network has learned.  _IEEE transactions on neural networks and learning systems_ , 28(11):2660–2673, 2016.
  * Montavon et al. [2018] Grégoire Montavon, Wojciech Samek, and Klaus-Robert Müller.  Methods for interpreting and understanding deep neural networks.  _Digital signal processing_ , 73:1–15, 2018.
  * Samek et al. [2017] Wojciech Samek, Thomas Wiegand, and Klaus-Robert Müller.  Explainable artificial intelligence: Understanding, visualizing and interpreting deep learning models.  _arXiv e-prints_ , pages arXiv–1708, 2017.
  * Liang et al. [2022] Weixin Liang, Girmaw Abebe Tadesse, Daniel Ho, L Fei-Fei, Matei Zaharia, Ce Zhang, and James Zou.  Advances, challenges and opportunities in creating data for trustworthy ai.  _Nature Machine Intelligence_ , 4(8):669–677, 2022.
  * Li et al. [2022a] Xuhong Li, Haoyi Xiong, Xingjian Li, Xuanyu Wu, Xiao Zhang, Ji Liu, Jiang Bian, and Dejing Dou.  Interpretable deep learning: Interpretation, interpretability, trustworthiness, and beyond.  _Knowledge and Information Systems_ , 64(12):3197–3234, 2022a.
  * Ma et al. [2018] Lei Ma, Felix Juefei-Xu, Fuyuan Zhang, Jiyuan Sun, Minhui Xue, Bo Li, Chunyang Chen, Ting Su, Li Li, Yang Liu, et al.  Deepgauge: Multi-granularity testing criteria for deep learning systems.  In _Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering_ , pages 120–131, 2018.
  * Pei et al. [2017] Kexin Pei, Yinzhi Cao, Junfeng Yang, and Suman Jana.  Deepxplore: Automated whitebox testing of deep learning systems.  In _proceedings of the 26th Symposium on Operating Systems Principles_ , pages 1–18, 2017.
  * Wang et al. [2020] Run Wang, Felix Juefei-Xu, Lei Ma, Xiaofei Xie, Yihao Huang, Jian Wang, and Yang Liu.  Fakespotter: A simple yet robust baseline for spotting ai-synthesized fake faces.  In Christian Bessiere, editor, _Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20_ , pages 3444–3451. International Joint Conferences on Artificial Intelligence Organization, 7 2020.  doi: 10.24963/ijcai.2020/476.  URL https://doi.org/10.24963/ijcai.2020/476.  Main track.
  * Cintas et al. [2022a] Celia Cintas, Skyler Speakman, Girmaw Abebe Tadesse, Victor Akinwande, Edward McFowland III, and Komminist Weldemariam.  Pattern detection in the activation space for identifying synthesized content.  _Pattern Recognition Letters_ , 153:207–213, 2022a.
  * Akinwande et al. [2020] Victor Akinwande, Celia Cintas, Skyler Speakman, and Srihari Sridharan.  Identifying audio adversarial examples via anomalous pattern detection.  In _Workshop on Adversarial Learning Methods for Machine Learning and Data Mining, KDD’20_ , 2020.
  * Kim et al. [2022] Hannah Kim, Girmaw Abebe Tadesse, Celia Cintas, Skyler Speakman, and Kush Varshney.  Out-of-distribution detection in dermatology using input perturbation and subset scanning.  In _2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI)_ , pages 1–4. IEEE, 2022.
  * Cintas et al. [2022b] Celia Cintas, Payel Das, Girmaw Abebe Tadesse, Brian Quanz, Skyler Speakman, and Pin-Yu Chen.  Towards creativity characterization of generative models via group-based subset scanning.  In _Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence (IJCAI-22) Special Track on AI, the Arts and Creativity_ , pages 4929–4935, 2022b.
  * Feinman et al. [2017] Reuben Feinman, Ryan R Curtin, Saurabh Shintre, and Andrew B Gardner.  Detecting adversarial samples from artifacts.  _arXiv e-prints_ , pages arXiv–1703, 2017.
  * Akhtar and Mian [2018] Naveed Akhtar and Ajmal Mian.  Threat of adversarial attacks on deep learning in computer vision: A survey.  _Ieee Access_ , 6:14410–14430, 2018.
  * Li et al. [2022b] Huiying Li, Shawn Shan, Emily Wenger, Jiayun Zhang, Haitao Zheng, and Ben Y Zhao.  Blacklight: Scalable defense for neural networks against $\\{$Query-Based$\\}$$\\{$Black-Box$\\}$ attacks.  In _31st USENIX Security Symposium (USENIX Security 22)_ , pages 2117–2134, 2022b.
  * You et al. [2018] Jiaxuan You, Bowen Liu, Zhitao Ying, Vijay Pande, and Jure Leskovec.  Graph convolutional policy network for goal-directed molecular graph generation.  _Advances in neural information processing systems_ , 31, 2018.
  * Tan et al. [2018] Wei Ren Tan, Chee Seng Chan, Hernan E Aguirre, and Kiyoshi Tanaka.  Improved artgan for conditional synthesis of natural image and artwork.  _IEEE Transactions on Image Processing_ , 28(1):394–409, 2018.
  * Irwin and Shoichet [2005] John J Irwin and Brian K Shoichet.  Zinc- a free database of commercially available compounds for virtual screening.  _Journal of chemical information and modeling_ , 45(1):177–182, 2005.
  * Sterling and Irwin [2015] Teague Sterling and John J Irwin.  Zinc 15–ligand discovery for everyone.  _Journal of chemical information and modeling_ , 55(11):2324–2337, 2015.
  * LeCun [1998] Yann LeCun.  The mnist database of handwritten digits.  _http://yann. lecun. com/exdb/mnist/_ , 1998.
  * Kim et al. [2023] Hannah H Kim, Celia Cintas, Girmaw Abebe Tadesse, and Skyler Speakman.  Spatially constrained adversarial attack detection and localization in the representation space of optical flow networks.  _IJCAI_ , 2023.
  * McFowland III et al. [2013] Edward McFowland III, Skyler D Speakman, and Daniel B Neill.  Fast generalized subset scan for anomalous pattern detection.  _The Journal of Machine Learning Research_ , 14(1):1533–1561, Jun 2013.
  * Cintas et al. [2020] Celia Cintas, Skyler Speakman, Victor Akinwande, William Ogallo, Komminist Weldemariam, Srihari Sridharan, and Edward McFowland.  Detecting adversarial attacks via subset scanning of autoencoder activations and reconstruction error.  In _IJCAI 2020_ , 2020.
  * Silverman [1981] Bernard W Silverman.  Using kernel density estimates to investigate multimodality.  _Journal of the Royal Statistical Society: Series B (Methodological)_ , 43(1):97–99, 1981.
  * Parzen [1962] Emanuel Parzen.  On estimation of a probability density function and mode.  _The annals of mathematical statistics_ , 33(3):1065–1076, 1962.
  * Freedman and Diaconis [1981] David Freedman and Persi Diaconis.  On the histogram as a density estimator: L 2 theory.  _Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete_ , 57(4):453–476, 1981.
  * Scott [2009] David W Scott.  Sturges’ rule.  _Wiley Interdisciplinary Reviews: Computational Statistics_ , 1(3):303–306, 2009.
  * Massey Jr [1951] Frank J Massey Jr.  The kolmogorov-smirnov test for goodness of fit.  _Journal of the American statistical Association_ , 46(253):68–78, 1951.
  * Speakman et al. [2018] Skyler Speakman, Srihari Sridharan, Sekou Remy, Komminist Weldemariam, and Edward McFowland.  Subset scanning over neural network activations.  _arXiv preprint arXiv:1810.08676_ , 2018.
  * Chen and Neill [2014] Feng Chen and Daniel B. Neill.  Non-parametric scan statistics for event detection and forecasting in heterogeneous social media graphs.  In _KDD ’14_ , pages 1166–1175, 2014.
  * Das et al. [2021] Payel Das, Brian Quanz, Pin-Yu Chen, Jae Wook Ahn, and Dhruv Shah.  Toward a neuro-inspired creative decoder.  In _International Joint Conference on Artificial Intelligence_. International Joint Conferences on Artificial Intelligence, 2021.
  * Goodfellow et al. [2015] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy.  Explaining and harnessing adversarial examples.  _stat_ , 1050:20, 2015.
  * Speakman et al. [2016] Skyler Speakman, Sriram Somanchi, Edward McFowland III, and Daniel B. Neill.  Penalized fast subset scanning.  _Journal of Computational and Graphical Statistics_ , 25(2):382–404, 2016.  doi: 10.1080/10618600.2015.1029578.  URL https://doi.org/10.1080/10618600.2015.1029578.


Leibniz algebras in which all centralisers of nonzero elements are zero
algebras

David A. Towers

Department of Mathematics, Lancaster University

Lancaster LA1 4YF, England

###### Abstract

This paper is concerned with generalising the results for Lie $CT$-algebras to
Leibniz algebras. In some cases our results give a generalisation even for the
case of a Lie algebra. Results on $A$-algebras are used to show every Leibniz
$CT$-algebra over an algebraically closed field of characteristic different
from 2,3 is solvable or is isomorphic to $sl_{2}(F)$. A characterisation is
then given for solvable Leibniz $CT$-algebras. It is also shown that the class
of solvable Leibniz $CT$-algebras is factor closed.

Mathematics Subject Classification 2020: 17A32, 17B05, 17B20, 17B30, 17B50.
Key Words and Phrases: Leibniz algebra, centraliser, $A$-algebra,
$CT$-algebra, solvable, completely solvable.

##  1 Introduction

An algebra $L$ over a field $F$ is called a Leibniz algebra if, for every
$x,y,z\in L$, we have

$[x,[y,z]]=[[x,y],z]-[[x,z],y].$

In other words, the right multiplication operator $R_{x}:L\rightarrow
L:y\mapsto[y,x]$ is a derivation of $L$. As a result such algebras are
sometimes called right Leibniz algebras, and there is a corresponding notion
of left Leibniz algebras, which satisfy

$[x,[y,z]]=[[x,y],z]+[y,[x,z]].$

Clearly, the opposite of a right (left) Leibniz algebra is a left (right)
Leibniz algebra, so, in most situations, it does not matter which definition
we use.

Every Lie algebra is a Leibniz algebra and every Leibniz algebra satisfying
$[x,x]=0$ for every element is a Lie algebra. They were introduced in 1965 by
Bloh ([3]) who called them $D$-algebras, though they attracted more widespread
interest, and acquired their current name, through work by Loday and
Pirashvili ([8], [9]). They have natural connections to a variety of areas,
including algebraic $K$-theory, classical algebraic topology, differential
geometry, homological algebra, loop spaces, noncommutative geometry and
physics. A number of structural results have been obtained as analogues of
corresponding results in Lie algebras.

The Leibniz kernel is the set $I=$ span$\\{x^{2}:x\in L\\}$. Then $I$ is the
smallest ideal of $L$ such that $L/I$ is a Lie algebra. Also $[L,I]=0$.

We define the following series:

$L^{1}=L,\;\;L^{k+1}=[L^{k},L]\;\;\;(k\geq 1)$

and

$L^{(0)}=L,\;\;L^{(k+1)}=[L^{(k)},L^{(k)}]\;\;\;(k\geq 0).$

Then $L$ is nilpotent of class n (resp. solvable of derived length n) if
$L^{n+1}=0$ but $L^{n}\neq 0$ (resp. $L^{(n)}=0$ but $L^{(n-1)}\neq 0$) for
some $n\in{\mathbb{N}}$. It is straightforward to check that $L$ is nilpotent
of class $n$ precisely when every product of $n+1$ elements of $L$ is zero,
but some product of $n$ elements is non-zero.

The centraliser of an element $x$ of an algebra $A$ is $C_{A}(x)=\\{a\in A\mid
ax=xa=0\\}$; if the algebra $A$ is clear we will simply write $C(x)$. An
algebra $A$ is called centraliser transitive, or a $CT$-algebra if $x\in C(y)$
and $y\in C(z)$ imply that $x\in C(z)$. Such algebras, where $A$ is a Lie
algebra, have been studied by Klep and Moravec ([7]) in 2010, by Arzhantsev,
Makedonskii, and Petravchuk ([7]) in 2011, and by Gorbatsevic ([5]) in 2016.
In the last two of these references they are shown to have applications to the
classification of finite-dimensional subalgebras in polynomial Lie algebras of
rank one and to Lie algebras of vector fields whose orbits are one-
dimensional. A similar notion for groups was defined and studied by Weisner
[13]) in 1925. Finite nonabelian simple $CT$-groups had been classified by
Suzuki ([11]) in 1957. He proved that every finite nonabelian simple
$CT$-group is isomorphic to some $PSL(2,2^{f})$, where $f>1$. Suzuki’s result
is considered to have been one of the keystones in the proof of the Odd Order
Theorem by Feit and Thompson ([4]).

Note that $x\in C(y)$ if and only if $y\in C(x)$. It is clear that
$CT$-algebras are subalgebra closed. The centre of an algebra $A$ is
$Z(A)=\\{z\in A\mid za=az=0\hbox{ for all }a\in A\\}$. We call $A$ a zero
algebra if $A=Z(A)$. The following lemma makes clear the title of the paper.

###### Lemma 1.1

The algebra $A$ is a $CT$-algebra if and only if $C(x)$ is a zero algebra for
all $0\neq x\in L$.

Proof. Let $A$ be a $CT$-algebra and let $y,z\in C(x)$. Then $y\in C(x)$ and
$x\in C(z)$, so $y\in C(z)$; that is $yz=zy=0$.

Conversely, suppose that $C(x)$ is a zero algebra for all $x\neq 0$, and let
$x\in C(y)$ and $y\in C(z)$. Then $x,z\in C(y)$, so $xz=zx=0$, whence $x\in
C(z)$. $\Box$

###### Lemma 1.2

If $A$ is a $CT$-algebra and $Z(A)\neq 0$, then $A$ is a zero algebra.

Proof. Let $0\neq x\in Z(A)$. Then $A=C(x)$ is a zero algebra. $\Box$

We call an algebra $A$ an $A$-algebra if every nilpotent subalgebra of $A$ is
a zero algebra. Then $CT$-algebras are a subclass of the class of
$A$-algebras.

###### Lemma 1.3

Every $CT$-algebra is an $A$-algebra.

Proof. Let $N$ be a nilpotent $CT$-algebra. Then $Z(N)\neq 0$, so the result
follows from Lemma 1.2.. $\Box$

The next section is concerned with generalising the results for Lie
$CT$-algebras to Leibniz algebras. In some cases our results give a
generalisation even for the case of a Lie algebra. Leibniz $A$-algebras were
studied by Towers in [12]. The results proved there are used to show every
Leibniz $CT$-algebra over an algebraically closed field of characteristic
different from 2,3 is solvable or is isomorphic to $sl_{2}(F)$; the previously
recorded result for Lie algebras asumed characteristic zero. A
characterisation is then given for solvable Leibniz $CT$-algebras. It is also
shown that the class of solvable Leibniz $CT$-algebras is factor closed.

From now on, $L$ will denote a finite-dimensional Leibniz algebra over a
general field $F$ (unless specified otherwise). The notation $\oplus$ will
denote an algebra direct sum, whereas $\dot{+}$ will denote a direct sum of
the underlying vector space structure alone.

##  2 Main results

The following lemma will prove useful.

###### Lemma 2.1

Let $L=A\dot{+}Fx$ be a Leibniz $CT$-algebra over a field $F$, where $A$ is a
zero ideal of $L$ and $x^{2}\in A$. Then, either $L$ is a zero algebra or
$R_{x}|_{A}$ has no zero eigenvalue in $F$ and $Ax=A$.

Proof. Suppose that $R_{x}|_{A}$ has a zero eigenvalue in $F$. Let $0\neq y\in
A$ be a corresponding eigenvector, so $yx=0$. If $xy=0$, we have that $L=C(y)$
is a zero algebra. So suppose that $xy\neq 0$. Then

$(xy)x=x(yx)+x^{2}y=0\hbox{ and }x(xy)=x^{2}y-(xy)x=0.$

Thus, $L=C(xy)$ is a zero algebra again.

Suppose next that $R_{x}|_{A}$ has no zero eigenvalue in $F$. Let
$L=L_{0}\dot{+}L_{1}$ be the Fitting decomposition of $L$ relative to $R_{x}$.
Then $L_{1}\subseteq A$. Suppose that $B=L_{0}\cap A\neq 0$ and that
$R_{x}^{n}(B)=0$, $R_{x}^{n-1}(B)\neq 0$. Let $0\neq y\in R_{x}^{n-1}(B)$.
Then $yx=0$ and $R_{x}|_{A}$ has a zero eigenvalue in $F$, a contradiction. It
follows that $B=0$ and so $A=L_{1}$ and $Ax=A$. $\Box$

Cyclic Leibniz algebras, $L$, are generated by a single element. In this case
$L$ has a basis $a,a^{2},\ldots,a^{n}(n>1)$ and product
$a^{n}a=\alpha_{2}a^{2}+\ldots+\alpha_{n}a^{n}$. Let $T$ be the matrix for
$R_{a}$ with respect to the above basis. Then $T$ is the companion matrix for
$p(x)=x^{n}-\alpha_{n}x^{n-1}-\ldots-\alpha_{2}x=p_{1}(x)^{n_{1}}\ldots
p_{r}(x)^{n_{r}}$, where the $p_{j}$ are the distinct irreducible factors of
$p(x)$. Then we have the following result.

###### Theorem 2.2

Let $L$ be a cyclic Leibniz algebra. Then the following are equivalent:

  * (i)

$L$ is a $CT$-algebra;

  * (ii)

$L$ is an $A$-algebra; and

  * (iii)

$\alpha_{2}\neq 0$, and then
$L=L^{2}\dot{+}F(a^{n}-\alpha_{n}a^{n-1}-\dots-\alpha_{2}a)$ and we can take
$p_{1}(x)^{n_{1}}=x$.

Proof. The equivalence of (ii) and (iii) is given by [12, Theorem 12]. Lemma
1.3 gives that (i) implies (ii), so it simply remains to show that (iii)
implies (i).

So suppose that $L$ is as described in (iii). Put
$b=a^{n}-\alpha_{n}a^{n-1}-\dots-\alpha_{2}a$. Then it is easy to check that
$b^{2}=0$ and $R_{b}|_{L^{2}}$ has no zero eigenvalue in $F$. Let $x=n+\lambda
b$ where $n\in L^{2}$ and $\lambda\in F$. Now $bL^{2}=bI=0$, so
straightforward calculations show that

$C(x)=\left\\{\begin{array}[]{lll}L^{2}&\hbox{if}&\lambda=0,n\neq 0\\\
Fb&\hbox{if}&\lambda\neq 0,n=0\\\ 0&\hbox{if}&\lambda\neq 0,n\neq
0\end{array}\right\\}$

Hence $L$ is a $CT$-algebra $\Box$

Note that the above result shows that we may not have $xA=A$ in Lemma 2.1.
For, if $L$ is a cyclic Leibniz $A$-algebra, then $Ix=I$, but $xI=0$.

###### Lemma 2.3

If $L$ is a Leibniz algebra and $N$ is a zero ideal of $L$, we can consider
$N$ as a right $L/N$-module under the action $n(x+N)=nx$ for all $x\in L$,
$n\in N$. Then, each element $R_{x+N}$ with $x\notin N$ has no zero eigenvalue
under this action if and only if $C_{L}(n)\subseteq N$ for all $n\in N$.

Proof. Suppose $R_{x+N}$ has a zero eigenvalue under the action, where
$x\notin N$. Then, there exists $0\neq n\in N$ such that $nx=0$. If $xn=0$,
then $x\in C_{L}(n)\setminus N$. If $xn\neq 0$, then $x\in C_{L}(xn)\setminus
N$, as in Lemma 2.1 above.

Conversely, if $x\in C_{L}(n)\setminus N$ for some $n\in N$, we have $nx=0$
and $R_{x+N}$ has a zero eigenvalue. $\Box$

###### Proposition 2.4

Let $L$ be a Leibniz algebra and $N$ be a zero ideal of $L$ such that $L/N$ is
a $CT$-algebra and $C_{L}(n)\subseteq N$ for all $0\neq n\in N$. Then $L$ is a
$CT$-algebra.

Proof. Let $0\neq x\in L$ and take $y,z\in C_{L}(x)$, so $xy=yx=xz=zx=0$. If
$y\in N$, then $x\in C_{L}(y)\subseteq N$. Hence $y,z\in C_{L}(x)\subseteq N$,
giving $yz=0$.

So assume that $y\notin N$. Then $yz\in N$, by the hypothesis. Now

$x(yz)=(xy)z-(xz)y=0\hbox{ and }(yz)x=y(zx)+(yx)z=0,$

so $x\in C_{L}(yz)\subseteq N$. Thus $C_{L}(x)\subseteq N$ and $yz=0$. The
result follows. $\Box$

Note that If $L$ is a nilpotent cyclic Leibniz algebra of dimension greater
than $1$, then $L/I$ is a Lie $CT$ algebra, but $L$ is not a $CT$-algebra.
Next we have the two main classification results.

###### Theorem 2.5

Let $L$ be a Leibniz $CT$-algebra over an algebraically closed field $F$ of
characteristic $\neq 2,3$. Then $L$ is solvable or is isomorphic to
$sl_{2}(F)$.

Proof. Since $F$ has cohomological dimension $0$, $L=R\dot{+}S$, where $R$ is
the radical of $L$ and $S$ is a direct sum of ideals isomorphic to
$sl_{2}(F)$, by Lemma 1.3 and [12, Theorem 2]. Suppose that $R\neq 0$. If
$S\neq 0$, it is clear from Lemma 1.1 that $S\cong sl_{2}(F)$, and there is an
element $x\in S$ such that $R_{x}$ is nilpotent. Then $R_{x}$ has a zero
eigenvalue and $A=N\dot{+}Fx$, where $N$ is the nilradical of $L$, must be a
zero algebra, by Lemma 2.1. Thus, $x\in C_{L}(N)$. But $C_{L}(N)$ is an ideal
of $L$, and so $S\subseteq C_{L}(N)$. Pick $0\neq y\in N$. Then $S\subset
C(y)$, which is a zero algebra. Hence $S=0$. $\Box$

###### Theorem 2.6

Let $L$ be a solvable Leibniz $CT$-algebra of derived length $n+1$. Then

  * (i)

$L=A_{n}\dot{+}A_{n-1}\dot{+}\ldots\dot{+}A_{0}$, where $A_{i}$ is an a zero
subalgebra of $L$ and $L^{(i)}=A_{n}\dot{+}A_{n-1}\dot{+}\ldots\dot{+}A_{i}$
for $0\leq i\leq n$;

  * (ii)

$L$ splits over the nilradical $N$, which equals $L^{(n)}$; and

  * (iii)

for every element $x\in L\setminus N$, $R_{x}|_{N}$ has no zero eigenvalue in
$F$, and $Nx=N$.

Proof. (i) We have that $L=A_{n}\dot{+}A_{n-1}\dot{+}\ldots\dot{+}A_{0}$ and
$L^{(i)}=A_{n}\dot{+}A_{n-1}\dot{+}\ldots\dot{+}A_{i}$, by [12, Corollary 1].

(ii) Also, $N=A_{n}\oplus(N\cap A_{n-1}\oplus\ldots\oplus N\cap A_{0})$ and
$Z(L^{(i)})=N\cap A_{i}$ for each $0\leq i\leq n$, by [12, Theorem 5]. Suppose
that $N\cap A_{i}\neq 0$ for some $0\leq i\leq n-1$. Let $0\neq x\in N\cap
A_{i}$. Then $C_{L}(x)\supseteq L^{(i)}$. But $L^{(i)}$ is not a zero algebra
for $0\leq i\leq n-1$, so we have a contradiction.

(iii) Let $x\in L\setminus N$. Then $L(x)=N\dot{+}Fx$ satisfies the hypothesis
of Lemma 2.1, since $x^{2}\in I\subseteq N$. If $L(x)$ is a zero algebra, then
$x\in C_{L}(N)=N$, by [12, Lemma 7], a contradiction. Hence $R_{x}|_{N}$ has
no zero eigenvalue in $F$, and $Nx=N$, by Lemma 2.1. $\Box$

###### Definition 1

A Leibniz algebra $L$ is called completely solvable if $L^{2}$ is nilpotent.

Over a field of characteristic zero, every solvable Leibniz algebra is
completely solvable, and so every solvable Leibniz $CT$-algebra has derived
length at most $2$. However, this is not the case over fields of positive
characteristic, even for Lie algebras, as the following example, which is
taken from [6, pages 52, 53], shows.

###### Example 2.1

Let

$e=\left[\begin{array}[]{rrrrrrr}0&1&0&.&.&.&0\\\ 0&0&1&0&.&.&0\\\
\vdots&&&&&&\vdots\\\ 0&.&.&.&.&0&1\\\ 1&0&.&.&.&.&0\end{array}\right],\hskip
5.69046ptf=\left[\begin{array}[]{rrrrr}0&0&0&\ldots&0\\\ 0&1&0&\ldots&0\\\
0&0&2&\ldots&0\\\ \vdots&&&&\vdots\\\ 0&0&0&\ldots&p-1\end{array}\right],$

let $F$ be a field of prime characteristic $p$ and put $L=Fe+Ff+F^{p}$ with
product $[a+{\bf x},b+{\bf y}]=[a,b]+({\bf x}b-{\bf y}a)$ for all $a,b\in
Fe+Ff$, ${\bf x},{\bf y}\in F^{p}$.

Then, straightforward calculations show that

$C(\alpha e+\beta f+{\bf x})=\left\\{\begin{array}[]{lll}Ff+F{\bf
x_{1}}&\hbox{if}&\alpha=\beta=0,{\bf x}={\bf x_{1}}\\\
F^{p}&\hbox{if}&\alpha=\beta=0,{\bf x}\neq{\bf x_{1}}\\\ F(\alpha e+\beta
f+{\bf x})&\hbox{if}&\alpha\neq 0\\\ F(\beta f+{\bf x})+F{\bf
x_{1}}&\hbox{if}&\alpha=0,\beta\neq 0,\end{array}\right\\}$

where ${\bf x_{1}}=(1,0,\ldots,0)$, and all of these are zero algebras.

###### Theorem 2.7

Let $L$ be a solvable Leibniz $CT$-algebra over an arbitrary algebraically
closed field $F$. Then the nilradical of $L$ has codimension at most $2$ in
$L$.

Proof. We have that $L=N\dot{+}A_{1}\dot{+}A_{0}$, by Theorem 2.6 and [12,
Theorem 14]. Suppose $A_{1}\neq 0$. Let $A$ be a minimal ideal of
$B=N\dot{+}A_{1}$, inside $N$. Then $A$ is an irreducible $B$-bimodule, and so
$A_{1}A=0$ or $ab=-ba$ for every $a\in A$, $b\in B$, by [2, Lemma 1.9]. In
either case, $A$ is a minimal right $A_{1}$-module and $A=Fn$ is one-
dimensional, by [10, Lemma 5]. But $C_{B}(A)\subseteq N$ has dimension at
least $\dim B-1$, so $\dim A_{1}\leq 1$. The same argument shows that $\dim
A_{0}\leq 1$, whence the result. $\Box$

If the field $F$ in the above result has characteristic zero, then the
codimension is at most one. However, over any field of characteristic $p>0$,
the codimension can be two, even if $L$ is a Lie algebra, as Example 2.1
shows.

If $L$ is a Leibniz algebra and $y\in L$, the left centraliser of $y$ in $L$,
is $C_{L}^{\ell}(y)=\\{x\in L\mid xy=0\\}$. It is easy to check that this a
subalgebra of $L$.

###### Theorem 2.8

Let $L$ be a completely solvable Leibniz $CT$-algebra. Then, either $L$ is a
zero algebra, or $L=N\dot{+}A_{0}$ where $N$ is the nilradical,
$N^{2}=A_{0}^{2}=0$, $R_{x}|_{N}$ has no zero eigenvalue and $Nx=N$ for all
$x\in A_{0}$. If $A_{0}$ and $A_{0}^{\prime}$ are two complements to $N$ in
$L$, then there exists $n\in N$ such that $(1+L_{n})(A_{0})=A_{0}^{\prime}$.

Proof. We have that $L=N\dot{+}A_{0}$ where $N$ is the nilradical and
$N^{2}=A_{0}^{2}=0$, by Theorem 2.6. Suppose that $L$ is not a zero algebra.
Then $A_{0}\neq 0$, $R_{x}|_{N}$ has no zero eigenvalue and $Nx=N$ for all
$x\in A_{0}$, by Lemma 2.1.

For every $y\in L\setminus N$ we have $N=Ny\subseteq Ly\subseteq
L^{2}\subseteq N$, so $N=Ly$. Pick any $x\in L$. Then $xy\in N=Ny$, so there
is an $n\in N$ such that $xy=ny$. Thus $(x-n)y=0$ and $x-n\in
C_{L}^{\ell}(y)$. It follows that $L=N+C_{L}^{\ell}(y)$. Moreover, $N\cap
C_{L}^{\ell}(y)=0$, since $y$ has no zero eigenvalue on $N$, so
$C_{L}^{\ell}(y)$ is a complement to $N$ in $L$.

Let $A_{0}$ be any complement to $N$ in $L$. Then $y=n^{\prime}+a$ for some
$n^{\prime}\in N=Ny$ and $a\in A_{0}$. Hence, there is an $n\in N$ such that
$n^{\prime}=-ny$ and $a=(1+L_{n})(y)$. Put $\theta=1+L_{n}$, so $\theta(y)=a$
and $\theta(C_{L}^{\ell}(y))\subseteq C_{L}^{\ell}(a)$. But
$A_{0}=C_{L}^{\ell}(a)$ from which the final claim follows. $\Box$

Note that the above result mirrors the corresponding result in Lie algebras
(see [7, Theorem 3]). However, in that result, the complements are conjugate
under the inner automorphism $1+ad\,n$. In our result, $1+L_{a}$ is not an
automorphism, in general, as the following example shows.

###### Example 2.2

Let $L$ be the two-dimensional solvable cyclic Leibniz algebra with basis
$a,a^{2}$ and $a^{2}a=a^{2}$. It is easy to see that this is a $CT$-algebra.
Put $\theta=1+L_{a^{2}}$. Then $\theta(a^{2})=a^{2}$, whereas
$\theta(a)\theta(a)=(a+a^{2})(a+a^{2})=a^{2}+a^{2}$.

It would have been better if we could have taken $\theta=1+R_{n}$, which is an
automorphism. However, if $N=I$ (as in Example 2.2 above),
$\theta(A_{0})=A_{0}$.

Finally, we have that solvable Leibniz $CT$-algebras are factor-closed.

###### Theorem 2.9

Let $L$ be a solvable Leibniz $CT$-algebra and let $J$ be an ideal of $L$.
Then $L/J$ is a $CT$-algebra.

Proof. Suppose that $J\not\subseteq N$ and let $x\in J\setminus N$. Then
$N=Nx\subseteq J$, by Lemma 2.1, so $J\subset N$ or $N\subseteq J$. We use
induction on the derived length $k$ of $L$. Suppose first that $k=2$, so $L$
is completely solvable. If $N\subseteq J$, we have that $L/N$ is a zero
algebra and so $L/J$ is a $CT$-algebra.

So suppose now that $J\subset N$. A similar argument to that used in Lemma 2.1
also shows that $J=Jx$ for all $x\notin N$. Let $x+J\in C_{L/J}(n+J)$ where
$n\in N\setminus J$ and suppose that $x\notin N$. Then $nx\in J=Jx$, so
$nx=jx$ for some $j\in J$. Thus $(n-j)x=0$. But now $n=j$, since $L$ is a
$CT$-algebra, and this is a contradiction. It follows that
$C_{L/J}(n+J)\subseteq N/J$. Now $L/N\cong(L/J)/(N/J)$ is a $CT$-algebra and
hence so is $L/J$, by Proposition 2.4.

So suppose the result holds whenever $k\leq m$ and let $L$ have derived length
$m+1$. Then $L=N\dot{+}B$ for some subalgebra $B$ of derived length $m$ of
$L$, by Theorem 2.6 (ii). Now $L/N\cong B$ is a $CT$-algebra, by the inductive
hypothesis. If $N\subseteq J$, then $L/J\cong(L/N)/(J/N)\cong B/B\cap J$,
which is a $CT$-algebra, by the inductive hypothesis. If $J\subset N$, then
$L/J$ is a $CT$-algebra as in paragraph two above. $\Box$

###### Corollary 2.10

Let $L$ be any Leibniz $CT$-algebra over an algebraically closed field of
characteristic $\neq 2,3$, and let $J$ be an ideal of $L$. Then $L/J$ is a
$CT$-algebra.

Proof. This is immediate from Theorems 2.5 and 2.9. $\Box$

## References

  * [1] I.V.Arzhantsev, E. A. Makedonskii, and A. P. Petravchuk, ‘Finite-dimensional subalgebras in polynomial Lie algebras of rank one’, Ukrainian Math. J. 63 (5) (2011), 708–712.
  * [2] D. Barnes, ’Some theorems on Leibniz algebras’, Comm. Alg. 39 (2011), 2463-2472.
  * [3] A. Bloh, ‘On a generalization of the concept of Lie algebra’. Dokl. Akad. Nauk SSSR. 165 (1965), 471–473.
  * [4] W. Feit, J. G. Thompson, ‘Solvability of groups of odd order’. Pacific J. Math. 13 (1963), 775–1029.
  * [5] V.V. Gorbatsevich, ‘Lie algebras with abelian centralizers’, Math. Notes 101 (5) (2017), 795-801; translated from Mat. Zametki 101 (5) (2017), 690-699.
  * [6] N. Jacobson, Lie algebras. New York: Dover Publ. (1979).
  * [7] I. Klep and P. Moravec, ‘Lie algebras with abelian centralizers’, Algebra Cooloq. 17(4) (2010), 629–636.
  * [8] J.-L. Loday, ‘Une version non commutative des algèbres de Lie: les algèbres de Leibniz’. Enseign. Math. (2) 39 (3–4) (1993), 269–293.
  * [9] J.-L. Loday and T. Pirashvili, ‘Universal enveloping algebras of Leibniz algebras and (co)homology’, Math. Annalen 296 (1) (1993) 139–158.
  * [10] H. Strade and R. Farnsteiner, ‘Modular Lie algebras and their representations’, New York: Marcel Dekker Inc. (1988).
  * [11] M. Suzuki, ‘The nonexistence of a certain type of simple groups of odd order’, Proc. Amer. Math. Soc. 8 (1957), 686–695.
  * [12] D.A.Towers, ‘Leibniz $A$-algebras’, Comm. Math. 38 (2020), 103-121.
  * [13] L. Weisner, ‘Groups in which the normaliser of every element except identity is abelian’, Bull. Amer. Math. Soc. 31 (1925), 413–416.