# Budgeted Embedding Table For Recommender Models

Yunke Qu  The University of QueenslandBrisbaneAustralia<EMAIL_ADDRESS>,
Tong Chen  The University of QueenslandBrisbaneAustralia<EMAIL_ADDRESS>,
Quoc Viet Hung Nguyen  Griffith UniversityGold CoastAustralia
<EMAIL_ADDRESS>and  Hongzhi Yin  The University of
QueenslandBrisbaneAustralia<EMAIL_ADDRESS>
(2023)

###### Abstract.

At the heart of contemporary recommender systems (RSs) are latent factor
models that provide quality recommendation experience to users. These models
use embedding vectors, which are typically of a uniform and fixed size, to
represent users and items. As the number of users and items continues to grow,
this design becomes inefficient and hard to scale. Recent lightweight
embedding methods have enabled different users and items to have diverse
embedding sizes, but are commonly subject to two major drawbacks. Firstly,
they limit the embedding size search to optimizing a heuristic balancing the
recommendation quality and the memory complexity, where the trade-off
coefficient needs to be manually tuned for every memory budget requested. The
implicitly enforced memory complexity term can even fail to cap the parameter
usage, making the resultant embedding table fail to meet the memory budget
strictly. Secondly, most solutions, especially reinforcement learning based
ones derive and optimize the embedding size for each each user/item on an
instance-by-instance basis, which impedes the search efficiency. In this
paper, we propose Budgeted Embedding Table (BET), a novel method that
generates table-level actions (i.e., embedding sizes for all users and items)
that is guaranteed to meet pre-specified memory budgets. Furthermore, by
leveraging a set-based action formulation and engaging set representation
learning, we present an innovative action search strategy powered by an action
fitness predictor that efficiently evaluates each table-level action.
Experiments have shown state-of-the-art performance on two real-world datasets
when BET is paired with three popular recommender models under different
memory budgets.

recommender systems, neural architecture search

††copyright: acmcopyright††journalyear: 2023††copyright:
acmlicensed††conference: Proceedings of the Seventeenth ACM International
Conference on Web Search and Data Mining(WSDM’24); March 4–8, 2024; Merida,
Mexico††booktitle: Proceedings of the Seventeenth ACM International Conference
on Web Search and Data Mining (WSDM ’24), March 4–8, 2024, Merida,
Mexico††price: 15.00††ccs: Information systems Recommender systems

##  1\. Introduction

Recommender systems (RSs) predict a user’s preference for an item according to
observed user-item interactions (Zhang et al., 2019; Wang et al., 2021) and
have been indispensable across numerous modern platforms such as e-commerce
and social media (Li and Karahanna, 2015; Zhou et al., 2018; Kywe et al.,
2012). In contexts where computations on local devices are essential, such as
federated recommendation (Muhammad et al., 2020) and Internet of Things (IoT)
services (Huang et al., 2019), the need for RSs with small memory footprints
becomes critical due to highly constrained computational resources. The main
bottleneck for downsizing RSs is often the embedding table, which represents
users and items with distinct embedding vectors to facilitate pairwise
similarity computation. As such, the size of the embedding table can quickly
exhaust the available memory, posing a significant challenge to achieving
scalable and resource-friendly RSs.

Such memory bottleneck is attributed to conventional embedding tables, which
typically allocate fixed and uniform embedding sizes to each user and item
(Joglekar et al., 2020; Chen et al., 2021b; Qu et al., 2023). Such a design
may require an excessive number of parameters in industry-scale applications,
e.g., the benchmark recommender in (Desai and Shrivastava, 2022) has more than
25 billion parameters in its embedding table and requires 100 GB memory. In
this regard, researchers have developed methods to achieve lightweight
embeddings while maintaining its expressiveness. One stream of methods aim to
prune redundant parameters of the embedding table. For example, (Lyu et al.,
2022) proposed OptEmbed, a novel embedding sparsification framework that
prunes redundant embeddings by learning an importance score for each feature.
These methods mostly optimize a loss function combining a recommendation term
and a sparsity regularizer. Apart from sparsification/pruning methods, another
line of works featuring reinforcement learning (RL) techniques have also
emerged. Those methods adopt an RL-based policy network to search for the
optimal embedding sizes for each user/item, where the embedding sizes (i.e.,
actions) can be selected from an action space consisting of either a
predefined set of values (Liu et al., 2020) or a continuous range (Qu et al.,
2023). Similar to sparsification methods, the reward functions in RL-based
lightweight embedding approaches also bind the recommendation quality term
with a memory complexity penalty. By tuning a trade-off coefficient among the
two terms, they can adjust the regularization strength on the parameter
consumption, thus achieving the desired memory efficiency.

Despite the possibility of balancing performance and embedding parameter size,
these two types of methods are still away from practical use for two reasons.
Firstly, such a desired balance is achieved by jointly optimizing an
additional space complexity constraint on top of the recommendation objective.
For example, in (Joglekar et al., 2020), this is formulated as the difference
between the budgeted embedding parameter size and the actual parameter
consumption of the compressed embedding table. However, such an implicit loss
term means that the resultant embedding table may not necessarily meet the
desired parameter budget every time (Chen et al., 2021b). Moreover, the
optimization quality is highly sensitive to the trade-off coefficient, which
needs to be heuristically tuned for each memory budget.

The second reason is that, to achieve optimal recommendation performance under
each memory budget, the embedding size is decided on an instance-by-instance
basis w.r.t. each user/item. Taking a recent RL-based method CIESS (Qu et al.,
2023) as an example, a policy network predicts the optimal action for every
single user and item, based on which the backbone RS is trained and evaluated
to calculate the reward, enabling training the policy network. On the one
hand, ideally, the policy network is expected to be iteratively updated w.r.t.
each instance (i.e., user/item), but this can be prohibitively inefficient
given the large number of users and items. On the other hand, performing only
one update per batch/epoch leads to slower convergence due to the aggregated
nature of the reward function, and may potentially result in inferior
generalization (Keskar et al., 2016).

In response to both challenges, we propose Budgeted Embedding Table (BET) for
RSs, an automated input embedding size search paradigm that can efficiently
generate size decisions and perform optimization at the embedding table level.
An overview of BET is provided in Figure 1. To address the first challenge, we
innovatively propose a bounded sampler that draws embedding sizes for all
users/items from a collection of probabilistic distributions, with the
guarantee that the total parameter usage is capped. As the embedding sizes for
all instances in the embedding table are produced all at once, we term this a
table-level action. This strategy frees us from the need for manually
designing and tuning the implicit parameter constraint in the reward/loss
function, and allows us to enforce the memory budget with a negligible cost.

To tackle the second challenge, we further pair the generated table-level
actions with a fitness prediction network to evaluate the quality of those
actions. Essentially, in each search iteration, after sampling several table-
level actions, we need to identify the ones that promise high recommendation
accuracy. A straightforward solution is to greedily evaluate each candidate
action by training and evaluating a backbone RS from scratch with the
suggested embedding sizes (Qu et al., 2023; Zhao et al., 2020; Joglekar et
al., 2020; Liu et al., 2020), which apparently impedes efficiency.

One may think of a surrogate function (Chen et al., 2021b; Cai et al., 2020)
that “predicts” the fitness of each action, instead of explicitly evaluating
it. Accurately predicting the performance of the table-level actions is
crucial yet another major challenge in this regard, as either overestimating
or underestimating an action’s performance will eventually cause suboptimal
embedding size decisions. To align with the evaluation of table-level actions
in BET, our fitness predictor innovatively utilizes a set-based formulation of
actions, and embeds each action with DeepSets (Zaheer et al., 2017), where the
learned high-quality action embeddings can efficiently provide reliable
performance feedback on each table-level action. Furthermore, the set-based
action representations can be inductively learned, which is essential for our
iterative search algorithm that constantly generates unseen actions that need
to be evaluated.

To summarize, our work presents the contributions below:

  * •

We identify the practicality bottlenecks of existing embedding size search
methods for lightweight RSs, namely the implicit and non-scalable parameter
size constraints for meeting different memory budgets, and the inefficient
embedding size generation and evaluation for every instance.

  * •

We propose BET, a novel table-level embedding size search approach featuring
an efficient action sampling strategy that generates table-level actions for
all users and items and a DeepSets-based fitness prediction network that
accurately predicts their performance to reduce action evaluation costs.

  * •

Extensive empirical comparisons have been carried out on two real-world
datasets by pairing state-of-the-art embedding size search algorithms with
various backbone RSs, which confirm the advantageous efficacy of BET.

Figure 1. Overview of BET.

##  2\. Preliminaries

To provide a detailed research background, we hereby discuss relevant research
topics.

###  2.1. Deep Recommender Systems

Researchers have proposed numerous deep recommender models to capture the
user-item relationships. The first line of deep recommender systems are based
on MLPs. For instance, Neural Collaborative Filtering (NCF) (He et al., 2017)
integrates a MLP with a matrix factorization component to learn the two-way
interactions between users and items. AutoRec (Sedhain et al., 2015) uses an
autoencoder to model the user-item interactions in a low-dimensional hidden
space and perform collaborative filtering. Lin et al. proposed a novel
sequential recommender system, Dual Contrastive Network (Lin et al., 2022a)
that generates self-supervised signals by considering both user and item
perspectives and refining representation learning through dual representation
contrastive learning. In addition to MLP-based methods, graph-based
architectures (Wang et al., 2019; He et al., 2020; Yu et al., 2022) have also
been studied. For example, Neural Graph Collaborative Filtering (NGCF) (Wang
et al., 2019) extends traditional neural collaborative filtering by modeling
user-item interactions with graph convolution networks. LightGCN (He et al.,
2020) takes inspiration from NGCF and simplifies it by removing its self-
connection, feature transformation and linear activation operations. Another
line of recommender systems are based on factorization machines. Factorization
Machines (FM) combine linear regression with matrix factorization techniques
to handle high-order interactions (Rendle, 2010). (Blondel et al., 2016) was
also created to mine higher-order interactions. Its deep variants - QNFM (Chen
et al., 2021a), DeepFM (Guo et al., 2017) and xLightFM (Jiang et al., 2021)
have been proposed too. Moreover, numerous methods have been proposed to
leverage heterogeneous data for recommendation. For instance, (Chen et al.,
2020b; Gong and Zhang, 2016; Li et al., 2017) have been devised to extract
information from user reviews or hashtags. (Lee et al., 2018; Wang et al.,
2017) have been proposed to exploit videos and images for recommendation.
Location information has also been utilized to assist location-aware
recommendations (Chen et al., 2017).

###  2.2. Lightweight Embeddings For Recommenders

Various classes of works have investigated learning compressed embeddings to
provide a low-memory alternative to traditional embedding tables in
recommender systems (Zheng et al., 2022). For examples, feature selection
search methods (Wang et al., 2022; Luo et al., 2019; Lin et al., 2022b) are
trained to learn importance scores, based on which they filter out the
unimportant feature fields. Embedding dimension search approaches (Liu et al.,
2021, 2020; Zhao et al., 2021; Qu et al., 2022; Kong et al., 2023; Qu et al.,
2023) improve traditional recommender systems with fixed and uniform sizes by
introducing mixed dimensions for each feature. Quantization-based methods
(Chen et al., 2020a; Ko et al., 2021; Kang et al., 2020; Li et al., 2023; Chen
et al., 2022a, b) reduces the precision of numerical values and store the
parameters with lower bits. Feature hashing approaches (Yan et al., 2021;
Zhang et al., 2020; Sethi et al., 2022; Liu et al., 2023; Shi et al., 2020; Li
et al., 2021; Aditya Desai, 2022; Desai and Shrivastava, 2022; X Liang, 2023)
utilize various hash tricks to transform input values to a smaller range, thus
compression the embedding table. Knowledge distillation techniques train a
smaller student model from a larger teacher model and have been used to learn
an embedding table of smaller sizes from larger ones (Xia et al., 2022, 2023).
BET achieves dynamic embedding sizes and aims to find the optimal embedding
size for each feature, so it should be classified as a embedding dimension
search approach.

##  3\. Proposed Approach

In a nutshell, BET has three components: (1) a pre-trained backbone
recommender with a full-size embedding table $G_{\Theta}(\cdot)$ parameterized
by $\Theta$; (2) an action fitness prediction network $F_{\Phi}(\cdot)$
parameterized by $\Phi$; and (3) a non-parametric action sampler $H(\cdot)$.
Figure 1 provides an overview of the workflow of BET. Under pre-specified
memory budgets, $H(\cdot)$ samples actions from a set of probabilistic
distributions in each iteration. Each action specifies the embedding sizes for
all users and items. Afterwards, $F_{\Phi}(\cdot)$ estimates the fitness of
each action and selects the best action predicted. $G_{\Theta}(\cdot)$ is
further finetuned by adopting the embedding sizes in the chosen action, and is
then is evaluated on a hold-out dataset to obtain the actual fitness score
(i.e., recommendation quality) of the action. This facilitates the
optimization of $F_{\Phi}(\cdot)$ by minimizing its prediction error, thus
leading us to optimal actions under each given memory budget after $T$
iterations. In the remainder of this section, we unfold the design of each
component.

###  3.1. Recommender with Embedding Masking

Let $\mathcal{U}$ and $\mathcal{V}$ denote a set of users $u$ and items $v$,
respectively. The embedding table in the recommender maps each integer-valued
user or item IDs to a real-valued embedding vector $\mathbf{e}_{n}$, where
$n\in\mathcal{U}\cup\mathcal{V}$ indexes either a user or an item. Note that
we only separately use $u$ and $v$ to index users and items when they need to
be distinguished, e.g., when performing pairwise ranking. The vertical
concatenation of all embedding vectors forms the embedding table $\mathbf{E}$
of size $(|\mathcal{U}|+|\mathcal{V}|)\times d_{max}$, where $d_{max}$ is the
maximum/full embedding size supported by $G_{\Theta}(\cdot)$. The embedding
sizes are adjusted via a binary mask
$\mathbf{M}\in\\{0,1\\}^{(|\mathcal{U}|+|\mathcal{V}|)\times d_{max}}$, which
is commonly adopted for embedding sparsification in lightweight recommender
systems (Liu et al., 2021; Lyu et al., 2022; Qu et al., 2023). Each row
$\mathbf{m}_{n}$ in $\mathbf{M}$ is a mask vector corresponding to user/item
$n$. Given a searched embedding size $d_{n}$, we set the first $d_{n}$
elements of $\mathbf{m}_{n}$ to ones the remaining $d_{max}-d_{n}$ dimensions
to zeros. Then, we perform element-wise multiplication of the embedding table
$\mathbf{E}$ and the binary mask matrix $\mathbf{M}$ to adjust the embedding
sizes during embedding look-up, i.e.,
$\mathbf{e}_{n}=(\mathbf{E}\odot\mathbf{M})[n]$ where $[n]$ retrieves the
$n$-th row of a matrix. $\mathbf{M}$ is adaptively updated based on the
sampled actions in every iteration. It is worth noting that, in the deployment
stage, only the heavily sparsified embedding table
$\mathbf{E}_{sparse}=\mathbf{E}\odot\mathbf{M}$ will be stored, where the cost
of storing zero-valued entries can be negligible with sparse matrix storage
techniques (Virtanen et al., 2020; Sedaghati et al., 2015). Once we obtain the
sparse embedding vectors, the recommender model $G_{\Phi}(\cdot)$ computes a
preference score $\hat{y}_{uv}$ representing pairwise user-item similarity:

(1)
$\hat{y}_{uv}=G_{\Theta}(\mathbf{e}_{u},\mathbf{e}_{v}),\;\;\;\;u\in\mathcal{U},v\in\mathcal{V},$

where $G_{\Theta}(\cdot)$ can be any latent factor-based recommender model
that supports pairwise scoring.

For optimizing the recommender, we adopt Bayesian Personalized Ranking (BPR)
loss (Rendle et al., 2009):

(2)
$\mathcal{L}_{BPR}=\sum_{(u,v,v^{\prime})\in\mathcal{D}_{train}}-\ln\sigma(\hat{y}_{uv}-\hat{y}_{uv^{\prime}})+\eta\|\Theta\|_{2}^{2},$

where $(u,v,v^{\prime})\in\mathcal{D}_{train}$ represents the training samples
where user $u$ chooses item $v$ over item $v^{\prime}$, while $\hat{y}_{uv}$
and $\hat{y}_{uv^{\prime}}$ respectively represent the predicted preferences
that user $u$ has on items $v$ and $v^{\prime}$. $\eta\|\Theta\|_{2}^{2}$
provides regularization to prevent overfitting, with a scaling factor $\eta$.
Our goal is to search for the optimal embedding sizes for all users and items
under strict memory budgets, hence the overall objective is defined as
follows:

(3)
$\min_{\Theta,\Phi}\mathcal{L}_{BPR}\;\;\;\text{s.t.}\;1-\frac{\|\mathbf{M}\|_{1,1}}{(|\mathcal{U}|+|\mathcal{V}|)\times
d_{max}}\leq c,$

where $\|\mathbf{M}\|_{1,1}$ counts the number of ones, or equivalently
retained parameters in the binary mask, $c$ ($0<c<1$) is the target sparsity
rate, i.e., the ratio between the numbers of active parameters in sparsified
and full embedding tables.

###  3.2. Budget-Aware Action Sampling

Hereby we introduce a novel table-level action sampling strategy. In each
search iteration, we randomly sample a set of candidate actions
$\mathcal{Q}=\\{a_{1},...,a_{m}\\}$. Each action assigns a continuous
embedding size $d_{n}\in\mathbb{N}^{+}_{\leq d_{max}}$ to every user or item
$n\in\mathcal{U}\cup\mathcal{V}$.

As we aim to avoid optimizing the selected action towards an implicit,
manually tuned memory cost term, we propose to directly draw a table-level
action that: (1) makes all generated embedding sizes conditioned on a certain
distribution; and (2) ensures the total parameter size of all embeddings is
capped at the budgeted parameter consumption. In short, (1) essentially
narrows the action space to mitigate the difficulty of searching from a huge
collection of completely random actions, while (2) guarantees that the table-
level action strictly meets the memory budget in the first place.

To generate an action $a$, we first determine two probability distributions
$P_{\mathcal{U}}$ and $P_{\mathcal{V}}$ respectively for users and items.
Meanwhile, to introduce inductive bias to the action sampler, instead of
resorting to a fixed distribution throughout the search process,
$P_{\mathcal{U}}$ and $P_{\mathcal{V}}$ are both randomly chosen from a set of
candidate distributions with on-the-fly parameterization. Specifically, we
utilize a power law distribution, a truncated exponential distribution, a
truncated normal distribution and a log normal distribution. For simplicity,
we let the two selected distributions respectively be parameterized by
$\beta_{\mathcal{U}}$ and $\beta_{\mathcal{V}}$. In each iteration, the
parameters are all determined uniformly at random:

(4)
$\beta_{\mathcal{U}},\beta_{\mathcal{V}}\sim\text{Uniform}(0,\beta_{max}),$

where $\beta_{max}$ is the maximal value allowed. Notably, the embedding sizes
of each field (i.e., users and items) are derived by independently sampling
from two distributions $P_{\mathcal{U}}$ and $P_{\mathcal{V}}$. This is
because each field may have distinct properties (Lyu et al., 2022) and
different distributions are needed to model the embedding sizes of users and
items.

After the distributions are determined, we sample $|\mathcal{U}|$ and
$|\mathcal{V}|$ probabilities respectively from $P_{\mathcal{U}}$ and
$P_{\mathcal{V}}$. The set of probabilities are respectively denoted as
$\\{p_{i}\\}_{i=1}^{|\mathcal{U}|}\sim P_{\mathcal{U}}$ and
$\\{p_{j}\\}_{j=1}^{|\mathcal{V}|}\sim P_{\mathcal{V}}$. We then normalize all
$p_{i}$ and $p_{j}$ as the following:

(5)
$\tilde{p}_{i}=\frac{p_{i}}{\sum_{i}^{|\mathcal{U}|}p_{i}},\,\,\,\,\tilde{p}_{j}=\frac{p_{j}}{\sum_{j}^{|\mathcal{V}|}p_{j}},$

where $\tilde{p}_{i}$/$\tilde{p}_{j}$ indicates the fraction of parameters
each user/item receives among all users/items. Next, we obtain the actual
embedding sizes by applying the fraction on the memory budget:

(6) $\begin{split}d_{i}=\lfloor\tilde{p}_{i}\cdot w\cdot
d_{max}\cdot(|\mathcal{U}|+|\mathcal{V}|)\rfloor,\;\;\;i=1,2,\cdots,|\mathcal{U}|,\\\
d_{j}=\lfloor\tilde{p}_{j}\cdot(1-w)\cdot
d_{max}\cdot(|\mathcal{U}|+|\mathcal{V}|)\rfloor,\;\;\;j=1,2,\cdots,|\mathcal{V}|,\end{split}$

where $w$ controls the proportion of parameters allocated to the users. In
practice, $w$ can be empirically defined, such as an even split (i.e., 0.5).
In BET, $w$ is sampled from a uniform distribution each time a table-level
action is generated, i.e., $w\sim\text{Uniform}(0,1)$, so as to probe more
embedding size allocation strategies in the iterative search process.

With $|\mathcal{U}|+|\mathcal{V}|$ embedding sizes derived, we now allocate
them to each user and item. Previous studies (Zhao et al., 2020; Lyu et al.,
2022; Sun et al., 2020) have shown that more frequent users and items tend to
be more informative than long-tail ones, hence generally benefit from a larger
embedding dimension which allows more information to be encoded. Therefore, we
rank the users by their frequency and allocate the $n$-th largest user
embedding size to the $n$-th frequent user. Similarly, we assign embedding
sizes to items based on their frequency. As such, we are capable of sampling
$m$ random table-level actions in each search iteration to facilitate the
subsequent steps.

###  3.3. Fitness Prediction Network

In each iteration $t$ we sample a set of $m$ actions, denoted as
$\mathcal{Q}_{t}$. To select the optimal action from $\mathcal{Q}_{t}$, a
straightforward approach is to pair each sampled action with a fresh
recommender, train it until convergence and select the action that yields the
best final recommendation quality on the validation set. However, this
approach faces inevitable computational burdens given the $m$ training and
evaluation processes needed per search iteration.

In BET, we utilize an action fitness predictor, which inductively learns
distinct embeddings of actions and predicts its fitness score (i.e.,
recommendation quality) based on the action embeddings. As discussed earlier,
learning expressive representations of different actions is a crucial yet non-
trivial task for effective fitness prediction, especially considering the need
for inductively composing embeddings for unseen actions in every new search
iteration.

Figure 2. Example of our set-based action formulation. Suppose we have three
legitimate embedding sizes $\\{16,32,64\\}$, four users (red) and two items
(green). Then, size 16 is a set containing user 1, user 2 and item 4, which
means their embedding sizes are 16. Similarly, the action specifies the
embedding sizes of user 3 and item 5 to be 32. Size 64 is not used by any
users/items. Figure 3. Overview of the DeepSets-based fitness predictor.

Thus, we propose a novel action fitness prediction network based on the notion
of set representation learning (Zaheer et al., 2017; Wagstaff et al., 2022).
Before delving into the fitness prediction network, we first detail the set-
based formulation of each action. Each action is regarded as a collection of
sets $a=\\{\mathcal{S}_{d}\\}_{d=1}^{d_{max}}$, where each set
$\mathcal{S}_{d}=\\{n|\forall n\in\mathcal{U}\cup\mathcal{V},d_{n}=d\\}$
stores the users and items that use the embedding size $d\in[1,d_{max}]$.
$\mathcal{S}_{d}=\emptyset$ if there is no user or item assigned to embedding
size $d$. A toy example with is given in figure 2. As such, this allows us to
build our fitness predictor upon DeepSets (Zaheer et al., 2017) owing to its
ability to learn permutation-variant representations of sets.

The fitness prediction network consists of four components: a user encoder
$\rho_{\mathcal{U}}(\cdot)$, an item encoder $\rho_{\mathcal{V}}(\cdot)$, an
embedding size encoder $\mu(\cdot)$, and a decoder $\pi(\cdot)$, all of which
bear a distinct MLP. Encoders
$\rho_{\mathcal{U}}(\cdot)$/$\rho_{\mathcal{V}}(\cdot)$ take some contextual
information about a user/item, and produce their corresponding
representations. Previous studies (Zhao et al., 2020; Qu et al., 2023) have
demonstrated the suitability of users’/items’ popularity and current embedding
sizes as the context information for embedding size search. We inherit this
design by establishing a hierarchical, set-compatible action representation
learning scheme. Specifically, $\rho_{\mathcal{U}}(\cdot)$ and
$\rho_{\mathcal{V}}(\cdot)$ respectively takes the user and item frequencies
to generate frequency-weighted embeddings:

(7)
$\begin{split}\boldsymbol{q}_{u}&=\rho_{\mathcal{U}}(\frac{f_{u}}{\max_{u^{\prime}\in\mathcal{U}}f_{u^{\prime}}}),\;\;\;u\in\mathcal{U},\\\
\boldsymbol{q}_{v}&=\rho_{\mathcal{V}}(\frac{f_{v}}{\max_{v^{\prime}\in\mathcal{V}}f_{v^{\prime}}}),\;\;\;v\in\mathcal{V},\end{split}$

where each $f_{u}$/$f_{v}$ is normalized by the maximum user/item frequency
observed.

As mentioned above, within an action $a$, each set $\mathcal{S}_{d}$ contains
all users and items with embedding size $d$. To obtain embedding
$\boldsymbol{s}_{d}$ of set $\mathcal{S}_{d}$, we firstly fuse the
representations of all involved users and items using mean aggregation, and
then append the explicit embedding size information:

(8)
$\boldsymbol{s}_{d}=\mu\Big{(}\Big{[}\frac{1}{|\mathcal{S}_{d}|}\sum_{n\in\mathcal{S}_{d}}\boldsymbol{q}_{n};\frac{d}{d_{max}}\Big{]}\Big{)},$

where $[;]$ represents concatenation. With the representations generated for
all $d_{max}$ sets, we derive the embedding for the entire action $a$ by
further merging all set representations:

(9)
$\boldsymbol{h}_{a}=\frac{1}{d_{max}}\sum_{d=1}^{d_{max}}\boldsymbol{s}_{d},$

where we use $\boldsymbol{h}_{a}$ to denote an action embedding. An additional
benefit from the set-based action embeddings is that, whenever the action
sampler draws any action that is new to the fitness predictor (which is
commonly the case in embedding size search), it allows for generating the
action embedding in an inductive fashion with the available context
information and the updated set formulation.

With each action embedding $\boldsymbol{h}_{a}$, the decoder $\pi(\cdot)$
takes $\boldsymbol{h}_{a}$ and outputs $\hat{r}_{a}$, which is the predicted
fitness score of action $a$:

(10) $\hat{r}_{a}=\pi(\boldsymbol{h}_{a}).$

We will detail how the entire fitness prediction network $F_{\Phi}(\cdot)$ is
optimized along with the search process in the next section.

###  3.4. Fitness Predictor Training

Though bearing a relatively simple structure, the fitness predictor still
needs to be trained with some ground truth samples. The fitness prediction
network $F_{\Phi}(\cdot)$ is optimized by minimizing the mean squared error
between the predicted fitness score $\hat{r}_{a}$ and the actual one $r_{a}$
with gradient descent:

(11) $\Phi=\underset{\Phi^{\prime}}{\text{argmin}}\sum_{\forall
a}(r_{a}-F_{\Phi^{\prime}}(a))^{2}=\underset{\Phi^{\prime}}{\text{argmin}}\sum_{\forall
a}(r_{a}-\hat{r}_{a})^{2}.$

The actual fitness score $r_{a}$, i.e., recommendation quality of the action,
can be evaluated on the backbone recommender $G_{\Theta}(\cdot)$ after being
finetuned with an action $a$ w.r.t. Eq. 2:

(12)
$r_{a}=\frac{\text{eval}(\mathbf{E}\odot\mathbf{M}_{a}|\mathcal{D}_{val})}{\text{eval}(\mathbf{E}|\mathcal{D}_{val})},$

where $\text{eval}(\cdot)$ evaluates the recommendation quality,
$\mathcal{D}_{val}$ is the validation set, $\mathbf{M}_{a}$ is the binary mask
corresponding to action $a$, and the denominator is the recommendation quality
of the pre-trained backbone recommender $G_{\Theta}(\cdot)$ with the full
embedding table. Hence, $r_{a}$ can be interpreted as the ratio between the
two recommendation accuracy obtained by the sparsified embeddings and the full
embeddings. The implementation of $\text{eval}(\cdot)$ can follow most
commonly used recommendation metrics like Recall@$k$ and NDCG@$k$ (He et al.,
2020; Rendle et al., 2009), where we adopt an ensemble of both metrics under
different values of $k$:

(13)
$\text{eval}(\mathbf{E}\odot\mathbf{M}|\mathcal{D}_{val})=\sum_{u\in\mathcal{U}}\frac{\sum_{k\in\mathcal{K}}\text{Recall@}k_{u}+\text{NDCG@}k_{u}}{2|\mathcal{K}||\mathcal{U}|},$

where we the choices of $k$ cover $\mathcal{K}=\\{5,10,20\\}$ in our paper.

The ground truth samples can be thus generated by gathering different action-
fitness pairs. Instead of constructing a pool of such training samples and
train $F_{\Phi}(\cdot)$ upfront, we incrementally add one training sample in
every search iteration $t$ to optimize $F_{\Phi}(\cdot)$ on the go. The
training sample is constructed by firstly identifying a promising action from
$\mathcal{Q}_{t}$, denoted by $a_{t}$, and then obtaining $r_{a_{t}}$ by
evaluating $a_{t}$ on the finetuned $G_{\Theta}(\cdot)$. Compared with most
RL-based embedding size search methods, the required number of fitness
evaluation with the recommendation model will be reduced from $m\times T$ to
merely $T$, where $T$ is the total number of iterations as well as the ground
truth samples for optimizing $F_{\Phi}(\cdot)$.

Algorithm 1 Embedding Size Search with BET

1:Initialize $\Theta$ and $\Phi$, set $\mathcal{A}\leftarrow\emptyset$;

2:Train $G_{\Theta}(\cdot)$ till convergence w.r.t. Eq. 2;

3:for $t=1,\cdots,T$ do

4: Draw distributions and their parameters w.r.t. Eq. 4;

5: Draw $m$ actions $\mathcal{Q}_{t}=\\{a_{1},...,a_{m}\\}$ with Eq. 5 and Eq.
6;

6: if $0\leq t\text{ mod }5\leq 2$ then

7: $a_{t}\leftarrow$ Strategy I;

8: else if $t\text{ mod }5=3$ then

9: $a_{t}\leftarrow$ Strategy II;

10: else if $t\text{ mod }5=4$ then

11: $a_{t}\leftarrow$ Strategy III;

12: end if

13: Finetune $\Theta$ w.r.t. $\textbf{E}\odot\textbf{M}_{a_{t}}$ and Eq. 2

14: $r_{a_{t}}\leftarrow$ Eq. 13

15: $\mathcal{A}\leftarrow\mathcal{A}\cup(a_{t},r_{a_{t}})$;

16: for $j=1,\cdots,N$ do

17: Randomly sample $(a_{j},r_{a_{j}})$ from $\mathcal{A}$;

18: $\hat{r}_{a_{j}}\leftarrow$ Eq. 10;

19: Update $\Phi$ w.r.t. Eq. 11;

20: end for

21:end for

22:Perform selective retraining and obtain $a^{*}\leftarrow$ Eq. 14.

###  3.5. Action Selection and Embedding Retraining

The pseudocode for BET is presented in Algorithm 1. Among its $T$ search
iterations, we utilize three different strategies to pick one action $a_{t}$
from the candidate action set $\mathcal{Q}_{t}$ (lines 4-12). Then, the
recommender $G_{\Theta}(\cdot)$ is finetuned by adopting $a_{t}$ and evaluated
on the validation set (lines 13-14), where the produced $(a_{t},r_{a_{t}})$
sample is popped into the population set $\mathcal{A}$ to facilitate training
the fitness prediction network $F_{\Phi}(\cdot)$ (lines 16-20). Finally, a
selective retraining process is conducted with the top-ranked actions in the
population in order to identify the best embedding size decision for all users
and items. We provide further details about action selection strategies and
retraining the sparsified embeddings below.

####  3.5.1. Diversifying the Action Space.

The iterative input embedding size search contains $T$ iterations. In each
iteration $t$, the selected action $a_{t}$ is added to the population set
$\mathcal{A}$ with its associated fitness score. For efficiency consideration,
in each iteration $t$, we only compute the actual fitness score for action
$a_{t}$, which is selected from candidate actions $\mathcal{Q}_{t}$ with one
of the three following strategies:

  1. I:

We use our fitness prediction network $F_{\Phi}(\cdot)$ to predict the fitness
score of each candidate actions, and greedily selects the action predicted to
have the highest fitness score, i.e.,
$a_{t}=\text{argmax}_{a\in\mathcal{Q}_{t}}F_{\Phi}(a)$. In essence,
$F_{\Phi}(\cdot)$ is a surrogate function that is trained to approximate Eq.
12 and is substantially more cost-effective than evaluating all $m$ actions
via recommender finetuning.

  2. II:

We randomly select one action from the $\mathcal{Q}_{t}$ to diversify the
training samples for $F_{\Phi}(\cdot)$ by preventing it from overfitting some
specific action distributions in the population $\mathcal{A}$.

  3. III:

We identify currently the best action (with the highest $r_{a}$) in
$\mathcal{A}$. All actions in $\mathcal{Q}_{t}$ and the best action are
inductively embedded into vector representations using Eq. 9. Then, by
calculating the pairwise Euclidean distance, we select action
$a_{t}\in\mathcal{Q}_{t}$ by locating the best action’s nearest neighbor in
the embedding space.

####  3.5.2. Selective Retraining for Sparsified Embeddings.

After $T$ search iterations, from the population we identify the top five
actions with the highest $r_{a}$ on the validation set, denoted as
$\mathcal{A}^{\prime}$. Each action $a\in\mathcal{A}^{\prime}$ is paired with
a fresh backbone recommender model, which is trained from scratch on the
training set until convergence. Next, we evaluate the trained recommenders and
select the final action $a^{*}$ that yields the best validation performance:

(14) $a^{*}=\underset{a\in\mathcal{A}^{\prime}}{\text{argmax}}\,\,r_{a},$

with which the sparsified embedding $\textbf{E}_{sparse}$ is obtained and
stored for inference.

##  4\. Experiments

This section details the experimental results of BET’s performance.

###  4.1. Recommenders and Comparative Methods

One can combine BET with different embedding-based representation learning
backbone recommender models. We tested the effectiveness of our approach with
three commonly used recommender models as the backbone recommender $F_{\Phi}$:
NCF (He et al., 2017), NGCF (Wang et al., 2019) and LightGCN (He et al.,
2020). We used the same settings as reported in their original works and
replaced the embedding table with one with adjustable embedding sizes to
demonstrate the versatility and generalizability of our method across various
backbone recommenders. We compare BET with the following model-agnostic
embedding size search algorithms:

  * •

PEP (Liu et al., 2021): It adaptively prunes the embedding parameters based on
a learned pruning threshold learned along with the model’s other parameters
during the training process.

  * •

ESAPN (Liu et al., 2020): It uses an RL agent which performs hard selection on
embedding sizes for users and items based on the memory complexity and
recommendation performance.

  * •

OptEmbed (Lyu et al., 2022): It prunes redundant embeddings based on learnable
pruning thresholds indicative of features’ importance and derives the optimal
embedding size for each field using evolutionary search.

  * •

CIESS (Qu et al., 2023): It is an RL-based method that chooses embedding sizes
for the users and items from a continuous domain.

  * •

Single and Uniform (SU): All users and items share equal and fixed embedding
sizes.

  * •

Single and Random (SR): The embedding sizes of the users and items are sampled
from a uniform distribution.

###  4.2. Evaluation Protocols

All the aforementioned methods are evaluated on two real-world datasets:
Gowalla (Liang et al., 2016) and Yelp2018 (Wang et al., 2019). The first has
1,027,370 interactions between 29,858 users and 40,981 items while the latter
has 1,561,147 interactions between 31,668 users and 38,048 items. We take 50%,
25% and 25% of the two datasets for training, validation, and test. To measure
the recommendation performance of the backbone recommender, Recall@$k$ and
NDCG@$k$ are used as evaluation metrics, where $k$ is set to either 5 or 20.
To test the recommendation performance of CIESS, PEP, SU and SR, three
different sparsity ratios ($c$) of 80%, 90% and 95% are used. For each method,
the compressed embedding table is ensured to have no more than $c\cdot
d_{max}\cdot(|\mathcal{U}|+|\mathcal{V}|)$ usable parameter and the maximal
embedding size is $d_{max}=128$. However, since ESAPN and OptEmbed are not
designed to precisely control the embedding sparsity and focus more on the
performance, this section only reports the performance associated with their
final embedding tables after pruning.

| LightGCN | NGCF | NCF
---|---|---|---
Method | Sparsity | R@5 | N@5 | R@20 | N@20 | Sparsity | R@5 | N@5 | R@20 | N@20 | Sparsity | R@5 | N@5 | R@20 | N@20
ESAPN | 75% | 0.0741 | 0.1173 | 0.1579 | 0.1329 | 83% | 0.0627 | 0.1060 | 0.1396 | 0.1193 | 74% | 0.0498 | 0.0718 | 0.1280 | 0.0959
OptEmb | 74% | 0.0591 | 0.0983 | 0.1314 | 0.1115 | 75% | 0.0510 | 0.0860 | 0.1227 | 0.1005 | 82% | 0.0330 | 0.0492 | 0.0877 | 0.0653
PEP | 80% | 0.0715 | 0.1189 | 0.1588 | 0.1347 | 80% | 0.0656 | 0.1049 | 0.1524 | 0.1245 | 80% | 0.0498 | 0.0704 | 0.1282 | 0.0957
SU | 0.0756 | 0.1219 | 0.1674 | 0.1400 | 0.0619 | 0.0961 | 0.1466 | 0.1170 | 0.0481 | 0.0662 | 0.1235 | 0.0916
SR | 0.0708 | 0.1150 | 0.1583 | 0.1323 | 0.0512 | 0.0815 | 0.1223 | 0.0982 | 0.0478 | 0.0656 | 0.1243 | 0.0910
CIESS | 0.0777 | 0.1263 | 0.1750 | 0.1461 | 0.0676 | 0.1084 | 0.1504 | 0.1251 | 0.0550 | 0.0744 | 0.1307 | 0.0997
BET | 0.0797 | 0.1303 | 0.1753 | 0.1483 | 0.0643 | 0.1039 | 0.1454 | 0.1204 | 0.0511 | 0.0723 | 0.1294 | 0.0973
$p$-value | 0.0024 | 0.0001 | 0.0391 | 0.0012 |  | 0.7815 | 0.2179 | 0.0514 | 0.0924 |  | 0.1962 | 0.1337 | 0.2578 | 0.0414
PEP | 90% | 0.0594 | 0.0992 | 0.1274 | 0.1102 | 90% | 0.0552 | 0.0914 | 0.1327 | 0.1081 | 90% | 0.0414 | 0.0631 | 0.1121 | 0.0839
SU | 0.0646 | 0.1041 | 0.1477 | 0.1217 | 0.0589 | 0.0935 | 0.1380 | 0.1115 | 0.0413 | 0.0548 | 0.1112 | 0.0796
SR | 0.0617 | 0.1013 | 0.1403 | 0.1171 | 0.0466 | 0.0764 | 0.1123 | 0.0911 | 0.0399 | 0.0536 | 0.1069 | 0.0769
CIESS | 0.0721 | 0.1193 | 0.1589 | 0.1353 | 0.0601 | 0.0980 | 0.1380 | 0.1140 | 0.0448 | 0.0613 | 0.1131 | 0.0841
BET | 0.0736 | 0.1212 | 0.1620 | 0.1376 | 0.0623 | 0.1012 | 0.1426 | 0.1177 | 0.0434 | 0.0648 | 0.1122 | 0.0845
$p$-value | 0.0165 | 0.0481 | 0.0091 | 0.0226 |  | 0.0005 | 0.0205 | 0.0601 | 0.0232 |  | 0.1454 | 0.0554 | 0.6608 | 0.0655
PEP | 95% | 0.0372 | 0.0666 | 0.0803 | 0.0712 | 95% | 0.0247 | 0.0471 | 0.0674 | 0.0554 | 95% | 0.0351 | 0.0591 | 0.0897 | 0.0712
SU | 0.0446 | 0.0727 | 0.1085 | 0.0872 | 0.0527 | 0.0870 | 0.1213 | 0.1007 | 0.0336 | 0.0478 | 0.0952 | 0.0682
SR | 0.0500 | 0.0834 | 0.1178 | 0.0975 | 0.0438 | 0.0739 | 0.1040 | 0.0859 | 0.0329 | 0.0489 | 0.0918 | 0.0674
CIESS | 0.0513 | 0.0853 | 0.1214 | 0.0997 | 0.0505 | 0.0856 | 0.1129 | 0.0964 | 0.0361 | 0.0567 | 0.0933 | 0.0717
BET | 0.0627 | 0.1037 | 0.1374 | 0.1171 | 0.0528 | 0.0883 | 0.1172 | 0.9980 | 0.0388 | 0.0616 | 0.0963 | 0.0756
$p$-value | 0.0001 | 0.0002 | 0.0001 | 0.0001 |  | 0.1896 | 0.0638 | 0.7269 | 0.3608 |  | 0.0321 | 0.1107 | 0.1104 | 0.0010
(a) Results on Gowalla
| LightGCN | NGCF | NCF
Method | Sparsity | R@5 | N@5 | R@20 | N@20 | Sparsity | R@5 | N@5 | R@20 | N@20 | Sparsity | R@5 | N@5 | R@20 | N@20
ESAPN | 73% | 0.0294 | 0.0696 | 0.0812 | 0.0773 | 71% | 0.0214 | 0.0538 | 0.0632 | 0.0603 | 76% | 0.0153 | 0.0342 | 0.0490 | 0.0430
OptEmb | 80% | 0.0183 | 0.0424 | 0.0534 | 0.0489 | 67% | 0.0133 | 0.0301 | 0.0412 | 0.0369 | 77% | 0.0076 | 0.0161 | 0.0238 | 0.0203
PEP | 80% | 0.0253 | 0.0605 | 0.0723 | 0.0682 | 80% | 0.0086 | 0.0167 | 0.0275 | 0.0223 | 80% | 0.0130 | 0.0236 | 0.0406 | 0.0327
SU | 0.0289 | 0.0665 | 0.0822 | 0.0758 | 0.0223 | 0.0517 | 0.0665 | 0.0603 | 0.0131 | 0.0226 | 0.0465 | 0.0351
SR | 0.0253 | 0.0588 | 0.0783 | 0.0698 | 0.0212 | 0.0499 | 0.0618 | 0.0573 | 0.0101 | 0.0183 | 0.0342 | 0.0268
CIESS | 0.0292 | 0.0692 | 0.0839 | 0.0783 | 0.0233 | 0.0566 | 0.0701 | 0.0652 | 0.0175 | 0.0377 | 0.0533 | 0.0474
BET |  | 0.0310 | 0.0726 | 0.0873 | 0.0816 |  | 0.0254 | 0.0601 | 0.0741 | 0.0685 |  | 0.0194 | 0.0451 | 0.0620 | 0.0554
$p$-value | 0.0008 | 0.0309 | 0.0004 | 0.0109 |  | 0.0412 | 0.0418 | 0.0082 | 0.0024 |  | 0.0387 | 0.0122 | 0.0483 | 0.0341
PEP | 90% | 0.0224 | 0.0531 | 0.0657 | 0.0610 | 90% | 0.0080 | 0.0153 | 0.0276 | 0.0215 | 90% | 0.0139 | 0.0242 | 0.0427 | 0.0335
SU | 0.0230 | 0.0544 | 0.0722 | 0.0648 | 0.0205 | 0.0528 | 0.0622 | 0.0592 | 0.0119 | 0.0201 | 0.0416 | 0.0309
SR | 0.0210 | 0.0476 | 0.0642 | 0.0573 | 0.0195 | 0.0449 | 0.0573 | 0.0524 | 0.0089 | 0.0170 | 0.0309 | 0.0242
CIESS | 0.0263 | 0.0649 | 0.0730 | 0.0705 | 0.0232 | 0.0551 | 0.0669 | 0.0619 | 0.0153 | 0.0350 | 0.0500 | 0.0442
BET | 0.0284 | 0.0674 | 0.0816 | 0.0762 | 0.0246 | 0.0586 | 0.0736 | 0.0676 | 0.0173 | 0.0400 | 0.0565 | 0.0499
$p$-value | 0.0045 | 0.0060 | 0.0051 | 0.0006 |  | 0.0012 | 0.0007 | 0.0102 | 0.0069 |  | 0.0234 | 0.0067 | 0.0426 | 0.0118
PEP | 95% | 0.0199 | 0.0496 | 0.0600 | 0.0561 | 95% | 0.0075 | 0.0143 | 0.0259 | 0.0206 | 95% | 0.0125 | 0.0244 | 0.0399 | 0.0330
SU | 0.0217 | 0.0494 | 0.0624 | 0.0571 | 0.0196 | 0.0496 | 0.0573 | 0.0545 | 0.0091 | 0.0174 | 0.0303 | 0.0238
SR | 0.0195 | 0.0465 | 0.0583 | 0.0540 | 0.0175 | 0.0415 | 0.0528 | 0.0486 | 0.0078 | 0.0145 | 0.0275 | 0.0216
CIESS | 0.0230 | 0.0534 | 0.6570 | 0.0640 | 0.0222 | 0.0540 | 0.0662 | 0.0613 | 0.0157 | 0.0383 | 0.0483 | 0.0446
BET | 0.0255 | 0.0596 | 0.0727 | 0.0676 | 0.0216 | 0.0505 | 0.0639 | 0.0588 | 0.0170 | 0.0404 | 0.0545 | 0.0491
$p$-value | 0.0403 | 0.0200 | 0.0305 | 0.0204 |  | 0.0327 | 0.0451 | 0.0064 | 0.0193 |  | 0.0399 | 0.0246 | 0.0094 | 0.0044
(b) Results on Yelp

Table 1. Performance of all methods on (a) Gowalla, and (b) Yelp2018 with the
sparsity ratio set to 80%, 90%, and 95%. The best results are highlighted.
Recall@$k$ and NDCG@$k$ are denoted by R@$k$ and N@$k$ for simplicity. Each
$p$-value is calculated via the paired $t$-test between BET and the best
baseline.

Sparsity | Selection Method | Yelp2018 | Gowalla
---|---|---|---
R@20 | N@20 | R@20 | N@20
80% | Random Selection | 0.0846 | 0.0788 | 0.1749 | 0.1473
Simple fitness prediction | 0.0835 | 0.0779 | 0.1739 | 0.1469
DeepSets fitness prediction | 0.0873 | 0.0816 | 0.1753 | 0.1483
90% | Random Selection | 0.0767 | 0.0717 | 0.1594 | 0.1357
Simple fitness prediction | 0.0768 | 0.0717 | 0.1590 | 0.1355
DeepSets fitness prediction | 0.0816 | 0.0762 | 0.1620 | 0.1376
95% | Random Selection | 0.0637 | 0.0583 | 0.1335 | 0.1143
Simple fitness prediction | 0.0673 | 0.0629 | 0.1349 | 0.1152
DeepSets fitness prediction | 0.0727 | 0.0676 | 0.1374 | 0.1171

Table 2. Performance of BET when different selection methods are used.

###  4.3. Implementation Notes for BET

Hereby we detail the implementation of BET. The number of search iterations
$T$ is 10. The maximal values of the distribution parameters $\beta_{max}$ are
set to 20 for power law distribution, 20 for truncated normal distribution, 5
for truncated exponential distribution and 0.5 for log normal distribution.
The maximal embedding size $d_{max}$ is 128 and the minimal embedding size
$d_{min}$ is 1. In each iteration, the pre-trained backbone recommender is
finetuned for 10 epochs (i.e., $h=10$) when the backbone recommender is
LightGCN or NCF. When NGCF is used, the backbone recommender is finetuned for
15 epochs (i.e., $h=15$). For finetuning, the initial learning rate is 0.03
and decays every 200 steps with a decay ratio of 0.98. The number of fitness
prediction updates in each iteration, or $N$, is set to 2. The user and item
encoders have input size 1, output size 16, and hidden size 16. The size
encoder has input size 17, output size 64, and hidden size 64. The decoder has
input size 64, output size 1, and hidden size 64. All the MLP-based encoders
and decoders have two layers with the Leaky RelU (Maas et al., 2013)
activation function.

###  4.4. Overall Performance Comparison

Table 1 shows the Recall@$k$ and NDCG@$k$ scores of various lightweight
embedding methods when they are paired with different backbone recommender
models. We also perform paired student’s $t$-test between BET and the best
baseline method. Compared with the baselines with adjustable sparsity ratios
(i.e., PEP, SU, SR, and CIESS), BET achieves better performance across all
three backbone recommenders and at nearly all specified sparsity levels (80%,
90%, and 95%). The associated $p$-values are below 5% most of the time so we
can reject the null hypothesis that the observed performance changes occurred
by chance in most scenarios.

Meanwhile, the sparsity ratios achieved by ESAPN and OptEmbed range from 67%
and 83%, often failing to meet the minimum target of 80%. Despite keeping a
relatively higher number of parameters compared to other methods, their
recommendation performance is consistently inferior to BET. For instance, when
used with LightGCN on the Gowalla dataset, ESAPN requires more memory
complexity (75% sparsity ratio) to achieve a comparable performance compared
to BET under an 90% sparsity condition.

In summary, BET demonstrates superior performance while maintaining the same
memory usage. Similarly, when comparing performance levels, BET proves to be
more efficient in memory utilization. Consequently, these findings highlight
the benefits of the ongoing embedding size exploration in BET.

###  4.5. Model Component Analysis

Selecting from the set of candidate actions in each iteration is a vital step
in BET, where we proposed a hybrid selection method that alternately uses
three selection strategies. To study the effectiveness of the proposed
selection method and the DeepSets-based fitness prediction network, we conduct
a quantitative study on BET and its two variants. The first variant uses
random selection only. The second variant incorporates a simplistic fitness
prediction network that only accepts the embedding sizes of each user and
item. Table 2 illustrates the performance of the original version of BET along
with its two variants in terms of their Recall@20 and NDCG@20 scores. Only the
performance of the best-performing backbone recommender LightGCN is reported
due to space limit. The results show that BET with a DeepSets-based fitness
prediction network has better performance than the two variants, implying that
our proposed selection method is effective and the DeepSets-based architecture
is able to learn the latent representation of actions.

###  4.6. Analysis of Hyperparameter

In this section, we inspect the influence of the core hyperparameters in BET
in terms of Recall@20 and NDCG@20. LightGCN shows the best performance so it
is chosen as the backbone recommender for demonstration.

####  4.6.1. Number of Sampled Actions $m$

In each iteration, we sample $m$ candidate actions to choose from. We tune BET
with $m\in\\{20,40,60,80,100\\}$ for both Yelp2018 and Gowalla. Then we show
how the performance changes with different values of $m$ w.r.t. Recall@20 and
NDCG@20 scores. As shown in Figure 4, although BET is relatively insensitive
to the value of $m$, better performance is achieved with $m=100$ on both
datasets.

(a) Effect of $m$ on Yelp

(b) Effect of $m$ on Gowalla

Figure 4. Sensitivity analysis of $m$ on Recall@20 and NDCG@20 with LightGCN
as the backbone recommender.

####  4.6.2. Number of Iterations $T$

(a) Effect of $T$ on Yelp

(b) Effect of $T$ on Gowalla

Figure 5. Sensitivity analysis of $T$ on Recall@20 and NDCG@20 with LightGCN
as the backbone recommender. Figure 6. Sensitivity analysis of $T$ on the
fitness prediction loss with LightGCN as the backbone recommender on Yelp2018
(left) and Gowalla (right).

BET iteratively samples, selects and evaluates actions in search of the
optimal one. One natural question that arises is how many iterations it needs
to find the action that leads to peak recommendation performance. This
question is crucial for understanding the effectiveness of BET. Therefore, we
perform selective retraining as described in Section 3.5.2 on the resulting
population $\mathcal{A}_{t}$ for each training iteration $t\in[0,T]$ and
report the best recommendation performance in terms of Recall@20 and NDCG@20.
We also report the average fitness prediction loss in each iteration. Figure 6
shows that fitness predictor converges within the first 15 iterations. While
the training loss of the fitness predictor is diminishing, BET witnesses a
substantial improvement in its performance in the first 10-15 iterations, as
shown in Figure 5. Then performance continues to climb slowly after the 15th
iteration until it reaches its peak around the 40th iteration and remains
unchanged after that. In conclusion, setting $T=50$ should suffice for BET to
find the optimal action for all sparsity ratios.

##  5\. Conclusion and Future Works

Latent factor recommenders typically employ vectorized embeddings with a
uniform and fixed size, resulting in suboptimal performance and excessive
memory complexity. To address this challenge, we have introduced BET, a model-
agnostic algorithm to enable the selection of customized embedding sizes at
the level of the embedding table for each user/item. This approach enhances
the representational power of the embeddings while minimizing memory costs.
Moving forward, we will endeavor to develop embedding size search algorithms
that determine embedding sizes for unseen users/items without the need to
retrain the algorithm from scratch.

## Acknowledgment

This work is supported by the Australian Research Council under the streams of
Future Fellowship (No. FT210100624), Discovery Project (No. DP190101985), and
Discovery Early Career Research Award (No. DE200101465 and No. DE230101033).

## References

  * (1)
  * Aditya Desai (2022) Anshumali Shrivastava Aditya Desai, Li Chou. 2022.  Random Offset Block Embedding (ROBE) for compressed embedding tables in deep learning recommendation systems. In _Proceedings of Machine Learning and Systems_. 762–778.
  * Blondel et al. (2016) Mathieu Blondel, Akinori Fujino, Naonori Ueda, and Masakazu Ishihata. 2016.  Higher-Order Factorization Machines. In _Proceedings of the 30th International Conference on Neural Information Processing Systems_. 3359–3367.
  * Cai et al. (2020) Han Cai, Chuang Gan, Tianzhe Wang, Zhekai Zhang, and Song Han. 2020.  Once-for-all: Train one network and specialize it for efficient deployment.  _ICLR_ (2020).
  * Chen et al. (2017) Cheng Chen, Xiangwu Meng, Zhenghua Xu, and Thomas Lukasiewicz. 2017\.  Location-Aware Personalized News Recommendation With Deep Semantic Analysis.  _IEEE Access_ 5 (2017), 1624–1638.
  * Chen et al. (2020a) Ting Chen, Lala Li, and Yizhou Sun. 2020a.  Differentiable Product Quantization for End-to-End Embedding Compression. In _Proceedings of the 37th International Conference on Machine Learning_. 1617–1626.
  * Chen et al. (2020b) Tong Chen, Hongzhi Yin, Guanhua Ye, Zi Huang, Yang Wang, and Meng Wang. 2020b.  Try This Instead: Personalized and Interpretable Substitute Recommendation. In _SIGIR_. 891–900.
  * Chen et al. (2021a) Tong Chen, Hongzhi Yin, Xiangliang Zhang, Zi Huang, Yang Wang, and Meng Wang. 2021a.  Quaternion Factorization Machines: A Lightweight Solution to Intricate Feature Interaction Modeling.  _IEEE Transactions on Neural Networks and Learning Systems_ (2021), 1–14.
  * Chen et al. (2021b) Tong Chen, Hongzhi Yin, Yujia Zheng, Zi Huang, Yang Wang, and Meng Wang. 2021b.  Learning Elastic Embeddings for Customizing On-Device Recommenders. In _SIGKDD_. 138–147.
  * Chen et al. (2022a) Yankai Chen, Huifeng Guo, Yingxue Zhang, Chen Ma, Ruiming Tang, Jingjie Li, and Irwin King. 2022a.  Learning Binarized Graph Representations with Multi-Faceted Quantization Reinforcement for Top-K Recommendation. In _SIGKDD_. 168–178.
  * Chen et al. (2022b) Yankai Chen, Huifeng Guo, Yingxue Zhang, Chen Ma, Ruiming Tang, Jingjie Li, and Irwin King. 2022b.  Learning Binarized Graph Representations with Multi-Faceted Quantization Reinforcement for Top-K Recommendation. In _KDD_. 168–178.
  * Desai and Shrivastava (2022) Aditya Desai and Anshumali Shrivastava. 2022.  The trade-offs of model size in large recommendation models : 100GB to 10MB Criteo-tb DLRM model. In _NeurIPS_. 33961–33972.
  * Gong and Zhang (2016) Yuyun Gong and Qi Zhang. 2016.  Hashtag recommendation using attention-based convolutional neural network.. In _IJCAI_. 2782–2788.
  * Guo et al. (2017) Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017.  DeepFM: a factorization-machine based neural network for CTR prediction.  _IJCAI_ (2017).
  * He et al. (2020) Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. 2020\.  Lightgcn: Simplifying and powering graph convolution network for recommendation. In _SIGIR_. 639–648.
  * He et al. (2017) Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017\.  Neural collaborative filtering. In _WWW_. 173–182.
  * Huang et al. (2019) Zhenhua Huang, Xin Xu, Juan Ni, Honghao Zhu, and Cheng Wang. 2019.  Multimodal representation learning for recommendation in Internet of Things.  _IEEE Internet of Things Journal_ 6, 6 (2019), 10675–10685.
  * Jiang et al. (2021) Gangwei Jiang, Hao Wang, Jin Chen, Haoyu Wang, Defu Lian, and Enhong Chen. 2021\.  XLightFM: Extremely Memory-Efficient Factorization Machine. In _SIGIR_. 337–346.
  * Joglekar et al. (2020) Manas R Joglekar, Cong Li, Mei Chen, Taibai Xu, Xiaoming Wang, Jay K Adams, Pranav Khaitan, Jiahui Liu, and Quoc V Le. 2020.  Neural input search for large scale recommendation models. In _SIGKDD_. 2387–2397.
  * Kang et al. (2020) Wang-Cheng Kang, Derek Zhiyuan Cheng, Ting Chen, Xinyang Yi, Dong Lin, Lichan Hong, and Ed H. Chi. 2020\.  Learning Multi-Granular Quantized Embeddings for Large-Vocab Categorical Features in Recommender Systems. In _WWW_. 562–566.
  * Keskar et al. (2016) Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Peter Tang. 2016.  On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima. In _ICLR_.
  * Ko et al. (2021) Yunyong Ko, Jae-Seo Yu, Hong-Kyun Bae, Yongjun Park, Dongwon Lee, and Sang-Wook Kim. 2021\.  MASCOT: A Quantization Framework for Efficient Matrix Factorization in Recommender Systems. In _2021 IEEE International Conference on Data Mining (ICDM)_. 290–299.
  * Kong et al. (2023) Shuming Kong, Weiyu Cheng, Yanyan Shen, and Linpeng Huang. 2023.  AutoSrh: An Embedding Dimensionality Search Framework for Tabular Data Prediction.  _TKDE_ 35, 7 (2023), 6673–6686.
  * Kywe et al. (2012) Su Mon Kywe, Ee-Peng Lim, and Feida Zhu. 2012.  A survey of recommender systems in twitter. In _International Conference on Social Informatics_. 420–433.
  * Lee et al. (2018) Joonseok Lee, Sami Abu-El-Haija, Balakrishnan Varadarajan, and Apostol Natsev. 2018.  Collaborative deep metric learning for video understanding. In _Proceedings of the 24th ACM SIGKDD International conference on knowledge discovery & data mining_. 481–490.
  * Li et al. (2017) Piji Li, Zihao Wang, Zhaochun Ren, Lidong Bing, and Wai Lam. 2017.  Neural rating regression with abstractive tips generation for recommendation. In _Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval_. 345–354.
  * Li et al. (2023) Shiwei Li, Huifeng Guo, Lu Hou, Wei Zhang, Xing Tang, Ruiming Tang, Rui Zhang, and Ruixuan Li. 2023.  Adaptive Low-Precision Training for Embeddings in Click-Through Rate Prediction.  _Proceedings of the AAAI Conference on Artificial Intelligence_ 37, 4 (2023), 4435–4443.
  * Li and Karahanna (2015) Siyuan Li and Elena Karahanna. 2015.  Online Recommendation Systems in a B2C E-Commerce Context: A Review and Future Directions.  _Journal of the Association for Information Systems_ 16 (2015), 72–107.
  * Li et al. (2021) Yang Li, Tong Chen, Peng-Fei Zhang, and Hongzhi Yin. 2021\.  Lightweight self-attentive sequential recommendation. In _CIKM_. 967–977.
  * Liang et al. (2016) Dawen Liang, Laurent Charlin, James McInerney, and David M. Blei. 2016.  Modeling User Exposure in Recommendation. In _WWW_. 951–961.
  * Lin et al. (2022a) Guanyu Lin, Chen Gao, Yinfeng Li, Y. Zheng, Zhiheng Li, Depeng Jin, and Yong Li. 2022a.  Dual Contrastive Network for Sequential Recommendation with User and Item-Centric Perspectives.  _ACM Transactions on Information Systems_ 1, 1 (2022).
  * Lin et al. (2022b) Weilin Lin, Xiangyu Zhao, Yejing Wang, Tong Xu, and Xian Wu. 2022b.  AdaFS: Adaptive Feature Selection in Deep Recommender System. In _Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining_. 3309–3317.
  * Liu et al. (2023) Han Liu, Yinwei Wei, Jianhua Yin, and Liqiang Nie. 2023\.  HS-GCN: Hamming Spatial Graph Convolutional Networks for Recommendation.  _TKDE_ 35, 6 (2023), 5977–5990.
  * Liu et al. (2020) Haochen Liu, Xiangyu Zhao, Chong Wang, Xiaobing Liu, and Jiliang Tang. 2020.  Automated embedding size search in deep recommender systems. In _SIGIR_. 2307–2316.
  * Liu et al. (2021) Siyi Liu, Chen Gao, Yihong Chen, Depeng Jin, and Yong Li. 2021.  Learnable Embedding sizes for Recommender Systems. In _ICLR_.
  * Luo et al. (2019) Yuanfei Luo, Mengshuo Wang, Hao Zhou, Quanming Yao, Wei-Wei Tu, Yuqiang Chen, Wenyuan Dai, and Qiang Yang. 2019\.  Autocross: Automatic feature crossing for tabular data in real-world applications. In _SIGKDD_. 1936–1945.
  * Lyu et al. (2022) Fuyuan Lyu, Xing Tang, Hong Zhu, Huifeng Guo, Yingxue Zhang, Ruiming Tang, and Xue Liu. 2022.  OptEmbed: Learning Optimal Embedding Table for Click-through Rate Prediction. In _CIKM_. 1399–1409.
  * Maas et al. (2013) Andrew L Maas, Awni Y Hannun, Andrew Y Ng, et al. 2013\.  Rectifier nonlinearities improve neural network acoustic models. In _ICML_.
  * Muhammad et al. (2020) Khalil Muhammad, Qinqin Wang, Diarmuid O’Reilly-Morgan, Elias Tragos, Barry Smyth, Neil Hurley, James Geraci, and Aonghus Lawlor. 2020\.  Fedfast: Going beyond average for faster training of federated recommender systems. In _SIGKDD_. 1234–1242.
  * Qu et al. (2022) Liang Qu, Yonghong Ye, Ningzhi Tang, Lixin Zhang, Yuhui Shi, and Hongzhi Yin. 2022\.  Single-Shot Embedding Dimension Search in Recommender System. In _SIGIR_. 513–522.
  * Qu et al. (2023) Yunke Qu, Tong Chen, Xiangyu Zhao, Lizhen Cui, Kai Zheng, and Hongzhi Yin. 2023\.  Continuous Input Embedding Size Search For Recommender Systems. In _SIGIR_. 708–717.
  * Rendle (2010) Steffen Rendle. 2010\.  Factorization machines. In _2010 IEEE International conference on data mining_. 995–1000.
  * Rendle et al. (2009) Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009.  BPR: Bayesian Personalized Ranking from Implicit Feedback. In _Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence_. 452–461.
  * Sedaghati et al. (2015) Naser Sedaghati, Te Mu, Louis-Noel Pouchet, Srinivasan Parthasarathy, and P Sadayappan. 2015\.  Automatic selection of sparse matrix representation on GPUs. In _ACM International Conference on Supercomputing_. 99–108.
  * Sedhain et al. (2015) Suvash Sedhain, Aditya Krishna Menon, Scott Sanner, and Lexing Xie. 2015.  AutoRec: Autoencoders Meet Collaborative Filtering. In _WWW_. 111–112.
  * Sethi et al. (2022) Geet Sethi, Bilge Acun, Niket Agarwal, Christos Kozyrakis, Caroline Trippel, and Carole-Jean Wu. 2022\.  RecShard: Statistical Feature-Based Memory Optimization for Industry-Scale Neural Recommendation. In _Proceedings of the 27th ACM International Conference on Architectural Support for Programming Languages and Operating Systems_. 344–358.
  * Shi et al. (2020) Hao-Jun Michael Shi, Dheevatsa Mudigere, Maxim Naumov, and Jiyan Yang. 2020.  Compositional embeddings using complementary partitions for memory-efficient recommendation systems. In _SIGKDD_. 165–175.
  * Sun et al. (2020) Yang Sun, Fajie Yuan, Min Yang, Guoao Wei, Zhou Zhao, and Duo Liu. 2020\.  A Generic Network Compression Framework for Sequential Recommender Systems. In _SIGIR ’20_. 1299–1308.
  * Virtanen et al. (2020) Pauli Virtanen et al. 2020\.  SciPy 1.0: fundamental algorithms for scientific computing in Python.  _Nature Methods_ 17, 3 (2020), 261–272.
  * Wagstaff et al. (2022) Edward Wagstaff, Fabian B Fuchs, Martin Engelcke, Michael A Osborne, and Ingmar Posner. 2022\.  Universal approximation of functions on sets.  _JMLR_ 23, 1 (2022), 6762–6817.
  * Wang et al. (2021) Shoujin Wang, Longbing Cao, Yan Wang, Quan Z Sheng, Mehmet A Orgun, and Defu Lian. 2021\.  A survey on session-based recommender systems.  _ACM Computing Surveys (CSUR)_ 54, 7 (2021), 1–38.
  * Wang et al. (2017) Suhang Wang, Yilin Wang, Jiliang Tang, Kai Shu, Suhas Ranganath, and Huan Liu. 2017\.  What Your Images Reveal: Exploiting Visual Contents for Point-of-Interest Recommendation. In _Proceedings of the 26th International Conference on World Wide Web_. 391–400.
  * Wang et al. (2019) Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019.  Neural graph collaborative filtering. In _SIGIR_. 165–174.
  * Wang et al. (2022) Yejing Wang, Xiangyu Zhao, Tong Xu, and Xian Wu. 2022\.  Autofield: Automating feature selection in deep recommender systems. In _WWW_. 1977–1986.
  * X Liang (2023) QVH Nguyen J Li H Yin X Liang, T Chen. 2023\.  Learning Compact Compositional Embeddings via Regularized Pruning for Recommendation. In _ICDM_.
  * Xia et al. (2022) Xin Xia, Hongzhi Yin, Junliang Yu, Qinyong Wang, Guandong Xu, and Quoc Viet Hung Nguyen. 2022.  On-Device Next-Item Recommendation with Self-Supervised Knowledge Distillation. In _SIGIR_. 546–555.
  * Xia et al. (2023) Xin Xia, Junliang Yu, Qinyong Wang, Chaoqun Yang, Nguyen Quoc Viet Hung, and Hongzhi Yin. 2023.  Efficient On-Device Session-Based Recommendation.  _ACM Transactions Information Systems_ 41, 4 (2023).
  * Yan et al. (2021) Bencheng Yan, Pengjie Wang, Jinquan Liu, Wei Lin, Kuang-Chih Lee, Jian Xu, and Bo Zheng. 2021.  Binary Code Based Hash Embedding for Web-Scale Applications. In _CIKM_. 3563–3567.
  * Yu et al. (2022) Junliang Yu, Hongzhi Yin, Xin Xia, Tong Chen, Lizhen Cui, and Quoc Viet Hung Nguyen. 2022\.  Are Graph Augmentations Necessary? Simple Graph Contrastive Learning for Recommendation. In _SIGIR_. 1294–1303.
  * Zaheer et al. (2017) Manzil Zaheer, Satwik Kottur, Siamak Ravanbhakhsh, Barnabás Póczos, Ruslan Salakhutdinov, and Alexander J Smola. 2017.  Deep Sets. In _NeurIPS_. 3394–3404.
  * Zhang et al. (2020) Caojin Zhang, Yicun Liu, Yuanpu Xie, Sofia Ira Ktena, Alykhan Tejani, Akshay Gupta, Pranay Kumar Myana, Deepak Dilipkumar, Suvadip Paul, Ikuhiro Ihara, et al. 2020\.  Model size reduction using frequency based double hashing for recommender systems. In _Fourteenth ACM Conference on Recommender Systems_. 521–526.
  * Zhang et al. (2019) Shuai Zhang, Lina Yao, Aixin Sun, and Yi Tay. 2019\.  Deep learning based recommender system: A survey and new perspectives.  _ACM Computing Surveys (CSUR)_ 52, 1 (2019), 1–38.
  * Zhao et al. (2021) Xiangyu Zhao, Haochen Liu, Wenqi Fan, Hui Liu, Jiliang Tang, Chong Wang, Ming Chen, Xudong Zheng, Xiaobing Liu, and Xiwang Yang. 2021\.  Autoemb: Automated embedding dimensionality search in streaming recommendations. In _2021 IEEE International Conference on Data Mining (ICDM)_. 896–905.
  * Zhao et al. (2020) Xiangyu Zhao, Chong Wang, Ming Chen, Xudong Zheng, Xiaobing Liu, and Jiliang Tang. 2020\.  AutoEmb: Automated Embedding Dimensionality Search in Streaming Recommendations.  _2021 IEEE International Conference on Data Mining (ICDM)_ (2020), 896–905.
  * Zheng et al. (2022) Ruiqi Zheng, Liang Qu, Bin Cui, Yuhui Shi, and Hongzhi Yin. 2022.  AutoML for Deep Recommender Systems: A Survey.  _ACM Transactions on Information Systems_ (2022).
  * Zhou et al. (2018) Meizi Zhou, Zhuoye Ding, Jiliang Tang, and Dawei Yin. 2018\.  Micro Behaviors: A New Perspective in E-Commerce Recommender Systems. In _Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining_. 727–735.


#  $n$-th Relative Nilpotency Degree
and Relative $n$-Isoclinism Classes

Rashid Rezaei  Department of Mathematics, Faculty of Mathematical Sciences
University of Malayer
Post Box: 657719, 95863, Malayer, Iran<EMAIL_ADDRESS>and
Francesco G. Russo  Departement of Mathematics
University of Palermo
via Archirafi 14, 90123, Palermo, Italy<EMAIL_ADDRESS>
###### Abstract.

P. Hall introduced the notion of isoclinism between two groups more than 60
years ago. Successively, many authors have extended such a notion in different
contexts. The present paper deals with the notion of relative $n$-isoclinism,
given by N. S. Hekster in 1986, and with the notion of $n$-th relative
nilpotency degree, recently introduced in literature.

###### Key words and phrases:

Nilpotency degree, relative $n$-th nilpotency degree, relative $n$-isoclinism
classes

###### 2010 Mathematics Subject Classification:

Primary: 20D60, 20P05; Secondary: 20D08, 20D15.

##  1\. Introduction and Statement of Results

Every locally compact topological group $G$ admits a left Haar measure
$\mu_{G}$, which is a positive Radon measure on a $\sigma$-algebra containing
Borel sets with the property that $\mu_{G}(xE)=\mu_{G}(E)$ for each element
$x$ of the measure space $G$ (see [13, Theorem 2.8]). The support of $\mu_{G}$
is $G$ and it is usually unbounded, but if $G$ is compact, then $\mu_{G}$ is
bounded. For this reason we may assume without ambiguity that a compact group
$G$ has a unique probability measure space, where $\mu_{G}$ is normalized,
that is, $\mu_{G}(G)=1$.

Let $G$ be a compact group with the normalized Haar measure $\mu_{G}$. On the
product measure space $G\times G$, it is possible to consider the product
measure $\mu_{G}\times\mu_{G}$. If

$C_{2}=\\{(x,y)\in G\times G\ |\ [x,y]=1\\},$

then $C_{2}=f^{-1}(1)$, where $f:G\times G\rightarrow G$ is defined via
$f(x,y)=[x,y]$. It is clear that $f$ is continuous and so $C_{2}$ is a compact
and measurable subset of $G\times G$. Therefore it is possible to define the
$commutativity$ $degree$ of $G$ as

$d(G)=(\mu_{G}\times\mu_{G})(C_{2}).$

We may extend $d(G)$ as follows. Suppose that $n\geq 1$ is an integer, $G^{n}$
is the product of $n$-copies of $G$ and $\mu^{n}_{G}$ that of $n$-copies of
$\mu_{G}$. The $n$-$th$ $nilpotency$ $degree$ of $G$ is defined to be

$d^{(n)}(G)=\mu^{n+1}_{G}(C_{n+1}),$

where

$C_{n+1}=\\{(x_{1},\ldots,x_{n+1})\in G^{n+1}\ |\
[x_{1},x_{2},...,x_{n+1}]=1\\}.$

Obviously, if $G$ is finite, then $G$ is a compact group with the discrete
topology and so the Haar measure of $G$ is the counting measure. Then we have
as special situation for a finite group $G$:

$d^{(n)}(G)=\mu^{n+1}_{G}(C_{n+1})=\frac{|C_{n+1}|}{|G|^{n+1}}.$

When $n=1$, literature can be found in [5, 6, 7, 8, 9, 14].

Let $G$ be a compact group and $H$ be a closed subgroup of $G$. It is possible
to define

$D_{2}=\\{(h,g)\in H\times G\ |\ [h,g]=1\\}.$

Then $D_{2}=\phi^{-1}(1)$, where $\phi:H\times G\rightarrow G$ is defined via
$\phi(h,g)=[h,g]$. It is clear that $\phi$ is continuous and so $D_{2}$ is a
compact and measurable subset of $H\times G$. Note that $\phi$ is the
restriction of $f$ under $H\times G$. This remark shows why $H$ has to be
required as closed subgroup of $G$. Then we may define the $relative$
$commutativity$ $degree$ of $H$ with respect to $G$ as

$d(H,G)=(\mu_{H}\times\mu_{G})(D_{2}).$

Considering

$D_{n+1}=\\{(h_{1},,...,h_{n},g)\in H^{n}\times G\ |\
[h_{1},h_{2},...,h_{n},g]=1\\},$

we define the $relative$ $n$-$th$ $nilpotency$ $degree$ of $H$ with respect to
$G$ as

$d^{(n)}(H,G)=(\mu^{n}_{H}\times\mu_{G})(D_{n+1}).$

The following notion is fundamental for stating our results (see for
terminology [1, 2, 6, 10, 11, 12, 13, 15]).

######  Definition 1.1.

Let $G_{1}$, $G_{2}$ be two groups, $H_{1}$ a subgroup of $G_{1}$ and $H_{2}$
a subgroup of $G_{2}$. A pair $(\alpha,\beta)$ is said to be a relative
$n$-isoclinism from $(H_{1},G_{1})$ to $(H_{2},G_{2})$ if we have the
following conditions:

  1. (i)

$\alpha$ is an isomorphism from $G_{1}/Z_{n}(G_{1})$ to $G_{2}/Z_{n}(G_{2})$
such that the restriction of $\alpha$ under $H_{1}/(Z_{n}(G_{1})\cap H_{1})$
is an isomorphism from $H_{1}/(Z_{n}(G_{1})\cap H_{1})$ to
$H_{2}/(Z_{n}(G_{2})\cap H_{2})$, that is, the map
$\alpha^{n+1}:\frac{H_{1}}{Z_{n}(G_{1})\cap
H_{1}}\times...\times\frac{H_{1}}{Z_{n}(G_{1})\cap
H_{1}}\times\frac{G_{1}}{Z_{n}(G_{1})}\rightarrow\frac{H_{2}}{Z_{n}(G_{2})\cap
H_{2}}\times...\times\frac{H_{2}}{Z_{n}(G_{2})\cap
H_{2}}\times\frac{G_{2}}{Z_{n}(G_{2})}$ is an isomorphism;

  2. (ii)

$\beta$ is an isomorphism from $[_{n}H_{1},G_{1}]$ to $[_{n}H_{2},G_{2}]$;

  3. (iii)

Considering for each $h_{1},...,h_{n}\in H_{1}$, $k_{1},...,k_{n}\in H_{2}$,
$g_{1}\in G_{1}$, $g_{2}\in G_{2}$ there exists a commutative diagram in which
the map

$\gamma(n,H_{1},G_{1}):((h_{1}(Z_{n}(G_{1})\cap
H_{1}),...,h_{n}(Z_{n}(G_{1})\cap
H_{1}),g_{1}Z_{n}(G_{1})))\in\frac{H_{1}}{Z_{n}(G_{1})\cap H_{1}}\times...$
$...\times\frac{H_{1}}{Z_{n}(G_{1})\cap
H_{1}}\times\frac{G_{1}}{Z_{n}(G_{1})}\mapsto[h_{1},...,h_{n},g_{1}]\in[_{n}H_{1},G_{1}]$

and the map

$\gamma(n,H_{2},G_{2}):((k_{1}(Z_{n}(G_{2})\cap
H_{2}),...,k_{n}(Z_{n}(G_{2})\cap
H_{2}),g_{2}Z_{n}(G_{2})))\in\frac{H_{2}}{Z_{n}(G_{2})\cap H_{2}}\times...$
$...\times\frac{H_{2}}{Z_{n}(G_{2})\cap
H_{2}}\times\frac{G_{2}}{Z_{n}(G_{2})}\mapsto[k_{1},...,k_{n},g_{2}]\in[_{n}H_{2},G_{2}],$

can be composed by the rule

$\gamma(n,H_{2},G_{2})\ \circ\ \alpha^{n+1}=\beta\ \circ\
\gamma(n,H_{1},G_{1}).$

It is easy to see that Definition 1.1 is well posed. In particular, two groups
$G_{1}$ and $G_{2}$ in Definition 1.1 are called $isoclinic$ if $n=1$,
$H_{1}=G_{1}$ and $H_{2}=G_{2}$.

P. Hall already pointed out that two groups which are isoclinic can allow us
to define the $isoclinism$ as an equivalence relation [11]. Then it is
possible to classify two groups with respect to their $isoclinism$ $class$.
See [10] as general reference on the topic. Successively, this crucial passage
was pointed out by J. Bioch in [1, 2] and by N. S. Hekster in [12] with
respect to generalizations of the notion of isoclinism. At the same way,
Definition 1.1 allows us to define an equivalence relation, which extends that
of P. Hall in [11], that of J. Bioch in [1, 2] and that of N. S. Hekster in
[12]. Two groups which satisfy Definition 1.1 are called $relative$
$n$-$isoclinic$ and we will write briefly $(H_{1},G_{1})\ _{\widetilde{n}}\
(H_{2},G_{2})$. It is easy to see that it is possible to classify two groups
with respect to their $relative$ $n$-$isoclinism$ $class$.

The main results of the present paper are listed below and they generalize [6,
Theorem 4.2].

######  Theorem 1.2.

Assume that $G_{1}$ and $G_{2}$ are two compact groups, $H_{1}$ is a closed
subgroup of $G_{1}$ and $H_{2}$ is a closed subgroup of $G_{2}$. If
$(H_{1},G_{1})\ _{\widetilde{n}}\ (H_{2},G_{2})$, then
$d^{(n)}(H_{1},G_{1})=d^{(n)}(H_{2},G_{2})$

######  Theorem 1.3.

Let $G$ be a compact group and $H$ be a closed subgroup of $G$. If
$G=HZ_{n}(G)$, then $d^{(n)}(H)=d^{(n)}(H,G)=d^{(n)}(G).$

##  2\. Proof of Main Theorems and Some Consequences

This section is devoted to describe our main results.

######  Lemma 2.1.

Assume that $G$ is a compact group, $H$ is a closed subgroup of $G$ and
$C_{G}([h_{1},...,h_{n}])$ is the centralizer of the commutator
$[h_{1},...,h_{n}]$ in $G$ for some elements $h_{1},...,h_{n}$ in $H$. Then

$d^{(n)}(H,G)=\int_{H}\ldots\left(\int_{H}\mu_{G}(C_{G}([h_{1},...,h_{n}]))d\mu_{H}(h_{1})\right)\ldots
d\mu_{H}(h_{n}),$

where

$\mu_{G}(C_{G}([h_{1},...,h_{n}]))=\int_{G}\chi_{{}_{D_{n+1}}}(h_{1},...,h_{n},g)d\mu_{G}(g),$

and $\chi_{{}_{D_{n+1}}}$ denotes the characteristic map of the set $D_{n+1}$.

###### Proof.

Since

$\mu_{G}(C_{G}([h_{1},...,h_{n}]))=\int_{G}\chi_{{}_{D_{n+1}}}(h_{1},...,h_{n},g)d\mu_{G}(g)$

we have by Fubini-Tonelli’s Theorem:

$\begin{array}[]{lcl}d^{(n)}(H,G)&=&(\mu^{n}_{H}\times\mu_{G})(D_{n+1})=\int_{H^{n}\times
G}\chi_{{}_{D_{n+1}}}(d\mu^{n}_{H}\times d\mu_{G})\vspace{.3cm}\\\
&=&\int_{H}...\left(\int_{H}\left(\int_{G}\chi_{{}_{D_{n+1}}}(h_{1},...,h_{n},g)d\mu_{G}(g)\right)d\mu_{H}(h_{1})\right)...\
d\mu_{H}(h_{n})\vspace{.3cm}\\\
&=&\int_{H}...\left(\int_{H}\mu(C_{G}([h_{1},...,h_{n}]))d\mu_{H}(h_{1})\right)...\
d\mu_{H}(h_{n}).\end{array}$

∎

We recall to convenience of the reader [4, Section 6, Theorem 3]. Let
$(\Omega_{1},\mathcal{A}_{1})$ and $(\Omega_{2},\mathcal{A}_{2})$ be two
measurable spaces. The mapping $X:\Omega_{1}\rightarrow\Omega_{2}$ is said to
be a $measurable$ $transformation$ from $(\Omega_{1},\mathcal{A}_{1})$ to
$(\Omega_{2},\mathcal{A}_{2})$ if
$X^{-1}(\mathcal{A}_{2})\subseteq\mathcal{A}_{1}.$ Now for a measurable
transformation $X$ from a measure space $(\Omega_{1},\mathcal{A}_{1},\mu)$ to
a measurable space $(\Omega_{2},\mathcal{A}_{2})$ one can check that the
measure $\mu$ induces a measure $\nu$ on $\mathcal{A}_{2}$ via
$\nu\\{A\\}=\mu\\{X^{-1}(A)\\},$ for all $A\in\mathcal{A}_{2}.$ This induced
measure is denoted by $\mu X^{-1}.$ Let $\phi$ be an isomorphism from a
compact group $G_{1}$ with Haar measure $\mu_{G_{1}}$ onto a compact group
$G_{2}$ with Haar measure $\mu_{G_{2}}$. It is clear that $\phi$ is a
measurable transformation from $G_{1}$ to $G_{2}$, then by the uniqueness of
the Haar measure on compact groups, we have that
$\mu_{G_{2}}=\mu_{G_{1}}\phi^{-1}$ and so

$(*)\hskip
56.9055pt\int_{G_{2}}fd\mu_{G_{2}}=\int_{G_{1}}(f\phi)d\mu_{G_{1}}.$

###### Proof of Theorem 1.2.

Let $(\alpha,\beta)$ be a relative $n$-isoclinism from $(H_{1},G_{1})$ to
$(H_{2},G_{2})$ as in Definition 1.1. Assume that $n=1$. Let $\mu_{H_{1}}$ be
a normalized Haar measure on $H_{1}$, $\mu_{H_{2}}$ on $H_{2}$, $\mu_{G_{1}}$
on $G_{1}$, $\mu_{G_{2}}$ on $G_{2}$, $\nu_{H_{1}}$ on $H_{1}/(H_{1}\cap
Z(G_{1}))$, $\nu_{H_{2}}$ on $H_{2}/(H_{2}\cap Z(G_{2}))$, $\nu_{G_{1}}$ on
$G_{1}/Z(G_{1})$ and $\nu_{G_{2}}$ on $G_{2}/Z(G_{2})$. Note that these
requirements are well posed, because $H_{1}$ is a closed subgroup of $G_{1}$
and $H_{2}$ is a closed subgroup of $G_{2}$.

With obvious meaning of symbols, we have

$\begin{array}[]{lcl}d(H_{1},G_{1})&=&\int_{H_{1}}\int_{G_{1}}\chi_{{}_{D_{2}}}(h_{1},g_{1})d\mu_{G_{1}}(g_{1})d\mu_{H_{1}}(h_{1})\vspace{.3cm}\\\
&=&\int_{H_{1}}\left(\int_{\frac{G_{1}}{Z(G_{1})}}\left(\int_{Z(G_{1})}\chi_{{}_{D_{2}}}(h_{1},z_{1}g_{1})d\mu_{Z(G_{1})}(z_{1})\right)d\nu_{G_{1}}(\bar{g_{1}})\right)d\mu_{H_{1}}(h_{1})\vspace{.3cm}\\\
&=&\int_{H_{1}}\left(\int_{\frac{G_{1}}{Z(G_{1})}}\left(\int_{Z(G_{1})}\chi_{{}_{D_{2}}}(h_{1},g_{1})d\mu_{Z(G_{1})}(z_{1})\right)d\nu_{G_{1}}(\bar{g_{1}})\right)d\mu_{H_{1}}(h_{1})\vspace{.3cm}\\\
&=&\int_{H_{1}}\left(\int_{\frac{G_{1}}{Z(G_{1})}}\chi_{{}_{D_{2}}}(h_{1},g_{1})\left(\int_{Z(G_{1})}d\mu_{Z(G_{1})}(z_{1})\right)d\nu_{G_{1}}(\bar{g_{1}})\right)d\mu_{H_{1}}(h_{1})\vspace{.3cm}\\\
&=&\int_{H_{1}}\left(\int_{\frac{G_{1}}{Z(G_{1})}}\chi_{{}_{D_{2}}}(h_{1},g_{1})d\nu_{G_{1}}(\bar{g_{1}})\right)d\mu_{H_{1}}(h_{1})\vspace{.3cm}\\\
\end{array}$

now by Fubini-Tonelli’s Theorem we have

$\begin{array}[]{lcl}=\int_{\frac{G_{1}}{Z(G_{1})}}\left(\int_{H_{1}}\chi_{{}_{D_{2}}}(h_{1},g_{1})d\mu_{H_{1}}(h_{1})\right)d\nu_{G_{1}}(\bar{g_{1}})\vspace{.3cm}\\\
\end{array}$

then we may proceed as before:

$\begin{array}[]{lcl}=\int_{\frac{G_{1}}{Z(G_{1})}}\left(\int_{\frac{H_{1}}{H_{1}\cap
Z(G_{1})}}\left(\int_{H_{1}\cap
Z(G_{1})}\chi_{{}_{D_{2}}}(h_{1}\zeta_{1},g_{1})d\mu_{Z(G_{1})\cap
H_{1}}(\zeta_{1})\right)d\nu_{H_{1}}(\bar{h_{1}})\right)d\nu_{G_{1}}(\bar{g_{1}})\vspace{.3cm}\\\
=\int_{\frac{G_{1}}{Z(G_{1})}}\left(\int_{\frac{H_{1}}{H_{1}\cap
Z(G_{1})}}\left(\int_{H_{1}\cap
Z(G_{1})}\chi_{{}_{D_{2}}}(h_{1},g_{1})d\mu_{Z(G_{1})\cap
H_{1}}(\zeta_{1})\right)d\nu_{H_{1}}(\bar{h_{1}})\right)d\nu_{G_{1}}(\bar{g_{1}})\vspace{.3cm}\\\
=\int_{\frac{G_{1}}{Z(G_{1})}}\left(\int_{\frac{H_{1}}{H_{1}\cap
Z(G_{1})}}\chi_{{}_{D_{2}}}(h_{1},g_{1})\left(\int_{H_{1}\cap
Z(G_{1})}d\mu_{Z(G_{1})\cap
H_{1}}(\zeta_{1})\right)d\nu_{H_{1}}(\bar{h_{1}})\right)d\nu_{G_{1}}(\bar{g_{1}})\vspace{.3cm}\\\
=\int_{\frac{G_{1}}{Z(G_{1})}}\left(\int_{\frac{H_{1}}{H_{1}\cap
Z(G_{1})}}\chi_{{}_{D_{2}}}(h_{1},g_{1})d\nu_{H_{1}}(\bar{h_{1}})\right)d\nu_{G_{1}}(\bar{g_{1}})\vspace{.3cm}\\\
\end{array}$

now the commutativity of the diagram in Definition 1.1 and the fact that
$\beta$ is an isomorphism of compact groups allow us to write

$\begin{array}[]{lcl}&=&\int_{\frac{G_{1}}{Z(G_{1})}}\left(\int_{\frac{H_{1}}{H_{1}\cap
Z(G_{1})}}\chi_{{}_{D_{2}}}(\gamma(1,H_{1},G_{1})(\bar{h_{1}},\bar{g_{1}}))d\nu_{H_{1}}(\bar{h_{1}})\right)d\nu_{G_{1}}(\bar{g_{1}})\vspace{.3cm}\\\
&=&\int_{\frac{G_{1}}{Z(G_{1})}}\left(\int_{\frac{H_{1}}{H_{1}\cap
Z(G_{1})}}\chi_{{}_{D_{2}}}(\beta(\gamma(1,H_{1},G_{1})(\bar{h_{1}},\bar{g_{1}})))d\nu_{H_{1}}(\bar{h_{1}})\right)d\nu_{G_{1}}(\bar{g_{1}})\vspace{.3cm}\\\
&=&\int_{\frac{G_{1}}{Z(G_{1})}}\left(\int_{\frac{H_{1}}{H_{1}\cap
Z(G_{1})}}\chi_{{}_{D_{2}}}(\gamma(1,H_{2},G_{2})(\alpha^{2}(\bar{h_{1}},\bar{g_{1}})))d\nu_{H_{1}}(\bar{h_{1}})\right)d\nu_{G_{1}}(\bar{g_{1}})\vspace{.3cm}\\\
\end{array}$

now we may apply $(*)$ above so

$\begin{array}[]{lcl}&=&\int_{\frac{G_{2}}{Z(G_{2})}}\left(\int_{\frac{H_{2}}{H_{2}\cap
Z(G_{2})}}\chi_{{}_{D_{2}}}(\gamma(1,H_{2},G_{2})(\bar{k_{1}},\bar{g_{2}}))d\nu_{H_{2}}(\bar{k_{1}})\right)d\nu_{G_{2}}(\bar{g_{2}})\vspace{.3cm}\\\
&=&\int_{\frac{G_{2}}{Z(G_{2})}}\left(\int_{\frac{H_{2}}{H_{2}\cap
Z(G_{2})}}\chi_{{}_{D_{2}}}(k_{1},g_{2})d\nu_{H_{2}}(\bar{k_{1}})\right)d\nu_{G_{2}}(\bar{g_{2}})\vspace{.3cm}\\\
\end{array}$

then we may proceed as before

$\begin{array}[]{lcl}&=&\int_{\frac{G_{2}}{Z(G_{2})}}\left(\int_{\frac{H_{2}}{H_{2}\cap
Z(G_{2})}}\chi_{{}_{D_{2}}}(k_{1},g_{2})\left(\int_{H_{2}\cap
Z(G_{2})}d\mu_{Z(G_{2})\cap
H_{2}}(\zeta_{2})\right)d\nu_{H_{2}}(\bar{k_{1}})\right)d\nu_{G_{2}}(\bar{g_{2}})\vspace{.3cm}\\\
&=&\int_{\frac{G_{2}}{Z(G_{2})}}\left(\int_{\frac{H_{2}}{H_{2}\cap
Z(G_{2})}}\left(\int_{H_{2}\cap
Z(G_{2})}\chi_{{}_{D_{2}}}(k_{1},g_{2})d\mu_{Z(G_{2})\cap
H_{2}}(\zeta_{2})\right)d\nu_{H_{2}}(\bar{k_{1}})\right)d\nu_{G_{2}}(\bar{g_{2}})\vspace{.3cm}\\\
&=&\int_{\frac{G_{2}}{Z(G_{2})}}\left(\int_{\frac{H_{2}}{H_{2}\cap
Z(G_{2})}}\left(\int_{H_{2}\cap
Z(G_{2})}\chi_{{}_{D_{2}}}(k_{1}\zeta_{2},g_{2})d\mu_{Z(G_{2})\cap
H_{2}}(\zeta_{2})\right)d\nu_{H_{2}}(\bar{h_{2}})\right)d\nu_{G_{2}}(\bar{g_{2}})\vspace{.3cm}\\\
&=&\int_{\frac{G_{2}}{Z(G_{2})}}\left(\int_{H_{2}}\chi_{{}_{D_{2}}}(k_{1},g_{2})d\mu_{H_{2}}(k_{1})\right)d\nu_{G_{2}}(\bar{g_{2}})\vspace{.3cm}\\\
&=&\int_{H_{2}}\left(\int_{\frac{G_{2}}{Z(G_{2})}}\chi_{{}_{D_{2}}}(k_{1},g_{2})d\nu_{G_{2}}(\bar{g_{2}})\right)d\mu_{H_{2}}(k_{1})\vspace{.3cm}\\\
\end{array}$
$\begin{array}[]{lcl}&=&\int_{H_{2}}\left(\int_{\frac{G_{2}}{Z(G_{2})}}\left(\int_{Z(G_{2})}\chi_{{}_{D_{2}}}(k_{1},g_{2})d\mu_{Z(G_{2})}(z_{2})\right)d\nu_{G_{2}}(\bar{g_{2}})\right)d\mu_{H_{2}}(k_{1})\vspace{.3cm}\\\
&=&\int_{H_{2}}\left(\int_{\frac{G_{2}}{Z(G_{2})}}\left(\int_{Z(G_{2})}\chi_{{}_{D_{2}}}(k_{1},g_{2}z_{2})d\mu_{Z(G_{2})}(z_{2})\right)d\nu_{G_{2}}(\bar{g_{2}})\right)d\mu_{H_{2}}(k_{1})\vspace{.3cm}\\\
&=&\int_{H_{2}}\int_{G_{2}}\chi_{{}_{D_{2}}}(k_{1},g_{2})d\mu_{G_{2}}(g_{2})d\mu_{H_{2}}(k_{1})\vspace{.3cm}\\\
&=&d(H_{2},G_{2}).\vspace{.3cm}\\\ \end{array}$

The result follows in this case.

If we assume that $n>1$, then we may repeat step by step the previous
argument, writing the corresponding integrals for the ($n+1$)-tuple
$(h_{1},h_{2},\ldots,h_{n},g_{1})$ of $H^{n}_{1}\times G_{1}$ and for the
($n+1$)-tuple $(k_{1},k_{2},\ldots,k_{n},g_{2})$ of $H^{n}_{2}\times G_{2}$.
Therefore the result follows completely.∎

The proof of Theorem 1.3 is mainly due to the next two lemmas, where the
notations of Definition 1.1 have been adopted. Note that there is not
hypothesis of compactness in the following two lemmas.

######  Lemma 2.2.

$(H_{1},G_{1})\ _{\widetilde{n}}\ (H_{2},G_{2})$ if and only if there exist
normal subgroups $N_{1}\leq Z_{n}(G_{1})$, $N_{2}\leq Z_{n}(G_{2})$ and two
isomorphisms $\alpha$ and $\beta$ such that $\alpha:G_{1}/N_{1}\rightarrow
G_{2}/N_{2},$ $\beta:[_{n}H_{1},G_{1}]\rightarrow[_{n}H_{2},G_{2}],$
$\alpha(H_{1}/(N_{1}\cap H_{1}))=H_{2}/(N_{2}\cap H_{2})$ for all $g_{1}\in
G_{1}$ and $h_{i}\in H_{1},$
$\beta([h_{1},...,h_{n},g_{1}])=[k_{1},...,k_{n},g_{2}],$ where
$g_{2}\in\alpha(g_{1}N_{1})$, $k_{i}\in\alpha(h_{i}(N_{1}\cap H_{1}))$ and
$1\leq i\leq n$.

###### Proof.

Assume that $(H_{1},G_{1})\ _{\widetilde{n}}\ (H_{2},G_{2})$. We may write
$N_{1}=Z_{n}(G_{1})$ and $N_{2}=Z_{n}(G_{2})$. From this, the result follows.

Conversely, one can show easily that
$\alpha(Z_{n}(G_{1})/N_{1})=Z_{n}(G_{2})/N_{2}$, therefore by third
isomorphism theorem, $\alpha$ induces the isomorphism $\alpha^{\prime}$ from
$G_{1}/Z_{n}(G_{1})$ to $G_{2}/Z_{n}(G_{2})$ by the rule
$\alpha^{\prime}(g_{1}Z_{n}(G_{1}))=g_{2}Z_{n}(G_{2})$, where
$g_{2}\in\alpha(g_{1}N_{1})$. Furthermore

$\alpha^{\prime}(h_{1}(Z_{n}(G_{1})\cap H_{1}))=h_{2}(Z_{n}(G_{2})\cap H_{2})$

in which $h_{2}\in\alpha(h_{1}(N_{1}\cap H_{1}))$. Hence
$(\alpha^{\prime},\beta)$ is a relative $n$-isoclinism from $(H_{1},G_{1})$ to
$(H_{2},G_{2})$.∎

######  Lemma 2.3.

Let $H$ be a subgroup of a group $G=HZ_{n}(G)$. Then

$(H,H)\ _{\widetilde{n}}\ (H,G)\ _{\widetilde{n}}\ (G,G).$

###### Proof.

We want to show that $(H,H)\ _{\widetilde{n}}\ (H,G)$. Let $G=HZ_{n}(G)$. We
may easily see that $Z_{n}(H)=Z_{n}(G)\cap H$. Thus
$H/Z_{n}(H)=H/(Z_{n}(G)\cap H)$ is isomorphic to
$HZ_{n}(G)/Z_{n}(G)=G/Z_{n}(G)$.

Therefore $\alpha:H/Z_{n}(H)\rightarrow G/Z_{n}(G)$ is an isomorphism which is
induced by the inclusion $i:H\rightarrow G.$ Furthermore, we can consider
$\alpha$ as isomorphism from $H/Z_{n}(H)$ to $H/(Z_{n}(G)\cap H)$.

On another hand, $[_{n}H,G]=[_{n}H,HZ_{n}(G)]=\gamma_{n+1}(H)$. By Lemma 2.2,
the pair $(\alpha,1_{\gamma_{n+1}(H)})$ allows us to state that $(H,H)\
_{\widetilde{n}}\ (H,G)$.

The remaining cases $(H,H)\ _{\widetilde{n}}\ (G,G)$ and $(H,H)\
_{\widetilde{n}}\ (H,G)$ follow by a similar argument. ∎

###### Proof of Theorem 1.3.

It follows from Theorem 1.2 and Lemma 2.3. ∎

We end with some consequences of the main results. We recall the following
lemma, which is used in many proofs of [5, 7, 8, 9].

######  Lemma 2.4 (See [6], Lemma 3.1).

Let $k$ be a positive integer. If $H$ is a closed subgroup of a compact group
$G$, then

$\mu_{G}(H)=\left\\{\begin{array}[]{lcl}\frac{1}{k},&&|G:H|=k\\\
0,&&|G:H|=\infty.\end{array}\right.$

######  Proposition 2.5.

Let $G$ be a compact group and $G/Z(G)$ be a $p$-group of order $p^{k}$, where
$p$ is a prime and $k\geq 2$ is an integer. Then
$d(G)\leq\frac{p^{k}+p-1}{p^{k+1}}$ and the equality holds if $G$ is isoclinic
to an extra-special $p$-group of order $p^{k+1}$.

###### Proof.

By Lemma 2.4 we have $\mu_{G}(Z(G))=\frac{1}{p^{k}}$ and
$\mu_{G}(C_{G}(x))\leq\frac{1}{p}$ for all $x\notin Z(G)$. Therefore

$\begin{array}[]{lcl}d(G)&=&\int_{G}\mu_{G}(C_{G}(x))d\mu_{G}(x)=\int_{Z(G)}\mu_{G}(C_{G}(x))+\int_{G-Z(G)}\mu_{G}(C_{G}(x))\vspace{.3cm}\\\
&\leq&\frac{1}{p}+(\frac{p-1}{p})\mu_{G}(Z(G))=\frac{1}{p}+(\frac{p-1}{p})\frac{1}{p^{k}}=\frac{p^{k}+p-1}{p^{k+1}}.\end{array}$

By Theorem 1.2, there is no loss of generality in assuming that $G$ is an
extra-special $p$-group of order $p^{k+1}$. For all $x\not\in Z(G)$ we may
consider the epimorphism $\varphi_{x}:G\rightarrow G^{\prime}$ defined by
$\varphi_{x}(y)=[x,y]$ in which the kernel is $C_{G}(x)$. Therefore
$\mu_{G}(C_{G}(x))=\frac{1}{p}$ and so the above inequality becomes
$d(G)=\frac{p^{k}+p-1}{p^{k+1}}$. ∎

An easy consequence has been shown below.

######  Proposition 2.6.

If $G$ is a compact group in which $Z(G)$ has infinite index in $G$, then
$d(G)\leq\frac{1}{2}$. In particular, if $G$ is isoclinic to an extra-special
$p$-group for some prime $p$, then $d(G)=\frac{1}{p}$.

###### Proof.

By Lemma 2.4 we have $\mu_{G}(Z(G))=0$ and $\mu_{G}(C_{G}(x))\leq\frac{1}{2}$
for all $x\not\in Z(G)$. From this, we have

$\begin{array}[]{lcl}d(G)=\int_{G}\mu_{G}(C_{G}(x))d\mu_{G}(x)\leq\frac{1}{2}+\frac{1}{2}\mu_{G}(Z(G))=\frac{1}{2}.\end{array}$

Following the same argument, if $G/Z(G)$ is an infinite $p$-group, then
$d(G)\leq\frac{1}{p}$. In particular, if $G$ is isoclinic to an extra-special
$p$-group, then we may argue as in the last part of the proof of Proposition
2.5, concluding that $\mu_{G}(C_{G}(x))=\frac{1}{p}$ for every $x\not\in
Z(G)$. Therefore the equality $d(G)=\frac{1}{p}$ holds and the result follows.
∎

## References

  * [1] J. C. Bioch, On $n$-isoclinic groups. Indag. Math. 38 (1976), 400–407.
  * [2] J. C. Bioch and R. W. van der Waall, Monomiality and isoclinism of groups. J. Reine Ang. Math. 298 (1978), 74–88.
  * [3] K. Chiti, M. R. R. Moghaddam and A. R. Salemkar, $n$-isoclinism classes and $n$-nilpotency degree of finite groups. Algebra Coll. 12 (2005), 225–261.
  * [4] Y. S. Chow and H. Teicher, Probability Theory: Independence, Interchangeability, Martingales. Springer, Berlin, 1988\.
  * [5] A. Erfanian, P. Lescot and R. Rezaei, On the relative commutativity degree of a subgroup of a finite group. Comm. Algebra 35 (2007), 4183–4197.
  * [6] A. Erfanian and R. Rezaei, On the commutativity degree of compact groups. Arch. Math. (Basel) 93 (2009), 201–212.
  * [7] P. X. Gallagher, The number of conjugacy classes in a finite group. Math. Z. 118 (1970), 175–179.
  * [8] R. M. Guralnick and G. R. Robinson, On the commuting probability in finite groups. J. Algebra 300 (2006), 509–528.
  * [9] W. H. Gustafson, What is the probability that two groups elements commute?. Amer. Math. Monthly 80 (1973), 1031–1304.
  * [10] M. Hall and J. K. Senior, The Groups of Order $2^{n}\ (n\leq 6)$. Macmillan, New York, 1964.
  * [11] P. Hall, The classification of prime-power groups. J. Reine Ang. Math. 182 (1940), 130–141.
  * [12] N. S. Hekster, On the structure of n-isoclinism classes of groups. J. Pure Appl. Algebra 40 (1986), 63–85.
  * [13] K. H. Hofmann and S. A. Morris, The Structure of Compact Groups. W. de Gruyter, Berlin, 1998.
  * [14] P. Lescot, Isoclinism classes and commutativity degrees of finite groups. J. Algebra 177 (1995), 847–869.
  * [15] D. J. S. Robinson, A course in the theory of groups. Springer, Heidelberg, 1982.


acoustics actuators addictions admsci aerospace agriculture agriengineering
agronomy ai algorithms allergies animals antibiotics antibodies antioxidants
applnano applmech applsci arts asc asi atmosphere atoms automation axioms
batteries bdcc behavsci beverages bioengineering biology biomedicines
biomimetics biomolecules biosensors bloods brainsci breath buildings cancers
carbon catalysts cells ceramics challenges chemengineering chemistry
chemosensors children civileng cleantechnol climate clockssleep cmd coatings
colloids computation computers condensedmatter cosmetics cryptography crystals
dairy data dentistry designs diagnostics diseases diversity drones
econometrics economies education ejbc ejihpe electricity electrochem
electronicmat electronics endocrines energies entropy environments
epidemiologia epigenomes est fermentation fibers fire fishes fluids foods
forecasting forests fractalfract fuels futureinternet futurephys galaxies
games gardens gastrointestdisord gels genealogy genes geohazards geosciences
geriatrics hazardousmatters healthcare hearts heritage highthroughput
horticulturae humanities hydrology ijd ijerph ijfs ijgi ijms ijtpp informatics
information infrastructures inorganics insects instruments inventions iot j
jcdd jcm jcp jcs jdb jfb jfmk jimaging jintelligence jlpea jmmp jmse jne jnt
jof joitmc journalmedia jpm jrfm jsan land languages laws life literature
logistics lubricants machines magnetochemistry make marinedrugs materials
mathematics mca medicina medicines medsci membranes metabolites metals
microarrays micromachines microorganisms minerals modelling molbank molecules
mps mti nanomaterials ncrna ijns neurosci neuroglia nitrogen notspecified
nutrients obesities oceans ohbm optics organics osteology particles pathogens
pharmaceuticals pharmaceutics pharmacy philosophies photonics physics plants
plasma pollutants polymers polysaccharides preprints proceedings processes
prosthesis proteomes psych psychiatryint publications quantumrep quaternary
qubs reactions recycling religions remotesensing reports reprodmed resources
risks robotics safety sci scipharm sensors separations sexes signals sinusitis
skins smartcities sna societies socsci soilsystems sports standards stats
surfaces surgeries suschem sustainability world symmetry systems technologies
telecom test tourismhosp toxics toxins transplantology tropicalmed universe
urbansci vaccines vehicles vetsci vibration viruses vision water wem wevj