# Continual self-training with bootstrapped remixing
for speech enhancement

###### Abstract

We propose RemixIT, a simple and novel self-supervised training method for
speech enhancement. The proposed method is based on a continuously self-
training scheme that overcomes limitations from previous studies including
assumptions for the in-domain noise distribution and having access to clean
target signals. Specifically, a separation teacher model is pre-trained on an
out-of-domain dataset and is used to infer estimated target signals for a
batch of in-domain mixtures. Next, we bootstrap the mixing process by
generating artificial mixtures using permuted estimated clean and noise
signals. Finally, the student model is trained using the permuted estimated
sources as targets while we periodically update teacher’s weights using the
latest student model. Our experiments show that RemixIT outperforms several
previous state-of-the-art self-supervised methods under multiple speech
enhancement tasks. Additionally, RemixIT provides a seamless alternative for
semi-supervised and unsupervised domain adaptation for speech enhancement
tasks, while being general enough to be applied to any separation task and
paired with any separation model.

Index Terms—  Self-supervised learning, speech enhancement, unsupervised
denoising, zero-shot learning, domain adaptation

##  1 Introduction

Neural networks have been found to be highly effective and widely applicable
to a large number of audio and speech problems, including speech enhancement,
where the goal is to improve the quality and intelligibility of degraded
speech signals. In recent years, several neural architectures have reported
state-of-the-art results for supervised [1], real-time [2] and semi-supervised
[3] speech enhancement tasks. Mostly driven by supervised learning, training
such models requires large amounts of audio data which are expected to closely
match the distribution of the test time input noisy recordings. While limited
supervised data might often be available, supervised speech enhancement
systems trained on them provides severely inferior performance due to the
mismatch with the actual test distribution.

To address these problems and to reduce reliance on purely supervised data,
several speech enhancement and audio source separation studies have shifted
their focus towards self-supervised methods [4]. In [5], a model was trained
to estimate the signal-to-noise ratio (SNR) of noisy mixture recordings and
assign a confidence value on each noisy segment. Next, a separation model was
trained on the noisy speech mixtures using a weighted reconstruction loss
function to filter out the contribution of noisy ground-truth speech
utterances. Lately, mixture invariant training (MixIT) has been proposed in
[6] which enables unsupervised training of separation models by generating
artificial mixtures of mixtures (MoMs) and letting the separation model
estimate and reassign the sources back to the ground-truth mixtures. MixIT has
been experimentally shown to provide a robust unsupervised solution for speech
enhancement under a variety of setups [6, 7, 8]. However, MixIT assumes access
to in-domain noise samples and slightly alters the input SNR distribution by
training on artificial MoMs with more than one noise sources [9].

On the other hand, teacher-student setups have shown significant improvements
in several audio processing tasks [10]. In [11], a student model was trained
at the outputs of a pre-trained MixIT model for solving the problem of the
artificially created input SNR mismatch between the train and test mixture
distributions. An energy threshold was used to reduce the number of sources
appearing in the noisy mixtures. Moreover, a student model can be adapted to a
given test set using regression over the pre-trained teacher’s estimates [12].
Closest to our work is the self-training framework, proposed in [13], for
semi-supervised singing voice separation where the teacher was pre-trained on
an out-of-domain (OOD) supervised data and used for predicting estimated
sources on the larger in-domain noisy dataset. The new self-labeled version of
the dataset was filtered for low-quality separated sources and stored for
offline training of a new student model using artificially generated mixtures
from the estimated self-labeled estimated sources. Although all the
aforementioned works assumed a frozen teacher, other works in automatic speech
recognition have shown significant benefits when updating the teacher model
using a moving mean teacher [14].

In this paper, we propose a self-training method capable of performing self-
supervised learning using large in-domain noisy datasets, while requiring only
an OOD pre-trained teacher model (e.g. MixIT on an OOD dataset). In contrast
to self-training methods in the literature which use ad-hoc filtering
procedures to enhance the quality of the teacher model estimates, our method
trains the student model by performing online remixing of the teacher’s
estimated sources. Moreover, instead of freezing the teacher model, RemixIT
treats the self-training process as a lifelong learning procedure by using
sequential and moving averaging update schemes which enables faster
convergence. Our experiments showcase the general applicability of our method
towards _self-supervised speech enhancement, semi-supervised OOD
generalization_ , and _zero-shot domain adaptation_. We also provide an
intuitive explanation of why our bootstrapped remixing process works under
minimal assumptions.

##  2 RemixIT Method

We present RemixIT for the general case of speech enhancement where the goal
is to reconstruct the clean speech from a noisy speech signal. Formally, we
train a separation model $f_{\mathcal{S}}$ which outputs $M$ sources for each
noisy recording from an input batch ${\mathbf{x}}\in\mathbb{R}^{B\times T}$
containing $B$ waveforms, each with $T$ samples in the time-domain:

$\begin{gathered}\widehat{\mathbf{s}},\widehat{\mathbf{n}}=f_{\mathcal{S}}({\mathbf{x}};{\bm{\theta}_{\mathcal{S}}}),\enskip{\mathbf{x}}=\mathbf{s}+{\textstyle\sum}_{i=1}^{M-1}\mathbf{n}_{i}=\widehat{\mathbf{s}}+{\textstyle\sum}_{i=1}^{M-1}\widehat{\mathbf{n}}_{i},\end{gathered}$
(1)

where $\widehat{\mathbf{s}},\mathbf{s}\in\mathbb{R}^{B\times T}$,
$\widehat{\mathbf{n}},\mathbf{n}\in\mathbb{R}^{(M-1)\times B\times T}$,
${\bm{\theta}_{\mathcal{S}}}$ are: the estimated speech signals, the clean
speech targets, the estimated noise signals, the noise targets and the
parameters of the model, respectively. In this work, we force the estimated
sources $\widehat{\mathbf{s}},\widehat{\mathbf{n}}$ to add up to the initial
input mixtures ${\mathbf{x}}$ using a mixture consistency layer [15].

${\bm{\theta}}^{(0)}_{\mathcal{T}}\leftarrow\textsc{PretrainTeacher}(f_{\mathcal{T}},{\cal
D}^{\prime})$

${\bm{\theta}}_{\mathcal{S}}\leftarrow\textsc{InitializeStudent}(f_{\mathcal{S}})$

for  _$k=0$ ; $k{+}{+}$; while $k<=K$_ do

for  _SampleBatch $\mathbf{m}\in{\cal
D}_{m},\enskip\mathbf{m}\in\mathbb{R}^{B\times T}$_ do

$\widetilde{\mathbf{s}},\widetilde{\mathbf{n}}\leftarrow
f_{\mathcal{T}}(\mathbf{m};{\bm{\theta}}^{(k)}_{\mathcal{T}})$ // Noisy
estimates

$\widetilde{\mathbf{m}}=\widetilde{\mathbf{s}}+\bm{\Pi}\widetilde{\mathbf{n}}$
// Bootstrapped remixing

$\widehat{\mathbf{s}},\widehat{\mathbf{n}}\leftarrow
f_{\mathcal{S}}(\widetilde{\mathbf{m}};{\bm{\theta}}^{(k)}_{\mathcal{S}})$ //
Student estimates

$\mathcal{L}_{\operatorname{RemixIT}}=\sum_{b=1}^{B}\left[{\cal
L}(\widehat{\mathbf{s}}_{b},\widetilde{\mathbf{s}}_{b})+{\cal
L}(\widehat{\mathbf{n}}_{b},\left[\bm{\Pi}\widetilde{\mathbf{n}}\right]_{b})\right]$

${\bm{\theta}}_{\mathcal{S}}\leftarrow\textsc{UpdateStudent}({\bm{\theta}}_{\mathcal{S}},\nabla_{{\bm{\theta}}_{\mathcal{S}}}\mathcal{L}_{\operatorname{RemixIT}})$

end for

${\bm{\theta}}^{(k+1)}_{\mathcal{T}}\leftarrow\textsc{UpdateTeacher}({\bm{\theta}}^{(k)}_{\mathcal{T}},{\bm{\theta}}_{\mathcal{S}})$

end for

Algorithm 1 RemixIT for the noisy dataset ${\cal D}_{m}$.

###  2.1 Mixture invariant training

MixIT [6] has proven its effectiveness under various self-supervised speech
enhancement settings [7, 8]. Specifically, MixIT assumes that the training
dataset consists of two portions $({\cal D}_{m},{\cal D}_{n})$, where ${\cal
D}_{m}$ is the part of the dataset which carries mixtures of speech and one
noise source while ${\cal D}_{n}$ contains isolated noise recordings. During
training, a new batch of artificial mixtures of mixtures (MoMs) is generated,
${\mathbf{x}}=\mathbf{s}+\mathbf{n}_{1}+\mathbf{n}_{2}$, by sampling a batch
of noisy speech recordings $\mathbf{m}\sim{\cal D}_{m}$ and a batch of clean
noise samples $\mathbf{n}_{2}\sim{\cal D}_{n}$, where
$\mathbf{m}=\mathbf{s}+\mathbf{n}_{1}$. The separation model always estimates
$M=3$ sources
($\widehat{\mathbf{s}},\widehat{\mathbf{n}}_{1},\widehat{\mathbf{n}}_{2}=f_{\mathcal{S}}({\mathbf{x}};{\bm{\theta}_{\mathcal{S}}})$)
and the following permutation invariant loss is minimized for the $b$-th input
MoM in the batch:

$\displaystyle\mathcal{L}_{\operatorname{MixIT}}^{(b)}=\underset{\bm{\pi}}{\min}[{\cal
L}(\widehat{\mathbf{s}}_{b}+\widehat{\mathbf{n}}_{\pi_{1},b},\mathbf{m}_{b})+{\cal
L}(\widehat{\mathbf{n}}_{\pi_{2},b},\mathbf{n}_{2,b})],\enskip\forall b,$ (2)

where $\bm{\pi}\in\\{(2,3),(3,2)\\}$ is the set of permutations of the output
noise slots. However, MixIT depends on in-domain isolated noise recordings
which makes it impractical for real-world settings. In most of the cases,
matching the real noise distribution with the available noise set ${\cal
D}_{n}$ is a strenuous task. The data augmentation proposed in [9] shows some
improvements when an extra noise source from an OOD noise distribution is
injected to the input MoM. Nevertheless, the performance of that method
depends on the level of distribution shift between the actual noise
distribution and ${\cal D}_{n}$.

###  2.2 Continual self-training with bootstrapped remixing

RemixIT does not assume access to in-domain information. Thus, we can only
draw mixtures from the in-domain noisy dataset
$\mathbf{m}=\mathbf{s}+\mathbf{n}$ ($m\sim{\cal D}_{m}$) where the noisy
speech recordings contain a single noise source each and thus,
$\mathbf{m},\mathbf{s},\mathbf{n}\in\mathbb{R}^{B\times T}$. RemixIT leverages
a student-teacher framework to bootstrap the remixing process by permuting the
previous noisy estimates, remixing them and using them as targets for
training. We summarize RemixIT in Algorithm 1.

####  2.2.1 RemixIT’s continual self-training framework

We assume that we can pre-train in a supervised or a self-supervised way a
teacher model $f_{\mathcal{T}}$ on an OOD dataset ${\cal D}^{\prime}$ which
meets the specifications of Equation 1. Now, the first step is to use the
teacher model to estimate some new noisy targets for a given mixture batch
$\mathbf{m}=\mathbf{s}+\mathbf{n}\in\mathbb{R}^{B\times
T},\enskip\mathbf{m}\sim{\cal D}_{m}$ as follows:

$\begin{gathered}\widetilde{\mathbf{s}},\widetilde{\mathbf{n}}=f_{\mathcal{T}}(\mathbf{m};{\bm{\theta}}^{(k)}_{\mathcal{T}}),\enskip\mathbf{m}=\mathbf{s}+\mathbf{n}=\widetilde{\mathbf{s}}+{\textstyle\sum}_{i=1}^{M-1}\widetilde{\mathbf{n}}_{i},\end{gathered}$
(3)

where $k$ denotes the optimization step. If the teacher network was obtained
by supervised (unsupervised via MixIT) OOD pre-training, we would have $M=2$
($M=3$) output slots. Next, we use these estimated sources to generate new
noisy mixtures as shown below:

$\begin{gathered}\widetilde{\mathbf{m}}=\widetilde{\mathbf{s}}+\widetilde{\mathbf{n}}^{(\bm{\Pi})}\in\mathbb{R}^{B\times
T},\enskip\widetilde{\mathbf{n}}^{(\bm{\Pi})}=\bm{\Pi}\widetilde{\mathbf{n}},\enskip\bm{\Pi}\sim\mathcal{P}_{B\times
B},\end{gathered}$ (4)

where $\bm{\Pi}$ is drawn uniformly from the set of all $B\times B$
permutation matrices. Now, we simply use the permuted target pairs to train
the student model $f_{\mathcal{S}}$ on the bootstrapped mixtures
$\widetilde{\mathbf{m}}$ as follows:

$\begin{gathered}\widehat{\mathbf{s}},\widehat{\mathbf{n}}=f_{\mathcal{S}}(\widetilde{\mathbf{m}};{\bm{\theta}}^{(k)}_{\mathcal{S}}),\enskip\widehat{\mathbf{s}},\widehat{\mathbf{n}}\in\mathbb{R}^{B\times
T}\\\ \mathcal{L}_{\operatorname{RemixIT}}^{(b)}={\cal
L}(\widehat{\mathbf{s}}_{b},\widetilde{\mathbf{s}}_{b})+{\cal
L}(\widehat{\mathbf{n}}_{b},\widetilde{\mathbf{n}}^{(\bm{\Pi})}_{b}),\enskip
b\in\\{1,\dots,B\\},\end{gathered}$ (5)

where the proposed loss function resembles a regular supervised setup with the
specified signal-level loss function ${\cal L}$. Our method does not
artificially alter the input SNR distributions similar to MixIT-like [6, 7, 9]
training recipes. Instead, the student model is trained on mixtures with the
same number of sources for the bootstrapped mixtures where the teacher model
had performed adequately. Unlike previous teacher-student methods which use
the same teacher-estimated source-pairs as the targets for the student network
[11, 12], the proposed bootstrapped mixtures increase the input mixture
diversity and allow faster model training. This is especially useful in
settings with a large distribution shift between teacher’s and student’s
training data. Moreover, in contrast to the self-training approach in [13],
where the teacher model is frozen and the inference on the in-domain dataset
${\cal D}_{m}$ is performed offline and the new self-labeled dataset is
stored, RemixIT employs a lifelong learning process. Our method is general
enough that could be paired with any online co-training method which
continuously updates the teacher’s weights in addition to the main student
training.

####  2.2.2 Error analysis under the Euclidean norm

We can express the errors produced by the student $\widehat{\mathbf{R}}_{\cal
S}$ and the teacher $\widetilde{\mathbf{R}}_{\cal T}$ w.r.t. the initial clean
targets as random variables:

$\begin{gathered}\widehat{\mathbf{R}}_{\cal
S}=\widehat{\mathbf{S}}-\mathbf{S},\enskip\widetilde{\mathbf{R}}_{\cal
T}=\widetilde{\mathbf{S}}-\mathbf{S},\enskip(\mathbf{S},\mathbf{N})\sim{\cal
D}\\\ \widehat{\mathbf{R}}_{\cal S}\sim P(\widehat{\mathbf{R}}_{\cal
S}|\widetilde{\mathbf{S}},\widetilde{\mathbf{N}},\bm{\Pi}),\enskip\widetilde{\mathbf{R}}_{\cal
T}\sim P(\widetilde{\mathbf{R}}_{\cal
T}|\mathbf{S},\mathbf{N}).\end{gathered}$ (6)

Now, we focus on the part of the objective function which is minimized at
every student optimization step w.r.t. the speech component. Assuming unit-
norm vector signals and using a signal-level loss ${\cal L}$ that minimizes
the squared error between the estimated and target signals, the student’s
RemixIT loss function is equivalent to:

$\displaystyle\mathcal{L}_{\operatorname{RemixIT}}$
$\displaystyle\propto{\mathbb{E}}\left[||\widehat{\mathbf{S}}-\widetilde{\mathbf{S}}||^{2}_{2}\right]={\mathbb{E}}\left[||(\widehat{\mathbf{S}}-\mathbf{S})-(\widetilde{\mathbf{S}}-\mathbf{S})||^{2}_{2}\right]$
(7)
$\displaystyle\propto\underbrace{{\mathbb{E}}\left[||\widehat{\mathbf{R}}_{\cal
S}||^{2}_{2}\right]}_{\text{Supervised
Loss}}+\underbrace{{\mathbb{E}}\left[||\widetilde{\mathbf{R}}_{\cal
T}||^{2}_{2}\right]}_{\text{Constant w.r.t. }{\bm{\theta}}_{\cal
S}}-2\underbrace{{\mathbb{E}}\left[\langle\widehat{\mathbf{R}}_{\cal
S},\widetilde{\mathbf{R}}_{\cal T}\rangle\right]}_{\text{Errors
correlation}}.$

Ideally, this loss could lead to the same optimization objective with a
supervised setup if the last inner-product term was zero.
$\langle\widehat{\mathbf{R}}_{\cal S},\widetilde{\mathbf{R}}_{\cal
T}\rangle=0$ could be achieved if the teacher produced outputs
indistinguishable from the clean target signals or the conditional error
distributions in Equation 6 were independent. Intuitively, as we continually
update the teacher model and refine its estimates, we minimize the norm of the
teacher error. Additionally, the bootstrapped remixing process forces the
errors to be more uncorrelated since the student tries to reconstruct the same
clean speech signals $\mathbf{s}$, similar to its teacher, but under a
different mixture distribution. Formally, the student tries to reconstruct
$\mathbf{s}$ when observing the bootstrapped mixtures
$\widetilde{\mathbf{m}}=\widetilde{\mathbf{s}}+\widetilde{\mathbf{n}}^{(\bm{\Pi})}$
while the teacher tries to reconstruct $\mathbf{s}$ from the initial input
mixtures $\mathbf{m}=\mathbf{s}+\mathbf{n}$.

##  3 Experimental Framework

###  3.1 Datasets

DNS-Challenge (DNS): We use the DNSChallenge 2020 benchmark dataset [16] which
covers a wide variety of noisy speech conditions. This dataset consists of
$64{,}649$ and $150$ pairs of clean speech and noise recordings for training
and testing, respectively. DNS is mainly used for showing the effectiveness of
RemixIT at leveraging vast amounts of noisy mixture recordings.

LibriFSD50K (LFSD): This dataset consists of a diverse set of speakers drawn
from the LibriSpeech [17] corpus and a wide variety of background noises from
FSD50K [18]. Specifically, $45{,}602$ and $3{,}081$ for training and
validation, correspondingly. We follow the same data generation procedure as
indicated in [8]. In this study, this dataset is mainly used for the OOD
unsupervised or semi-supervised pre-training of speech enhancement models.

WHAM!: We follow the same procedure as in [8] in order to generate noisy
mixtures using speakers and noise sources from the WHAM! [19] dataset. We use
this dataset as a medium-sized dataset with $20{,}000$ training noisy-speech
pairs and $3{,}000$ test mixtures.

VCTK: We use the test partition of the VCTK dataset proposed in [20] which
includes $586$ randomly generated noisy speech samples by mixing recordings
from the VCTK speech corpus [21] and the DEMAND [22] noisy data collection.

###  3.2 Separation model

We want to underline that the proposed method can be applied alongside any
separation model. In this work, we chose the Sudo rm -rf [23] architecture
since it achieves a good trade-off between speech enhancement quality and
time-memory computational requirements. Specifically, we use the Sudo rm -rf
variation with group communication [24] and the default parameters in [25],
which has shown promising results in speech enhancement tasks [8]. All models
have $U=8$ U-ConvBlocks except for the experiments where we increase the depth
of the new student networks to $16$ and $32$. We fix the number of output
slots to $M=3$ for MixIT models or $M=2$ otherwise.

###  3.3 RemixIT configurations

For the unsupervised RemixIT, we assume that the initial teacher model was
pre-trained using MixIT at a specified OOD dataset. For semi-supervised
RemixIT, we pre-train the teacher using conventional supervised training. We
also experiment with various online teacher updating protocols such as:

$\begin{gathered}\mathring{{\bm{\theta}}}^{(K)}_{\mathcal{T}}\coloneqq{\bm{\theta}}^{(K)}_{\mathcal{S}},\enskip\bar{{\bm{\theta}}}^{(k+1)}_{\mathcal{T}}\coloneqq\gamma{\bm{\theta}}^{(k)}_{\mathcal{S}}+(1-\gamma)\bar{{\bm{\theta}}}^{(k)}_{\mathcal{T}},\end{gathered}$
(8)

where $k$ denotes the training epoch index. For the sequentially updated
teacher, we replace the old teacher with the latest student every $K=20$
epochs. For the zero-shot domain adaptation experiments, we first set the
student to be the same as the teacher
${\bm{\theta}}^{(0)}_{\mathcal{S}}\coloneqq\bar{{\bm{\theta}}}^{(0)}_{\mathcal{T}}$
and then use the moving average teacher update with $\gamma=0.01$.

###  3.4 Training and evaluation details

Although we could use any valid signal-level loss function (see Equations 2,
5), we choose the negative scale-invariant signal to distortion ratio (SI-SDR)
[26]:

$\begin{gathered}\mathcal{L}(\widehat{y},y)=-\text{SI-
SDR}(\widehat{y},y)=-10\log_{10}\left(\nicefrac{{\|\alpha y\|^{2}}}{{\|\alpha
y-\widehat{y}\|^{2}}}\right),\end{gathered}$ (9)

where $\alpha=\widehat{y}^{\top}y/\|y\|^{2}$ makes the loss invariant to the
scale of the estimated source $\widehat{y}$ and the target signal $y$. We
train all models using the Adam optimizer [27] with a batch size of $B=2$ and
an initial learning rate of $10^{-3}$ which is divided by $2$ every $6$
epochs.

We evaluate the robustness of our speech enhancement models on the each test
set after $100$ epochs for the supervised trained and pre-trained teachers as
well as $60$ epochs for all the other configurations. Specifically, we report
the SI-SDR [26], the Short-Time Objective Intelligibility (STOI) [28] and the
Perceptual Evaluation of Speech Quality (PESQ) [29] for $16$kHz target
signals.

##  4 Results

Training Method and Model Details |  $\\#$Model | Available Training Data (%) |  Evaluation Metrics
---|---|---|---
| Params |  Clean Speech ${\cal D}_{s}$ |  Clean Noise ${\cal D}_{n}$ |  Mixture ${\cal D}_{m}$ | SISDR | PESQ | STOI
|  ($10^{6}$) | DNS | LFSD | DNS | LFSD | DNS | LFSD | (dB)
Input Noisy Mixture | - |  |  |  |  |  |  | 9.2 | 1.58 | 0.915
Unsupervised MixIT with Student ($U=8$) | In-domain | $0.79$ |  |  | 20% |  | 80% |  | 14.4 | 2.13 | 0.933
OOD noise | $0.79$ |  |  |  | 20% | 100% |  | 14.3 | 2.02 | 0.933
Extra noise [9] | $0.79$ |  |  |  | 50% | 100% |  | 14.5 | 2.03 | 0.930
Unsupervised RemixIT |  Teacher ($U=8$) | $0.79$ |  |  |  | 20% |  | 80% | 14.8 | 2.15 | 0.940
Student ($U=32$) | $0.97$ |  |  |  |  | 100% |  | 16.0 | 2.34 | 0.952
Semi-supervised RemixIT |  Teacher ($U=8$) | $0.56$ |  | 100% |  | 100% |  |  | 17.6 | 2.61 | 0.958
Student ($U=32$) | $0.73$ |  |  |  |  | 100% |  | 18.0 | 2.60 | 0.959
Supervised |  Student ($U=8$) | $0.56$ | 100% |  | 100% |  |  |  | 18.6 | 2.69 | 0.962
In-domain |  FullSubNet [2] | $5.6$ | 100% |  | 100% |  |  |  | 17.3 | 2.78 | 0.961
Table 1: Speech enhancement performance on the DNS test set using the proposed
RemixIT methods, unsupervised MixIT approaches [6, 9] and supervised in-domain
training with the same Sudo rm -rf model ($U=8$) and the state-of-the-art
FullSubNet model in the literature [2].

###  4.1 Continuous refinement of teacher’s estimates

Fig. 1: Speech enhancement performance on the DNS test set when using RemixIT
with different teacher update protocols. All approaches use the same teacher
architecture with $U=8$ which was pre-trained in a supervised way using the
training split of WHAM!. Notice that by sequentially updating the teacher
(orange solid line) every $20$ epochs and replacing it with the previous
student, we are able to obtain significant improvements over the methods which
use the same static teacher (gray and black dashed lines). In this semi-
supervised RemixIT setup, considering the large mismatch between WHAM! and DNS
datasets, the student model significantly outperforms the initial OOD pre-
trained teacher by showing an improvement of more than $1$ dB in terms of SI-
SDR performance.

Our experimental results validate our hypothesis that a speech enhancement
model could be trained faster and more effectively under a lifelong process
where the teacher would be continuously updated in parallel to the student.
The speech enhancement performance obtained by the sequentially updated and
the frozen teacher protocols are shown in Figure 1. We notice that all
protocols perform similarly until the $20$th epoch where the teacher is still
static for all strategies. However, after the $20$th epoch, the teacher model
is replaced with the latest student while the depth of the next student is
increased $8\rightarrow 16$. As a result, the student trained with the
sequential method scales better for the same number of epochs (e.g. $40$th
epoch) compared to the same size student ($U=16$) with a frozen teacher. We
have experimentally seen that the sequentially updated teacher scales better
than other protocols and this is the default strategy which we use across all
other experiments except for the zero-shot adaptation where we also show that
the running mean teacher updating scheme is also an effective option.

###  4.2 Self-supervised and Self-supervised speech enhancement

The speech enhancement results of the proposed method alongside supervised and
unsupervised baselines are summarized in Table 1. The percentage of the
available data denotes the portion of each disjoint splits from the DNS or the
LFSD paired data collections. For instance, the unsupervised RemixIT teacher
pre-training requires unsupervised MixIT using $80\%$ of the LFSD data pairs
to simulate the noisy recordings ${\cal D}^{\prime}_{m}$ and the other $20\%$
for the clean OOD noise recordings ${\cal D}^{\prime}_{n}$, whilst the regular
student training leverages the whole noisy DNS dataset.

Notice that unsupervised and semi-supervised RemixIT does not depend on clean
in-domain noise samples. Despite that, the unsupervised student model
significantly outperforms all MixIT-like approaches including the in-domain
training and the recently proposed extra noise augmentation where MoMs contain
$3$ noise sources [9] ($14.5$dB $\rightarrow 16.0$dB in terms of SI-SDR).
Moreover, the largest unsupervised student ($U=32$) outperforms its OOD MixIT
unsupervised teacher by a large margin across all evaluation metrics which
shows the efficacy of RemixIT for self-supervised settings. The proposed
method also yields noticeable gains for the semi-supervised case where the
student model performs comparably with in-domain supervised training using the
default Sudo rm -rf model with $U=8$ and a recent state-of-the-art model. We
want to underline that our method could be used with more complex models as
teachers, rather than the efficient Sudo rm -rf architecture, and provide even
higher quality speech enhancement performance.

###  4.3 Zero-shot domain adaptation

We show that RemixIT can also be used with low-resource datasets, where
training data are limited but one has access to a test dataset for adapting a
pre-trained model. In Figure 2, the performance improvement for the zero-shot
speech enhancement task is depicted with a variety of supervised and
unsupervised pre-trained networks on larger OOD datasets. Notably, RemixIT
yields improvements of up to $0.8$dB in terms of SI-SDR compared to the
uncalibrated pre-trained models while using a limited amount of in-domain
mixtures. The performance of our model is correlated with the amount of
available noisy mixtures and this is the reason we see the largest (smallest)
gains for the WHAM! (DNS) test partition which has $3{,}000$ (only $150$)
mixtures. Moreover, we also notice a large improvement in cases where there is
a large distribution shift between the training data and the mixtures in the
adaptation set (e.g. supervised training on WHAM! and adapting on the
relatively small DNS test set).

Fig. 2: Absolute gain in terms of SI-SDR when using RemixIT with an OOD pre-
trained model and a given test set to adapt (e.g. DNS, LFSD and WHAM!, from
left to right). We show our results on three different pre-trained Sudo rm -rf
models with $U=8$ and following MixIT unsupervised pre-training on LFSD
(blue/leftmost), supervised training on LFSD (yellow) and supervised pre-
training on WHAM! (green/rightmost).

##  5 Conclusion

We have proposed a novel continual self-training method for denoising and have
experimentally showed its benefits on several realistic speech enhancement
tasks. Our method depends only on the existence of in-domain noisy data and a
pre-trained model using purely out-of-domain data which might not necessarily
capture the in-domain distribution. The coupling of the bootstrap remixing
process with the continuously bi-directional teacher-student self-training
framework leads to significant improvements for zero-shot and self-supervised
speech enhancement as well as semi-supervised domain adaptation. In the
future, we aim to apply our method to other domains and denoising tasks as
well as provide stronger theoretical guarantees for the convergence of our
algorithm.

## References

  * [1] Ashutosh Pandey and DeLiang Wang,  “Dense cnn with self-attention for time-domain speech enhancement,”  IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 1270–1279, 2021.
  * [2] Xiang Hao, Xiangdong Su, Radu Horaud, and Xiaofei Li,  “Fullsubnet: A full-band and sub-band fusion model for real-time single-channel speech enhancement,”  in Proc. ICASSP, 2021, pp. 6633–6637.
  * [3] Umut Isik, Ritwik Giri, Neerad Phansalkar, Jean-Marc Valin, Karim Helwani, and Arvindh Krishnaswamy,  “Poconet: Better speech enhancement with frequency-positional embeddings, semi-supervised conversational data, and biased loss,”  in Proc. Interspeech, 2020, pp. 2487–2491.
  * [4] Yangyang Xia, Buye Xu, and Anurag Kumar,  “Incorporating real-world noisy speech in neural-network-based speech enhancement systems,”  arXiv preprint arXiv:2109.05172, 2021.
  * [5] Aswin Sivaraman, Sunwoo Kim, and Minje Kim,  “Personalized speech enhancement through self-supervised data augmentation and purification,”  in Proc. Interspeech, 2021, pp. 2676–2680.
  * [6] Scott Wisdom, Efthymios Tzinis, Hakan Erdogan, Ron J Weiss, Kevin Wilson, and John R Hershey,  “Unsupervised sound separation using mixture invariant training,”  in Proc. NeurIPS, 2020, vol. 33, pp. 3846–3857.
  * [7] Takuya Fujimura, Yuma Koizumi, Kohei Yatabe, and Ryoichi Miyazaki,  “Noisy-target training: A training strategy for dnn-based speech enhancement without clean speech,”  arXiv preprint arXiv:2101.08625, 2021.
  * [8] Efthymios Tzinis, Jonah Casebeer, Zhepei Wang, and Paris Smaragdis,  “Separate but together: Unsupervised federated learning for speech enhancement from non-iid data,”  in Proc. WASPAA, 2021, pp. 46–50.
  * [9] Koichi Saito, Stefan Uhlich, Giorgio Fabbro, and Yuki Mitsufuji,  “Training speech enhancement systems with noisy speech datasets,”  arXiv preprint arXiv:2105.12315, 2021.
  * [10] Ryo Aihara, Toshiyuki Hanazawa, Yohei Okato, Gordon Wichern, and Jonathan Le Roux,  “Teacher-student deep clustering for low-delay single channel speech separation,”  in Proc. ICASSP, 2019, pp. 690–694.
  * [11] Jisi Zhang, Cătălin Zorilă, Rama Doddipatla, and Jon Barker,  “Teacher-Student MixIT for Unsupervised and Semi-Supervised Speech Separation,”  in Proc. Interspeech, 2021, pp. 3495–3499.
  * [12] Sunwoo Kim and Minje Kim,  “Test-time adaptation toward personalized speech enhancement: Zero-shot learning with knowledge distillation,”  in Proc. WASPAA, 2021, pp. 176–180.
  * [13] Zhepei Wang, Ritwik Giri, Umut Isik, Jean-Marc Valin, and Arvindh Krishnaswamy,  “Semi-supervised singing voice separation with noisy self-training,”  in Proc. ICASSP, 2021, pp. 31–35.
  * [14] Yosuke Higuchi, Niko Moritz, Jonathan Le Roux, and Takaaki Hori,  “Momentum pseudo-labeling for semi-supervised speech recognition,”  in Proc. Interspeech, 2021, pp. 726–730.
  * [15] Scott Wisdom, John R Hershey, Kevin Wilson, Jeremy Thorpe, Michael Chinen, Brian Patton, and Rif A Saurous,  “Differentiable consistency constraints for improved deep speech enhancement,”  in Proc. ICASSP, 2019, pp. 900–904.
  * [16] Chandan KA Reddy et al.,  “The interspeech 2020 deep noise suppression challenge: Datasets, subjective testing framework, and challenge results,”  in Proc. Interspeech, 2020, pp. 2492–2496.
  * [17] Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur,  “Librispeech: an asr corpus based on public domain audio books,”  in Proc. ICASSP, 2015, pp. 5206–5210.
  * [18] Eduardo Fonseca, Xavier Favory, Jordi Pons, Frederic Font, and Xavier Serra,  “Fsd50k: an open dataset of human-labeled sound events,”  arXiv preprint arXiv:2010.00475, 2020.
  * [19] Gordon Wichern, Joe Antognini, Michael Flynn, Licheng Richard Zhu, Emmett McQuinn, Dwight Crow, Ethan Manilow, and Jonathan Le Roux,  “WHAM!: Extending Speech Separation to Noisy Environments,”  in Proc. Interspeech, 2019, pp. 1368–1372.
  * [20] Ritwik Giri, Umut Isik, and Arvindh Krishnaswamy,  “Attention wave-u-net for speech enhancement,”  in Proc. WASPAA, 2019, pp. 249–253.
  * [21] Junichi Yamagishi, Christophe Veaux, and Kirsten MacDonald,  “CSTR VCTK Corpus: English multi-speaker corpus for CSTR voice cloning toolkit (version 0.92),” 2019.
  * [22] Joachim Thiemann, Nobutaka Ito, and Emmanuel Vincent,  “The diverse environments multi-channel acoustic noise database (demand): A database of multichannel environmental noise recordings,”  in Proc. ICA, 2013.
  * [23] Efthymios Tzinis, Zhepei Wang, and Paris Smaragdis,  “Sudo rm-rf: Efficient networks for universal audio source separation,”  in Proc. MLSP, 2020, pp. 1–6.
  * [24] Yi Luo, Cong Han, and Nima Mesgarani,  “Ultra-lightweight speech separation via group communication,”  in Proc. ICASSP, 2021, pp. 16–20.
  * [25] Efthymios Tzinis, Zhepei Wang, Xilin Jiang, and Paris Smaragdis,  “Compute and memory efficient universal sound source separation,”  Journal of Signal Processing Systems, 2021.
  * [26] Jonathan Le Roux, Scott Wisdom, Hakan Erdogan, and John R Hershey,  “Sdr–half-baked or well done?,”  in Proc. ICASSP, 2019, pp. 626–630.
  * [27] Diederik P Kingma and Jimmy Ba,  “Adam: A method for stochastic optimization,”  arXiv preprint arXiv:1412.6980, 2014.
  * [28] Cees H Taal, Richard C Hendriks, Richard Heusdens, and Jesper Jensen,  “An algorithm for intelligibility prediction of time–frequency weighted noisy speech,”  IEEE Transactions on Audio, Speech, and Language Processing, vol. 19, no. 7, pp. 2125–2136, 2011.
  * [29] Antony W Rix, John G Beerends, Michael P Hollier, and Andries P Hekstra,  “Perceptual evaluation of speech quality (pesq)-a new method for speech quality assessment of telephone networks and codecs,”  in Proc. ICASSP, 2001, pp. 749–752.


* [94] D. Wands, K. A. Malik, D. H. Lyth and A. R. Liddle, “A New approach to the evolution of cosmological perturbations on large scales,” Phys. Rev. D 62 (2000), 043527 [arXiv:astro-ph/0003278 [astro-ph]].
  * [95] K. Inomata, K. Kohri, T. Nakama and T. Terada, “Enhancement of Gravitational Waves Induced by Scalar Perturbations due to a Sudden Transition from an Early Matter Era to the Radiation Era,” Phys. Rev. D 100 (2019) no.4, 043532 [arXiv:1904.12879 [astro-ph.CO]].
  * [96] K. Inomata, K. Kohri, T. Nakama and T. Terada, “Gravitational Waves Induced by Scalar Perturbations during a Gradual Transition from an Early Matter Era to the Radiation Era,” JCAP 10 (2019), 071 [arXiv:1904.12878 [astro-ph.CO]].
  * [97] I. Dalianis and C. Kouvaris, “Gravitational waves from density perturbations in an early matter domination era,” JCAP 07 (2021), 046 [arXiv:2012.09255 [astro-ph.CO]].
  * [98] J. Garcia-Bellido and E. Ruiz Morales, “Primordial black holes from single field models of inflation,” Phys. Dark Univ. 18, 47 (2017) [arXiv:1702.03901 [astro-ph.CO]].
  * [99] H. Motohashi and W. Hu, “Primordial Black Holes and Slow-Roll Violation,” Phys. Rev. D 96 (2017) no.6, 063503 [arXiv:1706.06784 [astro-ph.CO]].
  * [100] G. Ballesteros and M. Taoso, “Primordial black hole dark matter from single field inflation,” Phys. Rev. D 97, no. 2, 023501 (2018) [arXiv:1709.05565 [hep-ph]].
  * [101] K. Kefala, G. P. Kodaxis, I. D. Stamou and N. Tetradis, “Features of the inflaton potential and the power spectrum of cosmological perturbations,” Phys. Rev. D 104 (2021) no.2, 023506 [arXiv:2010.12483 [astro-ph.CO]].
  * [102] I. Dalianis, G. P. Kodaxis, I. D. Stamou, N. Tetradis and A. Tsigkas-Kouvelis, “Spectrum oscillations from features in the potential of single-field inflation,” Phys. Rev. D 104 (2021) no.10, 103510 [arXiv:2106.02467 [astro-ph.CO]].
  * [103] C. Fu, P. Wu and H. Yu, “Primordial Black Holes from Inflation with Nonminimal Derivative Coupling,” Phys. Rev. D 100 (2019) no.6, 063532 [arXiv:1907.05042 [astro-ph.CO]].
  * [104] I. Dalianis, S. Karydas and E. Papantonopoulos, “Generalized Non-Minimal Derivative Coupling: Application to Inflation and Primordial Black Hole Production,” JCAP 06 (2020), 040 [arXiv:1910.00622 [astro-ph.CO]].
  * [105] J. Fumagalli, S. Renaux-Petel and L. T. Witkowski, “Oscillations in the stochastic gravitational wave background from sharp features and particle production during inflation,” JCAP 08 (2021), 030 [arXiv:2012.02761 [astro-ph.CO]].
  * [106] I. Dalianis, “Constraints on the curvature power spectrum from primordial black hole evaporation,” JCAP 08 (2019), 032 doi:10.1088/1475-7516/2019/08/032 [arXiv:1812.09807 [astro-ph.CO]].
  * [107] C. T. Byrnes, P. S. Cole and S. P. Patil, “Steepest growth of the power spectrum and primordial black holes,” JCAP 06 (2019), 028 [arXiv:1811.11158 [astro-ph.CO]].
  * [108] G. Franciolini, A. Kehagias, S. Matarrese and A. Riotto, “Primordial Black Holes from Inflation and non-Gaussianity,” JCAP 1803 (2018) no.03, 016 [arXiv1801.09415[astro-ph.CO]].
  * [109] S. Passaglia, W. Hu and H. Motohashi, “Primordial black holes and local non-Gaussianity in canonical inflation,” Phys. Rev. D 99, no. 4, 043536 (2019) [arXiv:1812.08243 [astro-ph.CO]].
  * [110] V. Atal and C. Germani, “The role of non-gaussianities in Primordial Black Hole formation,” Phys. Dark Univ. 100275 [arXiv:1811.07857 [astro-ph.CO]].
  * [111] V. De Luca, G. Franciolini, A. Kehagias, M. Peloso, A. Riotto and C. Ünal, “The Ineludible non-Gaussianity of the Primordial Black Hole Abundance,” [arXiv:1904.00970 [astro-ph.CO]].
  * [112] C. Pattison, V. Vennin, H. Assadullahi and D. Wands, “Quantum diffusion during inflation and primordial black holes,” JCAP 1710 (2017) no.10, 046 [arXiv1707.00537[hep-th]].
  * [113] M. Biagetti, G. Franciolini, A. Kehagias and A. Riotto, “Primordial Black Holes from Inflation and Quantum Diffusion,” JCAP 1807 (2018) no.07, 032 [arXiv:1804.07124 [astro-ph.CO]].
  * [114] J. M. Ezquiaga and J. García-Bellido, “Quantum diffusion beyond slow-roll: implications for primordial black-hole production,” JCAP 1808 (2018) 018 [arXiv:1805.06731 [astro-ph.CO]].
  * [115] D. Cruces, C. Germani and T. Prokopec, “Failure of the stochastic approach to inflation beyond slow-roll,” JCAP 1903 (2019) no.03, 048 [arXiv:1807.09057 [gr-qc]].


# IoT Coverage Enhancement using Repetition in Energy Constrained Devices: an
Analytic Approach

Bisma Manzoor, , Bassel Al Homssi, ,
and Akram Al-Hourani The authors are with the School of Engineering, RMIT
University, Melbourne, VIC 3000, Australia. This research is supported by the
Australian Government Research Training Program (RTP) Scholarship.

###### Abstract

Novel Internet-of-Things (IoT) access technologies are emerging as part of the
next generation cellular networks. These technologies are specifically
oriented towards energy-limited IoT devices that are scattered far away from
their serving base station. One of the key methods of achieving deep coverage
is via repeated transmissions of data. However, repetition leads to higher
energy consumption and reduces the device battery-lifetime. Thus, a trade-off
between coverage and energy consumption exists and requires careful
investigation. This paper evaluates the effects of transmission repetition on
enhancing the probability of coverage and the cost of the incurred energy.
Using an empirical repetition profile based on traffic load and device
distance from the base station, we derive the coverage probability and energy
profile models for IoT links that utilize two different diversity combining
techniques. In particular, we focus on two common diversity combining that
are; (i) Selection Combining (SC), and (ii) Maximal Ratio Combining (MRC). We
utilize tools from stochastic geometry to formulate an analytic framework that
compares these two combining methods. This framework can aid network designers
in jointly maximizing the network coverage while minimizing the energy
expenditure of devices.

###### Index Terms:

Frame repetition, Internet-of-Things, stochastic geometry, energy-consumption,
diversity combining

##  I Introduction

Cellular Internet of Things (CIoT) connections are expected to grow to 5
billion by 2025 [1] due to the increasing number of use-cases from various
sectors such as agriculture, healthcare, industry. The 5G New Radio (NR) is
provisioned to support the growing traffic in the International Mobile
Telecommunications (IMT) network, that also includes CIoT. The use cases of 5G
are broadly classified into three categories (i) massive Machine-Type
Communication (mMTC), (ii) ultra-Reliable Low Latency Communication (uRLLC),
and (iii) enhanced Mobile Broadband (eMBB)[2]. IoT applications are envisioned
to fall under mMTC and uRLLC. mMTC supports discontinuous and delay-tolerant
applications, whereas uRLLC is meant for critical and high throughput
applications[3].

The massive number of connections in mMTC is due to the wide range of
applications across smart cities, smart agriculture, and others[4]. mMTC IoT
devices are expected to have a longer battery life compared to the traditional
cellular devices, and operate at lower power and long ranges. A class of
network technologies designed to support low-power IoT devices is termed as
Low-Power-Wide-Area-Networks (LPWAN) which provides wireless communication
services while meeting the service requirements of wide area coverage, and low
battery consumption for mMTC devices at the cost of lower bit rate than
typical cellular services.

Narrow band Internet of Things (NB-IoT) is one of the LPWAN technologies
recently developed by 3GPP to operate in the licensed frequency spectrum.
Unlike mobile broadband, NB-IoT supports low-power and high coverage by
maintaining low bit-rate. There is a multitude of methods available to enhance
the coverage, such as utilizing narrow-bandwidth, using spread spectrum
techniques, and employing frame repetitions [5]. Frame repetition is gaining
traction in new IoT access technologies such as NB-IoT. It is also utilized in
uRLLC for grant-free access mechanism [6] that allows a device to access time-
frequency resources without going through the traditional handshake process.
Repetitions allow the data to be transmitted multiple times, increasing the
detection-ability by enhancing the signal power at the receiver. However, it
comes at the cost of increased number of transmissions resulting in an overall
increase in network traffic and interference. In addition, even though
repetitions improve the coverage, they cause an extensive battery consumption
of the transmitting device. This is due to the transmitter staying active for
a prolonged duration while transmitting the repeated data. Furthermore, each
unsuccessful transmission leads to energy wastage in the transmitting device.
Thus, it is imperative to jointly analyze the coverage and energy wastage of
the network. Besides, various combining techniques exist to process the
repeated transmissions at the receiver. The diversity combiners operate at
different stages of the receiver. The repetitions are either combined before
signal processing which leads to an increase in total signal energy, or the
combination takes place after signal processing, resulting in an overall
improvement in the system performance[7].

A plethora of good literature is available on the energy consumption of IoT
devices [8, 9, 10, 11]. However, the energy wasted due to the failure of
repetitions does not seem to be analyzed previously. Besides, authors in [12,
13] present the analysis on combining techniques, and the work in [14] sheds
light on the combination scheme for IoT; however, the impact of interference
has not been taken into account. This work focuses on analyzing the repetition
scheme for the combining techniques that operate before signal processing
which are Selection Combining and Maximal Ratio Combining. The coverage
probability and energy dissipation analysis are carried out for both diversity
combining techniques. The performance is examined under the influence of
spatially co-related interference. The contributions of this paper are
summarized as follows,

  * •

It provides analytical models for the coverage probability with repetitions
using SC and MRC.

  * •

A mathematical model has been put forth to capture the energy expenditure of a
device during the process of implementing repetition.

  * •

Furthermore, the model aids in obtaining the ideal repetition profiles that
facilitate the maximum service availability and minimum energy dissipation.

##  II Background and Related Work

The concept of repetition is introduced by 3GPP [15] in release 13, for
cellular IoT access technologies to extend the coverage in mMTC networks
comprising of vast number of devices [16]. The devices using the repetition
scheme repeat the frames multiple times, which are accumulated at the receiver
to enhance the signal detection probability. The signal is then processed for
synchronization and channel estimation [5]. Ideally, the processing gain at
the receiver should be proportional to the number of repetitions used.
Nevertheless, practical signal combination is imperfect due to channel
impairments resulting in a lower gain than that of an ideal gain.

Signal propagation in a channel is subject to power losses due to multiple
factors including the losses due to natural wave expansion and the losses due
to multi-path fading. The average attenuation of the signal power as it
propagates for a certain distance is termed as the path-loss while the
fluctuations in the signal power due to the interactions with the environment
are caused by the multipath signal components, this is referred to as fading
[17]. In a non-stationary channel, the different repetitions encounter
different levels of fading [18], which is mitigated by combining the
repetitions at the receiver. As different combination techniques exist [19],
their performance analysis under variable signal impairments is of interest.
The most commonly used diversity combining methods are (i) SC (ii) MRC. In SC
technique, the receiver opts for the signal with the highest SNR. MRC is a
linear combining technique, where, all the signals are weighted in terms of
amplitude and phase, followed by the combination at the receiver, leading to
the gain in the signal power [20]. Although the performance is improved with
the aid of combining techniques, it is vital to investigate the impact of
interference on the received signals.

One of the main challenges to accurately examine network performance is to
incorporate the channel losses and the interference from the other active
devices located randomly in the network. Stochastic geometry captures the
spatial randomness of IoT devices and thus facilitates the analysis of key
performance indicators in a wireless network [21]. Using the point process to
define the location of the devices usually makes the analytic approach more
tractable [22]. Literature overview on combining techniques using stochastic
geometry is presented in [23, 24], where in the work in [23] provides the
performance analysis of the SC and MRC techniques using the Nakagami-Fading
Channels, while in [24] the authors provide the quantitative measures of the
performance of the linear combining techniques including SC and MRC, along
with the analytical expressions for SNR associated with the given combining
techniques are put forth. Nevertheless, the impact of unwanted power arising
from the spatially distributed nodes is not accounted for. Moreover, the work
in [14] provides a performance analysis of the combining strategies
implemented in cellular IoT technologies that are SC and MRC; however, it does
not account for the interference in the network. In contrast, although the
authors in [25, 26], employ stochastic geometry analysis with interference
correlation, which aids in profoundly characterizing the network behavior for
various combination techniques, there is a lack of investigation for the IoT
repetition scheme under such scenario.

Additionally, the repetition scheme impacts the active period of the device
leading to more energy consumption. An IoT device is said to be in an active
state when it is communicating with the network, else it is inactive in order
to preserve the battery [10]. To further enhance the battery life in IoT
devices, operational modes, i.e., Power Saving Mode (PSM) and extended
Discontinuous Reception (eDRX), are employed in cellular IoT technologies.
However, since the repetition scheme is employed in the active state, the
impact of repetitions on energy drainage during this state requires
investigation, especially if the repetitions fail to get delivered. Various
innovative mechanisms have been adopted to enhance the battery efficiency in
IoT devices. One such technique is the energy harvesting [27], where the IoT
devices harvest their energy from the external sources. The work in [27], have
put forth a two stage algorithm to maximize the energy efficiency in IoT
devices by harvesting energy from the radio-frequency signals. Authors in [28]
use the energy harvesting mechanism for an access scheme called irregular
repetition ALOHA (IRA) that utilizes varying repetitions. They put forth the
optimal transmission policy to maximize the system throughput. In another work
[29] authors have proposed a Irregular Repetition Slotted ALOHA, where the
nodes replicate the packets on a frame while utilizing the energy harvesting.
The authors obtain the optimized probability distributions for the number of
packet replicas. However, as aim of repetitions employed in the cellular IoT
is to achieve a high SNR and the detection probability at the receiver [30],
the above literature lacks the analysis of the energy efficiency from such
perspective. Furthermore, the literature on energy-efficient techniques
involving the repetitions in IoT devices is analyzed in [8, 9, 10], wherein
the predictive algorithm proposed in [9] aims at lowering energy expenditure
due to repetitions in an NB-IoT network by reducing the scheduling requests.
The work in [11] has highlighted the impact of repetitions on energy
consumption and hence, focused on the significance of implementing appropriate
network parameters, such as repetition rate. Nevertheless, the above works do
not investigate the energy wastage due to unsuccessful repetitions.

##  III System Model

###  III-A Geometric Model

Figure 1: Illustration of a typical IoT cell.

We consider a cell with a radius $R_{\text{c}}$ and a gNB located at its
center as illustrated in Fig.1. The IoT devices are assumed to be distributed
by an isotropic and homogeneous Poisson Point Process (PPP), denoted by
$\Phi\in\mathbb{R}^{2}$. The active devices have an initial base density of
$\lambda_{\text{o}}$ and are assumed to transmit messages of equal length at a
constant power $P_{\text{t}}$.

TABLE I: Notations and Symbols  Symbol |  Definition |  Value
---|---|---
$\Phi$ |  IoT devices point process |  -
$\lambda_{\text{o}}$ |  Initial base density |  2$/100^{2}$
$D_{\text{o}}$ |  Initial base duty cycle |  0.01
$r_{\text{o}}$ |  Distance of device from a gNB |  -
$\lambda(r)$ |  Effective density |  -
$D(r)$ |  Effective duty cycle |  -
$N$ |  Repetitions |  -
$\psi(r)$ |  Repetition profile function |  -
$a$ |  Steepness parameter of repetition profile |  -
$b$ |  Rate parameter of repetition profile |  -
$F_{I}$ |  CDF of the Interference |  -
$R_{\text{c}}$ |  Radius of the cell |  1000 m
$p|_{r{{}_{\mathrm{o}}}}(1)$ |  Probability of success for a single transmission given the device is at distance $r_{\mathrm{o}}$ |  -
$p|_{r{{}_{\mathrm{o}}}}(N)$ |  Probability of success for $N$ repetitions given the device is at distance $r_{\mathrm{o}}$ |  -
$\epsilon(1)$ |  Energy consumed for a single transmission |  -
$\epsilon({N})$ |  Energy consumed for $N$ transmissions |  -
$\epsilon_{\mathrm{w}}(N)$ |  Energy wasted for $N$ transmissions |  -
$\bar{\epsilon}_{\text{c}}$ |  Average energy wasted in a cell |  -
$f_{r}(r_{\mathrm{o}})$ |  Radial distance pdf |  -
B |  Bandwidth |  180 kHz
F |  Receiver noise figure |  3 dB
$\eta_{\epsilon}$ |  Power conversion factor |  4
$P_{\text{O}}$ |  Power overhead |  210 mW
$\alpha$ |  Path-loss exponent |  3.5†1
$\sigma^{2}$ |  Average noise power |  $-$118 dBm†2
$\dagger 1$ WINNER II channel model measurements [31]
$\dagger 2$ Calculated from noise figure of 3 dB and bandwidth 180 kHz

To enhance the signal coverage in this IoT network, we consider the repetition
mechanism, according to which the entire message is repeated several
times[15]. We utilize the mathematical framework developed in [32] in order to
implement the repetition profile that is based on the duty cycle and the
device’s distance from gNB. We assume $D_{\text{o}}$ is the base duty cycle in
a cell, given as $D_{\text{o}}=T_{\text{o}}/T_{\text{max}}$, where $T_{o}$ is
the duration of a single radio frame that is sent after every $T_{\text{max}}$
time. Since the devices are configured to repeat $N$ times, the effective duty
cycle becomes $D=NT_{\mathrm{o}}/T_{\mathrm{max}}=ND_{\mathrm{o}}$.

Moreover, the signal strength weakens for the devices located father from gNB,
and thus they tend to repeat more. This intuitively increases the duty cycle
of the devices. To incorporate this impact of distance, the duty cycle as a
function of distance is put forth as,

$D(r)=D_{\mathrm{o}}+(1-D_{\mathrm{o}})\psi(r)$ (1)

Hence, using the relation between the duty cycle and $N$ as previously put
forth, the repetition profile in terms of distance is given as follows,

$N(r)=\left\lceil{1+\frac{1-D_{\mathrm{o}}}{D_{\mathrm{o}}}\psi(r)}\right\rceil,$
(2)

where $\lceil.\rceil$ is the ceiling function, $D_{\text{o}}\leq D(r)\leq 1$,
and $\psi(r)$ is called the repetition profile function. This function is
selected such that $\psi(r)\in\left[0,1\right]$, and the duty cycle is bounded
between 0 and 1, i.e., $D(r)\in\left[0,1\right]$, since duty cycle cannot
exceed unity. Also $\psi(r)$ should be monotonically increasing, such that
$N(r)$ increases with the distance. A suitable function that satisfies the
above requirements is the logistic function given as,

$\psi(r)=\frac{1}{1+\exp\left(-\frac{r-b}{a}\right)},$ (3)

where $a$ and $b$ are the repetition profile function parameters of the
repetition profile. $a$ controls the steepness of repetitions, while, $b$
determines the mid-point of the logistic function. Repetition profiles with a
varying $N$ obtained at various repetition parameters is illustrated in Fig.
2. It can be seen that the number of repetitions used by the device increases
with respect to distance from gNB, and every repetition profile corresponds to
different number of repetitions. The model was also empirically observed in
NB-IoT technology [18]. Moreover, by utilizing the above given model, the
density at a given snapshot in a cell using a certain repetition profile is
defined as effective density, denoted as $\lambda(r)$, and is given as
$\lambda(r)=\lambda_{\mathrm{o}}D(r)$.

Figure 2: Repetitions $N(r)$ versus the radial distance at various repetition
profile parameters.

###  III-B Channel Model

For a device located at $x$, the received power at the gNB located at the
origin is $P_{r}=P_{\mathrm{t}}h~{}l(x)$, where $h$ is the channel fading
random variable and $l(x)$ is the path-gain[33]. Using the log-distance path-
loss model, $l(x)=||x||^{-\alpha}$ [34]. The strength of the signal at the
receiver is also subject to additional channel losses such as shadowing;
however, this work focuses on evaluating the performance under fading and
path-loss only. Furthermore, for various combining techniques, the success
probability at the receiver, which is dependent on the received signal power
and the interference from other devices, is analyzed for two-channel
conditions (i) Path-loss only where $h=1$ and the loss in the signal is only
due to the distance of the device from the gNB. (ii) Path-loss with Rayleigh
fading, which includes the path-loss along with the Rayleigh fading, where $h$
is assumed to be an exponentially distributed random variable with unity mean.
We first evaluate the interference characterization for both channel
conditions, followed by performance evaluation.

##  IV Interference Characterization

To preserve the simplicity of the IoT access technology, we assume a simple
ALOHA-like system where radio frames collide due to overlapping transmissions.
For a device being served at location $||x_{\text{o}}||$, all the other active
users are considered as a source of interference. In order to evaluate the
success probability, it is essential to characterize the aggregate
interference power from the active users as follows,

$\displaystyle
I=\sum_{x_{\text{i}}\in\Phi\text{\textbackslash}x_{\mathrm{o}}}P_{\mathrm{t}}h_{\mathrm{i}}||x_{\mathrm{i}}||^{-\alpha}=\sum_{x_{\mathrm{i}}\in\Phi\text{\textbackslash}x_{\mathrm{o}}}P_{\mathrm{t}}h_{\mathrm{i}}r_{\mathrm{i}}^{-\alpha},$
(4)

where $||x_{i}||$ is the location of the interferers located at a distance
$r_{i}$ from the gNB. The stochastic behavior of the interference is typically
characterized using its cumulative distribution function (CDF), however, such
function cannot be obtained in a closed-form for the proposed scenario.
Alternatively, we describe the stochastic behavior of the interference in
terms of the Laplace Transform (LT) of the CDF. For a known LT of the
interference, the CDF is calculated using numerical inversion of Laplace
transform [35],

$\displaystyle
F_{I}(x)=\mathcal{L}^{-1}\left[\frac{1}{s}\mathcal{L}_{I}(s)\right],$ (5)

where $\mathcal{L}_{I}(s)$ is the LT of the interference random variable $I$
evaluated at a point $s$ which is defined as
${\mathcal{L}_{X}(s)=\mathbb{E}[{\mathrm{exp}(-sX)}]}$, for a given variable
$X$. Using the above definition of Laplace transform, $\mathcal{L}_{I}(s)$ is
evaluated for $I$ from (4), as follows,

$\displaystyle\mathcal{L}_{I}(s)$
$\displaystyle=\mathbb{E}_{\Phi}\mathbb{E}_{h}\left[\mathrm{exp}\left(-s\sum_{x_{i}\in\Phi\setminus
x_{\mathrm{o}}}h_{i}r_{i}^{-\alpha}\right)\right]$
$\displaystyle\overset{(\mathrm{a})}{=}\mathbb{E}_{\Phi}\left[\prod_{x_{i}\in\Phi\setminus
x_{\mathrm{o}}}\mathbb{E}_{h}\left[\exp\left(-sh_{i}r_{i}^{-\alpha}\right)\right]\right],$
(6)

where (a) stems from the fact that $h_{i}$ is independent of the point process
and by converting the sum of exponents to product. Using the definition of the
probability generating functional on $\mathbb{R}^{2}$, for a function
$f:\mathbb{R}^{2}\rightarrow[0,1]$ as in [35],

$\displaystyle\mathbb{E}\left[\prod_{x\in\Phi}f(x)\right]=\mathrm{exp}\left(-\int_{\mathbb{R}^{2}}\left[1-f(x)\right]\lambda(\mathrm{d}x)\right)$
(7)

for the PPP, $\mathcal{L}_{I}(s)$ is given as,

$\displaystyle\mathcal{L}_{I}(s)=\mathrm{exp}\left(-\int_{\mathbb{R}^{2}}\left(1-\mathbb{E}_{h}\left[\exp\left(-shx^{-\alpha}\right)\right]\right)\lambda(\mathrm{d}x)\right)~{}.$
(8)

###  IV-A Interference Characterization with path-loss only

Considering the scenario for the channel when it is impacted only by path-
loss. Thus, ignoring the effect of fading and substituting the fading
component $h=1$, we re-write (8) in polar coordinates as follows,

$\displaystyle\mathcal{L}_{I}(s)=\exp\left(-2\pi\lambda_{\mathrm{o}}\int_{o}^{R_{\text{c}}}\left[1-\exp\left(-sr^{-\alpha}\right)\right]D(r)r\mathrm{d}r\right).$
(9)

Thus, the inverse Laplace Transform of (9) provides the CDF of the aggregate
interference power and it can be numerically obtained using the Euler
algorithms for numerical approximations of the inverse Laplace Transform [36].

###  IV-B Interference Characterization with Rayleigh Fading

In this scenario we assume that both the serving and the interfering signals
are impaired with path-loss and Rayleigh fading, which is denoted by an
exponential distribution random variable with a unity mean. The definition of
the moment generating function (MGF) of such an exponential variable is
${E}\left[\exp(-t)\right]=1/(1-t)$. Thus MGF for the fading variable $h$ at
$-sr_{\text{i}}^{-\alpha}$ is given as,
$\mathbb{E}_{h}\left[\exp(-shr_{i}^{-\alpha})\right]=1/(1+sr_{i}^{-\alpha})$
Substituting in (8), the Laplace transform of interference for Rayleigh fading
is given as in [32],

$\displaystyle\mathcal{L}_{I}(s)=\exp\left(-2\pi\lambda_{\mathrm{o}}\int_{0}^{R_{\mathrm{c}}}\frac{s}{s+r^{\alpha}}D(r)r\mathrm{d}r\right).$
(10)

Consequently, (9) and (10) are employed to calculate the success probability
in terms of coverage for the given combining techniques utilizing repetition
model. In case of the scenario when no repetitions are used in the cell,
$\psi(r)=0$, and $\lambda(r)$ reduces to $\lambda_{\mathrm{o}}$ in (9) and
(10).

##  V Performance Evaluation

This section introduces the performance evaluation of the two diversity
combining techniques (i) SC and (ii) MRC at the given repetition profiles
compared to the no repetition scenario. The values of the repetition profiles
parameters $a$ and $b$ can be optimized based on the evaluation of repetition
performance zones as defined in [32]. The combining techniques are
investigated under the spatially-correlated interference and channel
impairments that include path-loss and Rayleigh fading. The probability of
success at the receiver is assessed in terms of the Signal-to-Interference-
plus-Noise-Ratio (SINR), denoted by $\gamma$. It is the ratio of the received
signal power of the target device to the sum of total interference power from
interferes and noise power, given as
$\gamma=h_{\mathrm{o}}r_{\mathrm{o}}^{-\alpha}/(I+\sigma^{2})$, where
$r_{\mathrm{o}}=||x_{\mathrm{o}}||$ is the distance of the target device from
the gNB, $\sigma^{2}$ is the average noise power, $I$ is the aggregated
interference power from the active interferes. A transmission is considered to
be successful when $\gamma$ is above a given design threshold $\theta$ at the
receiver.

###  V-A No Repetition

This is a typical scenario where the devices are configured to transmit the
message only once. We abbreviate it as No Rep and use the same notation for
comparison. The probability of having the SINR of this single transmission
above a required receiver threshold is formulated as,

$\displaystyle
p_{\mathrm{\tiny{NoRep}}}=\mathbb{P}\left(\frac{h_{\mathrm{o}}r_{\mathrm{o}}^{-\alpha}}{I+\sigma^{2}}>\theta\right)=F_{I}\left(\frac{h_{\mathrm{o}}r_{\mathrm{o}}^{-\alpha}}{\theta}-\sigma^{2}\right),$
(11)

The above is considered as the probability of success and is later compared
with the combining techniques. For path-loss only, the probability of success
is obtained by evaluating $F_{I}$ using the inversion of Laplace transform
from (9) and by substituting ${D(r)=D_{o}}$. In case of the channel impaired
by fading, we assume both Rayleigh fading and the path-loss. Thus, the
probability of success is calculated with the Rayleigh fading component
$h_{\mathrm{o}}$ as an exponential distribution random variable with unity
mean. Further, $F_{I}$ in this case is evaluated using the inversion of
Laplace transform from (10), with $D(r)=D_{o}$, and is deconditioned over the
fading by utilizing the pdf of exponential distribution random variable given
by ${f_{h_{\mathrm{o}}}(x)=\exp(-x)}$. The concept is well defined in
literature as given in [21].

###  V-B Selection Combining

In this combination technique, at a given time, the receiver selects the
repetition with the highest SINR [24]. This is expressed as,

$\displaystyle\gamma_{\text{sc}}=\max(\gamma_{\text{1}},\gamma_{\text{2}}...\gamma_{\text{N}}),$

where, $\gamma_{\text{1}},\gamma_{\text{2}},..\gamma_{\text{N}}$ are the
individual SINRs of the repetitions 1 to N. Accordingly, in order to have a
successful repetition, there should be at least one signal with SINR above the
required threshold. Thus, the probability of success conditioned at a given
$r_{\mathrm{o}}$ is formulated as,

$\displaystyle
p|_{r{{}_{\text{o}}}}(N)=\mathbb{P}\left(\exists~{}n\in[1,N]:\gamma_{\text{n}}>\theta\right)$
(12)

Since the receiver selects the signal with the highest SINR, the probability
of the chosen signal to be below a given threshold is the product of
probabilities of SINRs of all signals to be below the threshold[37, 38]
accordingly the, probability of error is,

$\displaystyle
p_{e}{{}_{|r_{\mathrm{o}}}}(N)=\mathbb{P}\left[\gamma_{\text{n}}<\theta\right]^{N}=\left[1-p|_{r{{}_{\text{o}}}}(1)\right]^{N},$
(13)

where $p|_{r{{}_{\text{o}}}}(1)$ is the probability of success of a single
signal and $n\in[1,N]$. Therefore, the probability of success for SC is given
by

$\displaystyle{p_{|r_{\mathrm{o}}}(N)}$
$\displaystyle=1-p_{e}{{}_{|r_{\mathrm{o}}}}(N)$ (14)

which is evaluated for two channel scenarios as described next.

####  V-B1 SC with path-loss only

The analysis in this case presents the upper bound of the channel performance
since both the serving and interfering signals are impacted by path-loss only
channel. Thus, the probability of success for a signal at $r_{o}$ is,

$\displaystyle
p_{|r_{\mathrm{o}}}(1)=\mathbb{P}({\gamma>\theta}|r_{\mathrm{o}})$
$\displaystyle=F_{I}\left(\frac{r_{\mathrm{o}}^{-\alpha}}{\theta}-\sigma^{2}\right),$
(15)

where $F_{I}$ is the (CDF) of the interference $I$ and is obtained using (5)
and (9). Hence, given $N$ repetitions the success probability is evaluated
using (14) as follows,

$\displaystyle{p_{|r_{\mathrm{o}}}(N)}$
$\displaystyle=1-p_{e}{{}_{|r_{\mathrm{o}}}}(N)$
$\displaystyle=1-\left[1-F_{I}\left(\frac{r_{\mathrm{o}}^{-\alpha}}{\theta}-\sigma^{2}\right)\right]^{N},$
(16)

####  V-B2 SC with Path-loss and Fading

We take the Rayleigh fading model which represents the worst-case scenario
with no dominant path between the device and the gNB. Thus, the probability of
success at the receiver conditioned at a given $r_{o}$ and $h_{o}$ using (13)
is,

$\displaystyle p_{|r_{\mathrm{o}},h_{\text{o}}}(1)$
$\displaystyle=\mathbb{P}(\gamma>\theta|r_{\mathrm{o}},h_{\text{o}})$
$\displaystyle=\mathbb{P}\left(\frac{h_{o}r_{\mathrm{o}}^{-\alpha}}{I+\sigma^{2}}>\theta\right)$
$\displaystyle=F_{I}\left(\frac{h_{\mathrm{o}}r_{\mathrm{o}}^{-\alpha}}{\theta}-\sigma^{2}\right),$
(17)

where $h_{o}$ is independent of $r_{\mathrm{o}}$. Deconditioning at
$h_{\mathrm{o}}$ ,

$\displaystyle
p_{|r_{\mathrm{o}}}(1)=\int_{0}^{\infty}F_{I}\left(\frac{ur_{\mathrm{o}}^{-\alpha}}{\theta}-\sigma^{2}\right)f_{h_{\mathrm{o}}}(u)\mathrm{d}u,$
(18)

Thus, the probability of success for N repetitions, at $r_{\mathrm{o}}$ is
obtained using (14) as follows,

$\begin{split}{p_{|r_{\mathrm{o}}}(N)}&=1-p_{e}{{}_{|r_{\mathrm{o}}}}(N)\\\
&=1-\left[1-\int_{0}^{\infty}F_{I}\left(\frac{ur_{\mathrm{o}}^{-\alpha}}{\theta}-\sigma^{2}\right)f_{h_{\mathrm{o}}}(u)\mathrm{d}u\right]^{N},\end{split}$
(19)

where the CDF of the interference in this case is obtained by using (5) and
(10). $f_{h_{\text{o}}}(u)=\exp(-u)$ is the PDF of the exponential
distribution with unity mean.

The performance of the SC technique is depicted in Fig. 3, where Monte Carlo
simulations are compared to that of analytical equations. The probability of
success of SC combining technique surpasses the no repetition scenario for
both channel conditions. The path-loss only condition reveals a better
performance. Additionally, for both no repetition and SC, the signal power
declines with an increase in the distance between device and gNB, and so does
the probability of success. However for the SC, the performance of devices at
large distance shows an improved probability of success due to the increase in
the repetitions $N$ configured by the device. Since the number of repetition
$N$ is a discrete value, a stepping effect is observed in the probability of
success. Note that there is an optimal combination parameters $a$ and $b$ that
maximizes the coverage probability, as very low values of $N$ lead to high
frame losses and very high $N$ leads to elevated frame collisions. The
optimization concept is thoroughly elaborated in [32].

Figure 3: Conditional success probability vs $r_{\mathrm{o}}$ obtained for no
repetition and the two cases of Selection Combining from (V-B1) and (19),
respectively. The utilized repetition profile parameters are $a=50$, $b=1050$.

###  V-C Maximal Ratio Combining

The aim of this technique is to maximize the resulting SINR by combining the
$N$ repetition transmissions. To obtain an accurate analysis, all
transmissions in a repetition undergoes separate fading. The received
transmissions are added before further signal processing, leading to an
overall gain in the SINR. Implementing the geometric model as explained in
III-A, for a device at a given location transmitting $N$ repetitions, the
received signal at the gNB when using MRC is given as in eq. (13.17) [38],

$\displaystyle\gamma_{\mathrm{MRC}}=\sum_{n=1}^{N}\beta_{\mathrm{n}},$ (20)

where $\beta_{\text{n}}$ is the SNR of the each repetition. We extend the case
by including interference[26, 12, 13] from the other active devices. We assume
the average interference across all $n$ repetitions. Nevertheless, we account
for the spatial correlation of $I$ using the repetition profile and (4).

Thus, the SINR for a device at a distance $r_{\text{o}}$ and using $N$
repetitions is formulated as follows,

$\displaystyle\gamma(N)=\frac{\sum_{n=1}^{N}h_{\mathrm{o}_{n}}r_{o}^{-\alpha}}{I+\sigma^{2}},$
(21)

where $h_{\mathrm{o}_{n}}$ is the fading power gain of $n^{th}$ repetition and
$I$ is the aggregated interference power as provided in (4). The probability
of success for MRC under various channel losses is evaluated in next
subsections.

####  V-C1 MRC with path-loss only

Under the impact of the path-loss only, from (21), the probability of success
for $N$ repetitions, conditioned at a given $r_{\text{o}}$ reduces to,

$\displaystyle p_{|r_{\mathrm{o}}}(N)$
$\displaystyle=\mathbb{P}({\gamma(N)>\theta}|r_{\mathrm{o}},h)$
$\displaystyle=\mathbb{P}\left(\frac{\sum_{n=1}^{N}h_{\mathrm{o}_{n}}r_{\mathrm{o}}^{-\alpha}}{I+\sigma^{2}}>\theta\right)$
$\displaystyle\overset{(\mathrm{a})}{=}\mathbb{P}\left(\frac{N(r_{\mathrm{o}})r_{\mathrm{o}}^{-\alpha}}{I+\sigma^{2}}>\theta\right)~{},$
(22)

where (a) follows from the assumption of fading-less channel i.e.,
$h_{\mathrm{o}}$ = 1. Accordingly, the probability of success of $N$
repetitions is given as,

$\displaystyle p_{|r_{\mathrm{o}}}(N)$
$\displaystyle=F_{I}\left(\frac{N(r_{\mathrm{o}})r_{\mathrm{o}}^{-\alpha}}{\theta}-\sigma^{2}\right).$
(23)

$F_{I}$ is obtained from (5), where $\mathcal{L}_{I}(s)$ is used from (9).

Figure 4: Conditional success probability vs $r_{\mathrm{o}}$ for no
repetition and the two cases of MRC from (23) and (25), respectively. The
utilized repetition profile parameters are $a=50$, $b=1050$.

####  V-C2 MRC with Rayleigh Fading

Here we assume, the serving and the interfering signals are impacted by both
Rayleigh fading and path-loss. Since the fading impacts all the transmissions
in $N$ repetitions, it is significant to capture its impact on the resulting
SINR due signal combining at the receiver. Utilizing (21) the success
probability for $N$ repetitions, conditioned at $r_{\text{o}}$ and
$h_{\text{o}}$, is obtained as follows,

$\displaystyle p_{|r_{\mathrm{o}},h}(N)$
$\displaystyle=\mathbb{P}({\gamma(N)>\theta}|r_{\mathrm{o}},h)$
$\displaystyle=\mathbb{P}\left(\frac{\sum_{n=1}^{N}h_{\mathrm{o}_{n}}r_{\mathrm{o}}^{-\alpha}}{I+\sigma^{2}}>\theta\right)$
$\displaystyle=F_{I}\left(\frac{G_{\mathrm{o}}r_{\mathrm{o}}^{-\alpha}}{\theta}-\sigma^{2}\right)~{},$
(24)

where $G_{\text{o}}$ is the gamma random variable, and is equal to sum of the
exponential random variables
$h_{\mathrm{o}_{1}}+h_{\mathrm{o}_{2}}+..h_{\mathrm{o}_{3}}$ [26].
Deconditioning on $G_{\text{o}}$, the success probability for $N$ repetitions
is given as,

$\displaystyle
p_{|r_{\mathrm{o}}}(N)=\int_{o}^{\infty}F_{I}\left(\frac{ur_{\mathrm{o}}^{-\alpha}}{\theta}-\sigma^{2}\right)f_{G_{o}}(u)~{}\mathrm{d}u~{},$
(25)

where $f_{G_{\text{o}}}(u)$ is the PDF of the gamma distribution with number
of repetitions $N(r_{\mathrm{o}})$ and scale 1 and is given as follows,

$\displaystyle
f_{G_{o}}(u)=\frac{u^{N(r_{o})-1}\exp({-u})}{\Gamma\left(N\left(r_{o}\right)\right)}.$
(26)

Figure 5: Conditional success probability vs $r_{\mathrm{o}}$ for no
repetition, SC, and MRC, with path-loss and fading, at repetition profile
parameters $a=100$, $b=500$. Figure 6: Conditional success probability vs
$r_{\mathrm{o}}$ for no repetition, SC , and MRC, with path-loss and fading,
at repetition profile parameters $a=100$, $b=950$.

The performance of MRC technique compared to a no repetition scenario is
presented in Fig. 4. The output is illustrated for different repetition
profile function parameters $a$ and $b$ along with Monte Carlo simulation
showing a close match with the theoretical formula. The path-loss only
scenario aids in providing the better performance of success probability, and
the channel condition with path-loss and Rayleigh fading gives a lower success
probability. Furthermore, comparison of both combining techniques with the no
repetition scenario is illustrated in Fig. 5 and Fig. 6. Fig. 5 demonstrates
the enhancement provided by MRC over the SC technique at small value of $b$
indicting higher repetition values in a cell. Contrary, at $b=950$, in Fig. 6,
the overall repetitions used in a cell reduce and only the devices near the
cell edge employ extensive repetition. This reduces the impact of overall
interference and hence an adequate performance is obtained. In this case, both
techniques show a better performance than a no repetition scenario. As such
from a coverage perspective, MRC is beneficial in a cell with using high or
low repetitions while SC performs better only at low repetition profile.

On one hand, the execution of the repetition scheme yields an enhanced
coverage, however, on the other hand the device is consuming more battery.
Since most of the mMTC devices are battery operated and are expected to have a
battery life of about 10 years, they rely on efficient energy consumption
[39]. Thus, for the combining schemes showing acceptable coverage performance
under various repetition profiles, it is equivalently vital to examine the
energy expenditure of an IoT device for both combining schemes at given
repetition profiles.

##  VI Cell energy wastage due to repetitions

Among various operational modes of an IoT device, the ideal and Power Saving
Mode (PSM) modes aim at increasing the battery lifetime. During the connected
mode, a piece of information is conventionally transmitted once, however, if
the device is configured to repeat, the same message is sent over multiple
times. The prolonged transmission due to the repetitions significantly impacts
the energy consumption of a device. As such, this section focuses on the
energy consumption of the device with repeated uplink transmissions.

Figure 7: Illustration of the trade-off between probability of success and
energy consumption at repetition profile $a=100$ and $b=1050$.

We mathematically model the energy consumption of an IoT device during its
transmission mode. Assuming an IoT device transmitting a message with time-on-
air $T_{m}$ while utilizing the RF transmit power. The energy consumed by the
device during this single transmission is given as follows [40] ,

$\displaystyle\epsilon(1)$
$\displaystyle=(\eta_{\epsilon}P_{\mathrm{t}}+P_{\mathrm{O}})T_{m},$ (27)

where $\eta_{\epsilon}$ is the conversion factor of the power amplifier[40]
and values at least 1, $P_{\text{O}}$ is the overhead power consumed by the
communication module of the device while encoding [41]. Further, if the device
repeats the transmission $N$ times, it stays active for an extended duration.
Thus, the total energy consumed in case of $N$ transmissions is given as
follows,

$\epsilon(N)=\epsilon(1)N(r)=(\eta_{\epsilon}P_{\mathrm{t}}+P_{\mathrm{O}})T_{m}N(r),$
(28)

where $N(r)$ is the repetition profile evaluated using (2). As seen in the
previous sections, increase in number of transmissions enhance the probability
of success, however throughout this process the device consumes more energy.
An illustration of this trade-off is shown in Fig. 7. On one hand, an initial
increase in the distance between the device and the gNB reduces the
probability of success, however due to the increasing repetitions $N$, an
enhancement in probability of success is achieved. On the other hand, using
the same repetition profile, more energy is consumed by an IoT device with the
increasing values of $N$. Furthermore, a transmission is deemed successful
when at least a single packet out of the $N$ repetitions is delivered
successfully. Failure in delivery of repetitions leads to error and the energy
expended by the device is wasted. Thus, the probability of error and the
energy consumed aids in evaluating the energy wasted while using the
repetition scheme. Therefore, the wasted energy, for an IoT device at distance
on $r_{\text{o}}$ when no repetition is successfully delivered is put forth
as,

$\displaystyle\epsilon_{\mathrm{w}}(N)=\epsilon(N)p_{e}{{}_{|r_{\mathrm{o}}}}(N),$
(29)

Figure 8: Energy wastage for SC using (29) vs distance from the gNB, at three
different values of repetition profile parameter $b$ and a constant $a$ = 25.

where $p_{e}{{}_{|r_{\mathrm{o}}}(N)}$ is given from (13). The wasted energy
is a function of the i) number of repetitions $N$, and ii) the probability of
error, which depends on the distance of the device from gNB and as well as on
the repetition profile parameters $a$ and $b$. Thus, to examine the impact of
various repetition profile parameters on energy dissipation we assess the
average wasted energy across the cell over the distribution of the distances
of devices to the gNB. The expected wasted energy in a cell at various
repetition profile parameters is obtained by deconditioned (29) on
$r_{\text{o}}$ given as,

$\displaystyle\overline{\epsilon}_{\mathrm{c}}=\mathbb{E}\left[\epsilon_{{\mathrm{waste}}(N)}\right]=\int_{0}^{R}\epsilon_{{\mathrm{w}}(N)}f_{r}(r_{\mathrm{o}})~{}\mathrm{d}r~{},$
(30)

where $f_{r}(r_{\mathrm{o}})$ is the distance probability density function
(pdf) and is given as,

$\displaystyle
f_{r}(r_{\mathrm{o}})=\frac{\mathrm{d}}{\mathrm{d}r}\left[\frac{\text{avg. \\#
of points in $r_{\mathrm{o}}$}}{\text{avg. \\# of points in
$R_{\mathrm{c}}$}}\right]=\frac{2\pi
r_{\mathrm{o}}D(r)}{{\eta(R_{\mathrm{c}})}}~{},$ (31)

and $\eta$ is the cell constant given by eq (6) in [32]. The wasted energy for
SC as a function of distance at three repetition profiles is shown in Fig. 8.
For all repetition profiles including the no repetition scenario, the gradual
increase in the distance leads to an increase in the probability of error and
wasted energy. Nevertheless, the distinction between the repetition profiles
is due to the parameter $b$ which impacts the overall values of $N$ in a cell.
For the first profile at $b=1000$, the values of $N$ enhance near the cell
edge, as such the likelihood of error reduces, leading to drop in energy
wasted. The infliction point in the figure depicts a point where the decline
in probability of error due to enhancing $N$ overshadows the energy
consumption. For the second profile at $b=1300$ the overall values of $N$ in a
cell are less compared to the first profile. At these values a high success
probability is achieved, and hence minimum energy is wasted. It is interesting
to note that in this case the wasted energy is less than that of the no
repetition scenario. For the third repetition profile at $b=2000$ the value of
$N$ reduces to 1, thereby no enhancement in probability of success is
achieved. This phenomenon occurs due to the reason that on one hand, the large
values of $N$ lead to network collisions that results in less success
probability in SC, and on the other hand, the minimum value of $N$ entails no
repetition. Thus, there exists an optimal repetition profile in between, with
values of $N$ that lead to a high success probability and minimum wasted
energy. The wasted energy for MRC at similar repetition profiles is shown in
Fig. 9. However, in MRC, the wasted energy does not reduce below that of no
repetition scenario for any repetition profiles. As such, the cell average
wasted energy is evaluated only for SC at various $b$ as shown in Fig. 10. The
influence of the different values of the repetition profile parameter $b$ is
clearly visible on the average cell wasted energy. It is seen that, the
average wasted energy across the cell is less than that of no repetition
scenario for various values of parameter $b$.

Figure 9: Energy wastage obtained for MRC using (29) versus distance from the
gNB at three different values of repetition profile parameter $b$ and a
constant $a$ = 25. Figure 10: Average energy wasted across a cell using (30)
for SC vs repetition profile parameter $b$ in comparison to no repetition
scenario.

Furthermore, a comparison of the wasted energy at a high coverage probability
(97%) for both combining schemes is illustrated in Fig. 11. On one hand, this
coverage probability for MRC is achieved at large values of $N$ at a
repetition profile with parameters $b=500$ and $a=25$. However, the energy
wastage is higher compared to that of SC and no repetition, as demonstrated.
On the other hand, less energy is wasted while providing the same service by
SC at lower values $N$ for repetition profile parameters $b=960$ and $a=25$.
Hence, for a given coverage, SC is more energy-efficient than that of MRC. As
a result, application of combining techniques is subject to requirements of
coverage and energy efficiency.

Figure 11: Comparison of energy wastage between MRC and SC for high success
rate (97%) obtained at repetition profile parameters $a=25$, $b=500$ for MRC,
and $a=25$, $b=960$ for SC.

##  VII Conclusion

In this paper an analysis of two signal combining techniques, SC and MRC is
performed for IoT repetition scheme in terms of probability of success and
energy efficiency. The employed repetition mechanism is based on the distance
of the device from the gNB and the duty cycle. Both techniques are analyzed
for (i) path-loss only scenario and (ii) path-loss with fading scenario. All
cases are compared to that of the no repetition scenario. The evaluation
reveals that both techniques respond differently under various optimal
repetition profiles. In terms of coverage, MRC outperforms SC at high and low
repetition rates in a cell. Comparatively, SC shows an improved coverage only
for a low repetition profile. Furthermore, a mathematical model to capture the
prevalent energy wastage due to repetition failure was also presented. The
model demonstrates that SC is more energy-efficient as compared to MRC and
reveals the point of lowest energy wastage when a given repetition profile is
realized in a cell.

Therefore, in an IoT cell utilizing extensive repetition values, an enhanced
coverage can be achieved by using the MRC technique at the receiver. Moreover,
if low repetition values are implemented, both combining techniques are
suitable. Nevertheless, from an energy outlook, SC is a more appropriate
combining technique for energy-constrained environment.

## References

  * [1] Ericsson. Ericsson Mobility Report. [Online]. Available: {https://www.ericsson.com/4adc87/assets/local/mobility-report/documents/2020/november-2020-ericsson-mobility-report.pdf}
  * [2] ITU-R, “IMT Vision – Framework and overall objectives of the future development of IMT for 2020 and beyond,” Int. Telecommun. Union, Geneva, Switzerland, Tech. Rep. ITU-Recommendation M.2083-0, September 2015.
  * [3] S. Lien, S. Hung, D. Deng, and Y. J. Wang, “Efficient Ultra-Reliable and Low Latency Communications and Massive Machine-Type Communications in 5G New Radio,” in _GLOBECOM 2017 - 2017 IEEE Global Communications Conference_ , 2017, pp. 1–7.
  * [4] B. Al Homssi, A. Al-Hourani, K. Magowe, J. Delaney, N. Tom, J. Ying, H. Wolf, S. Maselli, S. Kandeepan, K. Wang, and K. M. Gomez, “A framework for the design and deployment of large-scale lpwan networks for smart cities applications,” _IEEE Internet of Things Magazine_ , vol. 4, no. 1, pp. 53–59, 2021.
  * [5] O. Libergo, M. Sundberg, E. Wang, J. Bergman, and J. Sachs, _Cellular Internet of Things Technologies, Standards, and Performance_. Academic Press, 2018.
  * [6] 3GPP-Third Generation Partnership Project, “UL grant-free transmission for URLLC,” 3GPP, Technical Specification (TS) R1-1705246, April 2016.
  * [7] P. Isomäki and J. Isoaho, “On diversity combining,” Turku Centre of Computer Science, Technical Report (TS), 12 2020.
  * [8] S. Popli, R. K. Jha, and S. Jain, “A Survey on Energy Efficient Narrowband Internet of Things (NBIoT): Architecture, Application and Challenges,” _IEEE Access_ , vol. 7, pp. 16 739–16 776, 2019.
  * [9] J. Lee and J. Lee, “Prediction-Based Energy Saving Mechanism in 3GPP NB-IoT Networks,” _Sensors_ , vol. 17, no. 9, p. 2008, Sep 2017.
  * [10] R. K. Singh, P. P. Puluckul, R. Berkvens, and M. Weyn, “Energy Consumption Analysis of LPWAN Technologies and Lifetime Estimation for IoT Application,” _Sensors_ , vol. 20, p. 4794, Aug 2020.
  * [11] F. Michelinakis, A. S. Al-Selwi, M. Capuzzo, A. Zanella, K. Mahmood, and A. Elmokashfi, “Dissecting energy consumption of nb-iot devices empirically,” _IEEE Internet of Things Journal_ , vol. 8, no. 2, pp. 1224–1242, 2021.
  * [12] A. Crismani, S. Toumpis, U. Schilcher, G. Brandner, and C. Bettstetter, “Cooperative Relaying Under Spatially and Temporally Correlated Interference,” _IEEE Transactions on Vehicular Technology_ , vol. 64, no. 10, pp. 4655–4669, 2015.
  * [13] A. M. Hunter, J. G. Andrews, and S. Weber, “Transmission capacity of ad hoc networks with spatial diversity,” _IEEE Transactions on Wireless Communications_ , vol. 7, no. 12, pp. 5058–5071, 2008.
  * [14] L. Dufrene, M. Crussiere, J. Helard, and J. Schwoerer, “Performance analysis of time diversity combining with frequency offset for cellular IoT,” in _2017 International Symposium on Wireless Communication Systems (ISWCS)_ , 2017, pp. 60–65.
  * [15] 3GPP-Third Generation Partnership Project, “Technical Report 45.820 v13.0.0, Cellular System Support for Ultra-Low Complexity and Low Throughput Internet of Things,” 3GPP, Report, 2016.
  * [16] B. Manzoor, A. Al-Hourani, K. G. Chavez, S. Kandeepan, and M. Ding, “2D versus 3D Geometric Modelling for Massive Access Networks in 5G-IoT applications,” in _2018 12th International Conference on Signal Processing and Communication Systems (ICSPCS)_ , 2018, pp. 1–6.
  * [17] International Telecommunication Union (ITU), “Guidelines for Evaluation of Radio Interface Technologies for IMT-Advanced,” ITU-R, Technical Report M.2135-1, 2009.
  * [18] B. Manzoor, A. Al-Hourani, B. Al Homssi, K. Magowe, S. Kandeepan, and K. G. Chavez, “Evaluating Coverage Performance of NB-IoT in the ISM-band,” in _2020 27th International Conference on Telecommunications (ICT)_ , 2020, pp. 1–5.
  * [19] T. Eng, Ning Kong, and L. B. Milstein, “Comparison of diversity combining techniques for rayleigh-fading channels,” _IEEE Transactions on Communications_ , vol. 44, no. 9, pp. 1117–1129, 1996.
  * [20] Y. Mo, C. GOURSAUD, and J.-M. Gorce, “Uplink Multiple Base Stations Diversity for UNB based IoT networks,” in _2018 IEEE Conference on Antenna Measurements and Applications (CAMA)_ , 09 2018, pp. 1–4.
  * [21] A. Al-Hourani and K. Sithamparanathan, “On modeling coverage and rate of random cellular networks under generic channel fading,” _Wireless Networks_ , 11 2016.
  * [22] R. K. Ganti and M. Haenggi, “Spatial and temporal correlation of the interference in ALOHA ad hoc networks,” _IEEE Communications Letters_ , vol. 13, no. 9, pp. 631–633, 2009.
  * [23] M. D. Yacoub, C. R. C. M. da Silva, and J. E. Vargas Bautista, “Second-order statistics for diversity-combining techniques in Nakagami-fading channels,” _IEEE Transactions on Vehicular Technology_ , vol. 50, no. 6, pp. 1464–1470, 2001.
  * [24] D. G. Brennan, “Linear diversity combining techniques,” _Proceedings of the IRE_ , vol. 47, no. 6, pp. 1075–1102, 1959.
  * [25] M. Haenggi, “Diversity Loss Due to Interference Correlation,” _IEEE Communications Letters_ , vol. 16, no. 10, pp. 1600–1603, 2012.
  * [26] R. Tanbourgi, H. S. Dhillon, J. G. Andrews, and F. K. Jondral, “Effect of Spatial Interference Correlation on the Performance of Maximum Ratio Combining,” _IEEE Transactions on Wireless Communications_ , vol. 13, no. 6, pp. 3307–3316, 2014.
  * [27] Z. Zhou, C. Zhang, J. Wang, B. Gu, S. Mumtaz, J. Rodriguez, and X. Zhao, “Energy-efficient resource allocation for energy harvesting-based cognitive machine-to-machine communications,” _IEEE Transactions on Cognitive Communications and Networking_ , vol. 5, no. 3, pp. 595–607, 2019.
  * [28] T. Akyıldız, U. Demirhan, and T. M. Duman, “Energy harvesting irregular repetition aloha with replica concatenation,” _IEEE Transactions on Wireless Communications_ , vol. 20, no. 2, pp. 955–968, 2021.
  * [29] U. Demirhan and T. M. Duman, “Irregular repetition slotted aloha with energy harvesting nodes,” _IEEE Transactions on Wireless Communications_ , vol. 18, no. 9, pp. 4505–4517, 2019.
  * [30] R. Harwahyu, R.-G. Cheng, W.-J. Tsai, J.-K. Hwang, and G. Bianchi, “Repetitions versus retransmissions: Tradeoff in configuring nb-iot random access channels,” _IEEE Internet of Things Journal_ , vol. 6, no. 2, pp. 3796–3805, 2019.
  * [31] P. Kyösti, J. Meinilä, L. Hentila, X. Zhao, T. Jämsä, C. Schneider, M. Narandzic, M. Milojević, A. Hong, J. Ylitalo, V.-M. Holappa, M. Alatossava, R. Bultitude, Y. Jong, and T. Rautiainen, “WINNER II channel models,” _IST-4-027756 WINNER II D1.1.2 V1.2_ , 02 2008.
  * [32] B. Manzoor, B. A. Homssi, A. Al-Hourani, and S. Kandeepan, “Optimal repetition rate for maximal coverage,” _IEEE Wireless Communications Letters_ , vol. 10, no. 4, pp. 800–804, 2021.
  * [33] A. Al-Hourani, S. Chandrasekharan, G. Baldini, and S. Kandeepan, “Propagation measurements in 5.8GHz and pathloss study for CEN-DSRC,” in _2014 International Conference on Connected Vehicles and Expo (ICCVE)_ , 2014, pp. 1086–1091.
  * [34] A. Al-Hourani and K. Gomez, “Modeling Cellular-to-UAV Path-Loss for Suburban Environments,” _IEEE Wireless Communications Letters_ , vol. 7, no. 1, pp. 82–85, 2018.
  * [35] A. Al-Hourani and M. Haenggi, “Performance of Next-Generation Cellular Networks Guarded With Frequency Reuse Distance,” _IEEE Transactions on Communications_ , vol. 67, no. 10, pp. 7277–7287, 2019.
  * [36] J. Abate and W. Whitt, “A Unified Framework for Numerically Inverting Laplace Transforms,” _INFORMS Journal on Computing_ , vol. 18, no. 4, pp. 408–421, 2006.
  * [37] A. Goldsmith, _Wireless Communications_. Cambridge University Press, 2005. [Online]. Available: https://app.knovel.com/hotlink/toc/id:kpWC000002/wireless-communications/wireless-communications
  * [38] A. F. Molisch, _Wireless Communications_. John Wiley and Sons Ltd., 2011.
  * [39] A. Elhaddad, H. Bruckmeyer, M. Hertlein, and G. Fischer, “Energy consumption evaluation of cellular narrowband internet of things (nb-iot) modules,” in _2020 IEEE 6th World Forum on Internet of Things (WF-IoT)_ , 2020, pp. 1–5.
  * [40] B. Al Homssi, A. Al-Hourani, S. Chandrasekharan, K. M. Gomez, and S. Kandeepan, “On the Bound of Energy Consumption in Cellular IoT Networks,” _IEEE Transactions on Green Communications and Networking_ , vol. 4, no. 2, pp. 355–364, 2020.
  * [41] A. Al-Hourani, S. Kandeepan, and E. Hossain, “Relay-Assisted Device-to-Device Communication: A Stochastic Analysis of Energy Saving,” _IEEE Transactions on Mobile Computing_ , vol. 15, no. 12, pp. 3129–3141, 2016.

|  Dr Bisma Manzoor (PhD, MEng, BEng) received the PhD degree in Electronic
and Electrical Engineering, with specialization in wirelesses communication
systems and technologies, from the School of Engineering, RMIT University,
Melbourne, Australia, in 2021. She received her Master’s degree in
Telecommunication and Network Engineering from RMIT University, Melbourne
Australia, in 2017. Dr Manzoor currently works as signal processing engineer
at Fleet Space Technologies, Australia. Her research interests include
modeling and optimization for the cellular Internet of Things, energy
efficiency in wireless networks, Internet of Things over satellite, and
Stochastic Geometry.
---|---
|  Dr Bassel Al Homssi (PhD, EIT, MSEng, BSEng) is a Research Fellow at both
RMIT University and Deakin University, Melbourne, Australia. Dr Al Homssi
received his Bachelor of Science and Master of Science in Electrical and
Electronic Engineering from the American University of Sharjah, Sharjah,
United Arab Emirates in 2014 and 2016 respectively. He also received his
Engineer in Training (EIT) certification from NCEES in 2014 and also received
his Master in Telecommunication from the University of Melbourne, Melbourne,
Australia in 2017. Dr Al Homssi received his PhD degree in 2020 from RMIT
University and was the lead network designer for North Melbourne Smart Cities
Network. He is currently working with the local Australian government on Smart
Cities in Melbourne, Australia. Dr Al Homssi is currently working on multiple
research projects related to satellite networks and MIMO communications for
next generation wireless systems.
---|---
|  Dr Akram Al-Hourani (PhD, BEng, MBA, SMIEE), is a Senior lecturer and
Telecommunication Program Manager at the School of Engineering, RMIT
University. Before joining RMIT in 2016, he has been with The University of
Melbourne as a Research Fellow. Dr Al-Hourani obtained his PhD in 2016 from
RMIT University. Prior joining academia, Dr Al-Hourani had extensively worked
in the ICT industry as an R&D engineer, radio network planning engineer and
then as an ICT program manager for several projects spanning over different
technologies; including mobile networks deployment, satellite networks, and
railway ICT systems. The total value of industry projects directed by Dr Al-
Hourani exceeded $190m. He has extensive industry / government engagement as a
chief investigator in multiple research projects related to The Internet-of-
Things (IoT), Smart Cities, Satellite / Wireless Communications. As a Lead
Chief Investigator, he provided key expertise in delivering the largest open
(Internet-of-Things) network in Victoria in collaboration with 5 local
governments “Northern Melbourne Smart Cities Network”, this project attracted
two prestigious industry awards from the Municipal Association of Victoria
(MAV) ”Technology Awards for Excellence 2020”, and the IoT Alliance Australia
(IoTAA) ”Smart Cities Award for 2020”. He published more than 75 journal
articles and conference proceedings, including 3 book chapters, Dr Al-Hourani
has won the IEEE Sensors Council Paper Award for his contribution in hand-
gesture recognition using neural networks. He is currently listed in Stanford
University’s list of top 2% scientists in both Career-Long list and Single
Year Impact list for 2020. Dr Al-Hourani is currently an Associate Editor in
IEEE Transactions of Aerospace and Electronics, and in Frontiers in Space
Technologies and an Editor in MDPI Remote Sensing Journal. His current
research interests include the Satellite Communications, Radars Systems, UAV
Communication Systems, Signal Processing / Machine Learning, and Stochastic
Geometry.
---|---