# On Neural Architectures for Deep Learning-based Source Separation of Co-
Channel OFDM Signals

###### Abstract

We study the single-channel source separation problem involving orthogonal
frequency-division multiplexing (OFDM) signals, which are ubiquitous in many
modern-day digital communication systems. Related efforts have been pursued in
monaural source separation, where state-of-the-art neural architectures have
been adopted to train an end-to-end separator for audio signals (as
1-dimensional time series). In this work, through a prototype problem based on
the OFDM source model, we assess—and question—the efficacy of using audio-
oriented neural architectures in separating signals based on features
pertinent to communication waveforms. Perhaps surprisingly, we demonstrate
that in some configurations, where _perfect_ separation is theoretically
attainable, these audio-oriented neural architectures perform poorly in
separating co-channel OFDM waveforms. Yet, we propose critical domain-informed
modifications to the network parameterization, based on insights from OFDM
structures, that can confer about $30$ dB improvement in performance.

Index Terms—  Single-channel source separation, deep learning, orthogonal
frequency-division multiplexing, Fourier features, neural architectures.

††footnotetext: Research was sponsored by the United States Air Force Research
Laboratory and the United States Air Force Artificial Intelligence Accelerator
and was accomplished under Cooperative Agreement Number FA8750-19-2-1000. The
views and conclusions contained in this document are those of the authors and
should not be interpreted as representing the official policies, either
expressed or implied, of the United States Air Force or the U.S. Government.
The U.S. Government is authorized to reproduce and distribute reprints for
Government purposes notwithstanding any copyright notation herein.
This work is also supported by the NSF under Grant No. CCF-2131115.
Alejandro Lancho has received funding from the EU’s Horizon 2020 research and
innovation programme under the Marie Sklodowska-Curie grant agreement No.
101024432.
The authors acknowledge the MIT SuperCloud and Lincoln Laboratory
Supercomputing Center for providing HPC resources that have contributed to the
research results reported within this paper.

##  1 Introduction

Source separation is a long-standing problem of interest in many engineering
applications. Particularly challenging is the _single-channel_ source
separation (SCSS) setting that has recently gained more interest [1, 2, 3, 4],
corresponding to an underdetermined scenario, in the absence of spatial
diversity. Most prominently, SCSS with audio signals has received much
attention [5, 6, 7]. State-of-the-art solutions benefit from deep learning
approaches, many of which propose novel neural architectures to achieve
improved separation ability. For example, the use of convolutional [8, 9, 10],
recurrent [11] and attention-based [12] layer structures in the separation
neural network (NN) have been proposed, with varying degree of success.
Implicit to these methods are strategies to exploit properties of typical
audio signals. In fact, it is believed that the features exploited by state-
of-the-art neural architectures are related to separability in the time-
frequency space [13].

Beyond audio applications, source separation finds important relevance to
other domains, such as in radio frequency (RF) and optical systems, for
communication and sensing applications [14, 15, 16, 17, 18, 19, 20]. Across
these domains, the raw data are 1-dimensional time series. Yet, the properties
of RF/optical waveforms differ from audio signals—e.g., these signals tend to
pack a large amount of information into a finite frequency band (or channel),
rendering them no longer sparse in the time-frequency space. In fact, they may
be overlapping in this space—described as “co-channel”. While audio-oriented
NNs can work on time series inputs, and could also be just as successful with
other modalities, e.g., demonstrated with seismic signals in [21]), it is
uncertain if the same neural architectures are also effective at separating
communication signals.

Of particular interest to our work is one such type of RF/optical signals,
which modulates information through orthogonal frequency-division multiplexing
(OFDM). While OFDM waveforms form a subclass of digitally modulated signals,
it is one of the most ubiquitous modulation found in modern data
communications, for both wireless systems (WiFi and 4G/LTE/5G) and optical
systems. We also empirically observe these waveforms to be a challenging class
of signals to tackle in data-driven SCSS [15, 22].

This work focuses on the SCSS of OFDM signals, and the relevant neural
architectural choices to capture informative features for signal separation.
In particular, we consider a prototype problem based on the OFDM model, posed
such that perfect separation of the signals is technically attainable through
the fast Fourier transform (FFT) algorithm with appropriately chosen
parameters. Yet, _without prior knowledge_ about the signal model, achieving
this is not a simple task. Under this setup, we study whether NN-based
approaches can learn to exploit the underlying OFDM structures for SCSS.

To the best of our knowledge, this work is the first to assess the performance
(and therefore, the ineffectiveness in some regimes) of neural architectures
from audio separation when applied to OFDM waveforms, serving as an important
benchmark. We also propose modifications, inspired by OFDM structures, that
confer orders of magnitude improvement to the separation performance. The key
takeaways are to demonstrate how distinct and challenging digital
communication signals can be for existing neural methods in SCSS, and to
propose judicious adaptations in advancing neural methods for time-domain
signals beyond the efforts in audio domain.

##  2 Problem Formulation

Consider an observed 1-dimensional mixture of signals

${\mbox{\boldmath$y$}}={\mbox{\boldmath$s$}}+{\mbox{\boldmath$b$}},$ (1)

where ${\mbox{\boldmath$s$}}\triangleq[s[0]\,\ldots\,s[N-1]]^{{\rm
T}}\in\mathbb{C}^{N}$ is our signal-of-interest (SOI) to be extracted, and
${\mbox{\boldmath$b$}}\triangleq[b[0]\,\ldots\,b[N-1]]^{{\rm
T}}\in\mathbb{C}^{N}$ is the interference (signal-not-of-interest). The goal
is to separate $s$ from $b$, namely, estimate $s$ from $y$ with minimum mean
squared error (MSE) as the criterion. We assume that the models for $s$ and
$b$ are not known; however, we have a dataset of $M$ independent, identically
distributed examples,
$\\{({\mbox{\boldmath$y$}}^{(i)},{\mbox{\boldmath$s$}}^{(i)})\\}_{i=1}^{M}$.
This setup naturally lends itself to a data-driven approach for the SCSS
problem.

In this work, we consider an SOI and an interference that are discrete-time
OFDM waveforms, formally expressed as

$\displaystyle
s[n]=\sum_{p=0}^{P-1}\sum_{k=0}^{K-1}g_{k,p}\,r[n-p\cdot(K+T_{\rm{cp}})-T_{\rm{cp}},\,k],$
(2) $\displaystyle
b[n]=\sum_{p=0}^{P-1}\sum_{k=0}^{K-1}h_{k,p}\,r[n-p\cdot(K+T_{\rm{cp}})-T_{\rm{cp}},\,k],$
$\displaystyle r[n,\,k]\triangleq\exp({\nicefrac{{j2\pi
kn}}{{K}}})\,\mathbbm{1}_{\left\\{-T_{\rm{cp}}\leq n<K\right\\}},$

for $n\in\\{0,\ldots,N-1\\}$, where $K\in 2\cdot\mathbb{N}$ is the total
number of orthogonal complex sinusoid terms (also termed as subcarriers; this
also corresponds to the FFT size). The coefficients $g_{k,p}\in\mathcal{G}$,
$h_{k,p}\in\mathcal{H}$ are the modulated symbols, and
$\mathcal{G},\mathcal{H}$ are their alphabets (constellations), respectively.
A cyclic prefix (CP) is typically added before an OFDM symbol. Hence, each
OFDM symbol is described for the interval $[-T_{\rm{cp}},K]$, where
$T_{\rm{cp}}\in\mathbb{N}$ is the CP length, and $K$ is the OFDM symbol length
(without CP). The signals span $P$ OFDM symbols, and their individual finite
support is reflected by the finitely supported function $r[n,\,k]$.

In this setting, the observed mixture can also be viewed as an OFDM waveform,
with the coefficients being elements from the superconstellation of the SOI’s
and interference’s symbols, i.e., the Minkowski sum
$\mathcal{A}\triangleq\mathcal{G}\oplus\mathcal{H}$, such that

$\displaystyle y[n]$
$\displaystyle=\sum_{p=0}^{P-1}\sum_{k=0}^{K-1}a_{k,p}\,r[n-p\cdot(K+T_{\rm{cp}})-T_{\rm{cp}},\,k],$
(3) $\displaystyle a_{k,p}$ $\displaystyle=g_{k,p}+h_{k,p},\quad
a_{k,p}\in\mathcal{A}.$

The existence of a surjective function $f:\mathcal{A}\rightarrow\mathcal{G}$,
i.e., every element in $\mathcal{A}$ can be uniquely associated with an
element in the SOI’s constellation $\mathcal{G}$, suffices for perfect
separability.111Alternatively, a surjective function
$f:\mathcal{A}\rightarrow\mathcal{H}$.

###  2.1 Special Case: Real-valued OFDM Signals

In order to focus on the core aspects of this problem, namely the underlying
Fourier structures and finite coefficient sets present in OFDM, we propose to
examine the following simplified (yet not simplistic) special case. Consider
(LABEL:eq:ofdmsources) with $P=1$, $T_{\rm{cp}}=N-K$ and $N\in
K\cdot\mathbb{N}$, namely,

$s[n]=\sum_{k=0}^{K-1}\underbrace{g_{k,0}}_{\triangleq
g_{k}}r[n-T_{\rm{cp}},k]=\sum_{k=0}^{K-1}g_{k}\exp({\nicefrac{{j2\pi
kn}}{{K}}}),$ (4)

and similarly for $b[n]$, $y[n]$ with coefficients $h_{k}$, $a_{k}$,
respectively, such that the (periodic extensions of the) SOI and interference
are discrete Fourier series. Further, we impose the conjugate symmetry
constraint on the coefficients $g_{k}$, $h_{k}$,

$\displaystyle\leavevmode\hbox to9.75pt{\vbox
to9.75pt{\pgfpicture\makeatletter\hbox{\hskip 4.87285pt\lower-4.87285pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{
{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor[named]{tikz@color}{rgb}{0,0,1}\definecolor[named]{.}{rgb}{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\pgfsys@color@rgb@stroke{0}{0}{1}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{1}\pgfsys@invoke{
}\pgfsys@stroke@opacity{0.9}\pgfsys@invoke{
}\pgfsys@fill@opacity{0.9}\pgfsys@invoke{
}{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor[named]{tikz@color}{rgb}{0,0,1}\definecolor[named]{.}{rgb}{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\pgfsys@color@rgb@stroke{0}{0}{1}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{1}\pgfsys@invoke{
}\pgfsys@stroke@opacity{0.9}\pgfsys@invoke{
}\pgfsys@fill@opacity{0.9}\pgfsys@invoke{
}{}\pgfsys@moveto{4.67285pt}{0.0pt}\pgfsys@curveto{4.67285pt}{2.58076pt}{2.58076pt}{4.67285pt}{0.0pt}{4.67285pt}\pgfsys@curveto{-2.58076pt}{4.67285pt}{-4.67285pt}{2.58076pt}{-4.67285pt}{0.0pt}\pgfsys@curveto{-4.67285pt}{-2.58076pt}{-2.58076pt}{-4.67285pt}{0.0pt}{-4.67285pt}\pgfsys@curveto{2.58076pt}{-4.67285pt}{4.67285pt}{-2.58076pt}{4.67285pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@stroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope}{{{{}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-2.0pt}{-2.57777pt}\pgfsys@invoke{
}\hbox{{\definecolor[named]{.}{rgb}{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\pgfsys@color@rgb@stroke{0}{0}{1}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{1}\pgfsys@invoke{
}\hbox{{\definecolor[named]{.}{rgb}{0,0,1}\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\footnotesize
1}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} }
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{{{}}}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}\;g_{0}=g_{K/2}=0\quad\leavevmode\hbox
to9.75pt{\vbox to9.75pt{\pgfpicture\makeatletter\hbox{\hskip
4.87285pt\lower-4.87285pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{
{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor[named]{tikz@color}{rgb}{0,0,1}\definecolor[named]{.}{rgb}{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\pgfsys@color@rgb@stroke{0}{0}{1}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{1}\pgfsys@invoke{
}\pgfsys@stroke@opacity{0.9}\pgfsys@invoke{
}\pgfsys@fill@opacity{0.9}\pgfsys@invoke{
}{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor[named]{tikz@color}{rgb}{0,0,1}\definecolor[named]{.}{rgb}{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\pgfsys@color@rgb@stroke{0}{0}{1}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{1}\pgfsys@invoke{
}\pgfsys@stroke@opacity{0.9}\pgfsys@invoke{
}\pgfsys@fill@opacity{0.9}\pgfsys@invoke{
}{}\pgfsys@moveto{4.67285pt}{0.0pt}\pgfsys@curveto{4.67285pt}{2.58076pt}{2.58076pt}{4.67285pt}{0.0pt}{4.67285pt}\pgfsys@curveto{-2.58076pt}{4.67285pt}{-4.67285pt}{2.58076pt}{-4.67285pt}{0.0pt}\pgfsys@curveto{-4.67285pt}{-2.58076pt}{-2.58076pt}{-4.67285pt}{0.0pt}{-4.67285pt}\pgfsys@curveto{2.58076pt}{-4.67285pt}{4.67285pt}{-2.58076pt}{4.67285pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@stroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope}{{{{}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-2.0pt}{-2.57777pt}\pgfsys@invoke{
}\hbox{{\definecolor[named]{.}{rgb}{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\pgfsys@color@rgb@stroke{0}{0}{1}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{1}\pgfsys@invoke{
}\hbox{{\definecolor[named]{.}{rgb}{0,0,1}\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\footnotesize
2}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} }
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{{{}}}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}\;g_{k}=g^{*}_{K-k},\,\forall
k\in\\{1,\ldots,\tfrac{K}{2}-1\\},$

and similarly for $h_{k}$, where $z^{*}$ denotes the complex conjugate of a
complex number $z$. Consequently, the waveforms generated by (4) are real-
valued:
${\mbox{\boldmath$s$}},{\mbox{\boldmath$b$}}\in\mathbb{R}^{N}\Rightarrow{\mbox{\boldmath$y$}}\in\mathbb{R}^{N}$.

The purpose of focusing on this special case is to test the ability of
candidate neural architectures to capture, or learn to exploit, the
subcarriers’ _orthogonality_ and the _discrete_ constellation set, with which
perfect separation is attainable.

Table 1: MSE (in decibels, dB) of the extracted SOI. Entries with MSE$<10^{-2}$ (i.e., $-20$ dB444An approximation of the best separation performance reported in [8, 10, 9, 11, 12].) are in red; improvement of our proposed architecture, compared to the best-performing benchmark method, is reported in parenthesis in the last row. |  Case 1 |  Case 2 |  Case 3 |  Case 4
---|---|---|---|---
|  Disjoint |  BPSK+BPSK |  BPSK+4-PAM (Mixed) |  4-PAM+4-PAM
Wave-U-Net [8] |  $-57.246$ dB |  $-46.827$ dB |  $-4.663$ dB |  $-4.665$ dB
Conv-TasNet [9] |  $-40.790$ dB |  $-12.179$ dB |  $-1.060$ dB |  $-1.009$ dB
Sudo-Rm-Rf [10] |  $-37.023$ dB |  $-26.493$ dB |  $-12.855$ dB |  $-11.495$ dB
Dual Path RNN [11] |  $-41.425$ dB |  $-27.302$ dB |  $-0.671$ dB |  $-0.542$ dB
DPTNet [12] |  $-36.825$ dB |  $-33.652$ dB |  $-3.548$ dB |  $-2.432$ dB
Modified Wave-U-Net |  $\mathbf{-65.526}$ dB |  $\mathbf{-47.558}$ dB |  $\mathbf{-47.377}$ dB |  $\mathbf{-41.156}$ dB
(Proposed) |  ($\downarrow 8.280$) |  ($\downarrow 0.731$) |  ($\downarrow 34.522$) |  ($\downarrow 29.661$)

##  3 Regimes of Methodology

Before we consider data-driven methods and neural architectures for separating
OFDM signals of the form (4), we briefly discuss conventional (informed)
model-based approaches for reference. Consider a frequency domain approach
(i.e., using FFT), where the SOI is reconstructed from the mixture’s spectral
coefficients, $\\{a_{k}\\}_{k=0}^{K-1}$. The first step requires an _exact_
FFT size, which can be unknown, that preserves the orthogonality of the
subcarriers. Otherwise, the spectral “leakage” from neighboring subcarriers
will typically result in a significantly larger superconstellation, due to the
loss of orthogonality (illustrated in Fig. 1(i)). Of course, for this, one has
to establish a realizable method that extracts the underlying spectral
coefficients—corresponding to the surjective mapping of the mixture’s symbol
superconstellation to the SOI’s constellation points (e.g. Fig. 1(ii)). The
existence of such a routine, albeit through explicit knowledge of source model
parameters, demonstrates a possible approach to achieve perfect signal
separation performance for (4). However, this does not correspond to a
practicable algorithm in a more general sense, e.g., with partial knowledge
about the source models.

|
---|---
(i) | (ii)
Fig. 1: Visualization of OFDM structure—(i) using the appropriate FFT size
leads to orthogonality between subcarriers; a mismatched FFT leads to a loss
of orthogonality at the subcarrier frequencies; (ii) for an appropriate choice
of discrete constellations, a surjective mapping of points from the
superconstellation to an SOI symbol can be obtained.

On the other hand, monaural separation architectures can be used for an end-
to-end signal separator _without_ explicitly requiring the source models.
Since the latent sources in the special case considered in Section 2.1 are
real-valued, we can adopt the neural architectures proposed in audio source
separation works, developed for real-valued time domain inputs.

For these audio-oriented methods, information pertaining to the sources being
a discrete Fourier series (or an OFDM waveform) is not explicitly utilized.
Clearly, an effective architecture for this SCSS problem ought to learn to
exploit properties relating to the OFDM’s subcarrier structure and discrete
symbol constellations from data. Intriguingly, we demonstrate that beyond a
limited regime of this problem, audio-based neural architectures fail to
separate OFDM waveforms that are in fact perfectly separable. We then propose
certain domain-informed modifications that revive these architectures,
consequently leading to successful separation—improving the figure of merit
(MSE) by orders of magnitudes.

For the rest of this work, we focus on computational experiments with
parameters $N=4096$, $K=64$, $K_{\rm{sc}}=28$, where $K_{\rm{sc}}$ corresponds
to the (maximum) number of unique nonzero coefficients (subcarriers) in this
model.222These parameters are based on 802.11n WiFi waveform properties [23].
We consider $4$ different cases of $g_{k}$, $h_{k}$, for
$k\in\\{1,\ldots,K_{\rm{sc}}\\}$:
Case 1: Disjoint frequency sets: $g_{k}=0$ when $h_{k}\neq 0$ and _vice versa_
, where nonzero indices are randomly chosen once, and stay fixed thereafter.
The nonzero coefficients are drawn from a random continuous uniform
distribution, $g_{k}\sim\mathcal{U}[-\sqrt{3},\sqrt{3}]$,
$h_{k}\sim\mathcal{U}[-4\sqrt{3},4\sqrt{3}]$.
Case 2: “BPSK333BPSK: Binary Phase Shift Keying; 4-PAM: 4 Pulse-Amplitude
Modulation; these are modulation schemes typical in digital communication
signals.-like” coefficients: $g_{k}\in\\{+1,-1\\}$ and $h_{k}\in\\{+4,-4\\}$.
Case 3: “Mixed” coefficients: $g_{k}\in\\{+1,-1\\}$ and
$h_{k}\in\\{\nicefrac{{+12}}{{\sqrt{5}}},$
$\nicefrac{{+4}}{{\sqrt{5}}},\nicefrac{{-4}}{{\sqrt{5}}},\nicefrac{{-12}}{{\sqrt{5}}}\\}$.
Case 4: “4-PAM33footnotemark: 3-like” coefficients:
$g_{k}\in\\{\nicefrac{{+3}}{{\sqrt{5}}},\nicefrac{{+1}}{{\sqrt{5}}},$
$\nicefrac{{-1}}{{\sqrt{5}}},\nicefrac{{-3}}{{\sqrt{5}}}\\}$ and
$h_{k}\in\\{\nicefrac{{+12}}{{\sqrt{5}}},\nicefrac{{+4}}{{\sqrt{5}}},\nicefrac{{-4}}{{\sqrt{5}}},\nicefrac{{-12}}{{\sqrt{5}}}\\}$.
We introduce the appropriate scaling factors on the source components such
that the SOI $s$ has unit average power. We set $g_{0}=h_{0}=0$, and
$g_{k}=h_{k}=0$ for $k\in[K_{\rm{sc}}+1,K/2]$, and recall that for $k>K/2$,
the coefficients are constrained to have conjugate symmetry. In all cases, the
average interference power is 16 times that of the average SOI power.

##  4 Neural Architectures for SCSS

We now describe the details of our experiments. We then present the separation
performance, and compare how they fare in the 4 different cases established
earlier.

###  4.1 Implementation Details

To the best of our knowledge, there are no established baseline NN methods for
this OFDM SCSS problem. Hence, part of our work is to train selected state-of-
the-art NNs from audio separation [8, 10, 9, 11, 12] for this problem, and
assess their performance to serve as our comparison benchmark. Asteroid, the
PyTorch-based audio source separation toolbox [24], is used for state-of-the-
art neural architectures, whereas Wave-U-Net and its modified version (our
proposed architecture, detailed later) are implemented in
PyTorch.555Repository containing code and implementation details:
https://github.com/RFChallenge/SCSS_OFDMArchitecture. The training and
validation sets comprise $90,000$ and $10,000$ independent realizations of
mixture-SOI pairs respectively. Adam optimizer with a learning rate $10^{-4}$
is used to train the respective NNs for $2\times 10^{3}$ epochs, with early
stopping after $100$ epochs of no improvement on validation. In Table 1 we
report the MSE performance of the selected neural architectures in the
reconstruction of the SOI, on a test set comprising $10^{3}$ examples.

###  4.2 Performance of Audio-Oriented NNs

As expected, the audio-oriented NN models are all effective in separating
signals with disjoint frequency sets (Case 1), i.e., masking/filtering the
frequencies that make up the SOI $s$. Yet, we see that many of these neural
architectures can still separate highly co-channel signals under some cases,
as in Case 2 with OFDM waveforms bearing BPSK symbols. The models’ success in
Case 2 indicates that the mechanisms of these audio-based neural methods have
the potential to separate signals that overlap in time and frequency, i.e.,
deviating from the proximity characteristics indicated in [13], thereby
reflecting the potential generalizability of some of these architectures.
Nonetheless, as we advance to consider more complex constellation, such as
4-PAM (Cases 3 and 4), these architectures are no longer as effective at
separating these signals when trained with the configuration considered here.

Table 2: MSE (in dB) of the extracted SOI using the modified Wave-U-Net with different first-layer kernel sizes. Kernel Size | MSE | Kernel Size | MSE
---|---|---|---
$W=15$ |  $-6.030$ dB | $W=65$ |  $-42.824$ dB
$W=21$ |  $-5.621$ dB | $W=71$ |  $-42.099$ dB
$W=31$ |  $-6.183$ dB | $W=81$ |  $-42.690$ dB
$W=51$ |  $-16.319$ dB | $W=101$ |  $-41.156$ dB
$W=63$ |  $-41.380$ dB | $W=201$ |  $-44.319$ dB

##  5 OFDM Domain-informed Architecture

We now propose modifications to one of the architectures based on insights
from OFDM signals. Thereafter, we review possible justifications for the
improvement attained by drawing connections to OFDM’s Fourier structures,
which in turn leads to guidelines for domain-informed parameterization.

###  5.1 Proposed Neural Architecture Modifications

Referring to the model-based approach, we seek an NN that is capable of
approximating an appropriately sized FFT operator (Fig. 1). Based on this
insight, a natural modification to the NN is to increase the number of filters
($20\times$ as many) and the receptive fields of these filters on the first
layer (kernel size $W=101$), which operates on the time-domain input. We
introduce these modifications to Wave-U-Net—the simplest among those
investigated. The last row of Table 1 reports the substantial improvement in
MSE due to these modifications.

To further lend credence to the role of first-layer kernel size, we show the
SCSS results on Case 4 using the modified Wave-U-Net with different sizes in
Table 2. Here, we see a significant improvement in separation performance when
kernel sizes $63$ and longer are used in the modified Wave-U-Net (recalling
that the true FFT size $K=64$).

###  5.2 Discussion on Fourier Structures

Referring to (4), each time-domain sample is a sum of statistically
independent random variables; by the central limit theorem (CLT), each of
these samples is marginally Gaussian distributed as $K\to\infty$. Note that
even when a (fixed) $W\ll K$ time samples are considered, they are
asymptotically jointly Gaussian, unless they are exactly a period away [25].
Yet, $K$ consecutive time-domain samples are not jointly Gaussian, as evident
from the discrete (non-Gaussian) coefficient set.666Indeed, the CLT cannot be
invoked in this case. A large receptive field in the NN, particularly on the
raw input, may be related to the window (FFT) size where one is able to
capture the underlying non-Gaussianity.

To further illustrate this, we consider the empirical marginal kurtosis of the
latent representation obtained by taking the $W$-order FFT of some length
$W$.777This is loosely based on kurtosis as a surrogate measure of non-
Gaussianity, as typical in algorithms, e.g., independent component analysis
(ICA) [26]. Fig. 2 shows the empirical kurtosis of the (real part of)
coefficients extracted across $2\times 10^{6}$ independent realizations of
$y$. We highlight two key observations: 1) spikes appear at multiples of
$K/2$; and 2) the empirical kurtosis stays close to $3$ when $W<K$ (resembling
that of Gaussian random variables), but increases for longer windows,
departing from Gaussianity. We note that the latter is true for (4), where a
growing window leads to more low-magnitude coefficients (with the same number
of non-zero subcarrier symbols), which in turn leads to a heavy-tailed
distribution of these latent coefficients.

These observations lead to two possible mechanisms through which a long first-
layer kernel is exploiting. First, a long kernel has the representation power
to approximate the FFT of the exact order or its harmonics (the red circle
data points in Fig. 2). Second, a significantly long kernel can process the
signal in a regime where it is non-Gaussian, but not necessarily exactly at
the FFT size (the two colored regions in Fig. 2). While the neural separation
mechanism and feature explainability are not the focus of this work, these are
potentially impactful aspects calling for further investigation.

We recognize that a deep NN ought to have a large effective receptive field
through its stacked layers, even if the kernel sizes of individual
convolutional layers are short [27]. Yet, we have observed that none of the
deep NN models considered are as effective in Case 4, in contrast to what is
achieved through significantly long kernel on the first layer, operating
_directly_ on the input itself. The disparity in performance remains a
question for further consideration.

Fig. 2: Empirical marginal kurtosis of the latent representation (real part of
FFT coefficients) versus window length $W$.

##  6 Concluding Remarks

Our work reveals key insights into strategies for neural architecture choices
for potential novel OFDM-based systems—specifically, parameterization (mildly)
informed by the FFT size, corresponding to sufficiently long kernel sizes on
the first convolutional layer. Next steps include revisiting the general model
(LABEL:eq:ofdmsources), which has more latent parameters that an appropriate
NN has to capture for SCSS. Of further interest is studying the mechanism
behind the modified Wave-U-Net architecture proposed, and how that translates
broadly to a more effective NN architecture for OFDM waveforms.

## References

  * [1] L. Benaroya, F. Bimbot, and R. Gribonval,  “Audio source separation with a single sensor,”  IEEE Trans. on Audio, Speech, and Language Process., vol. 14, no. 1, pp. 191–199, 2005.
  * [2] E. M. Grais, M. U. Sen, and H. Erdogan,  “Deep neural networks for single channel source separation,”  in Proc. IEEE Int. Conf. Acoust. Speech Signal Process. IEEE, 2014, pp. 3734–3738.
  * [3] Y.-m. Qian, C. Weng, X.-k. Chang, S. Wang, and D. Yu,  “Past review, current progress, and challenges ahead on the cocktail party problem,”  Front. Inf. Technol. Electron., vol. 19, no. 1, pp. 40–63, 2018\.
  * [4] R. Blouet, G. Rapaport, I. Cohen, and C. Fevotte,  “Evaluation of several strategies for single sensor speech/music separation,”  in Proc. IEEE Int. Conf. Acoust. Speech Signal Process. IEEE, 2008, pp. 37–40.
  * [5] D. Wang and J. Chen,  “Supervised speech separation based on deep learning: An overview,”  IEEE/ACM Trans. on Audio, Speech, and Language Process., vol. 26, no. 10, pp. 1702–1726, 2018.
  * [6] H. Purwins, B. Li, T. Virtanen, J. Schlüter, S.-Y. Chang, and T. Sainath,  “Deep learning for audio signal processing,”  IEEE J. Sel. Top. Signal Process, vol. 13, no. 2, pp. 206–219, 2019\.
  * [7] P.-S. Huang, M. Kim, M. Hasegawa-Johnson, and P. Smaragdis,  “Deep learning for monaural speech separation,”  in Proc. IEEE Int. Conf. Acoust. Speech Signal Process. IEEE, 2014, pp. 1562–1566.
  * [8] D. Stoller, S. Ewert, and S. Dixon,  “Wave-U-Net: A multi-scale neural network for end-to-end audio source separation,”  in Proc. 19th International Society for Music Information Retrieval Conference, 2018, pp. 334–340.
  * [9] Y. Luo and N. Mesgarani,  “Conv-TasNet: Surpassing ideal time–frequency magnitude masking for speech separation,”  IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 27, no. 8, pp. 1256–1266, 2019.
  * [10] E. Tzinis, Z. Wang, and P. Smaragdis,  “Sudo RM-RF: Efficient networks for universal audio source separation,”  in IEEE Int. Workshop Mach. Learn. Signal Process. IEEE, 2020, pp. 1–6.
  * [11] Y. Luo, Z. Chen, and T. Yoshioka,  “Dual-path RNN: Efficient long sequence modeling for time-domain single-channel speech separation,”  in Proc. IEEE Int. Conf. Acoust. Speech Signal Process. IEEE, 2020, pp. 46–50.
  * [12] J. Chen, Q. Mao, and D. Liu,  “Dual-path transformer network: Direct context-aware modeling for end-to-end monaural speech separation,”  in Proc. Interspeech, 2020, pp. 2642–2646.
  * [13] H. Li, K. Chen, L. Wang, J. Liu, B. Wan, and B. Zhou,  “Sound source separation mechanisms of different deep networks explained from the perspective of auditory perception,”  Applied Sciences, vol. 12, no. 2, 2022.
  * [14] MIT RLE,  “RF Challenge - AI Accelerator,” Accessed 2022-10-01.
  * [15] A. Lancho, A. Weiss, G. C. F. Lee, J. Tang, Y. Bu, Y. Polyanskiy, and G. W. Wornell,  “Data-driven blind synchronization and interference rejection for digital communication signals,”  arXiv preprint arXiv:2209.04871, 2022,  to appear in Proc. of IEEE Glob. Commun. Conf.
  * [16] K. K. Sharma and S. D. Joshi,  “Signal separation using linear canonical and fractional fourier transforms,”  Opt. Commun., vol. 265, no. 2, pp. 454–460, 2006.
  * [17] M. Zhao, X. Yao, J. Wang, Y. Yan, X. Gao, and Y. Fan,  “Single-channel blind source separation of spatial aliasing signal based on stacked-LSTM,”  Sensors, vol. 21, no. 14, 2021.
  * [18] M. Tao, J. Su, Y. Huang, and L. Wang,  “Mitigation of radio frequency interference in synthetic aperture radar data: Current status and future trends,”  Remote Sensing, vol. 11, no. 20, pp. 2438, 2019.
  * [19] P. Y. Ma, A. N. Tait, W. Zhang, E. A. Karahan, T. F. De Lima, C. Huang, B. J. Shastri, and P. R. Prucnal,  “Blind source separation with integrated photonics and reduced dimensional statistics,”  Opt. Lett., vol. 45, no. 23, pp. 6494–6497, 2020.
  * [20] X. Li, W.-D. Zhong, A. Alphones, C. Yu, and Z. Xu,  “Channel equalization in optical ofdm systems using independent component analysis,”  J. Light. Technol., vol. 32, no. 18, pp. 3206–3214, 2014.
  * [21] A. Novoselov, P. Balazs, and G. Bokelmann,  “SEDENOSS: SEparating and DENOising seismic signals with dual-path recurrent neural network architecture,”  J. Geophys. Res. Solid Earth, vol. 127, no. 3, pp. e2021JB023183, 2022.
  * [22] G. C. F. Lee, A. Weiss, A. Lancho, J. Tang, Y. Bu, Y. Polyanskiy, and G. W. Wornell,  “Exploiting temporal structures of cyclostationary signals for data-driven single-channel source separation,”  arXiv preprint arXiv:2208.10325, 2022,  to appear in Proc. IEEE Int. Workshop Mach. Learn. Signal Process.
  * [23] IEEE Standard for Information technology– Local and metropolitan area networks– Specific requirements– Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications Amendment 5: Enhancements for Higher Throughput, 2009.
  * [24] M. Pariente, S. Cornell, J. Cosentino, S. Sivasankaran, E. Tzinis, J. Heitkaemper, M. Olvera, F.-R. Stöter, M. Hu, J. M. Martín-Doñas, D. Ditter, A. Frank, A. Deleforge, and E. Vincent,  “Asteroid: the PyTorch-based audio source separation toolkit for researchers,”  in Proc. Interspeech, 2020.
  * [25] N. Shlezinger and R. Dabora,  “Frequency-shift filtering for OFDM signal recovery in narrowband power line communications,”  IEEE Trans. Commun., vol. 62, no. 4, pp. 1283–1295, 2014.
  * [26] P. Comon and C. Jutten,  Handbook of Blind Source Separation: Independent component analysis and applications,  Academic press, 2010.
  * [27] W. Luo, Y. Li, R. Urtasun, and R. Zemel,  “Understanding the effective receptive field in deep convolutional neural networks,”  Advances in neural information processing systems, vol. 29, 2016\.