* [224] P. Foggia, A. Saggese, N. Strisciuglio, and M. Vento, “Exploiting the deep learning paradigm for recognizing human actions,” in _2014 11th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS)_ , Aug 2014, pp. 93–98.
  * [225] P. Foggia, G. Percannella, A. Saggese, and M. Vento, “Recognizing human actions by a bag of visual words,” in _2013 IEEE International Conference on Systems, Man, and Cybernetics_. IEEE, 2013, pp. 2910–2915.
  * [226] F. Ofli, R. Chaudhry, G. Kurillo, R. Vidal, and R. Bajcsy, “Berkeley MHAD: A comprehensive multimodal human action database,” in _2013 IEEE Workshop on Applications of Computer Vision (WACV)_ , Jan 2013, pp. 53–60.
  * [227] K. H. Ali and T. Wang, “Learning features for action recognition and identity with deep belief networks,” in _2014 International Conference on Audio, Language and Image Processing_ , July 2014, pp. 129–132.
  * [228] P. Heckbert, “Fourier Transforms and the Fast Fourier Transform (FFT) Algorithm,” _Computer Graphics_ , vol. 2, pp. 15–463, 1995.
  * [229] Q. V. Le, W. Y. Zou, S. Y. Yeung, and A. Y. Ng, “Learning hierarchical invariant spatio-temporal features for action recognition with independent subspace analysis,” in _Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition_ , ser. CVPR ’11. Washington, DC, USA: IEEE Computer Society, 2011, pp. 3361–3368. [Online]. Available: http://dx.doi.org/10.1109/CVPR.2011.5995496
  * [230] I. Laptev, “On space-time interest points,” _International Journal of Computer Vision_ , vol. 64, no. 2-3, pp. 107–123, 2005.
  * [231] H. Wang, M. M. Ullah, A. Klaser, I. Laptev, and C. Schmid, “Evaluation of local spatio-temporal features for action recognition,” in _BMVC 2009-British Machine Vision Conference_. BMVA Press, 2009, pp. 124–1.
  * [232] G. W. Taylor, R. Fergus, Y. LeCun, and C. Bregler, “Convolutional learning of spatio-temporal features,” in _European conference on computer vision_. Springer, 2010, pp. 140–153.
  * [233] J. C. Niebles, H. Wang, and L. Fei-Fei, “Unsupervised learning of human action categories using spatial-temporal words,” _International journal of computer vision_ , vol. 79, no. 3, pp. 299–318, 2008.
  * [234] Y. Ke, R. Sukthankar, and M. Hebert, “Efficient visual event detection using volumetric features,” in _Tenth IEEE International Conference on Computer Vision (ICCV’05) Volume 1_ , vol. 1. IEEE, 2005, pp. 166–173.
  * [235] S. Nam, S. Park, J. Park, and T. Kim, “A single depth sensor based human activity recognition via deep belief network,” in _Proceedings of the 4th World Conference on Applied Sciences, Engineering and Technology 24-26 October 2015_ , 2015, pp. 015–019.
  * [236] F. J. Huang, Y.-L. Boureau, and Y. LeCun, “Unsupervised learning of invariant feature hierarchies with applications to object recognition,” in _2007 IEEE conference on computer vision and pattern recognition_. IEEE, 2007, pp. 1–8.
  * [237] M. Baccouche, F. Mamalet, C. Wolf, C. Garcia, and A. Baskurt, “Spatio-temporal convolutional sparse auto-encoder for sequence classification.” in _British Machine Vision Conference 2012 (BMVC’12)_ , 2012, pp. 1–12.
  * [238] M. F. Valstar, B. Jiang, M. Mehu, M. Pantic, and K. Scherer, “The first facial expression recognition and analysis challenge,” in _2011 IEEE International Conference on Automatic Face and Gesture Recognition (FG 2011)_. IEEE, 2011, pp. 921–926.
  * [239] D. Wu, W. Pan, L. Xie, and C. Huang, “An adaptive stacked denoising auto-encoder architecture for human action recognition.” _Applied Mechanics & Materials_, 2014.
  * [240] L. Xie, W. Pan, C. Tang, and H. Hu, “A pyramidal deep learning architecture for human action recognition,” _International Journal of Modelling, Identification and Control_ , vol. 21, no. 2, pp. 139–146, 2014.
  * [241] M. Hasan and A. K. Roy-Chowdhury, “Continuous learning of human activity models using deep nets,” in _European Conference on Computer Vision_. Springer, 2014, pp. 705–720.
  * [242] A. Budiman, M. I. Fanany, and C. Basaruddin, “Stacked denoising autoencoder for feature representation learning in pose-based action recognition,” in _2014 IEEE 3rd Global Conference on Consumer Electronics (GCCE)_. IEEE, 2014, pp. 684–688.
  * [243] B. Settles, “Active learning,” _Synthesis Lectures on Artificial Intelligence and Machine Learning_ , vol. 6, no. 1, pp. 1–114, 2012.
  * [244] M. Chen, Z. Xu, K. Weinberger, and F. Sha, “Marginalized stacked denoising autoencoders,” in _Proceedings of the Learning Workshop, Utah, UT, USA_ , vol. 36, 2012.
  * [245] F. Gu, F. Flórez-Revuelta, D. Monekosso, and P. Remagnino, “Marginalised stacked denoising autoencoders for robust representation of real-time multi-view action recognition,” _Sensors_ , vol. 15, no. 7, pp. 17 209–17 231, 2015.
  * [246] K. Yu, Y. Lin, and J. Lafferty, “Learning image representations from the pixel level via hierarchical sparse coding,” in _CVPR 2011_ , June 2011, pp. 1713–1720.
  * [247] R. Raina, A. Battle, H. Lee, B. Packer, and A. Y. Ng, “Self-taught learning: transfer learning from unlabeled data,” in _Proceedings of the 24th international conference on Machine learning_. ACM, 2007, pp. 759–766.
  * [248] J. Yang, K. Yu, and T. Huang, “Supervised translation-invariant sparse coding,” in _2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition_ , June 2010, pp. 3517–3524.
  * [249] J. Yang, K. Yu, Y. Gong, and T. Huang, “Linear spatial pyramid matching using sparse coding for image classification,” in _2009 IEEE Conference on Computer Vision and Pattern Recognition_ , June 2009, pp. 1794–1801.
  * [250] Y. Zhu, X. Zhao, Y. Fu, and Y. Liu, “Sparse coding on local spatial-temporal volumes for human action recognition,” in _Asian Conference on Computer Vision_. Springer, 2010, pp. 660–671.
  * [251] Z. Lu and Y. Peng, “Latent semantic learning by efficient sparse coding with hypergraph regularization,” in _Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence_ , ser. AAAI’11. AAAI Press, 2011, pp. 411–416. [Online]. Available: http://dl.acm.org/citation.cfm?id=2900423.2900488
  * [252] Z. Lu and Y. Peng, “Latent semantic learning with structured sparse representation for human action recognition,” _Pattern Recognition_ , vol. 46, no. 7, pp. 1799–1809, 2013.
  * [253] T. Guha and R. K. Ward, “Learning sparse representations for human action recognition,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 34, no. 8, pp. 1576–1588, 2012.
  * [254] A. Alfaro, D. Mery, and A. Soto, “Action recognition in video using sparse coding and relative features,” _arXiv preprint arXiv:1605.03222_ , 2016.
  * [255] I. Ullah and A. Petrosino, “A strict pyramidal deep neural network for action recognition,” in _International Conference on Image Analysis and Processing_. Springer, 2015, pp. 236–245.
  * [256] I. Ullah and A. Petrosino, “Spatiotemporal features learning with 3DPyraNet,” in _International Conference on Advanced Concepts for Intelligent Vision Systems_. Springer, 2016, pp. 638–647.
  * [257] H. Rahmani, A. Mian, and M. Shah, “Learning a deep model for human action recognition from novel viewpoints,” _arXiv preprint arXiv:1602.00828_ , 2016\.
  * [258] S. L. Phung and A. Bouzerdoum, “A pyramidal neural network for visual pattern recognition,” _IEEE Transactions on Neural Networks_ , vol. 18, no. 2, pp. 329–343, 2007.
  * [259] H. Rahmani, A. Mahmood, D. Huynh, and A. Mian, “Histogram of oriented principal components for cross-view action recognition,” 2016.
  * [260] J. Wang, X. Nie, Y. Xia, Y. Wu, and S.-C. Zhu, “Cross-view action modeling, learning and recognition,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2014, pp. 2649–2656.
  * [261] M. D. Rodriguez, J. Ahmed, and M. Shah, “Action MACH a spatio-temporal maximum average correlation height filter for action recognition,” in _2008 IEEE Conference on Computer Vision and Pattern Recognition_ , June 2008, pp. 1–8.
  * [262] A. Hyvärinen, J. Hurri, and P. O. Hoyer, “Independent component analysis,” in _Natural Image Statistics_. Springer, 2009, pp. 151–175.
  * [263] M. Mudrova and A. Prochazka, “Principal component analysis in image processing,” in _Proceedings of the MATLAB Technical Computing Conference, Prague_ , 2005.
  * [264] N. Srivastava, E. Mansimov, and R. Salakhutdinov, “Unsupervised learning of video representations using LSTMS,” _CoRR, abs/1502.04681_ , vol. 2, 2015\.
  * [265] Z. Luo, B. Peng, D.-A. Huang, A. Alahi, and L. Fei-Fei, “Unsupervised learning of long-term motion dynamics for videos,” _CoRR_ , vol. abs/1701.01821, 2017\.
  * [266] I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. C. Courville, and Y. Bengio, “Generative adversarial nets,” in _NIPS_ , 2014.
  * [267] A. Radford, L. Metz, and S. Chintala, “Unsupervised representation learning with deep convolutional generative adversarial networks,” _CoRR_ , vol. abs/1511.06434, 2015.
  * [268] C. Vondrick, H. Pirsiavash, and A. Torralba, “Generating videos with scene dynamics,” in _NIPS_ , 2016.

Hieu H. Pham is a Teaching Fellow at the College of Engineering and Computer
Science (CECS), VinUniversity, and a Research Fellow at VinUni-Illinois Smart
Health Center. He received his Ph.D. in Computer Science from the Toulouse
Computer Science Research Institute (IRIT), University of Toulouse, France, in
2019. Previously, he earned the Degree of Engineer in Industrial Informatics
from Hanoi University of Science and Technology (HUST), Vietnam, in 2016. His
research interests include Computer Vision, Machine Learning, Medical Image
Analysis, and their applications in Smart Healthcare. He is the author, co-
author of 30 scientific articles appeared in about 20 conferences and journals
such as Computer Vision and Image Understanding, Neurocomputing, International
Conference on Medical Image Computing and Computer-Assisted Intervention
(MICCAI), Medical Imaging with Deep Learning (MIDL), IEEE International
Conference on Image Processing (ICIP), and IEEE International Conference on
Computer Vision (ICCV). He is also currently serving as Reviewers for MICCAI,
ICCV, CVPR, IET Computer Vision Journal (IET-CVI), IEEE Journal of Biomedical
and Health Informatics (JBHI), and Nature Scientific Reports. Before joining
VinUniversity, Dr. Hieu worked at Vingroup Big Data Institute (VinBigData) as
a Research Scientist and Head of the Fundamental Research Team. With this
position, he led several research projects on Medical AI, including collecting
various types of medical data, managing and annotating data, and developing
new AI solutions for medical analysis.

Louahdi Khoudour, Director of Research, received his Ph.D. in Computer Vision
from the University of Lille in 1997 and Habilitation à Diriger des Recherches
(HDR) degree from the University of Paris in 2006. He worked at Ifsttar
(formerly INRETS: French National Institute on traffic and safety research)
for many years. He moved to CEREMA in 2011, where he is head of a research
group working on safety and security in transport.

Alain Crouzil received his Ph.D. degree in Computer science from the Paul
Sabatier University of Toulouse in 1997. He is currently an associate
professor and a member of the Traitement et Comprehension d’Images group of
Institut de Recherche en Informatique de Toulouse. His research interests
concern stereo vision, shape from shading, camera calibration, image
segmentation, change detection, and motion analysis.

Pablo Zegers received his B.S. and P.E. degrees in Engineering from the
Pontificia Universidad Catolica, Chile, in 1992, his M.Sc. from The University
of Arizona, USA, in 1998, and his Ph.D., also from The University of Arizona,
in 2002. He is currently an Associate Professor of the College of Engineering
and Applied Sciences of the Universidad de los Andes, Chile. His interests are
artificial intelligence, machine learning, neural networks, and information
theory. From 2006 to 2010 he was the Academic Director of this College, and
for a brief period at the end of 2010, the Interim Dean. He is a Senior Member
of the IEEE, and currently the Secretary of the Chilean IEEE section.

Sergio A. Velastin obtained his Ph.D. in 1982 from the University of
Manchester, UK. He is a professor of applied computer vision and was the
director of the Digital Imaging Research Centre at Kingston University until
2012. He then worked as a research professor at the University of Santiago
(Chile) and is currently UC3M-Marie Curie Research Professor at the University
Carlos III de Madrid, Spain, working on human action recognition. He is a
fellow of the IET and a Senior Member of the IEEE.
---


# Hardware Software Co-design of Statistical and Deep Learning Frameworks for
Wideband Sensing on Zynq System on Chip

Rohith Rajesh, Sumit J. Darak, Akshay Jain, Shivam Chandhok, and Animesh
Sharma This work is supported by the funding received from core research grant
(CRG) awarded to Dr. Sumit J. Darak from DST-SERB, GoI.Rohith Rajesh, Sumit J.
Darak and Animesh Sharma are with Electronics and Communications Department,
IIIT-Delhi, India-110020 (e-mail:
<EMAIL_ADDRESS>Jain is associated with
AMD, Hyderabad, India and Electronics and Communications Department, IIIT-
Delhi, India. email<EMAIL_ADDRESS>Chandhok is with INRIA, Universite
Grenoble Alpes, France

###### Abstract

With the introduction of spectrum sharing and heterogeneous services in next-
generation networks, the base stations need to sense the wideband spectrum and
identify the spectrum resources to meet the quality-of-service, bandwidth, and
latency constraints. Sub-Nyquist sampling (SNS) enables digitization for
sparse wideband spectrum without needing Nyquist speed analog-to-digital
converters. However, SNS demands additional signal processing algorithms for
spectrum reconstruction, such as the well-known orthogonal matching pursuit
(OMP) algorithm. OMP is also widely used in other compressed sensing
applications. The first contribution of this work is efficiently mapping the
OMP algorithm on the Zynq system-on-chip (ZSoC) consisting of an ARM processor
and FPGA. Experimental analysis shows a significant degradation in OMP
performance for sparse spectrum. Also, OMP needs prior knowledge of spectrum
sparsity. We address these challenges via deep-learning-based architectures
and efficiently map them on the ZSoC platform as second contribution. Via
hardware-software co-design, different versions of the proposed architecture
obtained by partitioning between software (ARM processor) and hardware (FPGA)
are considered. The resource, power, and execution time comparisons for given
memory constraints and a wide range of word lengths are presented for these
architectures.

###### Index Terms:

Deep learning, Hardware software co-design, Convolutional Neural Network, Zynq
System-on-chip, Sub-Nyquist Sampling, Orthogonal Matching Pursuit.

##  I Introduction

Wideband spectrum sensing (WSS) involves digitizing the wideband spectrum and
identifying available spectrum for resource allocation in wireless networks
[1, 2, 3]. WSS has gained significant importance in 5G and next-generation
wireless networks due to the introduction of spectrum sharing policies
replacing the conventional static spectrum allocation policies [4]. Spectrum
sharing allows the deployment of the wireless networks in licensed, shared,
and unlicensed spectrum [5]. The advantages include significant cost savings
for Telecom service operators since they can reduce licensed spectrum
requirement and corresponding exorbitant spectrum license fee, which is
usually in billions of dollars [6]. A broader spectrum enables the deployment
of upcoming heterogeneous services that demand a wide range of bandwidth,
quality-of-service, and latency constraints. Though the millimeter wave
spectrum (above 6 GHz) is being explored due to the availability of large
bandwidth, various studies have confirmed that it is not a feasible
alternative to the sub-6 GHz spectrum for reliable outdoor communication and
comprehensive network coverage.

Numerous spectrum measurement and utilization studies have shown that overall
utilization of the sub-6 GHz spectrum is poor even though most of the spectrum
is licensed [7, 8, 9]. For digitization of such a sparse wideband spectrum,
sub-Nyquist sampling (SNS) based WSS is an efficient and feasible alternative
to Nyquist sampling based WSS. This is because SNS needs low-speed analog-to-
digital converters (ADCs) compared to Nyquist rate ADCs in the latter [2, 10].
However, SNS-based WSS needs additional digital signal processing to recover
the SNS sampled spectrum so that it closely resembles the original spectrum
[11, 12]. Such recovery must be done accurately and should meet the stringent
area, power, and latency constraints.

Various signal recovery techniques for SNS have been reviewed in [13]. The
greedy-approach-based orthogonal matching pursuit (OMP) framework is popular
due to its lower complexity and faster execution time. Other applications of
OMP includes image processing, and radar systems [14, 15, 16, 17]. Recently,
advances in deep learning (DL) have been explored to improve the performance
of the OMP [18, 19, 20]. Most of these approaches augment the OMP by replacing
the matched filtering task with the DL. The OMP and its variants suffer from a
significant degradation in reconstruction performance, especially when the
spectrum sparsity is high. Also, they need prior knowledge of spectrum
sparsity which may not be available in a dynamic environment. From an
architecture perspective, in 5G and next-generation wireless networks,
distributed base station approach where radio unit (RU), distributed unit
(DU), and central unit (CU) may not be co-located is being explored [21, 22].
Such deployment demands the mapping of SNS algorithms on hardware platforms
such as system on chip (SoC) under the limited computing resource constraints
due to remote locations of the RU and DU.

The first contribution of this work is to efficiently map statistical signal
processing based OMP framework for SNS spectrum recovery on Zynq system-on-
chip (ZSoC). The ZSoC is the heterogeneous SoC comprising of software (SW),
i.e., dual-core Cortex ARM A9 processor, and hardware (HW), i.e., 7-series
field programmable gate array (FPGA). To address the drawbacks of OMP and make
the spectrum reconstruction agnostic to sparsity, we replace OMP with a
convolutional neural network (CNN) based deep learning (DL) architecture and
efficiently map it on the ZSoC platform as the second contribution. Via
hardware-software co-design, we explore different versions of the proposed DL
architecture obtained by partitioning between SW (ARM processor) and HW
(FPGA). Our study offers interesting insights which may not be visible in
conventional theoretical and simulation-based analysis. For these
architectures, we present the resource utilization, power consumption, and
execution time comparisons for given memory constraints and a wide range of
word lengths (WL) at the parameters and computation levels. We develop an end-
to-end application with a live graphical user interface (GUI) to demonstrate
the real-time WSS on the ZSoC platform.  Please refer to [23] for source
codes, hardware IPs, datasets and detailed tutorial used for generating the
results presented in this paper.

The rest of the paper is organized as follows. We discuss the OMP architecture
and experimental results in Section II. The DLWSS algorithm is presented in
Section III and corresponding architecture in Section IV. Section V presents
the performance analysis results and comparison of DLWSS with OMP followed by
complexity analysis in Section VI. Section VII concludes the paper.

##  II Spectrum Recovery via OMP for SNS based WSS

This section discusses the design and implementation of the OMP on the ZSoC.
We demonstrate the various drawbacks of OMP via experimental analysis and
results.

###  II-A Orthogonal Matching Pursuit (OMP) on ZSoC

OMP [24, 25, 26] is one of the most widely used sparse recovery algorithm for
SNS based WSS. It follows an iterative formulation where an occupied band that
is highly correlated with the residual matrix is identified. Then its
contribution is removed from the residual matrix to identify the next highly
correlated occupied band. This process is repeated for all occupied bands when
we have prior knowledge of the spectrum sparsity. Otherwise, we need to have a
stopping criterion based on estimated sparsity. The OMP pseudo-code is given
in Algorithm 1. Here, $K$ represents the number of ADCs used for SNS, $Q$ is
the number of snapshots produced by the ADC, and $N$ is the number of
frequency bands in the wideband spectrum. Further, $A$ is the sensing matrix
of dimension $K\times N$, $Y$ are the received SNS samples of dimension
$N\times Q$, and $\dagger$ represents a matrix pseudo-inverse. It comprises
four main steps, i.e., 1) Matching (Line 6-8); 2) Identification (Line 9-13);
3) Least squares (Line 14) and 4) Approximation (Line 15).

Algorithm 1 OMP algorithm for WSS

1:Sensing matrix A[K][N], aliased sub-Nyquist samples Y [K] [Q], Sparsity $S$

2:$\mathtt{occupied\\_bands\leftarrow[]}$

3:$\mathtt{A_{norm}\leftarrow}$ Column normalized A $\\#$Normalize A

4:$\mathtt{Res\leftarrow Y}$ $\\#$Initialize Residual

5:$\mathtt{iter\leftarrow 1}$

6:while $\mathtt{iter\leq S}$ do $\\#$No. of occupied bands

7: for j in columns($\mathtt{A_{norm}}$) do

8: $\mathtt{Z[j]\leftarrow norm(A_{norm}[\;:\;,j\;]^{T}\times Res)}$

9: end for

10: Append $\mathtt{argmax(Z)\>to\>occupied\\_bands}$

11: for j in $\mathtt{occupied\\_bands}$ do

12: Append column $\mathtt{A_{norm}[\;:\;,j\;]}$ to As

13: end for

14: $\mathtt{Res\leftarrow Res-As\times(As^{\dagger}\times Res)}$

15: $\mathtt{iter\leftarrow iter+1}$

16:end while

17:return $\mathtt{occupied\\_bands}$

We have realized the OMP in Algorithm 1 on ZSoC ZC706 comprising of Dual ARM
Cortex A9 processor and 7-series FPGA with 1090 units of 18kB Block RAMs, 80
DSP48E units, 218600 units of 6-input look-up-tables (LUTs) and 437200 flip-
flops (FFs). Table I shows the execution time, resource utilization, and power
consumption comparison for the two best possible realizations of the OMP
algorithm: 1) Only software (ARM), and 2) Software and hardware co-design
(ARM+FPGA). Note that we have optimized the code for software implementation
and carefully chosen the word length on hardware to minimize the execution
time, power consumption, and resource utilization without compromising the
functional accuracy. Interestingly, FPGA realization is slower than ARM
processors due to the sequential nature of the OMP algorithm and the need for
variable size matrix inversion operations. On FPGA, we need to implement
single matrix inversion architecture for the largest possible matrix size and
use the same for other matrices. Thus, the execution time of matrix inversion
operations in all stages is the same on FPGA, irrespective of matrix size. In
contrast, matrix inversion time in SW increases with the increase in matrix
size. Even though the OMP is realized on FPGA in the second case, we still
need an ARM processor for running the operating system and control operations
in remote edge deployment. Thus, the execution of the second realization has a
significant contribution from the data communication overhead between ARM and
FPGA. Also, the relatively small dimension of matrices means that the ARM
processor can execute the arithmetic operations faster due to efficient
caching and higher clock frequency than FPGA.

TABLE I: Comparison of execution time, resource and power statistics of OMP
realization on ZC706 Platform

Configuration |  | Execution Time
---
(Seconds)
| Resource Utilization
---
{BRAM,DSP,FF,LUT}
Total Power (W)
SW | 0.103 | – | 1.57
SW+HW | 0.189 | {44,258,17160,13070} | 3.664

###  II-B Functional Performance Analysis of OMP on ZSoC

In this section, we analyze the functional correctness of the OMP algorithm
realized on ZSoC. To begin with, we discuss the performance metrics used for
such analysis.

####  II-B1 Performance Metrics

We consider two well-know performance metrics: Detection accuracy of all bands
in percentage ($P_{d}^{AB}$) and detection accuracy of occupied bands in
percentage ($P_{d}^{OB}$) [27, 28]. The metric, $P_{d}^{AB}$, corresponds to
the fraction of frequency bands whose status is correctly detected by the
algorithm. The second metric, $P_{d}^{OB}$, corresponds to the fraction of
occupied bands whose status is correctly detected by the algorithm.
Mathematically, $P_{d}^{AB}$and $P_{d}^{OB}$are formulated as:

$P_{d}^{AB}=100*\frac{\sum_{n=1}^{N}(y_{pred}^{n}\,==\,y_{true}^{n})}{\sum_{n=1}^{N}(y_{true}^{n})}$

$P_{d}^{OB}=100*\frac{\sum_{n=1}^{N}(y_{pred}^{n}==1\,\&\,y_{true}^{n}==1)}{\sum_{n=1}^{N}(y_{true}^{n}==1)}$

Here, $N$ is the total number of frequency bands in the digitized spectrum,
$y_{pred}$ and $y_{true}$ are the predicted and ground-truth band status,
respectively (i.e.,, $0$ for vacant and $1$ for occupied band). It is worth
noting that $P_{d}^{OB}$is the preferred metric for a sparse spectrum
consisting of fewer occupied bands since $P_{d}^{AB}$may give a high value
even when the algorithm erroneously detects all bands as vacant. Similarly,
$P_{d}^{AB}$is the preferred metric for spectrum with lower sparsity
comprising the higher number of occupied bands. For such spectrum,
$P_{d}^{OB}$may be high even if the algorithm erroneously detects all bands as
occupied. We do not consider the spectrum recovery error as a performance
metric since it does not offer additional insights for WSS. There are
challenges in capturing a large number of samples from hardware for accurate
error calculation of spectrum recovery error in real-time.

####  II-B2 Dataset

Since SNS-based WSS is based on the underlining assumption of the sparse
spectrum, we have generated various datasets with sparsity ranging from
50%-100%. We consider the SNS with eight analog-to-digital converters (ADCs),
which allow the recovery of the spectrum when the number of occupied bands is
less than or equal to 8. We consider two groups of datasets:

  * •

Extremely Sparse Spectrum (ESS): This dataset contains the spectrum with the
number of occupied bands between 1 and 3.

  * •

Highly Sparse Spectrum (HSS): This dataset contains the spectrum with the
number of occupied bands between 4 and 7.

As discussed before, we use $P_{d}^{OB}$and $P_{d}^{AB}$as performance metrics
for ESS and HSS datasets, respectively.  Interested readers can refer to [23]
for datasets and source code used for the dataset generation.

####  II-B3 OMP without Sparsity Knowledge

As discussed in Algorithm 1, the number of iterations in OMP algorithm depends
on the sparsity, $S$ (Line 5) and this requirement limits its usefulness in
realistic applications. We consider the variation of OMP where sparsity
knowledge is not known by exploring the stopping criteria for residual, such
as $||Res||<\epsilon$ where $\epsilon$ is the convergence constant. The
convergence constant, $\epsilon$, depends on both the sparsity and SNR of the
digitized spectrum. To fix $\epsilon$, we assume prior knowledge of the SNR,
which is common in wireless systems since wireless receivers can easily
measure the SNR of the digitized spectrum. Next, via empirical analysis, we
study the correlation between SNR and residual, $||Res||$, to get the desired
performance. We use this value of $||Res||$ as an appropriate estimate for
$\epsilon$ for a given SNR. In Fig. 1, we compare the desired values of
$\epsilon$ for wide range of spectrum sparsity and SNRs. Note that we can not
have fixed $\epsilon$ for all SNRs. However, variation in $\epsilon$ for
different spectrum sparsity is insignificant, and we can use the average value
of $\epsilon$ for all spectrums with varying sparsity. We refer to this
algorithm as OMP-$\epsilon$.

In a wireless environment, the wireless channel may vary depending on the
deployment scenario. In Fig. 2, we study the effect of wireless channel on the
average value of $\epsilon$ obtained over wide range of SNRs,
$\epsilon_{SNR}$. As expected, the effect of the channel on the WSS is limited
since we do not need to recover the spectrum. Thus, we can fix a single value
of $\epsilon_{SNR}$ for all types of wireless channels.

Figure 1: Variation in $\epsilon$ estimates for wide range of spectrum
sparsity and SNRs. Figure 2: Average $\epsilon_{SNR}$ estimates for various
types of wireless channels.

####  II-B4 Performance Analysis and Drawbacks

In Fig. 3, we compare the performance of the OMP and OMP-$\epsilon$ for ESS
and HSS datasets for a wide range of SNRs. As expected, the performance
improves with the increase in SNR. In the case of ESS with $P_{d}^{OB}$as a
performance metric, both OMPs achieve 100% accuracy at high SNR due to
accurate estimation of $\epsilon$. However, the difference between OMP and
OMP-$\epsilon$ is around 10-40% at low SNR. In the case of HSS with
$P_{d}^{AB}$as a performance metric, overall accuracy is lower, and
OMP-$\epsilon$ can not meet the accuracy of OMP even at high SNR. This is
because the accurate selection of $\epsilon_{SNR}$ guarantees correct
detection of occupied bands but does not guarantee the correct detection of
the status of remaining bands. These results indicate that the state-of-the-
art OMP algorithm does not offer reliable performance for HSS even at high
SNR.

Figure 3: Performance comparisons of the OMP and OMP-$\epsilon$ for (a) ESS
and (b) HSS datasets for a wide range of SNRs.

OMP suffers from multiple drawbacks: 1) Prior knowledge of sparsity is needed.
If such knowledge is unavailable, it suffers from significant performance
degradation even with prior knowledge of SNR, 2) Poor performance in low SNR
for both types of spectrum, and 3) Poor performance even at high SNR for HSS.
These shortcomings motivates the search for potential alternatives for SNS
based WSS.

##  III DLWSS: Spectrum Recovery via Deep Learning for SNS based WSS

Previous efforts that aim to tackle SNS-based WSS using deep networks can be
broadly classified into two categories: 1) Iterative approaches [19, 18, 20]
that follow the OMP formulation and augment the frequency band status
detection with a deep network, and 2) Non-iterative approaches which utilize a
deep network for end-to-end spectrum recovery and frequency band status
detection [29]. Since the iterative approaches follow the algorithmic
formulation of OMP, they suffer from the same limitations as that of OMP. In
this paper, we focus on a non-iterative approach that can handle the signals
of varying sparsity and does not require sparsity information or any manually
adjusted convergence constant [29]. The proposed DLWSS algorithm aims to learn
an end-to-end model for WSS. DLWSS pipeline comprises two stages: 1) Pre-
processing and 2) Deep Network Prediction.

####  III-1 Pre-Processing

Algorithm 2 shows the steps involved in the Pre-processing stage of the DLWSS.
The pre-processing step receives the signal captured by the antenna and
digitized using the SNS-based analog front-end. It processes and normalizes
the digitized signal so the DL architecture can handle it. Specifically, this
involves computation of the pseudo-recovered spectrum $\tilde{X}$ using the
sensing matrix and sub-Nyquist samples. Since the complex input signal,
$\tilde{X}$ can not be fed directly to the DL architecture; it is converted to
a high dimensional real-valued signal as shown in Algorithm 2. In the end, the
signal is normalized for faster convergence of the training process.

Algorithm 2 Pre-Processing

1:Sensing matrix A[K][N], aliased sub-Nyquist samples Y[K][Q]

2:$A_{sq}$= $A^{*}\times A$ $\\#$Square Matrix $A_{sq}$

3:$P\times A_{sq}$ = $L\times D\times U$ $\\#$LU Factorization

4:$A_{sq}^{-1}$=$U^{-1}\times D^{-1}\times L^{-1}\times P^{-1}$

5:$A^{\dagger}$=$A_{sq}^{-1}\times A^{*}$ $\\#$Pseudo-inverse of $A$

6:$\tilde{X}\leftarrow A^{\dagger}\times Y$

7:$\tilde{X}_{d}\leftarrow$ Concatenate($\tilde{X}_{real},\tilde{X}_{imag}$)

8:$X_{n}\leftarrow$ Normalize($\tilde{X}_{d}$)

9:return $X_{n}$

####  III-2 Deep Network Prediction

The deep network block of the DLWSS receives the processed input samples from
Pre-processing block and predicts the status (vacant or occupied) of each
frequency band of the digitized spectrum. The DLWSS architecture is based on a
convolutional neural network (CNN) due to its ability to capture spatial
correlation in input signals which is integral for the spectrum sensing task.
Further, parameter sharing allows them to operate with fewer parameters,
enabling the network to be memory efficient and a good candidate for hardware
realization. Table II shows the architecture of the DL model, which comprises
three Convolutional (CV) layers and a single Fully Connected (FC) layer. All
intermediate activations are Rectified Linear Units (ReLU), and the activation
at the output layer is Sigmoid.

Similar to the deployment of any DL algorithm, DLWSS design has two phases,
training and inference. The training phase minimizes a loss function, which
measures the difference between the predicted and actual labels. Since more
than one frequency band can be occupied in a wideband spectrum, we formulate
the problem as a multi-label binary classification with binary cross-entropy
as the training loss function on final sigmoid outputs. After the training
mode, the model weights are frozen, and the CNN model is used in inference
mode to find the occupancy status of an unknown signal in real-time. Similar
to [29], we train the architecture for a dataset using a machine learning
framework (PyTorch in our case) and GPUs. After training, model architecture,
weights and parameters are extracted for realization on the ZSoC, followed by
inference on the new dataset.

TABLE II: CNN based deep network prediction Architecture

Layers | Filters | Kernel size | Input Shape | Output Shape
---|---|---|---|---
CV | 256 | 1x150 | 14x299x2 | 14x150x256
CV | 128 | 1x100 | 14x150x256 | 14x51x128
CV | 64 | 1x51 | 14x51x128 | 14x1x64
Flatten | - | - | 14x1x64 | 896x1
FC | - | - | 896x1 | 14

Figure 4: Proposed DLWSS Architecture on Zynq SoC.

##  IV Realization of DLWSS on ZSoC

This section presents the complete architecture and mapping of DLWSS on ZSoC,
and real-time demonstration via Linux-based graphical user interface (GUI)
deployed on ZSoC. In Fig. 4, various building blocks of the DLWSS such as Pre-
processing, CNN and FC layers, scheduler, DMA, interrupt, and GUI controller
are shown. For illustration, we have shown the Pre-processing, CNN, and FC
layer processing on FPGA. To enable this, we have developed AXI-stream
compatible hardware IPs for Pre-processing, CNN, and FC blocks and
interconnected them with PS via direct-memory access (DMA) in the scatter-
gather mode for efficient data transfers. Later in Section VI, we considered
various architectures via hardware-software co-design by moving the blocks
between ARM Processor and FPGA. We have deployed Linux based operating system
on PS, which takes care of various scheduling and controlling operations. It
also enables GUI development for real-time demonstration.

As discussed in Algorithm 2, Pre-processing stage involves large-size matrix
multiplication and inversion operations. We have modified Xilinx’s existing
matrix multiplication and matrix inversion reference examples to support the
complex number arithmetic since the baseband wireless spectrum is represented
using complex samples. The well-known lower-upper (LU) decomposition method is
selected for matrix inversion. We have included data buffers using block
memory to minimize the repeated data communication between SW and HW and
enable matrix operations of different sizes. While the overall algorithm is
sequential, we parallelize individual operations like Matrix Conjugate
Transpose and Matrix Multiplication on FPGA. Every element in the matrix is
parallelly processed to compute transpose, and every row column dot product in
matrix multiplication is performed in parallel to speed up the computation. In
addition, multiple instances of these IPs are integrated to get the desired
Pre-processing functionality, as shown in Fig. 4. In the end, a normalization
block is included to meet the input requirements of a deep neural network.

Next, we focus on mapping each CNN layer on the FPGA. As shown in Fig. 4, CNN
involves many multiply and accumulation operations on non-contiguous data,
i.e., frequent reading and writing from memory is needed. Depending on the CNN
layer dimensions, it may not be possible to store all weights and input data
in on-chip memory such as block RAM (BRAM) on FPGA due to limited size and
fewer read/write ports. For instance, if we store all the inputs and weights
of the CNN model considered in Table II in the BRAM with single-precision
floating (SPFL) number representation, we need a total of 116.4 mega Byte (Mb)
of BRAM, assuming the CNN output is written directly in the external DDR
memory. Such a large amount of on-chip memory is expensive and may not be
available in most SoC. Using external DDR memory leads to frequent data
communication overhead resulting in high latency. Thus, mapping the CNN layer
on FPGA requires careful sharing of data between external and on-chip memory
to get the desired latency. Underlining architecture can be layer-specific
depending on the various parameters of the CNN layer, resource, and latency
constraints.

Figure 5: Illustrative example demonstrating the memory tiling approach for
realization of the CNN on FPGA.

For the CNN model in Table II, we have explored the memory tiling approach in
which small tiles or blocks of weights and inputs are loaded into the on-chip
memory, and care has been taken to maximize the utilization of data currently
present in on-chip memory. We define 4 parameters, $<T_{o}$, $T_{i}$, $T_{r}$,
$T_{c}>$ i.e., the tiling factors of output channels, input channels, output
rows and output columns, respectively. For easier understanding, we present
illustrative example with tiling parameters $<1,2,2,2>$ in Fig. 5. The input
is of size $4\times 4\times 2$, i.e, 2 channels and it is convoluted with 2
filters of dimension $2\times 2\times 2$. Initially, we initialize an output
tile and load the input tile and weight tile required to compute the output
tile into the on-chip Block RAM. For instance, to compute a $2\times 2\times
1$ output tile, we need a $3\times 3\times 2$ input tile ($I_{tile}$) and one
filter of dimension $2\times 2\times 2$ ($W_{tile}$). After loading, the tiled
convolution is performed between $I_{tile}$ and $W_{tile}$ using a set of
parallel multipliers and adders, referred to as CNN Computations in Fig. 5.
Once all elements of the output tile are computed, the tile is sent back to
the DDR memory, and a similar process is repeated for computing the next
output tile. Here, we have shown the computation of one element of the output
tile. Depending on resource availability, latency constraints, and memory
ports, we can have CNN computations of all elements of a tile or even multiple
tiles in a parallel or serial-parallel fashion.

The tiling approach significantly reduces the on-chip memory requirement for
the CNN model in Table II. For instance, tiling with parameters
$<20,16,20,20>$ needs only 3.35Mb of BRAM (0.24Mb for output tile,1.464Mb for
weight tile and 1.650Mb for input tile) compared to 116.4 Mb in non-tiling
based architecture. This in turn allows efficient optimization of hardware IP
cores via pipelining and unrolling, resulting in significant improvement in
performance. We have explored a wide range of tiling parameters and
implemented these architectures on the ZSoC. Please refer to Section VI-C for
more details. We have explored a similar tiling approach for FC layers as
well. However, experimental analysis shows that FC layers do not need tiling
due to the smaller dimensions of inputs and weights.

The DLWSS model in Table II contains two types of non-linear activations; ReLU
and Sigmoid. As shown in Fig. 6, the realization of the ReLU on the FPGA is
simple as it needs only one comparator and multiplexer. The realization of the
Sigmoid on the FPGA can be done either using LUT based approach or by
polynomial approximation. The LUT-based approach is memory intensive, while
the polynomial-based approach involves many arithmetic and logical operations.
We have realized ReLU on FPGA and Sigmoid on ARM processor based on the
experimental analysis.

Figure 6: ReLU and Sigmoid activations TABLE III: HSCD Study of DLWSS Architecture on ZSoC No. | Blocks on HW | Blocks on SW | Execution Time (s) | {BRAM, DSP, FF, LUT} | {Total Power, Dynamic Power}(Watts)
---|---|---|---|---|---
1 |  | P+CV+FC+A | 30.8 | {0, 0, 0, 0} | {1.6, 1.2}
2 | CV | P+FC+A | 2.87 | {220, 719, 126741, 100250} | {3.205, 2.205}
3 | CV+FC | P+A | 2.869 | {249, 724, 167676, 117382} | {3.373, 2.369}
4 | P+CV | FC+A | 2.863 | {296, 818, 139857, 117738} | {3.391, 2.387}
5 | P+CV+FC | A | 2.863 | {325, 823, 180792, 134870} | {3.540, 2.533}
6 | P+CV+FC+A | - | 2.864 | {325, 853, 183173, 138230} | {3.567, 2.559}

An end-to-end application with a live graphical user interface (GUI) is
developed to demonstrate the real-time WSS on the ZSoC using the proposed
architectures. The application running on a Petalinux-based operating system
deployed on the ARM processor accelerates the computation of pre-processing
algorithm and the DL model on the FPGA. The real-time predictions are obtained
and stored on the SD-card by the application. The GUI application, shown in
Fig. 7, is deployed on the remote server and reads the contents of SD-card at
regular intervals. The GUI is developed using the Tkinter[30] framework and
provides visualization in real-time as and when the architecture predicts the
status of frequency bands in the received digitized spectrum.

Figure 7: GUI for real-time visualisation of SNS based WSS on ZSoC platform.

##  V Performance Analysis and Comparison with OMP

In this section, we compare the functionality of the OMP and DLWSS
architectures for different wireless channels, a wide range of SNRs, and
sparsity levels. We consider the effect of prior knowledge of sparsity on
performance. As discussed in Section II-B, we use $P_{d}^{OB}$and
$P_{d}^{AB}$as the performance metrics for ESS and HSS spectrums,
respectively. For all the experimental results presented in this section, we
consider the floating-point arithmetic based architecture realized on ZSoC
platform.

In Fig. 8, we compare the performance of OMP and DLWSS for three different
channels (AWGN, Rayleigh, and Rician). We assume the prior knowledge of the
spectrum sparsity in the case of OMP. For the ESS spectrum (Fig. 8 (a)), the
DLWSS performs better than OMP with an average performance gain of around 12
%. We also observed that DLWSS and OMP are robust to changes in channel
conditions, which is expected since channel conditions’ impact on the spectrum
sensing is fairly limited. For the HSS spectrum (Fig. 8(b)), the DLWSS
significantly outperforms the OMP algorithm with an average performance
improvement of around $24\%$.

Figure 8: Performance comparison of DLWSS and OMP on (a) ESS and (b) HSS
spectrum for different channel conditions. Sparsity level is known.

In practical deployments, spectrum occupancy changes dynamically with time and
hence, knowledge of spectrum sparsity is not available. In Fig. 9, we analyze
the performance of the DLWSS and OMP algorithms when spectrum sparsity is not
known. It is observed that the proposed DLWSS significantly outperforms the
OMP algorithm at all range of SNRs with an average improvement of around 36.4%
and 31.8% for for ESS and HSS dataset, respectively.

Figure 9: Performance comparison of DLWSS and OMP on (a) ESS and (b) HSS
spectrum for different channel conditions. Sparsity level is not known.

In Fig 10, we compare the impact of sparsity on the performance of DLWSS and
OMP algorithms when spectrum sparsity is unknown. We consider two SNRs: 0 dB
and 10 dB. We trained the DLWSS model using the dataset comprising an equal
number of samples from each sparsity level. It can be observed that the DLWSS
offers superior performance and significantly outperforms OMP as the sparsity
level increases. Thus, results in Fig. 8, Fig. 9 and Fig. 10 confirms the
superiority of the DLWSS over the OMP based approach.

Figure 10: Impact of sparsity level on the performance of the DLWSS and OMP
algorithms.

##  VI Complexity Analysis

In this section, we analyze the complexity of the DLWSS algorithms for
different architectures realized via hardware-software co-design, various word
lengths, and memory tiling approaches.

###  VI-A Hardware Software Co-Design (HSCD)

A heterogeneous SoC such as Zynq SoC from Xilinx contains ARM Cores as the
processing system (SW) and FPGA as the programmable logic (HW). An integral
aspect of developing an efficient mapping of the DLWSS on the ZSoC is to
design an optimal HSCD strategy to facilitate functionally accurate
architecture for the desired latency and resource constraints. Specifically,
we need to decide how to partition the algorithm between HW and SW and
minimize the data communication overhead between them. The HSCD is important
since sequential operations, scheduling tasks, and GUI are preferred on SW. At
the same time, FPGA can efficiently handle the task, which can be accelerated
via parallel processing. However, in certain situations, serial tasks are
preferred on HW, while parallel tasks are preferred on SW to avoid data
communication overhead between SW and HW. Furthermore, some operations may
offer speed up on HW. Still, such speed up may not be significant compared to
the algorithm’s overall execution time; hence, realizing such tasks on SW can
reduce FPGA size, cost, and power consumption. Such trade-offs demand a
detailed study of various HSCD architectures to design an architecture that
meets the cost, latency, and power constraints.

Table III shows the results of our HSCD study for the DLWSS architecture
comprising pre-processing (P), convolution (CV), fully connected (FC), and
activations (A) blocks. As shown in Row 1, mapping the entire architecture on
SW results in a high execution time of 30.8 seconds ($s$). As we shift more
blocks to HW, the execution time is reduced while resource and power
consumption increase (Rows 2-6). The CV layer is the most computational
complex unit in the DLWSS, and realizing it on the HW results in a significant
reduction in the execution time from 30.8$s$ to 2.87$s$.

Since the DLWSS model has fewer FC layers as compared to the CV layers,
shifting the FC layer to HW does not offer substantial improvement in
execution time compared to 2.7%, 0.6%, 9.4% and 7.8% increase in BRAM, DSP, FF
and LUT utilization, respectively. Similarly, shifting the pre-processing
algorithm (P) on HW leads to improvement in execution time of around 7 $ms$
compared to increase in the BRAM, DSP and LUT usage by 7%, 11% and 8%,
respectively with respect to previous architecture in Row 3. As we move FC
block to HW (Row 5), we notice that the impact of FC block is the same as seen
in Row 2, which is expected as adding the FC block adds a constant time and
constant resource utilization for all cases. In Row 6, we move Sigmoid based
activation block to HW which results in increase in the DSP and LUT
utilization by 3% and 1.5%, respectively, with around 1 $ms$ improvement in
execution time. Note that as we gradually shift more and more blocks from SW
to HW, the overall power consumption increases from 3.205W (Row 2) to 3.567W
(Row 6). Thus, from HSCD perspective, it is better to keep all blocks except
CV layers on the SW given high resource penalty in HW for small gain in
execution time.

###  VI-B Word Length Optimization

Conventionally, HW realization of the algorithm in floating-point arithmetic
offers good functional accuracy but incurs high resource utilization, power
consumption, and execution time. Since the extremely-large dynamic range
offered by floating-point arithmetic may not be needed for all the sub-blocks
of the algorithm, fixed-point arithmetic can potentially offer a significant
reduction in resource, power, and execution time without compromising on the
functional accuracy. In wireless applications such as DLWSS, the dynamic range
of inputs, weights, and activation is limited due to analog-to-digital
converters (ADCs); hence, fixed-point architectures are preferred. In this
section, we discuss the selection of appropriate word length for the part of
the algorithm realized on the HW, i.e., FPGA, and its impact on the functional
accuracy, resource utilization, execution time, and power consumption.

We use $W$ bits to represent each number in fixed point quantization. Out of
$W$ bits, we use $I$ bits to represent the integer part and ($W-I$) bits to
represent the fractional part. For example, the fixed-point representation of
a number, 3.25, needs only 6 bits with $I=3$ compared to 64 bits in double-
precision floating point (DPFP), 32 bits in single-precision floating point
(SPFP), and 16 bits in half-precision floating-point (HPFP). Thus, depending
on the dynamic range of the given variable, the appropriate selection of $W$
and $I$ can avoid loss in functional accuracy. To identify appropriate values
of $W$ and $I$ for the DLWSS architecture, we analyzed the dynamic range of
inputs, outputs, and intermediate outputs of various sub-blocks. For instance,
Table IV shows the analysis to determine the optimal integer width for
hardware realization of CV and FC layers of the DLWSS, where we infer the
model on samples of the dataset to estimate the ranges of intermediate
activations and the model weights. It can be observed that the minimum value
of $I$ is 9 and 2 bits for activation and weights, respectively. The value of
$W$ depends on the number of bits for accurate fractional number
representation to get the desired functional performance, and we select them
via heuristic experiments.

TABLE IV: Integer Word Length Selection for DLWSS Type | Layer | Minimum Value | Maximum Value | $I_{min}$
---|---|---|---|---
Activation | CV | -77.061 | 199.309 | 9
Activation | FC | -86.594 | 158.975 | 9
Weight | CV | -0.4812 | 0.9561 | 2
Weight | FC | -0.0661 | 0.0219 | 2
TABLE V: Functionality and Complexity Analysis of DLWSS Architectures for
Different Word Lengths at 10 dB SNR

No. |  | Activation
---
WL: <$W_{a},I_{a}$>
| Weights
---
WL: <$W_{w},I_{w}$>
Execution Time (s) |  ESS: $P_{d}^{OB}$ |  HSS: $P_{d}^{AB}$ | {BRAM, DSP, FF, LUT} | {Tot. Power, Dyn. Power}
1 | SPFP | SPFP | 2.28 | 100 | 100 | {256,879,150107,117023} | {4.218,3.944}
2 | HPFP | HPFP | 2.11 | 100 | 99.97 | {213,719,98614,74952} | {2.688,2.448}
3 | <29,9> | <16,2> | 2.28 | 100 | 100 | {248,399,75935,81119} | {2.523,2.285}
4 | <28,9> | <16,2> | 2.28 | 100 | 100 | {248,399,71907,74381} | {2.484,2.247}
5 | <27,9> | <16,2> | 2.30 | 100 | 99.98 | {243,399,59641,50259} | {2.353,2.119}
6 | <26,9> | <16,2> | 2.30 | 100 | 99.94 | {240,399,59641,50259} | {2.351,2.114}
7 | <25,9> | <16,2> | 2.30 | 100 | 99.94 | {240,399,53926,41443} | {2.299,2.065}
8 | <24,9> | <16,2> | 2.30 | 100 | 98.78 | {232,399,53926,41443} | {2.294,2.061}
9 | <23,9> | <16,2> | 2.28 | 87.38 | 79.45 | {232,399,53926,41443} | {2.294,2.061}
10 | <22,9> | <16,2> | 2.28 | 70.96 | 57.46 | {232,399,53926,41443} | {2.294,2.061}

We have designed and implemented DLWSS architectures of various WLs on ZSoC
and analyzed their performance to identify appropriate WL for fractional
number representation. In Table V, we compare 10 different DLWSS architectures
with fixed integer WL of 9 for activation and 2 for weights. Here, we have
fixed the WL of weights to <16,2> to identify the WL for activation. A similar
process is done to identify the WL of weights by fixing the WL of activation.
Corresponding details are omitted to avoid repetition of discussion. All these
results are evaluated for a fixed tile size of <20,16,20,20>. Please refer to
Section VI-C for more details about impact of tile size on resource
utilization.

To begin with, we consider DLWSS with SPFL WL in Row 1 of Table V. As
expected, it offers excellent functional accuracy in terms of the chosen
performance metrics. In Row 2, we consider DLWSS with HPFP WL. It offers
nearly identical functional performance as that of the SPFP architecture with
savings of around ${7\%}$ in BRAM, 17% in DSP, and 46 in % LUT and power
savings of ${0.47W}$. Also, there is a significant reduction of 0.17$s$ in
execution time.

Next, we have explored eight different DLWSS architectures via fixed-point
quantization, and corresponding results are given in Rows 3-10 of Table V. As
expected, there is a slight degradation in functional accuracy as WL
decreases. The DLWSS architecture with the WL of <23,9> or below in Rows 9-10
suffers from significant degradation in performance and should be avoided. The
DLWSS architecture in Row 7 with a WL of 25 offers functional performance same
as that of floating-point architectures with more than 50% savings in DSP,
FFs, and LUTs over HPFP architecture in Row 2 and over 60% savings in DSP,
FFs, and LUTs over SPFP architecture in Row 1. These savings can be further
improved using the DLWSS architecture with WL of <24,9> in Row 8 with minor
degradation in performance. Note that we have assumed that the inputs and
outputs are in SPFP format; hence, additional WL conversion inside the
architecture is needed. We can reduce the execution time further if the input
and output WLs are the same as the rest of the architectures.

In Table V, we have fixed the SNR to 10 dB. To analyze the architecture’s
performance at different SNRs, we have compared the $P_{d}^{OB}$for ESS and
$P_{d}^{AB}$for HSS for SNRs ranging from -20dB to 10dB in Fig. 11. It can be
observed that the performance degrades at higher SNR due to insufficient WL.

Figure 11: Impact of the WL on the functional accuracy of the DLWSS
architecture.

###  VI-C Impact of Memory Tiling Approach

As discussed in Section IV, memory tiling is essential to enable an efficient
implementation of the memory-intensive convolution layer operations on HW. In
this section, we discuss the impact of tiling size on the performance of the
DLWSS architecture realized on the SW and HW. In Fig. 12, we compare the
acceleration factor, i.e. the ratio of execution time on SW and HW, for
different tiling sizes on SW and HW realizations. It can be observed that the
acceleration factor increases with the increase in tiling size of the HW
architecture for a given tiling size of SW realization. Such behavior can be
attributed to the following:

  1. 1.

Higher tile size on HW means fewer accesses to external DDR memory since more
data is buffered in on-chip block RAM.

  2. 2.

Higher tile size allows more opportunities for intra-tile parallelization i.e.
dot products within a tile can be computed in parallel on HW.

Interestingly, an increase in the tiling size of the SW realization results in
degradation in the execution time of the DLWSS on SW. For instance,
acceleration factor is higher for SW tiling size of $<20,16,20,20>$ compared
to SW tiling size of $<2,2,2,2>$. This behavior is exactly reverse compared to
that of HW, and one possible reason is data caching in SW realization. Smaller
tile size allows tiles to be cached in the local data cache memory of ARM
cores, thereby limiting the number of accesses to external DDR memory. Since
cache size is significantly smaller than on-chip BRAM on HW, larger tile size
results in frequent cache flush requirements, resulting in execution time
degradation.

Though the tiling size does not impact functional accuracy, a larger tiling
size leads to higher resource utilization, as shown in Table VI. This is
because a larger tile size needs a higher amount of on-chip memory and enables
parallel computations due to the availability of more data on the chip. The
use of fixed-point arithmetic can help to reduce resource utilization
significantly. Thus, combining hardware-software co-design, WL, and tiling
parameters are vital to meet the given resource and execution time
constraints.

Figure 12: Impact of tile size on the execution time acceleration factor.
TABLE VI: Effect of Tiling and WL on resource utilization

Resources | Word-Length | Memory Tiling Parameters
---|---|---
<2,2,2,2> | <8,8,8,8> | <16,16,16,16> | <16,16,20,20> | <20,16,20,20>
BRAM | Floating Point | 14.9 | 67.9 | 219.96 | 219.96 | 255.72
Fixed Point | 14.5 | 63.5 | 203.5 | 203.5 | 239.5
DSP | Floating Point | 82 | 239 | 719 | 719 | 879
Fixed Point | 81 | 143 | 335 | 335 | 399
LUT | Floating Point | 35107 | 49097 | 100249 | 100687 | 117016
Fixed Point | 35652 | 37656 | 46824 | 47630 | 59259

##  VII Conclusions

In this paper, we designed and implemented statistical orthogonal matching
pursuit (OMP), and deep learning (DL) based algorithms on Zynq System-on-chip
(SoC) for wideband sensing applications. We have provided in-depth
experimental results and complexity comparisons among various architectures
obtained via hardware-software co-design, word-length optimization, and memory
tiling. Specifically, we demonstrated the drawbacks of conventional OMP
algorithms, such as poor performance at a low signal-to-noise ratio (SNR) and
the need for prior knowledge of sparsity. These drawbacks are addressed via a
novel DL-based approach. However, the DL architecture’s high resource
utilization and execution time is a concern. In the future, we also plan to
explore Neural architecture search (NAS) along with quantized model training
to reduce the complexity of DL architecture. We also plan to integrate the
proposed architecture with analog-front-end for experiments with real-radio
signals.

## References

  * [1] A. Ali and W. Hamouda, “Advances on spectrum sensing for cognitive radio networks: Theory and applications,” _IEEE Communications Surveys Tutorials_ , vol. 19, no. 2, pp. 1277–1304, 2017.
  * [2] C.-P. Yen, Y. Tsai, and X. Wang, “Wideband spectrum sensing based on sub-nyquist sampling,” _IEEE Transactions on Signal Processing_ , vol. 61, no. 12, pp. 3028–3040, 2013.
  * [3] M. Labib, V. Marojevic, J. H. Reed, and A. I. Zaghloul, “Extending lte into the unlicensed spectrum: Technical analysis of the proposed variants,” _IEEE Communications Standards Magazine_ , vol. 1, no. 4, pp. 31–39, 2017\.
  * [4] W. S. H. M. W. Ahmad, N. A. M. Radzi, F. S. Samidi, A. Ismail, F. Abdullah, M. Z. Jamaludin, and M. N. Zakaria, “5g technology: Towards dynamic spectrum sharing using cognitive radio networks,” _IEEE Access_ , vol. 8, pp. 14 460–14 488, 2020.
  * [5] L. Zhang, M. Xiao, G. Wu, M. Alam, Y.-C. Liang, and S. Li, “A survey of advanced techniques for spectrum sharing in 5g networks,” _IEEE Wireless Communications_ , vol. 24, no. 5, pp. 44–51, 2017.
  * [6] M. Matinmikko-Blue, S. Yrjölä, V. Seppänen, P. Ahokangas, H. Hämmäinen, and M. Latva-Aho, “Analysis of spectrum valuation elements for local 5g networks: Case study of 3.5-ghz band,” _IEEE Transactions on Cognitive Communications and Networking_ , vol. 5, no. 3, pp. 741–753, 2019.
  * [7] V. Valenta, R. Maršálek, G. Baudoin, M. Villegas, M. Suarez, and F. Robert, “Survey on spectrum utilization in europe: Measurements, analyses and observations,” in _2010 Proceedings of the Fifth International Conference on Cognitive Radio Oriented Wireless Networks and Communications_ , 2010, pp. 1–5.
  * [8] Y. Chen and H.-S. Oh, “A survey of measurement-based spectrum occupancy modeling for cognitive radios,” _IEEE Communications Surveys Tutorials_ , vol. 18, no. 1, pp. 848–859, 2016.
  * [9] M. López-Benítez and F. Casadevall, “Space-dimension models of spectrum usage for cognitive radio networks,” _IEEE Transactions on Vehicular Technology_ , vol. 66, no. 1, pp. 306–320, 2017.
  * [10] S. Stein Ioushua, O. Yair, D. Cohen, and Y. C. Eldar, “Cascade: Compressed carrier and doa estimation,” _IEEE Transactions on Signal Processing_ , vol. 65, no. 10, pp. 2645–2658, 2017.
  * [11] E. Crespo Marques, N. Maciel, L. Naviner, H. Cai, and J. Yang, “A review of sparse recovery algorithms,” _IEEE Access_ , vol. 7, pp. 1300–1322, 2019\.
  * [12] T. T. Cai and L. Wang, “Orthogonal matching pursuit for sparse signal recovery with noise,” _IEEE Transactions on Information Theory_ , vol. 57, no. 7, pp. 4680–4688, 2011.
  * [13] E. Crespo Marques, N. Maciel, L. Naviner, H. Cai, and J. Yang, “A review of sparse recovery algorithms,” _IEEE Access_ , vol. 7, pp. 1300–1322, 2019\.
  * [14] M. Yang and F. de Hoog, “Orthogonal matching pursuit with thresholding and its application in compressive sensing,” _IEEE Transactions on Signal Processing_ , vol. 63, no. 20, pp. 5479–5486, 2015.
  * [15] S. Zhang, J. Wu, D. Chen, S. Li, B. Yu, and J. Qu, “Fast frequency-domain compressed sensing analysis for high-density super-resolution imaging using orthogonal matching pursuit,” _IEEE Photonics Journal_ , vol. 11, no. 1, pp. 1–8, 2019.
  * [16] J. A. Becerra, M. J. Madero-Ayora, J. Reina-Tosina, C. Crespo-Cadenas, J. García-Frías, and G. Arce, “A doubly orthogonal matching pursuit algorithm for sparse predistortion of power amplifiers,” _IEEE Microwave and Wireless Components Letters_ , vol. 28, no. 8, pp. 726–728, 2018\.
  * [17] B. Mamandipoor, D. Ramasamy, and U. Madhow, “Newtonized orthogonal matching pursuit: Frequency estimation over the continuum,” _IEEE Transactions on Signal Processing_ , vol. 64, no. 19, pp. 5066–5081, 2016.
  * [18] H. Palangi, R. K. Ward, and L. Deng, “Convolutional deep stacking networks for distributed compressive sensing,” _Signal Process._ , vol. 131, pp. 181–189, 2017.
  * [19] H. Palangi, R. Ward, and L. Deng, “Using deep stacking network to improve structured compressed sensing with multiple measurement vectors,” in _2013 IEEE International Conference on Acoustics, Speech and Signal Processing_ , 2013, pp. 3337–3341.
  * [20] ——, “Distributed compressive sensing: A deep learning approach,” _IEEE Transactions on Signal Processing_ , vol. 64, no. 17, pp. 4504–4518, 2016.
  * [21] Y. Nakayama, D. Hisano, and K. Maruta, “Adaptive c-ran architecture with moving nodes toward beyond the 5g era,” _IEEE Network_ , vol. 34, no. 4, pp. 249–255, 2020.
  * [22] F. D. L. Coutinho, J. D. Domingues, P. M. C. Marques, S. S. Pereira, H. S. Silva, and A. S. R. Oliveira, “Towards the flexible and efficient implementation of the 5g-nr ran physical layer,” in _2021 IEEE Radio and Wireless Symposium (RWS)_ , 2021, pp. 130–132.
  * [23] R. Rajesh, “Source code and datasets,” https://drive.google.com/drive/folders/1NjVnkbUNwwITOi77UVTNq2v50sQ9_z7C?usp=sharing, 2022, [Online; accessed 5-August-2022].
  * [24] T. T. Cai and L. Wang, “Orthogonal matching pursuit for sparse signal recovery with noise,” _IEEE Transactions on Information Theory_ , vol. 57, no. 7, pp. 4680–4688, 2011.
  * [25] E. Crespo Marques, N. Maciel, L. Naviner, H. Cai, and J. Yang, “A review of sparse recovery algorithms,” _IEEE Access_ , vol. 7, pp. 1300–1322, 2019\.
  * [26] Y. Pati, R. Rezaiifar, and P. Krishnaprasad, “Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition,” in _Proceedings of 27th Asilomar Conference on Signals, Systems and Computers_ , 1993, pp. 40–44 vol.1.
  * [27] Z. Quan, S. Cui, A. H. Sayed, and H. V. Poor, “Wideband spectrum sensing in cognitive radio networks,” in _2008 IEEE International Conference on Communications_ , 2008, pp. 901–906.
  * [28] Y. L. Polo, Y. Wang, A. Pandharipande, and G. Leus, “Compressive wide-band spectrum sensing,” in _2009 IEEE International Conference on Acoustics, Speech and Signal Processing_ , 2009, pp. 2337–2340.
  * [29] S. Chandhok, H. Joshi, A. V. Subramanyam, and S. J. Darak, “Novel deep learning framework for wideband spectrum characterization at sub-nyquist rate,” 2020.
  * [30] F. Lundh, “An introduction to tkinter,” _URL: www. pythonware. com/library/tkinter/introduction/index. htm_ , 1999.