provided by an on-premises edge tier and the shared cloud tier only serves as
an exhaustive pattern matching search utility. We propose _Smartness at Edge_
(SAED mechanism) that offers intelligence in the form of semantic and
personalized search at the edge tier while maintaining privacy of the search
on the cloud tier. At the edge tier, SAED uses a knowledge-based lexical
database to expand the query and cover its semantics. SAED personalizes the
search via an RNN model that can learn the user’s interest. A word embedding
model is used to retrieve documents based on their semantic relevance to the
search query.

### 2 Problem Statement

Ideally, data owners desire a privacy-preserving cloud service that offers
semantic and personalized searchability in a real-time manner, without
overwhelming their resource-constrained (thin) client devices (e.g.,
smartphones). A large body of research has been undertaken on privacy-
preserving enterprise search services in the cloud [55, 57, 103, 53, 3] whose
goals are to protect user’s sensitive data from internal and external
attackers. However, most of these works fall short in retrieving search
results that are semantically relevant to the context and user’s interest
(i.e., personalized search) [3, 53]. In addition, these works often rely on
the client device and impose significant overhead on it to perform a secure
query processing or to encrypt/decrypt user documents.

To satisfy all of the aforementioned desires of a particular user, our main
idea in this research is to separate the intelligence aspect of the enterprise
search from its pattern matching aspect.

### 3 SAED: Smart Edge-Leveraged Enterprise Search System

#### 1 Architectural Overview

In this part, we provide a bird-eye view of the SAED system, that enables
intelligent and secure enterprise search on the cloud. The system is
structured around three tiers, shown in Figure 2, and explained as follows:

  * •

_Client tier_ (e.g., smartphone, tablet) contains a lightweight application
that provides a user interface for uploading documents and to search over them
in the cloud. Datasets are either uploaded by the user or by the organization
that owns the data.

  * •

_Edge tier_ extracts representative keywords of the documents being uploaded
to the cloud tier and builds an index on the cloud tier. Upon receiving a
search query from the client tier, the SAED system on the edge tier offers
intelligence by considering the query semantics and the user’s interest. The
edge tier is located in the client’s premises, hence, deemed as an honest and
secure system. To offer a secure enterprise search service, the edge tier
encrypts both the uploaded data and the search query. In addition, it decrypts
the result set before delivering it back to the client tier.

  * •

_Cloud tier_ contains numerous high-end servers that are utilized for storing
(encrypted) data and performing the large-scale computation required to
exhaustively search against the index [53, 3]. The index can be clustered
based on the underlying topics of its keywords (please refer to our prior
works [3, 1] for further details).

Figure 1: Architectural overview of the SAED system within edge tier and as
part of the three-tier enterprise search service. SAED provides semantic
search via identifying the query context and combining that with the user’s
interests. Then, Query Expansion and Weighting unit of SAED, respectively,
incorporate the semantic and assure the relevancy of the results. Solid and
dashed lines indicate the interactions from user to the cloud tier and from
the cloud tier to the user respectively.

In Figure 1, we depict the components of SAED and show the interactions
between them. At first, a user-provided search query is received by the _Query
Handler_ that keeps track of the user’s search history and initializes the
_Context Identifier_ unit whose job is to extract the context and disambiguate
the query phrase. Then, according to the extracted context, the query is
proactively expanded by the _Query Expansion_ unit and a _query set_ is
constructed. To achieve the personalized search, the _Interest Detector_ unit
of SAED leverages the user’s search history to recognize his/her interest and
weight each element of the query set (i.e., expanded queries) based on its
relatedness to the user interest. Once the pattern matching phase is
accomplished on the cloud tier, the resulted documents are returned to SAED on
the edge tier. Next, the _Ranking Unit_ utilizes the assigned weights to order
the retrieved documents based on their relevance to the user’s interest and
generates a retrieved document list, denoted as $D_{\theta}$, that is sent to
the user’s device. In the next parts, we elaborate on each unit of the SAED
system.

#### 2 Query Context Identification

Identifying the context of a given search phrase is vital to navigate the
search to the semantics intended by the user. Considering the example of cloud
computing as the search query, without a proper context identification the
returned document set can potentially include documents about sky and climate,
whereas, an efficient context identifier can recognize the right semantic and
navigate the search to the topics around distributed, edge, fog, and cloud
computing. In fact, identifying the context helps the Query Expansion unit to
form a query set diversified around relevant keywords that semantically
represent the search query and subsequently improve the relevancy of the
results.

Prior context identification works (e.g., [104, 105, 62]) have the following
shortcomings: _first,_ they often assume each keyword has the same importance
in the query and recognize the query context via averaging the embeddings of
its keywords. However, not all keywords in a query necessarily help in
identifying the context. For example, the keyword various in various cloud
providers does not bring any significance to the context and can be
eliminated. _Second,_ the embedding methods used by the existing works always
provide the same representation for a given keyword, irrespective of the
underlying context. This is particularly problematic for ambiguous keywords
whose meaning vary based on the query context. For instance, the embedding of
cloud in the aforementioned example should be different when it is used along
with the computing as opposed to when it is used along with the weather in a
given query. _Third,_ existing methods only consider the embeddings of the
common keywords, while discarding most of the name-entities (e.g., names and
locations) that do not exist in the vocabulary of Word2Vec [64, 106]. For
instance, consider best selling books of J.K. Rowling as the query; Book and
Sell are identified as the query context and J.K. Rowling is discarded.
However, our analysis suggests that the context of a short query phrase often
has contextual association with the discarded name-entities.

To overcome the shortcomings and identify the actual context of a given query,
we propose to take a holistic approach and extract the _semantic across query
keywords, proportionate to the importance of each keyword_. The main output of
the Context Identification unit is a set of keywords, denoted as _C_ , that
collectively represent the context of the query.

Specifically, to eliminate unimportant keywords that do not contribute to the
semantic of query $Q$, the Context Identification unit utilizes _Yake_ [107],
which is a unsupervised keyword extractor that discards unimportant keywords
of the query. The remaining keywords (i.e., the trimmed query, denoted as the
$Q^{\prime}$ set) are considered for context identification. To learn the true
semantic of $Q^{\prime}$, the unit leverages the Lesk algorithm [106] of
WordNet to disambiguate each keyword $q\in Q^{\prime}$. Lesk algorithm works
based on the fact that keywords in a given sentence (query) tend to imply a
certain topic. For keyword $q$, Lesk can determine its true semantics via
comparing the dictionary definitions of $q$ against other keywords in
$Q^{\prime}$ (i.e., $Q^{\prime}-\\{q\\}$). Let $c_{q}$ be the set of keywords
representing the context of $q$. Then, the context of $Q$ is determined as
$C=\cup_{\forall q\in Q^{\prime}}c_{q}$. Lastly, the Context Identifier
recognizes name-entities from $Q$ using WordNet and considers them as part of
the context, but in a separate set, denoted as $N$. The reason for considering
a separate set is that we apply a different treatment on $N$ and $C$ in the
other units of SAED.

Input : query $Q$

Output : $C$: set of keywords representing context of $Q$,

$N$: set of name-entity in $Q$

1

2

3Function contextIdentification($Q$):

4 $Q^{\prime}\leftarrow$ extract keywords from $Q$ using Yake alg.

5

6 foreach  _$q\in Q$_ do

7

8 if  _$q\in$ Name-entity_ then

9 $N\leftarrow N\cup\\{q\\}$

10

11 end if

12 else

13

14 if  _$q\in Q^{\prime}$_ then

15 $E_{q}\leftarrow$ define $q$ based on $Q^{\prime}-q$ using Lesk alg.

16 $c\leftarrow$ extract set of keywords of $E_{q}$ using Yake alg.

17 $C\leftarrow C\cup c$

18

19 end if

20

21 end if

22

23 end foreach

24

25 return $C,N$

26

27 end

28

Algorithm 4 Pseudo-code to detect the context of a given query in the Context
Identification unit of SAED.

Algorithm 4 provides a pseudo-code for identifying the context of incoming
query $Q$. The outputs of the pseudo-code are two sets, namely $C$ and $N$,
that collectively represent the context of $Q$. In Step 2 of the pseudo-code,
Yake algorithm is used to filter $Q$ by extracting its important keywords and
generate the $Q^{\prime}$ set. Name-entities of $Q$ are identified by checking
against WordNet and form the set $N$ (Steps 4–6). Next, in Steps 8–12, for
each keyword $q\in Q^{\prime}$, the Lesk algorithm is employed to disambiguate
$q$ and find its true definition with respect to the rest of keywords in
$Q^{\prime}$. Important keywords of the definitions form the context set ($C$)
for $Q$.

#### 3 Query Expansion Unit

The _Query Expansion_ unit is in charge of proactively expanding the query
keywords based on their relevant synonyms that are in line with their
identified context. Neglecting the query context and blindly considering all
the synonyms, as achieved in [104, 105, 62, 3], leads to finding irrelevant
documents. Accordingly, the unit leverages the context of $Q$ (i.e., $C$ and
$N$) to only find the set of synonyms, denoted as $P$, that are semantically
close to the query context.

Word2Vec [63] is a shallow neural network model that can be trained to
generate vector representation of keywords, such that the cosine similarity of
two given keywords indicates the semantic similarity between them.
Accordingly, to proactively expand each keyword $q\in Q$, the Query Expansion
unit instruments Word2Vec, pre-trained with Google News dataset [108], to form
the set of nominated synonyms, denoted as $s_{q}$. Let $s_{q}^{i}$ be a
synonym of $q$ (i.e., $s_{q}^{i}\in s_{q}$). Then, the similarity of
$s_{q}^{i}$ and the query context, denoted as $sim(s_{q}^{i},C)$, is defined
based on the sum of similarities with each element of $C$, as shown in
Equation 1.

$sim(s_{q}^{i},C)=\sum_{\forall C_{j}\in C}sim(s_{q}^{i},C_{j})$ (1)

Then, $s_{q}^{i}$ is chosen as an element of $P$, only if it is semantically
close enough to the query context. To determine the sufficient closeness, we
consider $sim(s_{q}^{i},C)$ to be greater than the mean of the pair-wise
similarity across all members of $s_{q}$ (i.e., $sim(s_{q}^{i},C)>\mu_{\forall
q\forall j}(sim(s_{q}^{j},C))$). We note that because the elements of $C$ and
$N$ represent the context of $Q$, they as well are added to $P$.

Algorithm 5 provides a high level pseudo-code for generating the expanded
query set $P$. In Steps 2–7 of the pseudo-code, the synonym set for each $q$
is generated. Next, the similarity between each word $s_{q}^{i}$ and $C$ is
calculated. The similarity values are used to calculate the mean similarity of
all nominated queries in Step 8. In Steps 9–15, expanded query set $P$ is
formed by including nominated synonyms whose semantic closeness is greater
than $\mu$. Lastly, in Step 16, set $P$ is expanded by including context set
and name-entities.

Input : $Q$, $C,N$

Output : $P$: the expanded query set

1

2

3Function  QueryExpansion($Q$, $C$, $N$)

4 foreach  _$q\in Q$_ do

5 $s_{q}\leftarrow$ use WordNet to obtain synonym set of $q$

6 foreach  _$s_{q}^{i}\in s_{q}$_ do

7 $sim(s_{q}^{i},C)\leftarrow\displaystyle\sum_{\forall C_{j}\in
C}sim(s_{q}^{i},C_{j})$

8

9 end foreach

10

11 end foreach

12 $\mu\leftarrow$ calculate mean $sim(s_{q}^{j},C)$ across all $q\in
Q,\forall s_{q}^{j}\in s_{q}$

13 foreach  _$q\in Q$_ do

14 foreach  _$s_{q}^{i}\in s_{q}$_ do

15 if  _$sim(s_{q}^{i},C) >\mu$_ then

16 Add $s_{q}^{i}$ to set $P$

17 end if

18

19 end foreach

20

21 end foreach

22 $P\leftarrow P\cup C\cup N$

23 return $P$

24

25 end

26

27

Algorithm 5 Pseudo-code to expand query based on the context in the Query
Expansion unit of SAED

#### 4 User Interest Detection

Detecting the user’s search interest is essential to deliver personalized
search. In SAED, interest detection is achieved by analyzing two factors: (A)
the user’s search history; and (B) the user’s reaction to the retrieved
results of prior search queries. This can be detected based on the results
chosen by the user or the time spent for browsing them.

Let $\Delta^{\prime}$ represent the whole resulted documents that are sent to
the user and $\tau$ represent the documents where the user is interested in.
We have $\tau\subseteq\Delta_{\prime}$. Accordingly, the user’s interest can
be derived from the topics of $\tau$. The Interest Detector unit uses an
existing document classification model [109], operating based on Naïve Biased
(NB) method, to determine the topics of $\tau$, denoted as $t_{\tau}$. We also
perform majority voting on $t_{\tau}$ to find the user’s main interest. The
process is repeated to store n-prior search interests data of the user. The
data is characterized as sequential as it is harvested from each successful
search. By analyzing the user’s prior search interests, the edge tier trains a
recurrent neural network-based prediction model [110] that can predict the
user’s search interest. In case of SAED, as the data does not contain long
dependency and to keep the model simple and to maintain real-timeliness,
instead of a stacked (i.e., deeper) model, we feed the harvested user-specific
historical search data to train a many-to-one vanilla RNN model [111].

#### 5 Weighting Unit

Once SAED learns the user interest, the next step to accomplish a context-
aware and personalized enterprise search is to determine the closeness of
contextually-expanded queries (i.e., elements of $P$) to the user’s interest.
In fact, not all expanded queries have the same significance in the
interpretation of the query. Accordingly, the objective of the _Weighting
unit_ is defined as quantifying the closeness of each expanded query to the
user’s interest. Later, upon completion of the search operation on the cloud
tier, the weights are used by the _Ranking unit_ of SAED to prune and sort the
result set.

Prior weighting schemes (e.g.,  [53, 3, 62, 59, 105]) often use the word
frequency-based approach (e.g., TF-IDF [3]) and discard the user interests.
Alternatively, the weighting procedure of SAED quantifies the importance of
each expanded query $p\in P$ based on two factors: (A) The _type_ of $p$,
which means if it directly belongs to the context ($C$ and $N$ sets) or is
derived from them; and (B) The _semantic similarity_ of $p$ to the user
interest.

In particular, those elements of $P$ that directly represent the query context
or name-entities (i.e., $\forall p|p\in P\cap(C\cup N)$) explicitly indicate
the user’s search intention, hence, weighting them should be carried out
irrespective of the user interest. A deeper analysis indicates that name-
entities that potentially exist in a query represent the search intention,
thus, biasing the search results to them can lead to a higher user
satisfaction. As such, the highest weight is assigned to $\forall p|p\in(P\cap
N$). The highest weight is determined by the domain expert, however, in the
experiments we consider it as $\eta_{max}=1$. We define the contribution of
$q\in Q$ as the ratio of the number of keywords added to $C$ because of $q$
(denoted $C_{q}$) to the cardinality of $C$. Let $\eta_{p}$ denote the weight
of $p\in P$. Then, for those elements of $P$ that are in the query context
(i.e., $\forall p\in(P\cap C)$), $\eta_{p}$ is calculated based on the
contribution of the query keyword $q$ corresponding to $p$. Equation 2
formally represents how $\eta_{p}$ is calculated.

$\eta_{p}=\frac{\eta_{max}\cdotp|C_{q}|}{|C|}$ (2)

The weight assignment for those $p$ that are derived from elements of $C$, as
explained in Section 3, (i.e., $\forall p|p\in P-(C\cup N)$) is carried out
via considering semantic similarity of $p$ with the user interest $\theta$.
That is, $\eta_{p}=sim(p,\theta)$.

Algorithm 6 provides the high level pseudo-code for distributing weight to the
expanded query set $P$. The algorithm considers $P,C,N$, and highest weight
value $\eta$ as the inputs. After assigning weights to $\forall p$
iteratively, it returns the weights mapped with corresponding $p$ as a hash
map denoted as $\varpi$. In Step 2 of the pseudo code, $\theta$ gets the
user’s search interest that is identified by leveraging a pre-trained document
classifier and a vanilla RNN model. In the following Step, hash map $\varpi$
is initialized to contain the weights that mapped with corresponding $p$.

Overall, in Steps 4–15, weight of each $p$ denoted as $\varpi_{p}$ is
calculated according to its type. Specifically, in Steps 5–7, $\varpi_{p}$,
where $p\in(P\cap N)$ is set by directly assigned $\eta$. In Steps 8–11, $p$,
where $p\in(P\cap C)$ is weighted based on its contribution towards context
$C$. At first, $q$ is determined that generates $p$ and weight $\varpi_{p}$ of
$p$ is calculated by the ratio between $\eta$ and total number of keywords
added in $C$ for the corresponding $q$. In the following Steps (12–14),
$\varpi_{p}$, where $p\in P-(C\cup N)$ is calculated by its semantic
similarity with $\theta$. Lastly, the algorithm is finished by returning hash
map $\varpi$ filled with weights corresponding to their $q$ (Step 16).

Input : $P$, $C$, $N$, $\eta$

Output : $\varpi$

1

2

3Function  weighting($P$, $C$, $N$, $\eta$)

4 $\theta\leftarrow$ predict a user’s search interest

5 $\varpi\leftarrow$ initialize hash map to store weights mapped with their
corresponding keywords

6

7 foreach  _$p\in P$_ do

8 if  _ $p\in N$_ then

9 $\varpi_{p}\leftarrow\eta$

10 end if

11

12 else if  _$p\in C$_ then

13 $\varpi_{p}\leftarrow\frac{\eta_{max}\cdotp|C_{q}|}{|C|}$

14

15 end if

16 else

17 $\varpi_{p}\leftarrow$ sim($p,\theta$) /*Compute similarity and store it in
hash map */

18

19 end if

20

21 end foreach

22

23 return $\varpi$

24

25 end

26

Algorithm 6 Pseudo-code to weight expanded query

#### 6 Ranking Unit

Once the expanded query set $P$ is formed, the cloud tier performs string
matching for each $p\in P$ across the index structure. We note that, if the
user chooses to perform a secure search, the elements of $P$ are encrypted
before delivered to the cloud tier. In addition, in our prior works [1], we
proposed methods for the cloud tier to cluster the index structure and perform
the pattern matching only on the clusters that are relevant to the query.

The cloud tier returns the resulted document set, denoted as $\Delta$, to the
edge tier where the Ranking unit of SAED ranks them based on the relevance and
the user’s interest and generates a document list, called $\Delta^{\prime}$ to
show to the user. For a document $\delta_{i}\in\Delta$, the ranking score,
denoted as $\gamma_{i}$, is calculated by aggregating the importance values of
each $p\in P$ within $\delta_{i}$ and with respect to its weight ($\eta_{p}$).
The importance of $p$ in $\delta_{i}$ is conventionally measured based on the
_TF-IDF_ score [112]. Accordingly, $\gamma_{i}$ is formally calculated based
on Equation 3.

$\gamma_{i}=\displaystyle\sum_{\forall p\in P}\Bigg{(}\eta_{p}\cdot
TF\textrm{-}IDF(p,\delta_{i})\Bigg{)}$ (3)

The TF-IDF score of $p$ in $\delta_{i}$ is defined based on the frequency of
$p$ in $\delta_{i}$ versus the inverse document frequency of $p$ across all
documents in $\Delta$. Details of calculating the tf-idf score can be found in
[112]. Once the Ranking unit calculates the ranking score for all
$\delta_{i}\in\Delta$, then the documents are sorted in the descending order
based on their ranks and thus, the document list $\Delta^{\prime}$ are formed
with each $\delta_{i}$ and displayed to the user.

### 4 SAED As a Pluggable Module Enterprise Search Solutions

The advantage of SAED is to be independent from the enterprise search service
deployed on the cloud tier. That is, using SAED neither interferes with nor
implies any change on the cloud-based enterprise search service. SAED can be
plugged into any enterprise search solution. It provides the search smartness
on the on-premises edge tier and leaves the cloud tier only for large-scale
pattern matching. The whole SAED solution reforms the enterprise search to be
semantic, personalized, and confidential services.

In this work, we set SAED to work both with AWS Kendra and S3BD. In the case
of using AWS Kendra, the Query Expansion unit sends the expanded query set $P$
to Kendra to search each keyword $p$ against the dataset on the Amazon cloud.
The resulted documents are received by SAED and ranked before being delivered
to the client tier. In the implementation, we only show top 10 documents from
the resulted list to the user. Similarly, we plugged SAED to S3BD to perform
confidential semantic search on the cloud. Because S3BD maintains an encrypted
index structure that has to be traversed against each search query, the
elements of $P$ had to be encrypted before handing them over to the cloud
tier. We also verified SAED when it is used along with AWS Kendra where the
dataset was encrypted. We noticed that SAED can achieve smart search even when
Kendra is set to work with encrypted dataset. The performance measurement and
analysis of using SAED along with AWS Kendra and S3BD are elaborated in the
next Section.

### 5 Performance Evaluation of SAED

#### 1 Experimental Set up

We have developed a fully working version of SAED and made it available
publicly in our Github111https://github.com/hpcclab/SAED-Security-At-Edge
page. To conduct a comprehensive performance evaluation of SAED on the
enterprise search solutions, we developed it to work with both S3BD [3] and
AWS Kendra [113]. S3BD already has the query expansion and weighting
mechanisms, but we deactivated them and set it to use the expanded queries
generated by SAED. In the experiments, the combination of SAED and S3BD is
shown as SAED+S3BD. Likewise, the combination of SAED and AWS Kendra is shown
as SAED+Kendra.

We evaluated SAED using two different datasets, namely Request For Comments
(RFC) and BBC that have distinct properties and volume. The reason we chose
the RFC dataset is that it is domain-specific and includes $4,951$ documents
about the Internet and wireless communication network. Alternatively, the BBC
dataset is more diverse. It includes $2,224$ news documents in five distinct
categories, including politics, entertainment, business, sports, and
technology.

To conduct a comprehensive evaluation, we used both systematic metrics and
human-based feedback as elaborated in Section 3. We deployed and experimented
SAED on a Virtual Machine (VM) within our local edge computing system. The VM
had two 10-core 2.8 GHz E5 Xeon processors with 64 GB memory and Ubuntu 18.4
operating system.

#### 2 Benchmark Queries

The datasets that we use to carry out the experiments are not featured with
any benchmark. Therefore, we required to develop benchmark queries for the
datasets before evaluating the performance of SAED. We developed $10$
benchmark queries, shown in Table 1, for each one of the two datasets. The
benchmark queries are proactively designed to explore the breadth and depth of
the datasets in question. In addition, some of the queries intentionally
contain ambiguous keywords to enable us examining the context detection
capability of SAED. For the sake of brevity, we provide one acronym for each
benchmark query (see Table 1). For each benchmark query, we collected at most
the top-20 retrieved documents. Then, the quality of the retrieved documents
were measured via both automated script and human-based users.

Table 1: Benchmark search queries developed for the RFC and BBC datasets.

BBC Dataset | RFC Dataset
---|---
European Commission (EC) | Network Information (NI)
Parliament Archives (PA) | Host Network Configuration (HNC)
Top Camera Phones 2020 (TCP) | Data Transfer (DT)
Credit Card Fraud (CCF) | Service Extension(SE)
Animal Welfare Bill (AWB) | Transport Layer (TL)
Piracy and Copyright Issues (PCI) | Message Authentication (MA)
Car and Property Market (CPM) | Network Access (NA)
Rugby Football League (RFL) | Internet Engineering (IE)
Opera in Vienna (OV) | Fibre Channel (FC)
Windows Operating System (WOS) | Streaming Media Service (SMS)

#### 3 Evaluation Metrics

We have to measure the search relevancy metric to understand how related the
resulted documents are with respect to the user’s query and how they meet the
his/her interests. For the measurement, we use TREC-Style Average Precision
(TSAP) score, described by Mariappan et al. [102]. TSAP provides a qualitative
score in a relatively fast manner and without the knowledge of the entire
dataset [3]. It works based on the precision-recall concept that is commonly
used for judging text retrieval systems. The TSAP score is calculated based on
$\sum_{i=0}^{N}r_{i}/N$, where $r_{i}$ denotes score for $i^{th}$ retrieved
document and $N$ denotes the cutoff number (total number of retrieved
documents). Since we consider $N=10$, we call the scoring metric as _TSAP@10_.

To determine $r_{i}$ for retrieved document
$\delta^{\prime}_{i}\in\Delta^{\prime}$, we conducted a human-based
evaluation. We engaged five volunteer students to judge the relevancy of each
retrieved document. For every search query, the volunteers labeled each
retrieved document as highly relevant, partially relevant, or irrelevant.
After performing majority voting based on the provided responses for document
$i$, the value of $r_{i}$ is determined as follows:

  * •

$r_{i}=1/i$ if a document is highly relevant

  * •

$r_{i}=1/2i$ if a document is partially relevant

  * •

$r_{i}=0$ if a document is irrelevant

We report TSAP@10 score to show the relevancy of results for each benchmark
query. In addition, mean TSAP score is reported to show the overall relevancy
across each dataset. As we set the top 10 documents to be retrieved for each
search, the highest possible for _TSAP@10_ score can be 0.292 [102].

In addition to the TSAP score, we measure _Mean F-1_ score too to compare the
search quality offered by the SAED-plugged enterprise search solutions against
the original enterprise search solutions (i.e., without SAED in place). The
F-1 score maintains a balance between the precision and recall metrics, which
is useful for unstructured datasets with non-uniform topic distribution.

#### 4 Evaluating Search Relevancy

The purpose of this experiment is to evaluate the search relevancy of
enterprise search systems that have SAED plugged into them and compare them
against the original (unmodified) systems. To evaluate the personalized
search, we set (assumed) technology as the user’s interest for both datasets.
We note that, in this part, the enterprise search solutions (S3BD and AWS
Kendra) are set to work in the plain-text datasets.

S3BD vs SAED+S3BD.  Figure 2(a) shows the TSAP@10 score for the RFC and BBC
datasets for the original S3BD and SAED+S3BD. The horizontal axes in both
subfigures show the benchmark queries and the vertical axes show the search
relevancy based on the TSAP@10 score.

(a) BBC dataset

(b) RFC dataset

Figure 2: Comparing TSAP@10 scores of SAED+S3BD and S3BD systems. Horizontal
axes show the benchmark queries.

(a) BBC dataset

(b) RFC dataset

Figure 3: Comparing TSAP@10 scores obtained from SAED+Kendra versus AWS Kendra
in searching benchmark queries.

In both Figure 2(a) and 2(b), we observe that for all queries in both
datasets, SAED+S3BD outperforms the S3BD system. In addition, we observe that
S3BD produces less relevant results for the BBC dataset compared to the RFC
dataset. This is because, unlike the RFC dataset, in several cases, the exact
keywords of the benchmark queries do not exist in the BBC dataset. The worst
case of these issues has occurred for the PCI query in S3BD, because its query
expansion procedure could not capture the complete semantics. In contrast,
SAED+S3BD is able to handle the cases where the exact keyword does not exist
in the dataset, thus, we see that it yields to a remarkably higher relevancy.

Even if we consider PCI as an outlier and exclude that from the analysis, in
Figure 2(a), we still notice that the TSAP@10 score of SAED+S3BD is on average
$41.2\%$ higher than S3BD. Although the difference between S3BD and SAED+S3BD
is less significant for the RFC dataset (in Figure 2(b)), we still notice some
$17\%$ improvement in TSAP@10 score. This is because RFC is a domain-specific
dataset and the exact keywords of queries can be found in the dataset, hence,
making use of smart methods to extract the semantic is not acute to earn
relevant results. From these results, we can conclude that SAED can be
specifically effective for generic datasets where numerous topics exist in the
documents.

AWS Kendra vs SAED+Kendra.

(a) Encrypted BBC dataset

(b) Encrypted RFC dataset

Figure 4: Comparing TSAP@10 scores obtained from SAED+Kendra vs AWS Kendra
systems in the encrypted domain.

In Figures 3(a) and 3(b), we report TSAP@10 score obtained from AWS Kendra
versus SAED+Kendra for BBC and RFC datasets, respectively. Specifically, in
Figure 3(a) (BBC dataset), a significant improvement (on average $26.5\%$) is
noticed in the TSAP@10 score of SAED+Kendra. However, unlike SAED+S3BD,
SAED+Kendra does not beat Kendra for all the queries. The reason Kendra
outperforms SAED+Kendra for AWB and CPM queries is that SAED injects extra
keywords and sends the expanded query set to AWS Kendra. Then, Kendra returns
documents that are related to the queries and to the expanded keywords. We
realized that the Ranking unit of SAED occasionally prioritizes documents that
include keywords of the expanded queries instead of those with the query
keywords.

Similar to the S3BD experiment, we observe that the relevancy resulted from
Kendra and SAED+Kendra is less significant for RFC. However, we still obtain
around $12\%$ improvement in TSAP@10 score according to Figure 3(b).

#### 5 Relevancy of Privacy-Preserving Enterprise Search

To examine the efficiency of SAED for privacy-preserving enterprise search
systems, we conducted experiments using encrypted BBC and RFC datasets. The
encrypted datasets were uploaded to the cloud tier and the expanded queries
were also encrypted and searched on the cloud tier via Kendra.

We use the TSAP@10 score, as shown in Figure 4(a) and 4(b), for the BBC and
RFC datasets, respectively. Figure 4(a) indicates that SAED+Kendra
substantially outperforms Kendra for all the benchmark queries. We can see
that for encrypted dataset Kendra cannot do anything except pattern matching
and returning documents that exactly include the encrypted query. Therefore,
searching for several queries (e.g., PA,TCP, CPM, etc.) does not retrieve any
documents. We notice that, in both systems, the highest TSAP@10 score is in
searching EC. The reason is the high number of documents in BBC that contain
the exact phrase European commission.

The reported TSAP@10 scores for the RFC dataset in Figure 4(b) shows a clear
improvement in compared with the BBC dataset. We observe that seven out of ten
queries provide an equal TSAP@10 scores in both systems. The reason that makes
Kendra competitive to SAED+Kendra is the exact availability of the benchmark
queries in RFC. However, for HNC and FC, the exact query keywords are not
present in the dataset, hence, Kendra fails to find any results.

#### 6 Discussion of the Relevancy Results

In Table 2, we report _mean F-1_ and _mean TSAP@10_ scores for the SAED-
plugged enterprise search systems along with their original versions upon
utilizing the datasets both in the plain-text and encrypted forms. From the
table, we notice that, regardless of the enterprise search system being
employed, a higher search relevancy is consistently achieved for the RFC
dataset as opposed to the BBC dataset.

The search relevancy is consistently improved when SAED+Kendra is used and it
provides on average of $23\%$ improvement in mean F-1 score and $21\%$ in the
mean TSAP@10 score. Although original S3BD is the underperformer, using
SAED+S3BD improves its mean F-1 and mean TSAP@10 scores by on average of
$40\%$ and $32\%$, respectively.

| BBC | RFC
---|---|---
Systems |  | Mean
---
F-1
| Mean
---
TSAP@10
| Mean
---
F-1
| Mean
---
TSAP@10
S3BD | 0.50 | 0.17 | 0.80 | 0.24
SAED+S3BD | 0.82 | 0.25 | 0.92 | 0.28
Kendra | 0.67 | 0.20 | 0.88 | 0.26
SAED+Kendra | 0.90 | 0.27 | 0.93 | 0.28
Kendra (Encry.) | 0.31 | 0.09 | 0.75 | 0.22
SAED+Kendra (Encry.) | 0.73 | 0.22 | 0.90 | 0.27

Table 2: Comparing the mean F-1 and the mean TSAP@10 scores obtained from
SAED-plugged enterprise search systems versus their original forms. The
highest resulted scores are shown in bold font.

In the encrypted domain, we notice that SAED+Kendra offers a substantially
higher (up to $130\%$) search relevancy for BBC dataset. As the exact keywords
of the given search queries are not present in the encrypted form of BBC
dataset, AWS Kendra fails to perform semantic search, rather does only a
pattern matching, which makes it an underperformer for this dataset. On the
other hand, search relevancy is improved for RFC dataset since mean F-1 and
mean TSAP@10 scores are improved by at least $20\%$. This is because, most of
the queries are present exactly in the dataset and Kendra retrieves most of
the relevant documents by relying only on pattern matching.

#### 7 Evaluating the Search Time

Figure 5 presents the total incurred search time of the experimented queries
for each dataset. The search time is calculated as the summation of the
elapsed time taken by a query to be processed (e.g., expansion, weighting) and
turnaround time until the result set is received. To eliminate the impact of
any randomness in the computing system, we searched each set of experimented
queries 10 times and reported the results in the form of box plots. The figure
indicates that S3BD system has the highest search time overhead for both
datasets which could impact real-time searchability in case of big data.
SAED+S3BD incurs less query processing time overhead compared to the original
(unmodified) S3BD system.

On the other hand, AWS Kendra causes the lowest time overhead for both
datasets compared to SAED+Kendra. SAED+Kendra causes around $4$ times more
time overhead compared to original Kendra. However, in the prior set of
experiments, we determine that SAED+Kendra achieves a substantially higher
search relevancy for most of the queries and, particularly, for datasets with
privacy constraints.

Figure 5: Search time comparison among S3BD, Kendra, SAED+S3BD, and
SAED+Kendra systems.

### 6 Summary

A context-aware, personalized, and privacy-preserving enterprise search
service is the need of the hour for data owners who wish to use cloud
services. Our approach to address this demand was to separate the search
intelligence and privacy aspects from the pattern matching aspect. We
developed SAED that achieves privacy and intelligence at the edge tier and
leaves the large-scale pattern matching for the cloud tier. SAED is pluggable
and can work with any enterprise search solution (e.g., AWS Kendra and S3BD)
without dictating any change on them. Utilizing edge computing on the user’s
premises preserves the user’s privacy and makes SAED a lightweight solution.
Leveraging recurrent neural network-based prediction models, WordNet database,
and Word2Vec, SAED proactively expands a search query in a proper contextual
direction and weights the expanded query set based on the user’s interest. In
addition, SAED provides the ability to perform semantic search while the data
are stored in the encrypted form on the cloud. In this case, the existing
enterprise search solutions just perform the pattern matching without knowing
the underlying data. Evaluation results, verified by human users, show that
SAED can improve the relevancy of the retrieved results by on average $\approx
24\%$ for plain-text and $\approx 75\%$ for encrypted generic datasets.

##  Chapter 4: Multi-Tenancy of Latency-Sensitive Deep Learning Applications
on Edge

### 1 Overview

In the prior chapter, we propose an enterprise search application, namely SAED
in the form of a trusted application for enabling secure search over
confidential data in the cloud. The SAED application spans across edge-to-
cloud continuum and consists of several microservices that run on edge to
perform the intelligent aspects of searching (i.e., query processing,
personalization, and ranking). Our investigation indicates that running a
number of microservices on the edge consumes a significant percentage of
resource, specifically, edge memory is exhausted and service(s) can either be
killed or failed to execute. Prior studies quantized the NN models to make
them lighter but without model management, the system cannot get actual
advantage of multi-tenant processing. Edge-MultiAI leverages NN model
compression techniques, such as model quantization, and dynamically loads NN
models for DL applications to stimulate multi-tenancy on the edge server. We
consider the problem and scale it up in order to come across a unified
solution of it.

Due to the robust uses of smart IoT-based systems, various application
requests (i.e., object detection, face recognition, NLP, and motion capture)
incoming from users’ devices execute on edge tier with low-latency constraint
on a daily basis. An exemplar use case of such IoT-based systems is SmartSight
[19], illustrated in Figure 1, that aims at providing ambient perception for
the blind and visually impaired people. The system operates based on a
smartglass (IoT device) and a companion edge server (e.g., smartphone). The
smartglass continuously captures the inputs via its sensors (e.g., camera and
microphone) and requests the edge server to process DL-based applications,
such as object detection to identify obstacles; face recognition to identify
acquainted people; speech recognition, and NLP to understand and react to the
user’s commands. To make SmartSight usable, the edge server has to
continuously execute multiple (a.k.a. _multi-tenant_) DL application to
process incoming requests with low-latency and high accuracy. It is noteworthy
that, although cloud datacenters can mitigate the inherent resource
limitations of the edge, due to the network latency overhead and data
confidentiality [14, 15, 16], offloading the latency-sensitive service
requests to the cloud is not a tractable approach in many use cases.

Figure 1: Bird-eye view of SmartSight, an IoT-based system that continuously
receives various inputs from the smartglass (IoT device) sensors, and
processes them via multi-tenant DL applications running on the edge server.

DL applications utilize bulky Neural Network (NN) models at their kernel to
infer on the inputs received from the sensors. The NN models have to be kept
in memory to enable low-latency (a.k.a. _warm-start_ [31]) inference
operations. Otherwise, because the NN model size is often huge, loading it
into the memory in an on-demand manner (a.k.a. _cold-start_) is
counterproductive and affects the latency constraint of the DL applications.
As the edge servers naturally have a limited memory size (e.g., 4 GB in the
case of Jetson Nano [32]), multi-tenant execution of DL applications on them
leads to a memory contention challenge across the processes [14, 33].

Accordingly, the main challenge of this study is to resolve the memory
contention across multi-tenant DL applications without compromising their
latency and accuracy constraints.

Table 1: Load time, inference time, and accuracy of popular NN models
individually running on Samsung Galaxy S20+ as the edge server.

NN Models |  | Bit
---
Width
| Size
---
(MB)
| Loading
---
Time (ms)
| Inference
---
Time (ms)
| Accu-
---
racy (%)
InceptionV3 | FP32 | 105 | 650 | 100 | 78.50
INT8 | 24 | 380 | 80 | 77.20
VGG16 | FP32 | 528 | 820 | 52 | 71.30
INT8 | 132 | 185 | 40 | 70.18
MobileNetV1 | FP32 | 89 | 600 | 15 | 70.56
INT8 | 23 | 192 | 8 | 65.70
MobileNetV2 | FP32 | 26 | 110 | 10 | 72.08
INT8 | 9 | 65 | 7.5 | 63.70
MobileNetV3 | FP32 | 14 | 80.3 | 7.80 | 74.04
INT8 | 8 | 47.45 | 6.21 | 71.32
MobileBERT | FP32 | 96 | 1100 | 62 | 81.23
INT8 | 26 | 890 | 40 | 77.08

In the deep learning context, there are techniques based on the idea of
approximate computing, such as quantization [114], that make the model edge-
friendly via compressing its NN model, hence, reducing its inference time and
accuracy. To understand the impact of such approximations, we conducted a
preliminary experiment using a Samsung Galaxy S20+ as the edge server; and
five popular DNN models, namely InceptionV3, VGG16, MobileNetV1, MobileNetV2,
MobileNetV3, MobileBERT, each one at two quantization (precision) levels,
namely FP32 and INT8 bit widths. In Table 1, we report the average loading
time, inference time, and accuracy for their individual executions. We observe
that: (A) for all the models, the loading time is 8—17$\times$ more than its
inference time; (B) Loading the high-precision model (FP32 bit width) occupies
$\approx$3.5$\times$ more memory than the low-precision (INT8 bit width) one;
and (C) Loading a low-precision model can reduce the inference accuracy by
around 3—6%. These results demonstrate that the model compression has a
considerable potential to mitigate the memory footprint of the DL
applications. Moreover, the model loading time invariably dominates the
inference time [115]. Accordingly, our hypothesis is that _the efficient use
of model compression and the edge memory can enhance the multi-tenancy and
inference time of DL applications without any major loss on their inference
accuracy_.

We propose each DL application to be equipped with multiple NN models with
different precision levels. The low-precision models have a small memory
footprint, hence, allowing for a higher multi-tenancy of DL applications with
their models loaded into the memory (i.e., warm-start inference) that enhances
the service latency. However, loading overly low-precision (over-quantized)
models to maximize multi-tenancy and warm-start inference is not viable,
because it reduces the inference accuracy and renders the multi-tenant DL
applications to be futile. On the contrary, loading high-precision (large) NN
models on a memory-limited edge system for an indefinite time period
unnecessarily occupies an excessive memory space that is detrimental for the
multi-tenancy and warm-start inference of other tenants. That is, other
tenants face a significant slow down (as noted in Table 1), because they
cannot keep their NN model in memory and have to load it from the storage
(i.e., cold-start) to perform the inference operation. Therefore, an ideal
solution for a multi-tenant edge system should be able to dynamically load a
suitable model from the set of models available to the application (a.k.a.
model zoo), such that it neither interrupts the execution of other
applications, nor causes a cold-start inference for them.

### 2 Problem Statement

The research question that we investigate is: _how to maximize the number of
warm-start inferences for multi-tenant DL applications on edge without
compromising the inference accuracy?_ The question indicates a trade-off
between two objectives: fulfilling the latency constraint of DL applications
and maintaining their inference accuracy. The former objective entails having
the NN models of DL applications loaded into the memory (i.e., warm-start
inference), whereas, the latter entails retaining high-precision NN models in
the memory.

For application $A_{i}\in A$ with $M_{i}=\\{m^{k}_{i}\ |\ 1\leq k\leq
q_{i}\\}$ as its model zoo, let $r_{i}(t)$ be a Boolean function that
represents an inference request for $A_{i}$ at time $t$ with value 1. Also,
let $m^{\ast}_{i}\subseteq M_{i}$ be an NN model of $A_{i}$ with size of
$s^{\ast}_{i}$ that is currently loaded in the memory. This means that, for
application $A_{j}$ that does not have any of its NN models currently in the
memory, we have $m^{\ast}_{j}=\varnothing$ and $s^{\ast}_{j}=0$. Then,
$M^{\ast}=\bigcup_{i=1}^{n}m^{\ast}_{i}$ represents the set of currently
loaded NN models that occupy $S^{\ast}=\sum_{i=1}^{n}s^{\ast}_{i}$ of the
memory space. A cold start event for the request arrives at time $t$ for
$A_{i}$, denoted $C_{i}(M^{\ast},t)$ and shown in Equation (1), occurs when
there is no NN model in memory for $A_{i}$ (i.e., $M_{i}\cap
M^{\ast}=\varnothing$).

$C_{i}(M^{\ast},t)=\begin{cases}r_{i}(t)&M_{i}\cap M^{\ast}=\varnothing\\\
0&otherwise\end{cases}$ (1)

Assume that utilizing $m^{\ast}_{i}\in M_{i}$ results in an inference accuracy
that we denote it as $\chi^{\ast}_{i}$. Then, based on Equation (2), for $n$
multi-tenant DL applications, we can formally state the objective function as
minimizing the total number of cold-start inferences, while maximizing the
accuracy of the inferences. In this case, the total memory size available for
the NN models (denoted $S$) serves as the constraint.

$\begin{gathered}\min\biggl{(}\int_{t}^{\infty}\sum\limits^{n}_{i=1}C_{i}(M^{\ast},t)\
dt\biggr{)}\
,\quad\max\biggl{(}\int_{t}^{\infty}\sum_{i=1}^{n}\chi^{\ast}_{i}(t)\
dt\biggr{)}\\\ \textrm{subject to:}\quad\quad\quad\forall t,\ \
\sum\limits_{i=1}^{n}s^{\ast}_{i}\leq S\quad\quad\quad\end{gathered}$ (2)

Note that optimal NN model management decisions do not have a greedy nature.
That is, minimizing the number of cold-start inferences at a given time $t$
does not necessarily lead to the minimum total number of cold-starts with
maximum accuracy during the entire applications’ lifetime. In other words, the
system may experience a cold-start at time $t$ to prevent multiple ones at a
later time. That is why, the objective function of Equation 2 includes
integrals over $t$ to the $\infty$ to encompass the impacts of the decisions
at $t$ on the future cold-starts and accuracy levels. In the objectives, the
NN models of application $A_{i}$ are only chosen from its model zoo ($M_{i}$),
thus, the accuracy ($\mu_{i}(t)$) and size functions ($s_{i}$) are discrete
functions. It is needless to say that _minimizing the number of cold-start
inferences_ is equivalent to _maximizing the number of warm-start events_
[116]. In the rest of this chapter, we use these two interchangeably.

### 3 Solution Statement and Contributions

To stimulate multi-tenancy on the limited edge memory, we develop a framework,
called Edge-MultiAI, that takes advantage of a model zoo for each DL
application and can dynamically swap the NN models of the applications. To
maximize the number of warm-starts with high inference accuracy across multi-
tenant DL applications, our approach is to proactively load the high-precision
NN models for the applications that are expected to receive inference
requests, while loading low-precision models for the others. We utilize the
recent memory usage information to predict the memory availability for the
next executions while not interrupting other active applications. We develop
model management heuristic policies that make use of the expected memory
availability and the usage pattern of multi-tenant DL applications to choose a
suitable NN model for the requester application right before the inference
operation, thereby, both the latency and inference accuracy of the application
are fulfilled.

### 4 Architectural Overview & System Design of Edge-MultiAI

Figure 2 illustrates the architectural overview of Edge-MultiAI that
facilitates multi-tenancy of DL applications on a resource-limited edge system
via enabling the applications to only swap their NN models, instead of the
entire application. The framework consists of three tiers: (i) Application
tier, (ii) NN model manager, and (iii) Memory tier.

Figure 2: Architectural overview of the Edge-MultiAI framework with three
tiers: Application, NN Model Manager, and Memory.

Application Tier. The incoming multi-modal inputs from the connected IoT
devices trigger execution of multi-tenant DL applications in the application
tier. The _model zoo_ for each DL application acts as a repository that
contains NN models with different compression levels (sizes) and inference
accuracy (a.k.a. various _precision levels_). The _model loader_ is
responsible for loading the chosen NN model from the model zoo into the edge
memory.

NN Model Manager. NN model manager comprises of three components: (i)
_application request predictor_ , (ii) _memory predictor_ , and (iii) _memory
optimizer_. “Application request predictor” collects historical requests to
each application and trains a lightweight (edge-friendly) many-to-one vanilla
recurrent neural network (RNN) time series prediction model, similar to the
one in [110], to periodically foresee the inference request arrivals for each
application. Upon arrival of each request, “memory predictor” is in charge of
predicting the memory availability based on the recent memory allocations in
the entire edge system. We leverage the historical memory allocation data and
train another many-to-one vanilla RNN time-series prediction model to predict
the available memory.

Memory optimizer interacts with the application “request predictor” and
“memory predictor” to receive: (A) the request arrival time for different
applications plus the information of their model zoo; and (B) the memory
availability information. Then, the memory optimizer feeds the received
information to an NN model management policy that determines the highest
possible precision NN model that can be loaded to serve the inference request
of a DL application with the minimum impact (in terms of the prediction
accuracy or latency) on the execution of other applications. Upon facing
memory shortage for an arriving inference request, the memory optimizer
scavenges the memory allocated to the NN models of other applications via
either loading a lower-precision model or forcing them to cold-start. After
procuring adequate memory, the memory optimizer informs the “model loader” to
load the appropriate NN model of the requested application.

Memory Tier. The tier includes the “memory spaces” allocated to the
applications; and a “memory manager” that keeps track of the currently loaded
models, the available memory spaces, and the current status of the
applications. The memory manager communicates these information to the NN
Model Manager to efficiently allocates them to the arriving requests.

### 5 Heuristics to Manage Models of Multi-tenant Applications

#### 1 Overview

Recall that the aim of NN model management policy is to minimize the number of
cold-start inferences and maximize the inference accuracy for multi-tenant DL
applications on the edge servers. To that end, the memory optimizer strives to
maximize the time to retain the loaded models in the edge memory. However, due
to limitations in the available memory space, it is not possible to retain the
highest precision NN model of all applications in the memory. To resolve this
memory contention, the NN models of the applications that are unlikely to be
requested in the near future should be assigned a lower priority to remain in
the memory. Furthermore, Edge-MultiAI makes it possible to dynamically load NN
models for the applications. This means that, upon predicting time $t$ as the
inference request time for a given DL application, Edge-MultiAI can be
instructed to load the high-precision NN model of that application immediately
before performing the inference. Similarly, in the face of a memory shortage,
for the application(s) that are unlikely to be requested at time $t$, Edge-
MultiAI can be instructed to unload their NN models or, more interestingly,
replace them with a lower precision one.

However, we know that the request arrivals are inherently uncertain [19] and
no prediction model can precisely capture the exact request time for an
application. To capture the uncertainty, we consider a request time window,
denoted as $\Delta$, around each predicted request time. The value of $\Delta$
is obtained from profiling past request predictions and calculating the mean
difference of actual arrival time and the predicted ones across all
applications. In addition, there is a time overhead, denoted as $\theta_{i}$,
to load the chosen NN model of an application $A_{i}$ into the memory. In sum,
to prevent a cold-start for $A_{i}$ that is predicted to perform inference at
time $t$, as shown in Figure 3, the NN model has to be loaded at time
$(t_{i}-\Delta-\theta_{i})$ and kept in memory until $(t_{i}+\Delta)$.

Figure 3: A sample scenario of inference requests for five multi-tenant
applications, namely $A_{1}$ to $A_{5}$. Each pulse represents the time window
within which an inference request is expected. Solid lines expresses the event
that has already happened and dashed lines after “now” are the request
predictions.

Furthermore, there is uncertainty in predictions of “no request” for an
application at a given time. That is, at time $t$, there can be an inference
request for an application that was predicted not to have an request at that
time. To make the system robust against this type of uncertainty and to avoid
cold-start inferences in these circumstances, an ideal policy should load low-
precision NN models for these applications. Hence, an unpredicted inference
request can be still served as a warm-start by the low-precision model and the
latency constraint is maintained.

In this work, the set of applications whose NN models are retained in memory
outside of their predicted request time window are called the _minimalist_
set, and denoted as $A^{\prime}$. Similarly, the set of applications that are
in their request time window and we load a high-precision model for them are
called _maximalist_ , and denoted as $A^{\ast}$. To resolve the memory
contention, the policy can be based on scavenging memory from the minimalist
applications to procure the required memory space for the maximalist ones.
That is, in the event that application $A_{i}$ is predicted to have an
inference at time $t_{i}$, it becomes a member of $A^{\ast}$ set at time
$t_{i}-\Delta-\theta_{i}$, and then becomes a member of $A^{\prime}$ set after
$t_{i}+\Delta$; thus, its model can be evicted from the memory in the event
the memory space is needed for another maximalist application. The NN model
eviction is only permitted from $A^{\prime}$ set and we aim at retaining a
low-precision model for the applications in this set. However, due to high
inference demand, $A^{\prime}$ have to unload their models (i.e., switch to
cold-start) to free space for the model of the applications that are in the
maximalist set. In an extreme situation, if $A^{\prime}$ is empty, or the
scavenged memory from $A^{\prime}$ cannot procure sufficient space to load the
suitable model for application $A_{i}$, the next (smaller) model for $A_{i}$
is considered, and the aforementioned steps are repeated. Ultimately, if the
scavenged memory space is inadequate for the lowest precision model of
$A_{i}$, an inference failure occurs.

The memory contention problem can be reduced to the classic binary Knapsack
optimization problem [117] where from a collection of items, each one with a
weight and a value, we need to select items such that the total value is
maximized, while the total weight is bounded to a limit. This problem is known
to be NP-Complete,hence, we can rely on the heuristic-based solutions for it
[118]. In the next part, we discuss four NN model management (a.k.a. _NN model
eviction_) policies to manage the memory for multi-tenant DL applications such
that the number of warm-start inferences is maximized without any major impact
on the inference accuracy.

#### 2 Policy 1: Largest-First Model Eviction (LFE)

In this policy, to allocate memory for the NN model of a maximalist process,
we first evict NN models from set ($A^{\prime}$ that occupy the highest memory
space, until there is enough space to allocate the high-precision NN model of
$A^{\ast}$. For that purpose, members of $A^{\prime}$ are sorted based on the
size of their currently loaded NN model in the descending order. In the event
that evicting all the NN models of $A_{\prime}$ does not free enough memory
space to allocate the NN model of the request, a lower precision NN model
(smaller in size) is tried for allocation. This procedure continues until a
model from the model zoo can be allocated in the memory; otherwise, the edge
system is not able to serve that request at that time.

#### 3 Policy 2: Best-Fit Model Eviction (BFE)

The limitation of LFE is to evict the largest NN models of the minimalist
applications, irrespective of the exact memory requirement. This means that
adopting LFE can free more memory space than the actual requirement. To tackle
the issue, we implement the BFE policy where applications in the minimalist
set are sorted based on the difference between their model sizes and the
actual memory requirement. Then, the NN model with a minimum difference is
chosen for eviction. The memory requirement for a maximalist application is
first calculated based on its highest precision (largest) NN model to gain the
highest inference accuracy. However, in the event that evicting the NN models
of all the minimalist applications do not free enough memory space to allocate
the desired NN model, BFE iteratively selects the next high-precision model
from the model zoo of the requested application.

#### 4 Policy 3: Warm-Start-aware Best-Fit Model Eviction (WS-BFE)

Let $A_{i}\in A^{\ast}$ an application that is currently in the maximalist
set, and $A_{j}\in A^{\prime}$ an application that is currently in the
minimalist set. It is technically possible that the predicted request time
window of $A_{i}$ overlaps with the one for $A_{j}$. In this case, LFE and BFE
policies potentially choose to evict the NN model of $A_{j}$ in favor of the
$A_{i}$ model. This is because both of these policies are backward-looking and
ignore the fact that $A_{j}$ can be requested soon after evicting its NN
model. Such an eviction decision increases the likelihood of a cold-start
inference and to avoid that, we develop WS-BFE that assigns the lowest
eviction priority to those applications in $A^{\prime}$ that have overlapping
time window with $A_{i}$.

In our early experiments, we realized that another reason for cold-start
inferences is due to uncertain nature of request arrivals. That is, a
minimalist application is unexpectedly requested. To minimize the likelihood
of cold-start inference in these circumstances, we implement WS-BFE to
_replace_ the evicted NN model with the lowest-precision (i.e., smallest) NN
model of that application. As such, in the event of an unpredicted request the
minimalist applications, there is a low-precision model available to carry out
a warm-start inference.

#### 5 Policy 4: Intelligent Warm-Start-aware Best-Fit Eviction (iWS-BFE)

To make WS-BFE robust against uncertainties in the application request time
prediction, we enhance it by applying the Bayesian theory and proposing a new
policy, called iWS-BFE. This policy is inspired from the widely-adopted LRU-K
cache management policy [119] that considers _the least recently used (i.e.,
requested) applications are not likely to be requested in the near future_.
Similarly, iWS-BFE only considers members of $A^{\prime}$ as eviction
candidates, denoted by $E^{\prime}$, that are not recently requested. Figure
3, shows a scenario of predicted request times for $A_{1}$—$A_{5}$. To procure
memory for $A_{1}$, we have $A^{\prime}=\\{A_{2},A_{3},A_{5}\\}$. Because
$A_{3}$ was requested during the “history window” ($H$), it is likely to be
requested in the near future. Hence, iWS-BFE, chooses
$E^{\prime}=\\{A_{2},A_{5}\\}$ for eviction. The value of $H$ is determined
based on the mean request inter-arrival time of all applications.

In addition to considering LRU, iWS-BFE also makes use of the request
prediction, provided by Edge-MultiAI. That is, it considers the most
appropriate application for eviction as the one that has not been recently
requested, and is predicted to be requested the latest in future. However, the
request time predictions are uncertain, and the system can receive an
unexpected request from members of $E^{\prime}$ in the current request window.
To make iWS-BFE robust against such uncertainty, we calculate the probability
of an unexpected request. For application $A_{j}\in E^{\prime}$, let $r_{j}$
denote an unexpected request. Then, the probability of $r_{j}$ occurring
during the current request window (i.e., [$t,t+\Delta$]) is defined as
$P(r_{j}|A_{i}\in A^{\ast})$. The application that is likely to be requested
unexpectedly is not an optimal choice for eviction. Therefore, in Equation 3,
to calculate the fitness score of $A_{j}$ for eviction (denoted
$Score(A_{j})$), we consider $1-P(r_{j}|A_{i}\in A^{\ast})$. To take the
predicted request time of $A_{j}$ into consideration, we calculate the
distance between its predicted request time and the current time (i.e.,
$t_{j}-t_{i}$). To confine the value between [0,1], we normalize the distance
based on the latest predicted distance across all $k$ applications.

$\begin{gathered}Score(A_{j})=\frac{t_{j}-t_{i}}{\displaystyle\max_{k\in
E^{\prime}}(t_{k}-t_{i})}\cdotp\bigl{[}1-P(r_{j}|A_{i}\in
A^{\ast})\bigr{]}\end{gathered}$ (3)

The pseudo-code of the iWS-BFE policy is provided in Algorithm 7. It begins
with an initial set of eviction candidates, called $\tau\subseteq A^{\prime}$,
that is formed based on the applications that were not requested during the
history window ($H$). From $\tau$, in Step $3$, a list of eviction candidates
(denoted $E$) whose elements do not overlap with the request window of active
application ($A_{i}$) is derived. Next, in Step 4, we use Equation 3 to
calculate the fitness score for each $E_{k}\in E$ and then, build a max-heap
tree of $E$ based on the fitness scores (Step 5). In Steps 6—10, the policy
iteratively retrieves the application with the highest fitness score (i.e.,
the max-heap root, denoted $w$) and foresees the amount of memory that can be
scavenged upon replacing its loaded model with the lowest-precision one. Once
the policy finds enough memory to be scavenged such that the NN model of
$A_{i}$ (denoted $m_{i}$) can be loaded, in Step 13, it enacts all the NN
model replacement decisions and then loads $m_{i}$ in Step 14. In the event
that the scavenged memory is insufficient, the policy switches to the next NN
model for $A_{i}$ that has a lower size and accuracy (Step 17). In the worst
case that even the smallest NN model of $A_{i}$ cannot fit in the memory, the
inference request fails (Step 17) [120].

1

2

3Function iWS-BFE($A^{\prime}$, $A^{\ast}$, $A_{i}$, $H$)

4 $\tau\leftarrow$ Select $\forall A^{\prime}_{j}\in A^{\prime}$ not requested
during $H$

5 $E\leftarrow$ Determine $\forall A^{\prime}_{j}\in\tau$ non-overlapping with
request window of $A_{i}$

6

7 $\forall E_{k}\in E$ calculate fitness score using Equation 3

8 Build max-heap tree of $E$ based on fitness score

9

10 while  _$size(m_{i}) >$ available memory_ do

11 $w\leftarrow$ Extract root of the max-heap tree

12 If $w=\emptyset$ then break the loop

13 Measure memory scavenged by replacing model of $w$ with its lowest-
precision one

14 Add scavenged amount to _available memory_

15

16 end while

17

18 if  _$size(m_{i})\leq$ available memory_ then

19 Enact NN model replacement(s) decisions

20 Scavenge the leftover memory to load $m_{i}$

21

22 end if

23 else

24

25 If there is no model left to check then the inference request fails

26 Repeat Step 6—10 with the next (smaller) model

27

28 end if

29

30 end

31

32

Algorithm 7 Pseudo-code for iWS-BFE NN model eviction policy

### 6 Performance Evaluation

#### 1 Experimental Setup and Evaluation Metrics

To evaluate the efficacy of Edge-MultiAI and its NN model eviction policies,
we benchmarked five different DL applications, namely face recognition, speech
recognition, image classification, next sentence prediction, and text
classification, and recorded their real characteristics, including the model
size, and the inference accuracy (shown in Table 2). We have developed the E2C
simulator that enables modeling the IoT-based systems with different
characteristics and configurations, and is available publicly for the
community access through our Github page111Github page of the E2C simulator:
https://github.com/hpcclab/E2C-Sim.git. The simulator has implemented all of
the NN model eviction policies, and the user can quickly deploy and examine
any one of them.

Application | NN Model |  | Bit
---
Width
| Size
---
(MB)
| Accuracy
---
(%)
Face recognition | VGG-Face | FP32 | 535.1 | 90.2
FP16 | 378.8 | 82.5
INT8 | 144.2 | 71.8
Image classification | VIT-base-patch16 | FP32 | 346.4 | 94.5
FP16 | 242.2 | 81.3
INT8 | 106.7 | 72.2
Speech recognition | S2T-librisspeech | FP32 | 285.2 | 89.7
FP16 | 228.0 | 77.2
INT8 | 78.4 | 68.0
Sentence prediction |  Paraphrase-Mini LM-L12-v2 | FP32 | 471.3 | 88.2
FP16 | 377.6 | 81.7
INT8 | 98.9 | 76.2
Text classification | Roberta-base | FP32 | 499.0 | 91.1
FP16 | 392.2 | 82.4
INT8 | 132.3 | 76.6
|  |  |  |

Table 2: Application-specific models with different precision variants that
are experimented. Figure 4: The impact of Edge-MultiAI and its iWS-BFE
eviction policy on satisfying the requested multi-tenancy. The large graph
represents the summative analysis via increasing the mean of multi-tenancy
requested in the horizontal axis, and showing the percentage of requests that
were satisfied in the vertical axis. For each case, the smaller graph more
granularly represents the number of concurrent requests issued and fulfilled
during the simulation time.

The simulator also enables us to generate workload traces that include the
request arrival times for each application during the simulation time. We
configure the actual workload to include an equal number of requests for the
five applications, and the inter-arrival times between requests for each
application are distributed exponentially within the workload. To study the
uncertainty exists in the inference request predictions, in the evaluations,
we generate two sets of workloads, one includes the predicted arrival times
for the multi-tenant applications, and the other one includes the actual
arrival times of the applications. The distribution of request arrivals in the
actual workload deviates from the distribution of requests in the predicted
workload. The degree of deviation between the two is measured based on the
Kullback-Leibler (KL) [121] divergence. We explore the impact of this
deviation in the experiments of next subsections.

Our evaluation metrics are: (A) The _degree of multi-tenancy_ under different
request arrival intensity; (B) The _inference latency_ ; (C) the _inference
accuracy_ ; and (D) The _robustness_ metric to measure the tolerance of
different eviction policies against the uncertainty exists in the request
predictions.

#### 2 Impact of Edge-MultiAI on the Degree of Multi-tenancy

This experiment is to examine the efficacy of Edge-MultiAI in satisfying the
incoming requests to the edge server. To that end, as shown in Figure 4, we
increased the workload intensity, via the mean number of concurrent requests
issued, and in each case measured the _multi-tenancy satisfaction rate_ ,
which is the percentage of warm-start inferences out of the total incoming
requests during the simulation time. We examined two cases: (A) without any
solution to stimulate multi-tenancy (called, no policy); and (B) with Edge-
MultiAI and its iWS-BFE policy in place. The experiment was repeated 10 times
and the average rate and 95% confidence intervals for each data point is
reported.

The experiment shows that the degree of multi-tenancy achieved by adopting
Edge-MultiAI and its iWS-BFE is remarkably higher than the situation where
Edge-MultiAI is not in place. The smaller graphs show that this superiority
occurs consistently during the simulation time. We also notice that the impact
of employing Edge-MultiAI is more effective for higher degrees of multi-
tenancy. In particular, we can see that with the mean degree of multi-tenancy
is 5, using Edge-MultiAI and its iWS-BFE policy achieves $\approx$130% higher
satisfaction rate than no policy when mean requested degree of multi-tenancy
is larger than 2. This experiment justifies the efficacy of Edge-MultiAI and
the NN model management in stimulating multi-tenancy of DL applications.

#### 3 Impact of the Eviction Policies on the Cold-Start Inference

The purpose of this experiment is to

Figure 5: Measuring the percentage of cold-start inferences of multi-tenant
applications resulted from the proposed eviction policies. The horizontal axis
shows the deviation between predicted and actual inference request times.

analyze the impact of different NN model eviction policies on the number of
cold-start inferences. For that purpose, we measure percentage of cold-start
inferences caused by employing different eviction policies, particularly, upon
varying the deviation of request prediction from the actual requests.

The results, illustrated in Figure 5, show that LFE and BFE perform poorly and
cause a remarkable number of cold-start inferences, whereas, WS-BFE and iWS-
BFE mitigate the cold-start inferences by at least 65%. This is because, in
LFE and BFE, upon evicting an NN model, its corresponding application suffers
from a cold-start inference in the event of an unpredicted request. In
contrast, in WS-BFE and iWS-BFE, the evicted model is replaced with a low-
precision one, hence, unpredicted calls to the corresponding application do
not lead to cold-start inferences. It is noteworthy that, regardless of the
employed policy, the percentage of cold-start inferences rises upon increasing
the deviation between predicted and actual request times. Nonetheless, we see
that even under 90% deviation, iWS-BFE still substantially outperforms other
policies. On average, it yields 102% less cold-start in compare to LFE and
BFE, and 40% less than WS-BFE.

Figure 6: Measuring the normalized inference accuracy of applications resulted
from employing the different eviction policies.

#### 4 Impact of the Eviction Policies on the Inference Accuracy

In this experiment, we analyze the average inference accuracy caused by
employing different model eviction policies. Because the accuracy largely
varies across different applications, we perform min-max normalization on the
accuracy values. Also, for the cold-start inferences, in the accuracy
measurements, we consider the accuracy provided by the NN model after it is
loaded into the memory.

Figure 6 shows the normalized mean inference accuracy obtained from employing
different NN model eviction policies upon changing the deviation between
predicted and actual request times. According to the figure, LFE and BFE
policies outperform WS-BFE. This is because, these two policies do not retain
the low-precision models in the memory. Therefore, their inference requests
either lead to a cold-start (that was explored in the previous experiment), or
they load high-precision models that provide a high inference accuracy.
Nonetheless, we observe that iWS-BFE outperforms LFE and BFE in most of the
cases, except the one with 90% deviation. The reason for the higher inference
accuracy of iWS-BFE is that, it nominates cold-start candidates intelligently,
based on their probability of future invocations. This results indicate the
importance of the scoring (described in Equation 3) on efficiently nominating
cold-start candidates. It is noteworthy that the higher inference accuracy of
LFE and BFE at 90% deviation comes with the cost of substantially higher cold-
start inferences that are detrimental to the “usability” of the IoT-based
systems.

#### 5 Bi-Objective Analysis of NN Model Eviction Policies

Recall that the NN model management for multi-tenant applications in a
resource-limited edge system is a bi-objective optimization problem that aims
at minimizing the number of cold-start inferences and maximizing the inference
accuracy. However, these two are generally conflicting objectives and there is
not a single optimal solution that can satisfy both objectives. Instead, there
could be a range of solutions that dominate other solutions. To analyze which
one of the studied policies dominate others, in Figure 7, we plot the
percentage of cold-start inferences versus the model error (defined as
100-accuracy) for different policies and $\Delta$ values. Let $D$ and $\sigma$
be the mean and standard deviation of residuals of predicted versus actual
request times. Then, $\Delta=D\pm\alpha\cdotp\sigma$ ranges by changing the
value of $0\leq\alpha\leq 2$. The deviation of actual versus predicted
workload in this experiment is 30%.

Figure 7: Bi-objective analysis of the different model selection policies.

For each policy, the colored area shows the cold-start inferences and model
error rate that are dominated by that policy. An ideal policy should approach
the graph origin (i.e., resulting in zero cold-start and zero model error). In
Figure 7, we observe that Edge-MultiAI dominates other policies and form the
Pareto-front, particularly with $\alpha=1.02$. We can conclude that the iWS-
BFE policy can significantly improve the usability of the systems via causing
fewer cold-start inferences and offering a higher inference accuracy.

#### 6 Analyzing Robustness against Uncertainties

The goal of this experiment is to study how the eviction policies of Edge-
MultiAI make the IoT-based system robust against the uncertainty exists
between the predicted and actual application request predictor. We define the
_robustness metric_ , shown in Equation 4, to encompass the ratio of warm-
start inferences (denoted $\varpi_{i}$) to the total number of requests
(denoted $\gamma_{i}$), and the mean prediction accuracy ($\psi_{i}$) of each
application $i$ throughout the simulation period.

$R=\frac{1}{n}\cdotp\displaystyle\sum^{n}_{i=1}\bigg{[}{\frac{\varpi_{i}}{\gamma_{i}}\cdotp\psi_{i}}\bigg{]}$
(4)

Figure 8: Robustness of the system against uncertainty in the prediction of
inference requests.

Figure 8 represents the robustness score achieved by adopting the proposed
policies and no policy (a.k.a. baseline) against uncertainties in the
inference request prediction. We observe that deploying Edge-MultiAI with any
policy provides more robustness than the circumstance where Edge-MultiAI is
not in place (no policy). We also notice that the robustness value
consistently drops because the rate of inference failure and cold-starts rise
for higher deviations. We observe that WS-BFE and iWS-BFE are more robust
against deviation than the LFE and BFE. This is because, LFE and BFE do not
replace their NN models with a lower-precision one upon eviction, which leads
to cold-start inferences for the applications.

#### 7 Evaluating the Fairness of NN Model Eviction Policies

In this experiment, our goal is to examine whether the achievements of Edge-
MultiAI and its policies, explored in the previous experiments, is fairly
distributed across all applications, or some applications benefit more than
the others. To that end, we analyze the distribution of cold-start inference
and accuracy across different DL applications. The name and the NN model
characteristics of the examined DL applications are listed in Table 2. Figures
9 and 10, respectively, express the percentage of cold-start inferences and
inference accuracy for each application upon using various NN model eviction
policies. It is noteworthy that in Figure 9, “no policy” indicates the
situation where Edge-MultiAI is not in place, and in Figure 10, “maximum”
serve as the benchmark, by showing the use of highest-precision NN model for
each application. While Figure 9 shows that WS-BFE and iWS-BFE remarkably
outperform the other policies across all the applications, Figure 10
illustrates that, particularly for iWS-BFE, the outperformance does not come
with the cost of lower inference accuracy for the applications. More
importantly, in both figures, we observe that, for each policy, the percentage
of cold-start inferences and accuracy do not fluctuate significantly from one
application to the other. This shows that policies are not biased to any
particular DL application. Specifically, the rate of cold-start inferences and
the accuracy are fairly distributed across different applications.

Figure 9: The percentage of cold-start inferences using different NN model
eviction policies versus no policy. Figure 10: The inference accuracy obtained
from the different policies. The “maximum” is the benchmark, showing the
accuracy of the highest-precision model for each application.

### 7 Summary

Smart IoT-based systems often desire continuous execution of multiple latency-
sensitive Deep Learning (DL) applications. The edge servers serve as the
cornerstone of such IoT-based systems, however, their resource limitations
hamper the continuous execution of multiple (multi-tenant) DL applications.
The research aims to stimulate the degree of multi-tenancy of such
applications without compromising their latency and accuracy objectives.

We developed a framework, called Edge-MultiAI, to facilitate multi-tenancy of
DL applications via enabling swapping only their NN models. The framework was
also equipped with model management policies, particularly iWS-BFE, to choose
suitable models for eviction and loading to edge memory, such that the
percentage of warm-start inferences is maximized without any major loss in the
inference accuracy of the applications. Evaluation results indicate that Edge-
MultiAI can improve the degree of multi-tenancy by $2\times$, and iWS-BFE can
increase warm-start inferences by 60%. They also show how different policies
are robust against uncertainty in the inference request predictions. Last but
not the least, the experiments show that the policies are not biased to a
certain application in their decisions.

##  Chapter 5: Conclusion and Future Research Directions

This chapter summarizes the research and major findings of this dissertation.
Additionally, research topics that have surfaced during this research but have
not been covered in this dissertation are brought up and discussed. These
potential pathways for the future can be investigated further by other
researchers working in this field.

### 1 Discussion

In this dissertation, our main objective was to enable confidential computing
across edge-to-cloud continuum by maintaining data integrity and
confidentiality during executions that span across the continuum. We provide
three trusted applications to perform secure clustering and semantic searching
over confidential data without revealing any meaningful information to any
off-premise tiers. In addition, for model management of DL applications, we
develop a framework that can effectively facilitate multi-tenancy of DL
applications via enabling swapping only their NN models.

In Chapter 2, we developed solutions for topic-based clustering of both static
(ClustCrypt and S-ClusPr) and dynamic unstructured encrypted big datasets (SD-
ClusPr and FD-ClusPr). The proposed solutions approximate the number of
clusters for a dataset within a feasible time complexity. For that purpose,
they leverage the tokens’ co-occurrences to measures the tendency of each
token to stay with or segregate from other tokens and use that to estimate the
number of clusters. Next, we develop a probabilistic approach to determine the
center of each cluster and disseminate encrypted tokens to the most topically
related cluster. Experimental evaluations reveal that for static datasets,
S-ClusPr can improve the clustering coherency on average by $65\%$. Similarly,
for semi-dynamic and dynamic datasets, SD-ClusPr and FD-ClusPr can improve the
coherency by $55\%$. By incorporating ClustCrypt and ClusPr within the context
of a secure semantic search system, we learned that the more coherent and
accurate topic-based clustering can improve the relevancy of search results.

In Chapter 3, we propose an open-source generic pluggable module, namely SAED
into existing search services (e.g., AWS kendra, S3BD) to perform context-
aware, personalized, and secure search without dictating any change on them.
SAED can search over the data that is either plain-text or encrypted using
client side encryption before outsourcing to the cloud (i.e., AWS S3). Upon
verified by human users, experimental evaluations indicate SAED can improve
the relevancy of the retrieved results by on average $\approx 24\%$ for plain-
text and $\approx 75\%$ for encrypted datasets.

Our solution in Chapter 3 entailed continuously and simultaneously maintaining
multiple DL models that process confidential user data on the trusted edge
tier. This was challenging considering the memory limitations on the edge
tier. Moreover, such ML models could not be outsourced to Clouds because of
the user’s privacy. As such in Chapter 4, we propose a framework, namely Edge-
MultiAI that that operates based on the idea of approximate computing and
ushers the NN models of the DL applications into the edge memory such that the
degree of multi-tenancy and the number of warm-starts are maximized. Edge-
MultiAI leverages NN model compression techniques, such as model quantization,
and dynamically loads NN models for DL applications to stimulate multi-tenancy
on the edge server. We also devise a model management heuristic for Edge-
MultiAI, called _iWS-BFE_ , that functions based on the Bayesian theory to
predict the inference requests for multi-tenant applications, and uses it to
choose the appropriate NN models for loading, hence, increasing the number of
warm-start inferences. We evaluate the efficacy and robustness of Edge-MultiAI
under various configurations. Evaluation results indicate that Edge-MultiAI
can improve the degree of multi-tenancy by $2\times$, and iWS-BFE can increase
warm-start inferences by 60%. They also show how different policies are robust
against uncertainty in the inference request predictions. Last but not the
least, the experiments show that the policies are not biased to a certain
application in their decisions.

### 2 Future Research Directions

Based on our findings during the exploration of AI-driven confidential
computing paradigm across the edge-to-cloud continuum, there are several
points where the work could be expanded upon that were not covered in this
dissertation.

#### 1 Hierarchical Clustering of unstructured Data

We can employ _active learning_ to enable the automatic hierarchical
clustering of tokens with similar topics [122, 123, 124]. The active learning
paradigm was inspired by situations in which it is simple to collect enormous
quantities of unlabeled data (i.e., pictures and videos downloaded from the
internet, speech signals obtained from recordings made with microphones, and
so on), but it is difficult or expensive to gain their labels.

We can exploit the meaning of the deciphered tokens and their distributions in
the available clusters. With this information, we incorporate Wikipedia
knowledge to formulate hierarchical relationships among the tokens across the
confidential dataset. Later, for new a token, we measure the relatedness
between the token and the representation of each topic to propagate the
hierarchy.

#### 2 Building Classifier from the Encrypted Clusters

Clustering is a classic unsupervised learning that groups a massive amount of
unlabeled data. We can employ _active learning_ on the clusters to build a
classifier that potentially increases the use-cases in trusted computing for
unstructured data paradigm. Active learning can leverage the knowledge while
querying on cluster to measure the relatedness with the new token to form a
decision boundary of a classifier. The resultant classifier offers
substantially lower cost than traditional supervised learning [122].

#### 3 Introducing Elasticity in Confidential Search

The current implementation of SAED framework needs dependency of a connected
edge server to facilitate the searching. One idea is to including flexibility
in the framework which will reduce the burden of edge communication with
acceptable performance degradation. For instance, when the user is on the move
and does not have access to the edge, SAED should shrink to the bare minimum
search intelligence and vice versa.

#### 4 Adding Energy in Model Management Schemes

In Edge-MultiAI, NN model management for multi-tenant applications in a
resource-limited edge system is a bi-objective optimization problem that aims
at minimizing the number of cold-start inferences and maximizing the inference
accuracy. Since the edge servers have limited energy, often use battery,
considering energy is crucial to assign a job on an edge. Otherwise, due to
dead battery, the system could be abruptly switched off that leads to
execution failure. Subsequently, we can add the energy as a third objective
into the problem. In this way, we frame the problem as maximizing the warm-
starts and total accuracy with memory size and energy budget as constraints.

#### 5 Cloud Offloading for Latency-tolerant Applications

We only consider low-latency applications in Edge-MultiAI. We can also add
offloading option to the cloud tier in the framework. In this way, the system
could decide whether processing a task locally (probably with a few cold-
starts) is more beneficial (in terms of latency and energy consumption) or
offloading it to the cloud server.

## References

  * [1] S. Zobaed, S. Ahmad, R. Gottumukkala, and M. A. Salehi, “Clustcrypt: Privacy-preserving clustering of unstructured big data in the cloud,” in _Proceedings of the 21st International Conference on High Performance Computing and Communications (HPCC)_ , 2019, pp. 609–616.
  * [2] B. Seth, S. Dalal, V. Jaglan, D.-N. Le, S. Mohan, and G. Srivastava, “Integrating encryption techniques for secure data storage in the cloud,” _Transactions on Emerging Telecommunications Technologies_ , vol. 33, no. 4, p. e4108, 2022.
  * [3] W. Jason and M. A. Salehi, “S3BD: secure semantic search over encrypted big data in the cloud,” _Journal of Concurrency and Computation:Practice and Experience (CCPE)_ , vol. 28, no. 11, December 2018.
  * [4] W. Sun, W. Lou, Y. T. Hou, and H. Li, “Privacy-preserving keyword search over encrypted data in cloud computing,” in _Secure cloud computing_. Springer, 2014, pp. 189–212.
  * [5] Y. Zhu and X. Li, “Privacy-preserving k-means clustering with local synchronization in peer-to-peer networks,” _Journal of Peer-to-Peer Networking and Applications_ , vol. 13, no. 6, 2020.
  * [6] J. Al-Muhtadi, B. Shahzad, K. Saleem, W. Jameel, and M. A. Orgun, “Cybersecurity and privacy issues for socially integrated mobile healthcare applications operating in a multi-cloud environment,” _Jounral of Health informatics journal_ , vol. 25, no. 2, pp. 315–329, May 2019.
  * [7] “The 15 biggest data breaches of the 21st century,” www.csoonline.com/article/2130877/data-breach/the-biggest-data-breaches-of-the-21st-century.html, November, 2021\.
  * [8] S. Zobaed and M. A. Salehi, “Big data in the cloud,” in _Encyclopedia of Big Data_ , L. A. Schintler and C. L. McNeely, Eds. Springer, 2018.
  * [9] “Every Single Yahoo Account Was Hacked,” www.money.cnn.com/2017/10/03/technology/business/yahoo-breach-3-billion-accounts/index.html, Accessed May, 2020.
  * [10] “Cloud leak: How a verizon partner exposed millions of customer accounts,” www.upguard.com/breaches/verizon-cloud-leak, Accessed April, 2020.
  * [11] C. Shepherd, G. Arfaoui, I. Gurulian, R. P. Lee, K. Markantonakis, R. N. Akram, D. Sauveron, and E. Conchon, “Secure and trusted execution: Past, present, and future-a critical review in the context of the internet of things and cyber-physical systems,” in _2016 IEEE Trustcom/BigDataSE/ISPA_ , 2016, pp. 168–177.
  * [12] Z. Ning, J. Liao, F. Zhang, and W. Shi, “Preliminary study of trusted execution environments on heterogeneous edge platforms,” in _Proceedings of Symposium on Edge Computing (SEC)_ , 2018, pp. 421–426.
  * [13] “Confidential computing – the next frontier in data security,” https://confidentialcomputing.io/wp-content/uploads/sites/85/2021/10/Everest_Group_-_Confidential_Computing_-_The_Next_Frontier_in_Data_Security_-_2021-10-19.pdf, Accessed July 5, 2022.
  * [14] S. Deng, H. Zhao, W. Fang, J. Yin, S. Dustdar, and A. Y. Zomaya, “Edge intelligence: The confluence of edge computing and artificial intelligence,” _Journal of IEEE Internet of Things_ , vol. 7, no. 8, pp. 7457–7469, 2020\.
  * [15] S. M. Zobaed, M. A. Salehi, and R. Buyya, “Saed: Edge-based intelligence for privacy-preserving enterprise search on the cloud,” in _2021 IEEE/ACM 21st International Symposium on Cluster, Cloud and Internet Computing (CCGrid)_. IEEE, 2021, pp. 366–375.
  * [16] R. F. Hussain, A. Pakravan, and M. A. Salehi, “Analyzing the performance of smart industry 4.0 applications on cloud computing systems,” in _proceeginds of 22nd IEEE International Conference on High Performance Computing and Communications (HPCC)_ , 2020, pp. 11–18.
  * [17] S. Ahmad, S. Zobaed, R. Gottumukkala, and M. A. Salehi, “Edge computing for user-centric secure search on cloud-based encrypted big data,” in _Proceedings of the 21st International Conference on High Performance Computing and Communications (HPCC)_ , August 2019, pp. 662–669.
  * [18] F. Almalki, S. H. Alsamhi, R. Sahal, J. Hassan, A. Hawbani, N. Rajput, A. Saif, J. Morgan, J. Breslin _et al._ , “Green iot for eco-friendly and sustainable smart cities: future directions and opportunities,” _Mobile Networks and Applications_ , pp. 1–25, 2021.
  * [19] A. Mokhtari, M. A. Hossen, P. Jamshidi, and M. A. Salehi, “FELARE: fair scheduling of machine learning applications on heterogeneous edge systems,” in _Proceedings of International Conference On Cloud Computing_ , July 2022\.
  * [20] “Protecting data using client-side encryption,” https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingClientSideEncryption.html, Accessed November, 2022.
  * [21] H. Pham, J. Woodworth, and M. Amini Salehi, “Survey on secure search over encrypted data on the cloud,” _Journal of Concurrency and Computation: Practice and Experience_ , vol. 31, no. 17, p. e5284, 2019.
  * [22] K. Zhang, J. Long, X. Wang, H.-N. Dai, K. Liang, and M. Imran, “Lightweight searchable encryption protocol for industrial internet of things,” _Transactions on Industrial Informatics_ , vol. 17, no. 6, pp. 4248–4259, 2020.
  * [23] M. Zhang, Y. Chen, and J. Huang, “Se-ppfm: A searchable encryption scheme supporting privacy-preserving fuzzy multikeyword in cloud systems,” _Journal of Systems_ , vol. 15, no. 2, pp. 2980–2988, 2020.
  * [24] M. A. Salehi, T. Caldwell, A. Fernandez, E. Mickiewicz, E. W. Rozier, S. Zonouz, and D. Redberg, “RESeED: a secure regular-expression search tool for storage clouds,” _Journal of Software: Practice and Experience_ , vol. 47, no. 9, pp. 1221–1241, September 2017.
  * [25] D. C. G. Valadares, M. S. L. da Silva, A. E. M. Brito, and E. M. Salvador, “Achieving data dissemination with security using fiware and intel software guard extensions (sgx),” in _Proceedings of IEEE Symposium on Computers and Communications (ISCC)_. IEEE, 2018, pp. 1–7.
  * [26] M. Kehoe, “Mapping security requirements to enterprise search - part 1: Defining specific security requirements,” http://www.ideaeng.com/security-eprise-search-p1-0304, 2009, [Online; accessed 16-February-2020].
  * [27] “Announcing amazon kendra: Reinventing enterprise search with machine learning,” https://aws.amazon.com/about-aws/whats-new/2019/12/announcing-amazon-kendra-reinventing-enterprise-search-with-machine-learning/, 2019, [Online; accessed 16-February-2020].
  * [28] “Azure cognitive services,” https://azure.microsoft.com/en-us/services/cognitive-services, Accessed April, 2022.
  * [29] W. Yu, F. Liang, X. He, W. G. Hatcher, C. Lu, J. Lin, and X. Yang, “A survey on the edge computing for the internet of things,” _Journal of IEEE access_ , vol. 6, pp. 6900–6919, 2017.
  * [30] C. Gong, F. Lin, X. Gong, and Y. Lu, “Intelligent cooperative edge computing in internet of things,” _Journal of Internet of Things_ , vol. 7, no. 10, pp. 9372–9382, 2020.
  * [31] Q.-V. Dang and C.-L. Ignat, “dtrust: a simple deep learning approach for social recommendation,” in _proceedings of 3rd International Conference on Collaboration and Internet Computing (CIC)_ , 2017, pp. 209–218.
  * [32] “Jetson nano developer kit,” https://developer.nvidia.com/embedded/jetson-nano-developer-kit, Accessed November, 2022.
  * [33] A. Gholami, S. Kim, Z. Dong, Z. Yao, M. W. Mahoney, and K. Keutzer, “A survey of quantization methods for efficient neural network inference,” in _Journal of Low-Power Computer Vision_. Chapman and Hall/CRC, 2021, pp. 291–326.
  * [34] L. Zhou, Y. Zhu, and A. Castiglione, “Efficient k-nn query over encrypted data in cloud with limited key-disclosure and offline data owner,” _Computers & Security_, vol. 69, pp. 84–96, 2017.
  * [35] T. Tarpey, “Linear transformations and the k-means clustering algorithm: applications to clustering curves,” _Journal of the american statistician_ , vol. 61, no. 1, pp. 34–40, 2007.
  * [36] W. Sun, B. Wang, N. Cao, M. Li, W. Lou, Y. T. Hou, and H. Li, “Privacy-preserving multi-keyword text search in the cloud supporting similarity-based ranking,” in _Proceedings of the 8th symposium on Information, computer and communications security_ , ser. SIGSAC’13, May 2013, pp. 71–82.
  * [37] K. Gai and M. Qiu, “Blend arithmetic operations on tensor-based fully homomorphic encryption over real numbers,” _Transactions on Industrial Informatics_ , vol. 14, no. 8, pp. 3590–3598, August 2018.
  * [38] K. Xing, C. Hu, J. Yu, X. Cheng, and F. Zhang, “Mutual privacy preserving $k$-means clustering in social participatory sensing,” _Transactions on Industrial Informatics_ , vol. 13, no. 4, pp. 2066–2076, 2017.
  * [39] H. Pang and B. Wang, “Privacy-preserving association rule mining using homomorphic encryption in a multikey environment,” _Systems Journal_ , vol. 15, no. 2, pp. 3131–3141, 2020.
  * [40] P. Li, J. Li, Z. Huang, C.-Z. Gao, W.-B. Chen, and K. Chen, “Privacy-preserving outsourced classification in cloud computing,” _Journal of Cluster Computing_ , vol. 21, no. 1, pp. 277–286, March 2018\.
  * [41] C. Wang, A. Wang, X. Liu, and J. Xu, “Research on k-means clustering algorithm over encrypted data,” in _Proceedings of International Symposium on Cyberspace Safety and Security_ , December 2019, pp. 182–191.
  * [42] L. Ducas and D. Micciancio, “FHEW: Bootstrapping homomorphic encryption in less than a second,” in _Advances in Cryptology_ , ser. EUROCRYPT’15, April 2015, pp. 617–640.
  * [43] J. Vaidya and C. Clifton, “Privacy-preserving k-means clustering over vertically partitioned data,” in _Proceedings of the 9th international conference on Knowledge discovery and data mining_ , 2003, pp. 206–215.
  * [44] D. X. Song, D. Wagner, and A. Perrig, “Practical techniques for searches on encrypted data,” in _Proceedings of international symposium on security and privacy. S &P 2000_. IEEE, 2000, pp. 44–55.
  * [45] D. Boneh, G. D. Crescenzo, R. Ostrovsky, and G. Persiano, “Public key encryption with keyword search,” in _Proceedings of international conference on the theory and applications of cryptographic techniques_. Springer, 2004, pp. 506–522.
  * [46] N. Cao, Z. Yang, C. Wang, K. Ren, and W. Lou, “Privacy-preserving query over encrypted graph-structured data in cloud computing,” in _Proceedings of the 31st International Conference on Distributed Computing Systems_ , ser. ICDCS ’11, Washington, DC, USA, 2011, pp. 393–402.
  * [47] R. A. Popa, C. M. Redfield, N. Zeldovich, and H. Balakrishnan, “Cryptdb: processing queries on an encrypted database,” _Journal of Communications of the ACM_ , vol. 55, no. 9, pp. 103–111, 2012.
  * [48] E. W. Rozier, S. Zonouz, and D. Redberg, “Dragonfruit: Cloud provider-agnostic trustworthy cloud data storage and remote processing,” in _Proceedings of 19th Pacific Rim International Symposium on Dependable Computing_. IEEE, 2013, pp. 172–177.
  * [49] M. Ahmadian, F. Plochan, Z. Roessler, and D. C. Marinescu, “Securenosql: An approach for secure search of encrypted nosql databases in the public cloud,” _Jounal of International Journal of Information Management_ , vol. 37, no. 2, pp. 63–74, 2017.
  * [50] M. Kettouch, C. Luca, and M. Hobbs, “Semild: mediator-based framework for keyword search over semi-structured and linked data,” _Journal of Intelligent Information Systems_ , vol. 52, no. 2, pp. 311–335, 2019.
  * [51] M. Naseriparsa, M. S. Islam, C. Liu, and L. Chen, “Xsnippets: Exploring semi-structured data via snippets,” _Journal of Data & Knowledge Engineering_, vol. 124, p. 101758, 2019.
  * [52] R. Curtmola, J. Garay, S. Kamara, and R. Ostrovsky, “Searchable symmetric encryption: improved definitions and efficient constructions,” in _Proceedings of the 13th ACM conference on Computer and communications security_ , ser. CCS ’06, 2006, pp. 79–88.
  * [53] J. Woodworth, M. A. Salehi, and V. Raghavan, “S3c: An architecture for space-efficient semantic search over encrypted data in the cloud,” in _Proceedings of the 4th International Conference on Big Data_ , ser. Big Data’16, December 2016, pp. 3722–3731.
  * [54] S. Zobaed and M. Amini Salehi, “Privacy-preserving clustering of unstructured big data for cloud-based enterprise search solutions,” _Journal of Concurrency and Computation: Practice and Experience_ , vol. 34, no. 22, p. e7160, 2022.
  * [55] J. Li, Q. Wang, C. Wang, N. Cao, K. Ren, and W. Lou, “Fuzzy keyword search over encrypted data in cloud computing,” in _Proceedings of the 29th International Conference on Computer Communications_ , ser. INFOCOM ’10, March 2010, pp. 1–5.
  * [56] T. Moataz, A. Shikfa, N. Cuppens-Boulahia, and F. Cuppens, “Semantic search over encrypted data,” in _Proceedings of International Conference on Telecommunications (ICT)_ , May 2013, pp. 1–5.
  * [57] X. Sun, Y. Zhu, Z. Xia, and L. Chen, “Privacy preserving keyword based semantic search over encrypted cloud data,” _Journal of Security and Its Applications_ , vol. 8, no. 3, May 2014.
  * [58] F. Liang, R. Qiang, and J. Yang, “Exploiting real-time information retrieval in the microblogosphere,” in _Proceedings of the 12th ACM/IEEE-CS joint conference on Digital Libraries_ , June 2012, pp. 267–276.
  * [59] Y. Wang, H. Huang, and C. Feng, “Query expansion with local conceptual word embeddings in microblog retrieval,” _IEEE Transactions on Knowledge and Data Engineering_ , October 2019.
  * [60] S. Deerwester, S. T. Dumais, G. W. Furnas, T. K. Landauer, and R. Harshman, “Indexing by latent semantic analysis,” _Journal of the American society for information science_ , vol. 41, no. 6, pp. 391–407, September 1990\.
  * [61] K. Albishre, Y. Li, and Y. Xu, “Effective pseudo-relevance for microblog retrieval,” in _Proceedings of the Australasian Computer Science Week Multiconference_ , January 2017, pp. 1–6.
  * [62] F. Diaz, B. Mitra, and N. Craswell, “Query expansion with locally-trained word embeddings,” in _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , August 2016, pp. 367–377.
  * [63] T. Mikolov, K. Chen, G. Corrado, and J. Dean, “Efficient estimation of word representations in vector space,” www.pub-tools-public-publication-data.storage.googleapis.com/pdf/41224.pdf, 2013\.
  * [64] G. A. Miller, “Wordnet: a lexical database for english,” _Journal of Communications of the ACM_ , vol. 38, no. 11, pp. 39–41, 1995.
  * [65] M. Song, I.-Y. Song, X. Hu, and R. B. Allen, “Integration of association rules and ontologies for semantic query expansion,” _Journal of Data & Knowledge Engineering_, vol. 63, no. 1, pp. 63–75, October 2007.
  * [66] V. Nakade, A. Musaev, and T. Atkison, “Preliminary research on thesaurus-based query expansion for twitter data extraction,” in _Proceedings of the Southeast Regional Conference_ , ser. ACMSE’18, March 2018, pp. 1–4.
  * [67] C. H. Leung, Y. Li, A. Milani, and V. Franzoni, “Collective evolutionary concept distance based query expansion for effective web document retrieval,” in _Proceedings of the International Conference on Computational Science and Its Applications_ , June 2013, pp. 657–672.
  * [68] J. Chen and X. Ran, “Deep learning with edge computing: A review,” _Proceedings of the IEEE_ , vol. 107, no. 8, pp. 1655–1674, 2019.
  * [69] M. S. Murshed, C. Murphy, D. Hou, N. Khan, G. Ananthanarayanan, and F. Hussain, “Machine learning at the network edge: A survey,” _Journal of ACM Computing Surveys (CSUR)_ , vol. 54, no. 8, pp. 1–37, 2021.
  * [70] Z. Zhou, X. Chen, E. Li, L. Zeng, K. Luo, and J. Zhang, “Edge intelligence: Paving the last mile of artificial intelligence with edge computing,” _Journal of the IEEE Access_ , vol. 107, no. 8, pp. 1738–1762, 2019.
  * [71] X. Wang, Y. Han, V. C. Leung, D. Niyato, X. Yan, and X. Chen, “Convergence of edge computing and deep learning: A comprehensive survey,” _IEEE Communications Surveys & Tutorials_, vol. 22, no. 2, pp. 869–904, 2020.
  * [72] J. Mao, X. Chen, K. W. Nixon, C. Krieger, and Y. Chen, “Modnn: Local distributed mobile computing system for deep neural network,” in _Proceedings of Design, Automation & Test in Europe Conference & Exhibition (DATE)_. IEEE, 2017, pp. 1396–1401.
  * [73] K. Ko, Y. Son, S. Kim, and Y. Lee, “Disco: A distributed and concurrent offloading framework for mobile edge cloud computing,” in _Proceedings of 9th international conference on ubiquitous and future networks (ICUFN)_ , 2017, pp. 763–766.
  * [74] R. Hadidi, J. Cao, M. Woodward, M. S. Ryoo, and H. Kim, “Distributed perception by collaborative robots,” _IEEE Robotics and Automation Letters_ , vol. 3, no. 4, pp. 3709–3716, 2018.
  * [75] L. Xie, S. Wang, A. Markham, and N. Trigoni, “Towards monocular vision based obstacle avoidance through deep reinforcement learning,” _arXiv preprint arXiv:1706.09829_ , 2017.
  * [76] S. Yao, Y. Zhao, A. Zhang, L. Su, and T. Abdelzaher, “DeepIoT: Compressing deep neural network structures for sensing systems with a compressor-critic framework,” in _Proceedings of the 15th ACM Conference on Embedded Network Sensor Systems_ , 2017, pp. 1–14.
  * [77] Y. Ma, D. Xiang, S. Zheng, D. Tian, and X. Liu, “Moving deep learning into web browser: How far can we go?” in _Proceedings of the 19th World Wide Web Conference (WWW)_ , May 2019, pp. 1234–1244.
  * [78] S. R. Paladhi, R. M. Kumar, A. D. Reddy, C. Vinayak, and T. Pusphavathi, “Enhanced possibilistic c-means clustering on big data while ensuring security,” in _International Conference on Computer Networks and Communication Technologies_. Springer, 2019, pp. 583–588.
  * [79] P. K. Samantaray, N. K. Randhawa, and S. L. Pati, “An efficient multi-keyword text search over outsourced encrypted cloud data with ranked results,” in _Computational Intelligence in Data Mining_. Springer, 2019, pp. 31–40.
  * [80] W. Diffie and M. Hellman, “New directions in cryptography,” _Transactions on Information Theory_ , vol. 22, no. 6, pp. 644–654, November 1976.
  * [81] F. Can and E. A. Ozkarahan, “Concepts and effectiveness of the cover-coefficient-based clustering methodology for text databases,” _Journal of ACM Trans. Database Syst._ , vol. 15, no. 4, pp. 483–517, December 1990.
  * [82] D. R. Cutting, D. R. Karger, J. O. Pedersen, and J. W. Tukey, “Scatter/gather: A cluster-based approach to browsing large document collections,” in _ACM SIGIR Forum_ , vol. 51, no. 2, August 2017, pp. 148–159.
  * [83] S. Aggarwal, N. Agarwal, and M. Jain, “Performance analysis of uncertain k-means clustering algorithm using different distance metrics,” in _Computational Intelligence: Theories, Applications and Future Directions-Volume I_. Springer, 2019, pp. 237–245.
  * [84] X. Liu and W. B. Croft, “Cluster-based retrieval using language models,” in _Proceedings of the 27th International ACM SIGIR Conference on Research and Development in Information Retrieval_ , ser. SIGIR ’04, July 2004, pp. 186–193.
  * [85] C. C. Aggarwal, A. Hinneburg, and D. A. Keim, “On the surprising behavior of distance metrics in high dimensional space,” in _Proceedings of the 8th International conference on database theory_ , ser. ICDT ’01, January 2001, pp. 420–434.
  * [86] A. L. Mary and K. S. Kumar, “A density based dynamic data clustering algorithm based on incremental dataset,” _Journal of Computer Science_ , vol. 8, no. 5, pp. 656–664, 2012.
  * [87] “The Museum of Modern Art Data,” tapoueh.org/blog/2018/07/batch-updates-and-concurrency, Accessed May ,2020.
  * [88] “Twitter stream api dataset,” github.com/shreybatra/Twitter-Stream-API-Dataset, Accessed March, 2020.
  * [89] “Statistics Solution,” www.statisticssolutions.com/using-chi-square-statistic-in-research/, Accessed July 15, 2020.
  * [90] R. A. Popa, C. M. S. Redfield, N. Zeldovich, and H. Balakrishnan, “Cryptdb: Protecting confidentiality with encrypted query processing,” in _Proceedings of the 23rd ACM Symposium on Operating Systems Principles_ , ser. SOSP ’11, October 2011, pp. 85–100.
  * [91] D. Wang and P. Wang, “Offline dictionary attack on password authentication schemes using smart cards,” in _Information Security_. Springer International Publishing, 2015, pp. 221–237.
  * [92] M. Naehrig, K. Lauter, and V. Vaikuntanathan, “Can homomorphic encryption be practical?” in _Proceedings of the 3rd ACM Workshop on Cloud Computing Security Workshop_ , ser. CCSW ’11, October 2011, pp. 113–124.
  * [93] M. George, S. Kamara, and T. Moataz, “Structured encryption and dynamic leakage suppression,” in _Proceedings of Annual International Conference on the Theory and Applications of Cryptographic Techniques_ , October 2021, pp. 370–396.
  * [94] “Common Crawl on Amazon Web Services (AWS),” https://aws.amazon.com/public-datasets/common-crawl/, Accessed February, 2021.
  * [95] “RFC (request for comments) series,” old.datahub.io/dataset/rfcs, Accessed March, 2020.
  * [96] “BBC news classification,” www.kaggle.com/c/learn-ai-bbc, Accessed March, 2020.
  * [97] O. Medelyan, E. Frank, and I. H. Witten, “Human-competitive tagging using automatic keyphrase extraction,” in _Proceedings of the 14th Conference on Empirical Methods in Natural Language_ , ser. EMNLP ’09, August 2009, pp. 1318–1327.
  * [98] P. Berkhin, “A survey of clustering data mining techniques,” in _Grouping multidimensional data_. Springer, 2006, pp. 25–71.
  * [99] G. A. Miller, “Wordnet: a lexical database for english,” _Journal of Communications of the ACM_ , vol. 38, no. 11, pp. 39–41, November 1995.
  * [100] M. Z. Rodriguez, C. H. Comin, D. Casanova, O. M. Bruno, D. R. Amancio, L. d. F. Costa, and F. A. Rodrigues, “Clustering algorithms: A comparative approach,” _Journal of PloS one_ , vol. 14, no. 1, p. e0210236, 2019.
  * [101] B. C. Kwon, B. Eysenbach, J. Verma, K. Ng, C. De Filippi, W. F. Stewart, and A. Perer, “Clustervision: Visual supervision of unsupervised clustering,” _IEEE transactions on visualization and computer graphics_ , vol. 24, no. 1, pp. 142–151, 2017.
  * [102] A. K. Mariappan, R. M. Suresh, and V. S. Bharathi, “A comparative study on the effectiveness of semantic search engine over keyword search engine using tsap measure,” _Journal of Computer Applications EGovernance and Cloud Computing Services_ , pp. 4–6, December 2012.
  * [103] M. Amini Salehi, T. Caldwell, A. Fernandez, E. Mickiewicz, D. Redberg, E. W. D. Rozier, and S. Zonouz, “RESeED: Regular Expression Search over Encrypted Data in the Cloud,” in _Proceedings of the 7th International Cloud conference_ , ser. Cloud ’14, June 2014, pp. 673–680.
  * [104] A. Silva and M. Mendoza, “Improving query expansion strategies with word embeddings,” in _Proceedings of the ACM Symposium on Document Engineering_ , September 2020.
  * [105] S. Kuzi, A. Shtok, and O. Kurland, “Query expansion using word embeddings,” in _Proceedings of the 25th international on conference on information and knowledge management_ , 2016.
  * [106] C. Fellbaum, “Wordnet: An electronic lexical resource,” in _The Oxford Handbook of Cognitive Science_. Routledge, 2017, pp. 301–314.
  * [107] R. Campos, V. Mangaravite, A. Pasquali, A. M. Jorge, C. Nunes, and A. Jatowt, “YAKE! collection-independent automatic keyword extractor,” in _Proceedings of the 40th European Conference on Information Retrieval_ , March 2018, pp. 806–810.
  * [108] A. Khatua, A. Khatua, and E. Cambria, “A tale of two epidemics: Contextual word2vec for classifying twitter streams during outbreaks,” _Journal of Information Processing & Management_, vol. 56, no. 1, pp. 247–257, 2019.
  * [109] Z. Kastrati, A. S. Imran, and S. Y. Yayilgan, “The impact of deep learning on document classification using semantically rich representations,” _Journal of Information Processing & Management_, vol. 56, no. 5, pp. 1618–1632, 2019.
  * [110] X. Pang, Y. Zhou, P. Wang, W. Lin, and V. Chang, “An innovative neural network approach for stock market prediction,” _The Journal of Supercomputing_ , vol. 76, no. 3, pp. 2098–2118, 2020.
  * [111] “Vanilla recurrent neural network,” http://calvinfeng.gitbook.io/machine-learning-notebook/supervised-learning/recurrent-neural-network/recurrent_neural_networks, Accessed September,2020.
  * [112] F. Viegas, S. Canuto, C. Gomes, W. Luiz, T. Rosa, S. Ribas, L. Rocha, and M. Gonçalves, “Cluwords: exploiting semantic word clustering representation for enhanced topic modeling,” in _Proceedings of the 12th International Conference on Web Search and Data Mining_ , January 2019, pp. 753–761.
  * [113] “Amazon Kendra,” www.aws.amazon.com/kendra, Accessed April 10, 2020.
  * [114] Y. Zhou, S.-M. Moosavi-Dezfooli, N.-M. Cheung, and P. Frossard, “Adaptive quantization for deep neural network,” in _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 32, no. 1, 2018.
  * [115] D. G. Samani and M. A. Salehi, “Exploring the impact of virtualization on the usability of deep learning applications,” in _Proceedings of 22nd IEEE International Symposium on Cluster, Cloud and Internet Computing (CCGrid)_ , 2022, pp. 442–451.
  * [116] J. Manner, M. Endreß, T. Heckel, and G. Wirtz, “Cold start influencing factors in function as a service,” in _Proceedings of the 12th IEEE/ACM International Conference on Utility and Cloud Computing Companion (UCC)_ , 2018, pp. 181–188.
  * [117] X. Chen, Y. Wu, and Y. Han, “Fepim: Contention-free in-memory computing based on ferroelectric field-effect transistors,” in _Proceedings of 26th IEEE Asia and South Pacific Design Automation Conference (ASP-DAC)_ , 2021, pp. 114–119.
  * [118] T. V. Christensen, “Heuristic algorithms for np-complete problems,” _Project report, Institute of Informatics and mathematical Modelling, Technical University of Denmark_ , 2007.
  * [119] E. J. O’neil, P. E. O’neil, and G. Weikum, “The LRU-K page replacement algorithm for database disk buffering,” _Journal of Acm Sigmod Record_ , vol. 22, no. 2, pp. 297–306, 1993.
  * [120] A. Mokhtari, C. Denninnart, and M. A. Salehi, “Autonomous task dropping mechanism to achieve robustness in heterogeneous computing systems,” in _Proceedings of International Parallel and Distributed Processing Symposium Workshops (IPDPSW)_ , 2020, pp. 17–26.
  * [121] T. Van Erven and P. Harremos, “Rényi divergence and kullback-leibler divergence,” _IEEE Transactions on Information Theory_ , vol. 60, no. 7, pp. 3797–3820, 2014.
  * [122] S. Dasgupta and D. Hsu, “Hierarchical sampling for active learning,” in _Proceedings of the 25th international conference on Machine learning_ , 2008, pp. 208–215.
  * [123] P. Kumar and A. Gupta, “Active learning query strategies for classification, regression, and clustering: a survey,” _Journal of Computer Science and Technology_ , vol. 35, no. 4, pp. 913–945, 2020.
  * [124] F. Min, S.-M. Zhang, D. Ciucci, and M. Wang, “Three-way active learning through clustering selection,” _International Journal of Machine Learning and Cybernetics_ , vol. 11, no. 5, pp. 1033–1046, 2020.

received his Bachelor of Science in computer science and engineering in the
fall of 2015 from Islamic University of Technology (IUT), Bangladesh. He
started his professional career in December of 2015, as a System Engineer in
one of the top tech giants named “Huawei Technologies Ltd”. After around a
year and half, he planned to enrich his academic knowledge by pursuing higher
education. Hence, started his Ph.D. journey in computer science in the fall of
2017 at the University of Louisiana at Lafayette. received his M.Sc. degree in
computer science in the spring of 2019 during his Ph.D. journey. His research
interests are: Natural language processing, Data analytics, and Cloud
computing.