∎ 11institutetext: Department of Computer Applications, National Institute of
Technology, Kurukshetra, Haryana, INDIA
11email<EMAIL_ADDRESS>
11email<EMAIL_ADDRESS>
This article has been accepted in Springer New Generation Computing Journal
$\copyright$ 2022 Springer Nature. Personal use of this material is permitted.
Permission from Springer must be obtained for all other uses in any current or
future media, including reprinting/republishing this material for advertising
or promotional purposes, creating new collective works, for resale or
redistribution to servers or lists, or reuse of any copyrighted component of
this work in other works. This work is freely available for survey and
citation.

# A Differential Approach for Data and Classification Service based Privacy-
Preserving Machine Learning Model in Cloud Environment

Rishabh Gupta  Ashutosh Kumar Singh

###### Abstract

The massive upsurge in computational and storage has driven the local data and
machine learning applications to the cloud environment. The owners may not
fully trust the cloud environment as it is managed by third parties. However,
maintaining privacy while sharing data and the classifier with several
stakeholders is a critical challenge. This paper proposes a novel model based
on differential privacy and machine learning approaches that enable multiple
owners to share their data for utilization and the classifier to render
classification services for users in the cloud environment. To process owners’
data and classifier, the model specifies a communication protocol among
various untrustworthy parties. The proposed model also provides a robust
mechanism to preserve the privacy of data and the classifier. The experiments
are conducted for a Naive Bayes classifier over numerous datasets to compute
the proposed model’s efficiency. The achieved results demonstrate that the
proposed model has high accuracy, precision, recall, and F1-score up to 94%,
95%, 94%, and 94%, and improvement up to 16.95%, 20.16%, 16.95%, and 23.33%,
respectively, compared with state-of-the-art works.

###### Keywords:

Cloud computing Machine learning Differential privacy Classification Privacy-
preserving

##  1 INTRODUCTION

Cloud computing has the capability to provide an ample amount of data storage,
computation, analysis, and sharing services to organizations without revealing
its implementation and platform details [1]. Due to these illimitable
services, any organization outsources its data and model from the local to the
cloud platform [2], [3]. Nowadays, machine learning has gained a lot of
attention in a wide range of real-world applications, including image
recognition, spam detection, financial market analysis, and recommendation
systems [4], [5]. The machine learning classifier is extensively utilized in
these applications. Thereafter, the emerging challenges include handling the
privacy of the model and sensitive data [6]. The cloud stores and computes
massive amounts of collected data and the classification model that are
outsourced from various owners without interaction with each other. The owners
hesitate to share their data and model with the cloud for computation and
storage since a third party runs it [7]. The owners lose control of their
outsourced data, model and are unaware of its extraction from the cloud
platform [8]. According to a Cisco survey, 76% of owners have no clue about
their data utilization by other parties [9]. The cloud may misuse and provide
both data and the model to other parties for different purposes [10], [11].
Due to these reasons, the privacy-preserving of data and the model has become
a challenge for any organization. Therefore, it is essential to protect the
data and classification model by applying some privacy strengthening process
before transferring them to an untrusted cloud server.

To address the challenges mentioned above, we propose a Differential Approach
for data and classification service-based Privacy-preserving Machine Learning
Model (DA-PMLM) in the cloud environment. In the proposed model,
$\epsilon$-differential privacy protection is considered on the owner’s side
because they do not aspire to disclose actual data, and the model [12, 13, 14,
15]. Fig. 1 outlines a bird-eye view of the proposed work and emphasizes our
consistent contributions to protect the data as well as the classification
model from unauthorized parties. The owners inject different statistical noise
into data and the model according to various applications and queries. The
achieved data and model are uploaded to the cloud platform, and classification
services are provided. The obtained machine learning model is applied over
collected data for classification. The cloud platform receives the classified
data from the classification model and distributes it to the data owners or
users rather than other parties. It classifies tasks and allows data sharing
in the cloud environment. The main contributions of the proposed work have
been described as follows:

Figure 1: Bird eye view of DA-PMLM

  * $\bullet$

DA-PMLM permits several data owners and the classifier owner to share their
data and machine learning model without hindrance. The Laplace mechanism has
been used to keep the data and model private before sharing.

  * $\bullet$

DA-PMLM uses two clouds: cloud1 and cloud2. Cloud1 handles the data storage
and sharing process, whereas cloud2 carries out the classification tasks over
accumulated data from various owners. To secure data and model with improving
privacy, all entities are deemed untrustworthy.

  * $\bullet$

DA-PMLM ensures the level of security because the value of the privacy budget
is assigned according to the requirement of protection.

  * $\bullet$

Implementation and evaluation are performed using various datasets, which
shows that DA-PMLM outperforms the state-of-art works in terms of privacy,
accuracy, precision, recall, and f1-score.

Organization: Section 2 describes the related work followed by the proposed
model and the process of preserving data in Sections 3 and 4, respectively.
The privacy-preserving of the model and the data classification steps are
presented in Section 5 and Section 6, respectively. The illustration of the
proposed model is defined in Section 7. The operational design and
computational complexity are discussed in Section 8. The experimental results
with statistical analysis are shown in Section 9, followed by the conclusion
and future work in Section 10. Table 1 demonstrates the list of symbols with
their descriptive terms that have been used throughout this manuscript.

Table 1: List of Terminologies with their Explanatory Terms $CO$: | Classifier Owner |  $n^{**}$: | Training objects |  $P$: | Precision
---|---|---|---|---|---
UE: | Untrusted Entity |  $D_{i}$: | Actual data |  $N_{i}$: | Noise
$CM$: | Classification Model |  $RU_{id}$: | Request Users |  $D_{i}^{N}$: | Synthetic Data
$CA$: | Classification Accuracy |  $\check{D}_{t^{{}^{\prime}}}^{N}$: | Training Data |  $R$: | Recall
$\hat{D}_{i}^{N}$: | Preprocessed Data |  $\mathbb{C}^{L}$: | Label Vector |  $\check{D}_{t^{{}^{\prime\prime}}}^{N}$: | Testing Data
$\Delta$ f: | Absolute distance |  $n$: | No. of data object |  $FP$: | False Positive
DP: | Differential Privacy |  $CS$: | Cloud Storage |  Pr: | Probability
$\vartheta$: | Scaling parameter |  $n^{***}$: | Testing objects |  $\epsilon$: | Privacy budget
$\hat{R}$: | Random function |  f: | Query function |  $n^{*}$: | count of classes
$CM^{N}$: | Synthetic Model |  $CF$: | Classifier |  $FS$: | F1-Score
$CSP$: | Cloud Service Provider |  $C1$: | Cloud1 |  $C2$: | Cloud2
$\sigma$: | Standard deviation |  $\mu$: | Mean |  $\Lambda$: | Data attribute

##  2 Related Work

To protect the data, and machine learning model using the differential privacy
mechanism has been categorized at the two levels: (a) At the data level, (b)
At the model level.

###  2.1 At Data Level

To protect data privacy, Li et al. [16] proposed a differentially private
scheme called Privacy-preserving Machine Learning with Multiple data providers
(PMLM) with improved computational efficiency and data analysis accuracy. The
PMLM scheme uses public-key encryption with a double decryption algorithm (DD-
PKE) to transform the encrypted data into a randomized dataset and
$\epsilon$-differential privacy to make the data private. But the PMLM scheme
suffered from less accuracy as well as limited data sharing. Wang et al. [17]
proposed a distributed agent-based privacy-preserving framework, namely DADP,
for real-time spatial statistics data collection and publication with an
untrusted server. A distributed budget allocation mechanism and an agent-based
dynamic grouping mechanism were developed to achieve global w-event
$\epsilon$-differential privacy in a distributed manner. In DADP, crowd-
sourced data is aggregated, and then the noise is added to it using the
Laplace mechanism. However, it was considered a semi-centralized setting and
resulted in a more complex system because it initiated a batch of trusted
proxies (Agents) and anonymous connection technology to protect the privacy of
users under an untrusted server. A privacy-preserving deep learning model,
namely PDLM, is presented in [18] that protects training data by encrypting it
with multiple keys using a public-key distributed two trapdoors (DT-PKC)
cryptosystem. The proposed scheme minimized the storage cost. However, it
necessitates more calculations on the ciphertext, resulting in decreased
efficiency. Hassan et al. [19] proposed an efficient privacy-preserving scheme
based on machine learning to protect data privacy. The partially homomorphic
encryption technique was utilized to re-encrypt data, and a differential
privacy mechanism was used to add noise to the data. It permits all
participants to publicly verify the validity of the ciphertext using a
unidirectional proxy re-encryption (UPRE) scheme that reduces computational
costs. But the proposed scheme has limited data sharing. A local differential
privacy (LDP)-based classification algorithm for data centers is introduced
[20] with high efficiency and feasible accuracy. Sensitive information is
protected by adding the noise using the Laplace mechanism in the pattern
mining process. They devised a method for determining the level of privacy
protection. However, the proposed method does not allow for data sharing. To
prevent information leakage, a framework, namely, noising before model
aggregation federated learning (NbAFL), was proposed by Wei et al. [21].
Before the model aggregation, a differential privacy mechanism was utilized to
add the noise to clients’ local parameters. Nevertheless, this framework
requires a significant amount of noise to add and sacrifices model utility. An
adaptive privacy-preserving federated learning framework is introduced in
[22], which protects gradients by inserting the noise with varying privacy
budgets. They devised a randomized privacy-preserving adjustment technology
(RPAT) to enhance the accuracy, but the proposed framework makes the
classification process computationally costly. A machine learning and
probabilistic analysis-based model, namely MLPAM, is presented in [23]. It
supports various participants to share their data safely for different
purposes using encryption, machine learning, and probabilistic approaches.
This scheme reduced the risk associated with the leakage for prevention
coupled with detection, but it does not provide privacy for the classifier. To
protect the Spatio-temporal aggregated data in real-time, Xiong et al. [24]
proposed a privacy-preserving framework based on the local differential
approach. The authors also devised a generalized randomized response (GRR)
framework for obtaining reliable aggregated statistics continuously while
maintaining user’s privacy. The proposed framework improves data usage but
exposes the user’s privacy by adding independent identically distributed noise
to the associated data. Liu et al. [25] proposed a differential privacy for
local uncertain social network (DP-LUSN) model that protects the social
network’s community structured data. DP-LUSN increased the data utilization by
minimizing the noise effect on a single edge. However, DP-LUSN only operates
on static social networks. Sharma et al. [26] proposed a Differential Privacy
Fuzzy Convolution Neural Network framework, namely DP-FCNN, which protects the
data and query processing by adding the noise using the Laplace mechanism. The
authors used the lightweight Piccolo algorithm to encrypt the data and the
BLAKE2s algorithm to extract the key attributes from the data. However, DP-
FCNN increases the computational overhead.

###  2.2 At Model Level

A privacy-preserving Naive Bayes learning scheme with various data sources
using $\epsilon$-differential privacy and homomorphic encryption is presented
in [27]. This scheme enables the trainer to train the Naive Bayes classier
over the dataset provided jointly by the different data owners. But still,
adversaries can forge and manipulate the data in this scheme. Gao et al. [28]
proposed a privacy-preserving Naive Bayes classifier scheme to avoid
information leakage under the substitution-then-comparison (STC) attack. They
adopted a double-blinding technique to preserve the privacy of Naive Bayes.
The proposed scheme reduced both the communication and computation overhead.
But this scheme is unable to obtain the discovery of truth that protects
privacy. A privacy-preserving outsourced classification in the cloud computing
(POCC) framework was introduced in [29]. It protects the privacy of sensitive
data under various public keys using a fully homomorphic encryption proxy
technique without leakage. However, encryption for outsourced data can protect
privacy against unauthorized behaviors. It also makes effective data
utilization, such as search over encrypted data, a complicated issue. Liu et
al. [30] proposed a private decision tree algorithm based on the noisy maximal
vote. An effective privacy budget allocation strategy was adopted to make the
balance between the true counts and noise. It was constructed as an ensemble
model with differential privacy to boost the accuracy and improve the
stableness. In the proposed algorithm, the privacy analysis was performed only
on each separate tree and not on the ensemble as a whole. A differentially
private gaussian processes classification (GPC) model was presented in [31],
which adds noise to the classifier to provide privacy. The authors used the
Laplacian approximation method to determine GPC’s sensitivity and scaled noise
to outputs generated from other dataset sections. In this manner, the total
noise can be decreased. However, the proposed model is not sparse. Wang and
Zhang [32] proposed a differential privacy version of convex and nonconvex
sparse classification approach based on alternating direction method of
multiplier (ADMM) algorithm with mild conditions on the regularizers. By
adding exponential noise to stable stages, they were able to turn the sparse
problem into a multistep iteration process and accomplish privacy protection.
However, estimating the noise introduced to stochastic algorithms in the
proposed approach is problematic. A differentially private ensemble learning
method for classification, referred to as DPEL, was presented in [33] which
provides privacy protection while ensuring prediction accuracy. The authors
applied the Bag of Little Bootstrap and the Jaccard similarity coefficient
techniques to enhance the ensemble’s diversity. A privacy budget allocation
strategy was designed to make the differentially private base classifiers by
adding noise. But the DPEL method is affected by limited data sharing. Table 2
provides an overview of the literature review.

Table 2: Tabular sketch of the literature review

Model/Scheme /Framework |  Workflow & Implementation |  Outcomes |  Drawback
---|---|---|---
A machine learning-based privacy-preserving model for data protection [16] |  • To prevent data leakage, additively homomorphic encryption and differential privacy techniques were employed • Abalone, Wine, Cpu, Glass, and Krkopt data sets were used for experiments |  • Improve the accuracy of data analysis and the efficiency of computations • The model seems to be more protected as per the security analysis |  For the classifier, it does not provide privacy
A distributed privacy-preserving task allocation framework [17] |  • $w$ event and $\epsilon$-Differential Privacy mechanisms were used to protect data • Python language and taxi trajectory, nice rider datasets were used for experiments |  • Less Mean Absolute Error (MAE) and Mean Relative Error (MRE) • Enhance the overall throughput of the whole vehicular network |  A trade-off between accuracy and the overall payment of crowd-sensing server
A cloud-based Deep Learning model for data privacy [18] |  • A homomorphic encryption mechanism was adopted to encrypt the data • To assess PDLM performance, a LeNet deep learning model was utilized |  • Achieve high accuracy up to 94% • Perform the tasks of privacy-preserving training and classification appropriately |  The use of multiple keys for big data reduced the efficiency of the machine learners
A privacy-preserving with public verifiability scheme [19] |  • Data is protected by applying differential privacy and partially homomorphic encryption techniques • Java pairing based cryptography (JPBC) library was used for experiments |  • Lower computational as well as communication costs • The security analysis verifies that the scheme is more secure under the random oracle model |  With homomorphic encryption, the issue of ciphertext expansion arise
A classification algorithm satisfies local differential privacy [20] |  • Internet of Multimedia Things (IoMT) produces the sensitive information • Laplace noise is injected into the data during pattern mining |  • High efficiency, reliability, and precision • Less MAE, mean squared error (MSE) metrics |  IoMT devices need higher latency and bandwidth
A framework protects training data based on differential privacy [21] |  • Inject the noise into the parameters of the local learning model • This framework was assessed by utilizing multi-layer perception and MNIST datasets |  • A trade-off between federated learning performance and privacy • Acquire high privacy using random mini-batches |  Less accuracy due to injected noise in the parameters
A secure multi-party computation strategy by adaptive privacy-preserving [22] |  • The Laplace mechanism was adopted to add noise with different privacy budget • TensorFlow library and MNIST dataset were used for experiments |  • Achieve high accuracy up to 88.46% • Less computation and transmission overhead |  Data perturbation may degrade the data utility
A secure data sharing model for determining guilty entities against data leakage [23] |  • The attribute based encryption and differential privacy were used to preserve the privacy of multiple participants’ data • The pyhton language and Glass, Iris, Wine, and Balance Scale datasets were utilized for experiments |  • Perform secure data distribution among users by an effective allocation mechanism • Achieve high accuracy up to 97% |  High computational and communication costs while transferring the data
A real-time spatio-temporal data aggregation method [24] |  • LDP was employed for data protection • MATLAB R2018a programming language and Geolife, Taxi Service datasets were used to conduct the experimentations |  • Less MAE, Mean MAE (MMAE), Mean KL divergence (MKLD), and Mean RelativeError (MRE) • Acquire adequate trade-off of privacy and utility |  High computation and communication costs due to generalized randomized response
A local differential privacy scheme for social network publishing [25] |  • The perturbed local method was applied to yield a synthetic network • The experiments are performed using Python language, and WebKB, Citation, and Cora datasets |  • Decrease network structure information’s loss • This model provides a substantial degree of privacy |  It does not support dynamic social networks
A fuzzy convolution neural network framework based on Differential Privacy for data security [26] |  • Data is encrypted with the help of a 128-bit block cipher • Java Development toolkit (JDK) 1.8, Weka tools, and Adult and Heart disease dataset were used for experimentations |  • The comparative analysis shows that it takes less run-time up to 14 ms • More effective in terms of scalability and accuracy |  Not suitable for hybrid machine learning algorithms
A privacy-preserving learning algorithm based on differential privacy[27] |  • The differential privacy and homomorphic encryption mechanisms were used for data preservation • A GMP library was adopted to execute cryptographic operations |  • Achieve secure data sharing without breaking privacy • Require less computational time for training model |  Less efficiency because of encryption
A privacy-preserving scheme for protecting the trained model [28] |  • Additively homomorphic encryption was used to encrypt the data • The experiments were conducted over Breast Cancer Wisconsin (Original), Statlog Heart datasets |  • Less the computation overhead because of the offline stage of the server • Accomplish suitable privacy-preserving training tasks |  It does not provide efficient information sharing
A framework for protecting the outsourced classification model [29] |  • Data was encrypted by using the multikey homomorphic encryption proxy • The experiments were carried out using a naive bayes classifier |  • More effective for data utilization • Achieve high-level data protection |  Less efficiency due to additions, multiplications, and the nonlinear computations over encrypted data
A method for learning a differentially private decision tree [30] |  • The differentially-private decision tree was trained by leveraging the noisy maximal vote • The experiments were carried out over Mushroom and Adult datasets |  • Improve the learned classifier’s utility • Provide high accuracy up to 81.19% |  The ensemble model’s performance can not be maximized
A privacy-preserving model for data classification [31] |  • A variant-noise mechanism was used to perturb the classifier with the different scaled noise • The experiments are conducted over an actual oil dataset provided by NCRG of Aston University |  • Achieve elevated accuracy up to 75.00% • Retain the classifier’s accuracy |  Noise inevitably decreases the accuracy
A privacy-preserving for convex and nonconvex sparse classification [32] |  • The sparse problem was transformed into a multistep iteration process through ADMM algorithm • The logistic regression and KDDCup99 dataset were used for experiments |  • More effective and efficient for performing sensitive data analysis • Accomplish appropriate privacy-preserving training and classification tasks |  It does not share the data effectively in several environments
A privacy-preserving method for preventing leakage in the process of classification [33] |  • Laplace noise is injected into the classifier’s objective function • The experiments were run over Wisconsin, User knowledge, Mushroom, Adult, and Bank Market datasets |  • Perform a trade-off between privacy and accuracy • Acquire high accuracy up to 94.66% |  It does not provide the data protection

In the light of the aforementioned discussion, it can be concluded that
earlier works limited data sharing and injected the noise into either data or
classification model and/or protected them by employing diverse encryption
techniques followed by the machine learning-based classification, resulting in
decreased privacy, accuracy, and higher computing cost. In addition, the prior
models only supported a single owner or untrustworthy entity. Unlike the
previous works, DA-PMLM ensures that both data and the classification model
are protected before being shared. The proposed model enables numerous data
and classifier owners to securely share outsourced data and classification
model while assuming all entities to be untrustworthy.

##  3 PROPOSED MODEL

The proposed model (Fig. 2) consists of four entities: Data Owners
($DO_{id}$), Classifier Owner ($CO$), Cloud Service Provider ($CSP$), and
Request Users ($RU_{id}$), which are described as follows in terms of
intercommunication and vital information flow:

  1. 1)

$DO_{id}$: An entity that yields the data and appeals to $CSP$ for storing,
computing, and sharing services. To protect the data, $DO_{id}$ injects the
noise into data using the $\epsilon$-differential privacy mechanism prior send
it to $CSP$. $DO_{id}$ permits $CSP$ to share the data with $RU_{id}$. Since
it is assumed that $DO_{id}$ can not misuse its data, it may disclose the data
of other owners. Therefore, $DO_{id}$ is not considered a trusted entity.

  2. 2)

$CO$: An entity that has a classification model (CM) and offers the services
to perform the classification tasks through $CSP$. Before passing CM to $CSP$,
$CO$ injects the noise utilizing the $\epsilon$-differential privacy mechanism
to retain the privacy and stores it. $CO$ is viewed as an untrustworthy entity
in the proposed model.

  3. 3)

$CSP$: An entity that collects all the data from $DO_{id}$ and CM from $CO$.
It provides storage, computation, and data sharing services to $DO_{id}$,
$CO$, and $RU_{id}$. It also offers the classification services using CM to
$DO_{id}$, and $RU_{id}$. $CSP$ trains CM using the machine learning algorithm
over collected data and obtains classified data from CM. These accessed
results are shared between $DO_{id}$, or $RU_{id}$. $CSP$ is introduced as a
semi-trusted entity in the proposed model, as it strictly follows the protocol
but is curious to learn the information. In our model, the Cloud Platform (CP)
consists of two clouds: Cloud1 (C1) and Cloud2 (C2). The data is stored in
Cloud Storage (CS) at C1 while the classification model is kept in Classifier
(CF) at C2. $CSP$ is the only entity that acts as a bridge among $DO_{id}$,
$CO$, and $RU_{id}$. It stores the data and model to perform the data sharing
and classification tasks.

  4. 4)

$RU_{id}$: An entity that requests $CSP$ to obtain the owner’s data, and
classification service. $RU_{id}$ receives the data from C1 and accesses the
CM from C2 through the $CSP$. $RU_{id}$ is treated as an untrusted entity in
our model.

Figure 2: Proposed DA-PMLM architecture

Consider the data owners $\mathbb{DO}$ = {$DO_{1}$, $DO_{2}$, $\dots$,
$DO_{n}$} having data $\mathbb{D}$, such as $\mathbb{D}$ $\in$ {$D_{1}$,
$D_{2}$, $\dots$, $D_{n}$}. It is essential that $\mathbb{DO}$ must share
$\mathbb{D}$ among the other parties, such as $CSP$, and several request users
$\mathbb{RU}$ = {$RU_{1}$, $RU_{2}$, $\dots$, $RU_{m}$} for storage,
computation, and data usage purposes. Due to the sensitive data, $\mathbb{DO}$
do not aspire to disclose $\mathbb{D}$ to unauthorized parties. As a result,
each $\mathbb{DO}$ procures synthetic data $\mathbb{D^{N}}$ =
{$D_{1}^{N_{1}}$, $D_{2}^{N_{2}}$, $\dots$, $D_{n}^{N_{n}}$} by injecting
noise $\mathbb{N}$ = {$N_{1}$, $N_{2}$, $\dots$, $N_{n}$} into sensitive data
{$D_{1}$, $D_{2}$, $\dots$, $D_{n}$} using $\epsilon$-differential privacy to
make the data private before sharing (details discussion in Section 4). DA-
PMLM makes the use of Differential Privacy (DP) mechanism to inject $N_{1}$,
$N_{2}$, $\dots$, $N_{n}$ into $D_{1}$, $D_{2}$, $\dots$, $D_{n}$,
respectively, because DP is the most appropriate mechanism for generating
$N_{1}$, $N_{2}$, $\dots$, $N_{n}$ and sharing data $D_{1}^{N_{1}}$,
$D_{2}^{N_{2}}$, $\dots$, $D_{n}^{N_{n}}$ without sacrificing or revealing
sensitive and personal information about individual owners [34]. Each
$DO_{1}$, $DO_{2}$, $\dots$, $DO_{n}$ delivers $D_{1}^{N_{1}}$,
$D_{2}^{N_{2}}$, $\dots$, $D_{n}^{N_{n}}$ to $CSP$ that store it on CS at C1
for computation, and sharing purposes. Due to the lack of storage space and
the need to handle multiple requests, $CO$ aspires to store CM on the cloud.
Nevertheless, $CO$ injects noise $\mathbb{\hat{N}}$ into CM to protect privacy
before transferring it (as described in Section 5). $CO$ sends the synthetic
model ($CM^{N}$) to $CSP$ that keeps it on CF at C2. $CSP$ utilizes the stored
data $D_{1}^{N_{1}}$, $D_{2}^{N_{2}}$, $\dots$, $D_{n}^{N_{n}}$ to accomplish
the classification tasks to make a fit $CM^{N}$, and also share it with
$RU_{1}$, $RU_{2}$, $\dots$, $RU_{m}$. $CSP$ also uses the stored model
$CM^{N}$ to handle the query of $DO_{id}$, and $RU_{id}$. $DO_{1}$, $DO_{2}$,
$\dots$, $DO_{n}$ and $RU_{1}$, $RU_{2}$, $\dots$, $RU_{m}$ can make any query
and send it to $CSP$. Afterward, $CSP$ interacts with CF and achieves the
results of these queries from $CM^{N}$. The obtained results are sent to the
corresponding entity $DO_{1}$, $DO_{2}$, $\dots$, $DO_{n}$, and $RU_{1}$,
$RU_{2}$, $\dots$, $RU_{m}$ by $CSP$.

##  4 Privacy-Preserving of Data

Data owners ($DO_{1}$, $DO_{2}$, $\dots$, $DO_{n}$) injects the noise
($N_{1}$, $N_{2}$, $\dots$, $N_{n}$) into data ($D_{1}$, $D_{2}$, $\dots$,
$D_{n}$), respectively, using $\epsilon$-differential privacy. A random
function $\hat{R}$ is referred to as $\epsilon$-differential privacy [35] if
for any combination of dataset $D$ and its neighbor $D^{{}^{\prime}}$, and for
all $\vartheta$ $\subseteq$ Range($\hat{R}$) are defined using Eq. (1):

$Pr[\hat{R}(D)=\vartheta]\leq exp(\epsilon)\times
Pr[\hat{R}(D^{{}^{\prime}})=\vartheta]$ (1)

where $Pr[.]$ is the probability function that has been applied to the
function $\hat{R}$ and demonstrates the privacy disclosure risk. The privacy
budget ($\epsilon$) is defined by the results of statistical computation,
analysis, and the individual’s information that has to be kept private. To
achieve a higher level of privacy protection, the value of $\epsilon$ should
be reduced. A numeric query function ($f$) that is utilized to map the data
set $D$ into real space with $d$-dimensional, assigned as $f$: $D$
$\rightarrow$ $R^{d}$. The sensitivity of $f$ for all the combinations of
dataset $D$ and $D^{{}^{\prime}}$ is defined using Eq. (2):

$\Delta f=\max\limits_{D,D^{{}^{\prime}}}\parallel
f(D)-f(D^{{}^{\prime}})\parallel_{P_{1}}$ (2)

where $\parallel$ . $\parallel_{P_{1}}$ is norm. The sensitivity $\Delta f$ is
only depend on the query function $f$ and finds the maximum gap of query
results on neighboring datasets. For any function $f$: $D$ $\rightarrow$
$R^{d}$, the Laplace mechanism $F$ is defined using Eq. (3). The mechanism $F$
takes input $\chi$, and $\epsilon$ $>$ 0, a numeric query function $f$, and
computes the output. It is based on the sensitivity of $f$ and receives the
statistical noise from a Laplace distribution. Afterward, the obtained noise
is injected into the datasets.

$F(\chi)=f(\chi)+(Lap_{1}(\varpi),Lap_{2}(\varpi),\dots,Lap_{d}(\varpi))$ (3)

where the noise $Lap_{j}$($\varpi$) ($j$ $\in$ [$1$, $d$]), and $\varpi$
belongs to $R^{+}$ comes from a Laplace distribution, whose the probability
density function is calculated using Eq. (4).

$N=\frac{1}{2\varpi}\cdot(exp(\frac{-|\chi|}{\varpi}))$ (4)

where $N$ is a noise vector. The noise $N_{1}$, $N_{2}$, $\dots$, $N_{n}$ is
generated taking the sample from the Laplace distribution with scaling
parameter $\varpi$ = $\Delta f/\epsilon$. The parameter $\varpi$ is under the
control of $\epsilon$. The generated noise vector $N_{1}$, $N_{2}$, $\dots$,
$N_{n}$ is injected into the corresponding $D_{1}$, $D_{2}$, $\dots$, $D_{n}$
as $D_{i}^{N_{i}}$ = $D_{i}$ \+ $N_{i}$, where $i$ $\in$ [$1$, $n$]. After
adding $N_{1}$, $N_{2}$, $\dots$, $N_{n}$, $DO_{1}$, $DO_{2}$, $\dots$,
$DO_{n}$ obtains the Synthetic Data $D_{1}^{N_{1}}$, $D_{2}^{N_{2}}$, $\dots$,
$D_{n}^{N_{n}}$. $DO_{1}$, $DO_{2}$, $\dots$, $DO_{n}$ sends data
$D_{1}^{N_{1}}$, $D_{2}^{N_{2}}$, $\dots$, $D_{n}^{N_{n}}$ to $CSP$ that
stores it on CS at C1, shares it among $RU_{1}$, $RU_{2}$, $\dots$, $RU_{m}$
and performs the classification task over it.

##  5 Privacy-Preserving of Classification Model

To protect the privacy of $CM$, $CO$ injects the noise $\mathbb{\hat{N}}$ into
$CM$ before sharing it on the cloud. In DA-PMLM, $CO$ has a Naive Bayes (NB)
classifier, which is used to perform the classification tasks. The NB
classifier takes the input vector $\mathbb{D^{N}}$ = {$x_{1}$, $x_{2}$,
$\dots$, $x_{d}$}, which belongs to $R^{d}$, and classify $d$ into a class in
the form of discrete set $\mathbb{C}$ = {$C_{1}$, $C_{2}$, $\dots$, $C_{n}$}.
The functionality of the NB classifier based on the Bayes decision rule [36]
is to select the class $\mathbb{C}$ with the highest posterior probability of
$\mathbb{D^{N}}$ and measures it by using Eq. (5).

$\mathbb{C^{L}}=\operatorname*{argmax}_{i\in
n}p(\mathbb{C}=C_{i}|\mathbb{D^{N}}=x)$ (5)

The highest posterior probability’s value
($p(\mathbb{C}=C_{i},\mathbb{D^{N}}=x)$) is equal to the factorization of
$p(\mathbb{C}$ = $C_{i})$ $\prod_{j=1}^{d}$
$P(\mathbb{D}^{N}=x_{j}|\mathbb{C}=C_{i})$. Each component of $x$ is
conditionally independent based on the attribute conditional assumption. It
can also be expressed as ($log$ $p(\mathbb{C}=C_{i})$ = $\sum_{j=1}^{d}$ $log$
$p(\mathbb{D}^{N}=x_{j}$)) in a logarithm form. The conditional probabilities
$P(\mathbb{D}^{N}=x_{j}|\mathbb{C}=C_{i})$ is computed using the mean
($\mu_{i,j}$), and variance ($\sigma_{i,j}^{2}$), which are calculated for
class $j$ using the values of attribute $X$ from the training set. As a
result, the sensitivity of both $\mu_{i}$, and standard deviation
($\sigma_{i,j}$) are computed. It is assumed that the feature value of
$\mathbb{D}_{j}^{N}$ is bounded in range [$g_{i}$, $h_{i}$]. The sensitivity
of $\mu_{i,j}$ is calculated using ($h_{i}$ \- $g_{i}$)/($n$+1), and the
sensitivity of $\sigma_{i,j}$ is computed using $n$ $*$ ($h_{i}$ \- $g_{i}$) /
($n$+1). After calculating the $\mu_{i,j}$ and $\sigma_{i,j}$, $CO$ generates
the noise using the Eqs.1 to 4. The obtained noise $\hat{N}_{i}$ and
$\hat{\hat{N}}_{i}$ are injected into $\mu_{i}$ and $\sigma_{i}$,
respectively, using Eqs. 6 and 7.

$\mu_{i}^{{}^{\prime}}=\mu_{i}+\hat{N}_{i}$ (6)
$\sigma_{i}^{{}^{\prime}}=\sigma_{i}+\hat{\hat{N}}_{i}$ (7)

$CO$ uses $\mu_{i}^{{}^{\prime}}$, and $\sigma_{i}^{{}^{\prime}}$ to calculate
$p(\mathbb{C}=C_{i},\mathbb{D^{N}}=x)$. Afterword, synthetic model $CM^{N}$ is
sent to $CSP$ to handle the queries of $DO_{1}$, $DO_{2}$, $\dots$, $DO_{n}$,
and $RU_{1}$, $RU_{2}$, $\dots$, $RU_{m}$. Therefore, unauthorized users or
any attacker are unable to acquire the sensitive classification model from
$CSP$; the proof is provided in the following theorem:

######  Theorem 5.1.

In the proposed model, the privacy-preserving mechanism of the classification
model satisfies the parallel composition of $\epsilon$-differential privacy.

###### Proof.

Let $\hat{R}_{1}$, $\hat{R}_{2}$, $\dots$, $\hat{R}_{n}$ be $n$ mechanisms,
where each mechanism $\hat{R}_{i}$ ($i$ $\in$ [1, $n$]) provides
$\epsilon$-differential privacy. Let $\mu_{1}$, $\mu_{2}$, $\dots$, $\mu_{n}$
and $\sigma_{1}$, $\sigma_{2}$, $\dots$, $\sigma_{n}$ are $n$ arbitrary mean
and standard deviation to calculate the conditional probability attributes.
For a new mechanism $\hat{R}$, the sequence of
$\hat{R}$($\hat{R}_{1}$($\mu_{1}$), $\hat{R}_{2}$($\mu_{2}$), $\dots$,
$\hat{R}_{n}$($\mu_{n}$)) and $\hat{R}$($\hat{R}_{1}$($\sigma_{1}$),
$\hat{R}_{2}$($\sigma_{2}$), $\dots$, $\hat{R}_{n}$($\sigma_{n}$)) provide
$max_{1\leq i\leq n}$ $\epsilon$\- differential privacy. $CO$ generates the
Laplace noise $\hat{N}_{i}$ and $\hat{\hat{N}}_{i}$ according to sensitivity
($\Delta f$), privacy budget ($\epsilon$) to inject the produced noise into
$\mu_{1}$, $\mu_{2}$, $\dots$, $\mu_{n}$ and $\sigma_{1}$, $\sigma_{2}$,
$\dots$, $\sigma_{n}$, respectively, which satisfy $Pr$[$\hat{R}(\mu)$ =
$\vartheta$] $\leq$ $exp$($\epsilon$) $\times$
$Pr$[$\hat{R}(\mu^{{}^{\prime}})$ = $\vartheta$], and $Pr$[$\hat{R}(\sigma)$ =
$\vartheta$] $\leq$ $exp$($\epsilon$) $\times$
$Pr$[$\hat{R}(\sigma^{{}^{\prime}})$ = $\vartheta$]. It can be observed that
the mechanism $\hat{R}$ provides privacy to the classification model. Hence
this privacy-preserving mechanism of the classification model satisfies the
parallel composition of $\epsilon$-differential privacy. ∎

##  6 Data Classification

$CSP$ receives the synthetic data $D_{1}^{N_{1}}$, $D_{2}^{N_{2}}$, $\dots$,
$D_{n}^{N_{n}}$ from $DO_{1}$, $DO_{2}$, $\dots$, $DO_{n}$ and prepossesses it
by applying the normalization function shown in Eq. (8), where $D_{i}^{N_{i}}$
is training sample, $\mu$, and $\sigma$ are the mean and the standard
deviation of the training sample, respectively.

$\check{D}_{i}^{N_{i}}=\frac{(D_{i}^{N_{i}}-\mu)}{\sigma}$ (8)

$CSP$ achieves the prepossessed data $\check{\mathbb{D}}_{i}^{N_{i}}$ =
{$\check{D}_{1}^{N_{1}}$, $\check{D}_{2}^{N_{2}}$, $\dots$,
$\check{D}_{n}^{N_{n}}$}, which belongs to $n^{*}$ is less than equal to $n$
classes $\mathbb{C}$ = {$C_{1}$, $C_{2}$, $\dots$, $C_{n^{*}}$}, for all
$\mathbb{C}_{i}$ = $\mathbb{D}$, $i$ $\in$ [$1$, $n^{*}$] and $\mathbb{C}_{i}$
$\cap$ $\mathbb{C}_{k}$ = $\phi$, where $k$ $\in$ [$1$, $n^{*}$] $\wedge$
$i\neq k$. Fig. 3 depicts the stepwise process for data classification in
which the prepossessed data $\check{\mathbb{D}}_{i}^{N_{i}}$ =
{$\check{D}_{1}^{N_{1}}$, $\check{D}_{2}^{N_{2}}$, $\dots$,
$\check{D}_{n}^{N_{n}}$} is split into training data
$\check{\mathbb{D}}_{\grave{t}}^{N}$ = {$\check{D}_{\grave{t},1}^{N_{1}}$,
$\check{D}_{\grave{t},2}^{N_{2}}$, $\dots$,
$\check{D}_{\grave{t},n^{**}}^{N_{n^{**}}}$}, and testing data
$\check{\mathbb{D}}_{\grave{\grave{t}}}^{N}$ =
{$\check{D}_{\grave{\grave{t}},1}^{N_{1}}$,
$\check{D}_{\grave{\grave{t}},2}^{N_{2}}$, $\dots$,
$\check{D}_{\grave{\grave{t}},n^{***}}^{N_{n^{***}}}$} fulfill the following
conditions a) $\check{\mathbb{D}}_{\grave{t}}^{N}$ $\cup$
$\check{\mathbb{D}}_{\grave{\grave{t}}}^{N}$ = $\check{\mathbb{D}}^{N}$; b)
$\check{\mathbb{D}}_{\grave{t}}^{N}$ $\cap$
$\check{\mathbb{D}}_{\grave{\grave{t}}}^{N}$ = $\phi$; c) $n^{**}$, $n^{***}$
$\leq$ $n$; and d) $n^{**}$ = $n\times z$, and $n^{***}$ = $n\times(1-z)$,
where $z$ belongs to $\mathbb{Z}$ or the value of $z$ can lie between 0 and 1
for $CM^{N}$.

Figure 3: Classification flow for shared data and model

The utilization of training data $\check{D}_{\grave{t},1}^{N_{1}}$,
$\check{D}_{\grave{t},2}^{N_{2}}$, $\dots$,
$\check{D}_{\grave{t},n^{**}}^{N_{n^{**}}}$ is to train $CM^{N}$, whereas the
accuracy of $CM^{N}$ is assessed by the testing data
$\check{D}_{\grave{\grave{t}},1}^{N_{1}}$,
$\check{D}_{\grave{\grave{t}},2}^{N_{2}}$, $\dots$,
$\check{D}_{\grave{\grave{t}},n^{***}}^{N_{n^{***}}}$. Data objects
$\check{D}_{\grave{\grave{t}},1}^{N_{1}}$,
$\check{D}_{\grave{\grave{t}},2}^{N_{2}}$, $\dots$,
$\check{D}_{\grave{\grave{t}},n^{***}}^{N_{n^{***}}}$ are provided to $CM^{N}$
in the testing process to determine the classes. $CM^{N}$ examines
$\check{D}_{\grave{\grave{t}},1}^{N_{1}}$,
$\check{D}_{\grave{\grave{t}},2}^{N_{2}}$, $\dots$,
$\check{D}_{\grave{\grave{t}},n^{***}}^{N_{n^{***}}}$, and generates the
output as the class label vector $\mathbb{C}^{L}$ = {$C_{1}^{L}$, $C_{2}^{L}$,
$\dots$, $C_{n^{***}}^{L}$}. Using $C_{1}^{L}$, $C_{2}^{L}$, $\dots$,
$C_{n^{***}}^{L}$, the Classification Accuracy (CA) of $CM^{N}$ is measured by
applying the Eq. (9). In this Eq., $C_{1}^{L}$, $C_{2}^{L}$, $\dots$,
$C_{n^{{}^{\prime\prime\prime}}}^{L}$ indicates the number of correctly
identified samples and $C_{1}^{L}$, $C_{2}^{L}$, $\dots$, $C_{n^{***}}^{L}$ is
the total number of sample in test data, where $n^{{}^{\prime\prime\prime}}$
$\in$ [1, $n^{***}$].

$CA=\frac{C_{1}^{L},C_{2}^{L},\dots,C_{n^{{}^{\prime\prime\prime}}}^{L}}{C_{1}^{L},C_{2}^{L},\dots,C_{n^{***}}^{L}}$
(9)

The precision (P), and recall (R) are computed using Eq. (10) and (11),
respectively.

$P=\frac{(C_{1}^{L},C_{2}^{L},\dots,C_{n^{{}^{\prime\prime\prime}}}^{L})\cap(C_{1}^{L},C_{2}^{L},\dots,C_{n^{***}}^{L})}{C_{1}^{L},C_{2}^{L},\dots,C_{n^{{}^{\prime\prime\prime}}}^{L}}$
(10)
$R=\frac{(C_{1}^{L},C_{2}^{L},\dots,C_{n^{{}^{\prime\prime\prime}}}^{L})\cap(C_{1}^{L},C_{2}^{L},\dots,C_{n^{***}}^{L})}{C_{1}^{L},C_{2}^{L},\dots,C_{n^{***}}^{L}}$
(11)

The F1-Score (FS) is measured using Eq. (12).

$FS=2\times\frac{(P\times R)}{(P+R)}$ (12)

##  7 Illustration

Let’s assume that the proposed model consists of twenty owners: $DO_{id}$ =
{$DO_{1}$, $DO_{2}$, $\dots$, $DO_{20}$} possess data $D$ = {$D_{1}$, $D_{2}$,
$\dots$, $D_{20}$}. Data $D_{1}$ contains the information (4.2575, 7.1754,
-1.7685, -2.6898, 0), $D_{2}$ contains the information (0.7536, 0.9137,
6.1110, -4.0517, 0), $\dots$, $D_{20}$ contains the information (6.2122,
7.6552, -2.4515, -1.4451, 0). Before sending it to $CSP$, $DO_{1}$, $DO_{2}$,
$\dots$, $DO_{20}$ generate the noise $N_{1}$, $N_{2}$, $\dots$, $N_{20}$ by
applying the Eqs. (1) to (4). The obtained noise $N_{1}$ incorporates 0.6569,
$N_{2}$ incorporates 0.8015, $\dots$, $N_{20}$ incorporates 0.6959. Table 3
shows the synthetic data (SD) that are produced after injecting $N_{1}$,
$N_{2}$, $\dots$, $N_{20}$ into actual data (AD).

Table 3: Synthetic data v/s Actual data

Data Owners | $\Lambda_{1}$ | $\Lambda_{2}$ | $\Lambda_{3}$ | $\Lambda_{4}$ | $\Lambda_{5}$
---|---|---|---|---|---
$SD_{1}$ | $AD_{1}$ | $SD_{2}$ | $AD_{2}$ | $SD_{3}$ | $AD_{3}$ | $SD_{4}$ | $AD_{4}$
$DO_{1}$ | 4.9144 | 4.2575 | 7.8223 | 7.1754 | -1.1116 | -1.7685 | -2.0329 | -2.6898 | 0
$DO_{2}$ | 1.5551 | 0.7536 | 1.7152 | 0.9137 | 6.9125 | 6.1110 | -3.2502 | -4.0517 | 0
$DO_{3}$ | 1.3112 | 1.0978 | -2.6567 | -2.8701 | 2.8156 | 2.6022 | -1.8123 | -2.0257 | 0
$DO_{4}$ | 2.9124 | 2.1312 | -2.3231 | -3.1043 | 3.4122 | 2.631 | -2.9158 | -3.6970 | 1
$DO_{5}$ | 5.1451 | 4.9639 | 3.4256 | 3.2444 | -1.6823 | -1.8635 | 1.2151 | 1.0339 | 1
$DO_{6}$ | 1.0132 | 0.7983 | 5.7588 | 5.5439 | 0.4724 | 0.2575 | -0.6125 | -0.8274 | 0
$DO_{7}$ | 2.5156 | 1.9033 | 9.1772 | 8.5649 | -0.7336 | -1.3459 | -0.7353 | -1.3476 | 0
$DO_{8}$ | 4.4157 | 4.2046 | 8.7779 | 8.5668 | -4.4035 | -4.6146 | -0.8064 | -1.0175 | 0
$DO_{9}$ | 1.9877 | 1.6659 | 1.1038 | 0.7820 | 2.3946 | 2.0728 | 0.8629 | 0.5411 | 1
$DO_{10}$ | 2.9935 | 2.5039 | 7.6625 | 7.1729 | 0.1539 | -0.3357 | -1.0175 | -1.5071 | 1
$DO_{11}$ | -2.6128 | -2.9317 | 10.8430 | 10.5241 | 2.5462 | 2.2273 | -2.7926 | -3.1115 | 0
$DO_{12}$ | 2.4121 | 2.012 | 8.7261 | 8.3260 | -3.0030 | -3.4031 | -0.5724 | -0.9725 | 0
$DO_{13}$ | 3.6651 | 2.7939 | -3.3924 | -4.2636 | 3.4896 | 2.6184 | 1.4771 | 0.6059 | 0
$DO_{14}$ | 1.7219 | 1.5800 | 3.0646 | 2.927 | 0.8747 | 0.7328 | 0.5861 | 0.4442 | 1
$DO_{15}$ | 1.4512 | 1.2401 | 2.8473 | 2.6362 | 4.3439 | 4.1328 | 1.6524 | 1.4413 | 1
$DO_{16}$ | 2.7159 | 2.5497 | -4.0257 | -4.1919 | 8.3428 | 8.1766 | -2.1086 | -2.2748 | 0
$DO_{17}$ | 3.2153 | 2.7866 | 11.0272 | 10.5985 | -2.3564 | -2.7851 | -2.1113 | -2.5400 | 0
$DO_{18}$ | 1.7121 | 1.5903 | 7.8902 | 7.7684 | -1.3663 | -1.4881 | -1.5650 | -1.6868 | 0
$DO_{19}$ | 1.2192 | 0.9812 | -1.1117 | -1.3496 | 2.7118 | 2.4739 | -1.4445 | -1.6824 | 1
$DO_{20}$ | 6.9081 | 6.2122 | 8.3511 | 7.6552 | -1.7556 | -2.4515 | -0.7492 | -1.4451 | 1

To preserve the privacy of CM, $CO$ also generates the noise $\hat{N}_{1}$,
$\hat{N}_{2}$ using Eqs. (1) to (4). It is supposed that the value of
$\hat{N}_{1}$ is 0.5654, and $\hat{N}_{2}$ is 0.3651. Initially, $CO$ measures
the sensitivity of mean ($\mu$), and standard deviation ($\sigma$) by adopting
Eq. (5). It is observed that the value of $\mu$ is 16.3258, and the value of
$\sigma$ is 9.6894. Afterward, $\hat{N}_{1}$, and $\hat{N}_{2}$ are injected
into $\mu$ and $\sigma$, respectively, by applying Eqs. (6) and (7). The
synthetic model $CM^{N}$, and SD are uploaded to cloud platform. $CSP$
preprocesses SD using Eq. (8) and acquires the preprocessed data as
$\check{D}^{N}$ as (1.2456, 2.4523, -0.4452, 1.3425), (-2.5434, -0.8145,
1.9315, 1.3155), $\dots$, (-1.3141, 2.5145, -1.3145, -1.0485). The $CM^{N}$
accepts 30 input data, and provides 30 output labels. The CA of $CM^{N}$ is
calculated by using Eq. (9). If $CM^{N}$ identifies 18 correct labels then
$CM^{N}$’s CA is 60% ((18/30) * 100), where 30 is the total number of test
items. The P of $CM^{N}$ is measured by applying Eq. (10). If $C_{1}^{L}$,
$C_{2}^{L}$, $\dots$, $C_{n^{{}^{\prime\prime\prime}}}^{L}$ is 10,
$C_{1}^{L}$, $C_{2}^{L}$, $\dots$, $C_{n^{***}}^{L}$ is 30, and common labels
among them is 6 then $CM^{N}$’s P is 60% ((6/10) * 100). The R of $CM^{N}$ is
computed by adopting Eq. (11). The $CM^{N}$’s R is 20% ((6/30) * 100). The FS
of $CM^{N}$ is estimated by using Eq. (12). The $CM^{N}$’s FS is 30% (2 * (P *
R) / (P \+ R)), where P is 60%, and R is 20%.

##  8 Operational Design and Computational Complexity

The operational summary of the proposed model is described by Algorithm 1,
which protects the data as well as the model and performs the classification
task. Initially, the list of data ($D$), noise ($N$), and preprocessed data
$\check{D}^{N}$ are initialized. To preserve the privacy of the owner’s data
$D$, $N$ is generated. The obtained $N$ is used to construct the synthetic
data after injecting it into $D$. To protect the classification model, the
sensitivity of $\mu$ is calculated. Based on this sensitivity, the scale
factor is measured for $\mu$. Using the scale factor, the noise is generated
and adds it into $\mu$. Similarly, the sensitivity of $\sigma$ is also
computed, and the scale factor is determined using this sensitivity.

Input: Actual data $D$, Noise vector $N$, Privacy parameter $\epsilon$

Output: unknown class label $C_{i}^{L}$, CA, P, R, and FS

1 Initialize data $D$ := {$D_{1}$, $D_{2}$, $\dots$, $D_{n}$}, $N$ :=
{$N_{1}$, $N_{2}$, $\dots$, $N_{n}$}, $\check{D}_{1}^{N}$ = {$(x_{1},y_{1})$,
$(x_{2},y_{2})$, $\dots$, $(x_{\ddot{i}},y_{\ddot{j}})$}

2 for $i$ = $1$, $2$, $\dots$, $n$ do

3 $N_{i}$ = Lap(0,1)

4 $D_{i}^{N}$ = $D_{i}$ \+ $N_{i}$

5 for each attribute $X_{j}$ do

6 Compute the sensitivity $S^{\mu}$ with mean $\mu$ with bound [$g_{i}$,
$h_{i}$]

7 Scale factor for $\mu$ ($Sf^{\mu}$) $\leftarrow$ $S^{\mu}$ / $\epsilon$

8 $\mu_{i}^{{}^{\prime}}$ = $\mu_{i}$ \+ Lap(0, $Sf^{\mu}$)

9 Compute the sensitivity $S^{\sigma}$ with standard deviation $\sigma$ with
bound [$g_{i}$, $h_{i}$]

10 Scale factor for $\sigma$ ($Sf^{\sigma}$) $\leftarrow$ $S^{\sigma}$ /
$\epsilon$

11 $\sigma_{i}^{{}^{\prime}}$ = $\sigma_{i}$ \+ Lap(0, $Sf^{\sigma}$)

12 Use $\mu_{i}^{{}^{\prime}}$, and $\sigma_{i}^{{}^{\prime}}$ to compute
$P(\mathbb{D}^{N}=x_{j}|\mathbb{C}=C_{i})$

13 end for

14 for each class $c_{i}$ do

15 Count $nc_{i}^{{}^{\prime}}$ $\leftarrow$ $nc_{i}$ \+ Lap (0,1)

16 Use $nc_{i}^{{}^{\prime}}$ to compute the prior $P(c_{i})$

17 end for

18 $C_{i}^{L}$ = $argmax_{c}$ $P(C)$ $\prod$ $P(X_{j}\mid c)$

19 end for

20 CA = (#$Correctly$ $classified$ $sample$ / #$test$ $sample$) * 100

21 P = {relevant data labels $\cap$ retrieved data labels} / {retrieved data
labels}

22 R = {relevant data labels $\cap$ retrieved data labels} / {relevant data
labels}

FS = $2*(\textit{P}*\textit{R})/(\textit{P}+\textit{R})$

Algorithm 1 DA-PMLM operational summary

Based on this scale factor, noise is produced and added into $\sigma$. The
synthetic $\mu$, and $\sigma$ are used to compute
$P(\mathbb{D}^{N}=x_{j}|\mathbb{C}=C_{i})$. The test data is given and
received class labels from the classification model. Using these class labels,
the classification parameter values are measured in a privacy-preserving
manner.

In algorithm 1, steps 2 to 19 perform the classification over synthetic data
using a synthetic model, whose time complexity depends on the noise
generation, noise addition, and classifier use. To preserve the privacy of the
data, noise is generated in step 3 and added using the Laplace mechanism in
step 4, which takes $\mathcal{O}(n^{2})$ time, where $n$ is the total rows in
the dataset. The classification model is protected using steps 5 to 18. The NB
classifier takes $\mathcal{O}(ni)$ time. The computation time for the
sensitivity of $\mu$ and $\sigma$ is $\mathcal{O}(n^{2})$ time. Steps 20 to 23
require $\mathcal{O}(1)$ time. Therefore, the total time complexity of DA-PMLM
is $\mathcal{O}(n^{2})$.

##  9 Performance Evaluation

###  9.1 Experimental Setup

The experiments are conducted on a system equipped with Intel (R) Core (TM)
i5-4210U CPU and 3.60 GHz clock speed. The computation machine is deployed
with 64-bit Ubuntu along with 8 GB of main memory. The classification tasks
are accomplished using Python 2.7.15 programming language. The NB classifier
has been used over testing data.

###  9.2 Datasets and Classification Parameters

Heart Disease, Iris, Balance Scale, and Nursery datasets, all of which were
obtained from the UCI Machine Learning Repository [37] to train $CM$. These
datasets have 75, 5, 5, 9 attributes and 303, 150, 625, 12960 instances. The
description of the datasets is shown in Table 4.

The 9/10 of the data from the complete dataset is utilized as training data,
while the remaining is used as test data for the purpose of training the CM.
The machine learning tasks are accomplished over the clean data, DA-PMLM, PMLM
[16], MLPAM [23], DPEL [33], and PDLM [18]. We have used the Laplace mechanism
to generate the noise. However, DA-PMLM, PMLM [16], MLPAM [23], as well as
DPEL [33] schemes contain noise, and PDLM [18] scheme is based on a
homomorphic encryption technique. The results of CM are evaluated using test
data, and CA, P, R, and FS are computed from these results.

Table 4: Basic information of four datasets Dataset | #Instances | #Features | #Classes | Samples in | Samples in
---|---|---|---|---|---
|  |  |  | training set | test set
Heart Disease | 303 | 75 | 2 | 272 | 31
Iris | 150 | 5 | 3 | 135 | 15
Balance Scale | 625 | 5 | 3 | 562 | 63
Nursery | 12960 | 9 | 5 | 11664 | 1296

###  9.3 Results

The $CM$ acquires the classification results including CA, P, R, and FS over
the clean data, DA-PMLM, PMLM [16], MLPAM [23], DPEL [33], and PDLM [18] as
shown in Figs. 4(a)-(d) to 7(a)-(d). In DA-PMLM, the maximal value of CA is
93.33% on the Iris dataset with 2.0 privacy budget. The minimal value of CA is
33.87% on the Nursery dataset with 0.01 privacy budget. The average value of
CA is 70.04%, 74.97%, 61.44%, and 46.28% over Heart Disease, Iris, Balance
Scale, and Nursery datasets, respectively. The greatest value of P is 94.44%
on the Iris dataset with 2.0 privacy budget. The lowest value of P is 31.57%
on the Nursery dataset with 0.01 privacy budget. The average value of P is
70.53%, 72.67%, 59.66%, and 44.89% over Heart Disease, Iris, Balance Scale,
and Nursery datasets, respectively. The maximum value of R is 93.33% on the
Iris dataset with 2.0 privacy budget.

0.010.050.10.51.01.52.0$0$$0.2$$0.4$$0.6$$0.8$Privacy BudgetAccuracyClean
DataPMLM [16], MLPAM [23]DPEL [33]PDLM [18]DA-PMLM (a) Heart Disease

0.010.050.10.51.01.52.0$0$$0.2$$0.4$$0.6$$0.8$$1$Privacy BudgetAccuracyClean
DataPMLM [16], MLPAM [23]DPEL [33]PDLM [18]DA-PMLM  (b) Iris

0.010.050.10.51.01.52.0$0$$0.2$$0.4$$0.6$$0.8$Privacy BudgetAccuracyClean
DataPMLM [16], MLPAM [23]DPEL [33]PDLM [18]DA-PMLM (c) Balance

0.010.050.10.51.01.52.0$0$$0.2$$0.4$$0.6$$0.8$Privacy BudgetAccuracyClean
DataPMLM [16], MLPAM [23]DPEL [33]PDLM [18]DA-PMLM  (d) Nursery

Figure 4: Accuracy of CM in DA-PMLM

The minimum value of R is 33.87% on the Nursery dataset with 0.01 privacy
budget. The average value of R is 70.04%, 74.97%, 61.44%, and 46.28% over
Heart Disease, Iris, Balance Scale, and Nursery datasets, respectively. The
highest value of FS is 93.33% on the Iris dataset with 2.0 privacy budget. The
lowest value of FS is 28.66% on the Nursery dataset with 0.01 privacy budget.
The average value of FS is 69.30%, 73.13%, 58.43%, and 43.64% over Heart
Disease, Iris, Balance Scale, and Nursery datasets, respectively. The
performance of the datasets decreases in order: Iris $>$ Heart Disease $>$
Balance Scale $>$ Nursery datasets.

0.010.050.10.51.01.52.0$0$$0.2$$0.4$$0.6$$0.8$Privacy BudgetPrecisionClean
DataPMLM [16], MLPAM [23]DPEL [33]PDLM [18]DA-PMLM  (a) Heart Disease

0.010.050.10.51.01.52.0$0$$0.2$$0.4$$0.6$$0.8$$1$Privacy BudgetPrecisionClean
DataPMLM [16], MLPAM [23]DPEL [33]PDLM [18]DA-PMLM  (b) Iris

0.010.050.10.51.01.52.0$0$$0.2$$0.4$$0.6$$0.8$Privacy BudgetPrecisionClean
DataPMLM [16], MLPAM [23]DPEL [33]PDLM [18]DA-PMLM  (c) Balance

0.010.050.10.51.01.52.0$0$$0.2$$0.4$$0.6$$0.8$$1$Privacy BudgetPrecisionClean
DataPMLM [16], MLPAM [23]DPEL [33]PDLM [18]DA-PMLM  (d) Nursery

Figure 5: Precision of CM in DA-PMLM

0.010.050.10.51.01.52.0$0$$0.2$$0.4$$0.6$$0.8$$1$Privacy BudgetRecallClean
DataPMLM [16], MLPAM [23]DPEL [33]PDLM [18]DA-PMLM  (a) Heart Disease

0.010.050.10.51.01.52.0$0$$0.2$$0.4$$0.6$$0.8$$1$Privacy BudgetRecallClean
DataPMLM [16], MLPAM [23]DPEL [33]PDLM [18]DA-PMLM  (b) Iris

0.010.050.10.51.01.52.0$0$$0.2$$0.4$$0.6$$0.8$$1$Privacy BudgetRecallClean
DataPMLM [16], MLPAM [23]DPEL [33]PDLM [18]DA-PMLM (c) Balance

0.010.050.10.51.01.52.0$0$$0.2$$0.4$$0.6$$0.8$Privacy BudgetRecallClean
DataPMLM [16], MLPAM [23]DPEL[33]PDLM [18]DA-PMLM (d) Nursery

Figure 6: Recall of CM in DA-PMLM

0.010.050.10.51.01.52.0$0$$0.2$$0.4$$0.6$$0.8$$1$Privacy BudgetFSClean
DataPMLM [16], MLPAM [23]DPEL [33]PDLM [18]DA-PMLM (a) Heart Disease

0.010.050.10.51.01.52.0$0$$0.2$$0.4$$0.6$$0.8$$1$Privacy BudgetFSClean
DataPMLM [16], MLPAM [23]DPEL [33]PDLM [18]DA-PMLM (b) Iris

0.010.050.10.51.01.52.0$0$$0.2$$0.4$$0.6$$0.8$Privacy BudgetFSClean DataPMLM
[16], MLPAM [23]DPEL[33]PDLM [18]DA-PMLM (c) Balance

0.010.050.10.51.01.52.0$0$$0.2$$0.4$$0.6$$0.8$Privacy BudgetFSClean DataPMLM
[16], MLPAM [23]DPEL [33]PDLM [18]DA-PMLM (d) Nursery

Figure 7: F1-Score of CM in DA-PMLM

###  9.4 Comparison

The results of the experiments are compared to clean data, PMLM [16], MLPAM
[23], DPEL [33], and PDLM [18], which are also deployed on the same platform
(Figs. 4(a)-(d) to 7(a)-(d)). The parameters (CA, P, R, and FS) results for
DA-PMLM are less than the results of Clean, PMLM [16], MLPAM [23], and DPEL
[33] in all the cases because of the noise addition in the data and
classification model. From Table 5, it is observed that the highest difference
for CA between DA-PMLM and DPEL is 13.35% on the Nursey dataset with 0.5
privacy budget. The lowest difference is found 0.0% on the Heart Disease
dataset with 0.1, 0.5, 1.0, 1.5, and 2.0 privacy budget, and on the Iris
dataset with 0.1 privacy budget. Besides, the maximum gap for P is 17.16% on
the Nursery dataset with 0.5 privacy budget, but the lowest differences are
found 0.11% on the Heart Disease dataset with 2.0 privacy budget. The maximum
decrement for R of DA-PMLM is 13.35% from DPEL on the Nursey dataset with 0.5
privacy budget. In contrast, the smallest decrement is 0.0% on the Heart
Disease dataset with 0.1, 0.5, 1.0, 1.5, and 2.0 privacy budget, and on the
Iris dataset with 0.1 privacy budget. The highest difference for FS is 14.05%
on the Iris dataset with 0.01 privacy budget, while the lowest difference for
FS is 0.1% on the Heart Disease dataset with 1.0 privacy budget.

Additionally, the parameters (CA, P, R, and FS) results for DA-PMLM are less
than the results of PMLM and MLPAM in all the cases due to noise addition in
the model. The maximum gap for CA is 9.25% on the Nursey dataset with 1.0
privacy budget, but the smallest gap is found 1.0% on the Heart Disease with
0.05 privacy budget. Similarly, the highest difference for P is 19.33% on the
Iris with 0.01 privacy budget, while the lowest difference is found 0.16% on
the Heart Disease dataset with 0.05 privacy budget. The R of DA-PMLM the
maximum decrement by 8.72% from PMLM, MLPAM on the Nursery dataset with 1.0
privacy budget, whereas the smallest decrement is 1.0% on the Heart Disease
dataset with 0.05 privacy budget. The highest difference for FS is 13.29% on
the Iris dataset with 0.05 privacy budget, but the lowest difference for FS is
0.18% on the Balance dataset with 0.5 privacy budget.

Table 5: Accuracy, precision, recall, and f1-score; DA-PMLM v/s state-of-the-
art works [16], [23], [33] and the baseline schemes

Dataset Epsilon Decrements in the value of parameters Accuracy Precision
Recall F1-Score Clean data [16], [23] [33] Clean data [16], [23] [33] Clean
data [16], [23] [33] Clean data [16], [23] [33] 0.01 25.81 3.23 3.23 24.37
1.97 2.29 25.81 3.23 3.23 25.44 2.88 3.05 0.05 22.58 1 3.22 21.62 0.16 4.11
22.58 1 3.22 22.96 1.62 4.46 Heart 0.1 16.13 1.11 0 17.21 4.08 1.34 16.13 1.11
0 17.71 2.91 1.62 Disease 0.5 12.91 2.81 0 13.17 3.59 1.43 12.91 2.81 0 13.57
3.6 0.65 1.0 9.68 1.21 0 9.55 1.29 0.21 9.68 1.21 0 9.73 1.04 0.1 1.5 6.46
1.27 0 6.27 3.16 1.94 6.46 1.27 0 7.66 2.66 1.41 2.0 3.23 1.32 0 2.32 1.07
0.11 3.23 1.32 0 4.13 2.55 1.22 0.01 40 6.67 6.67 51.11 19.33 14.64 40 6.67
6.67 46.71 13.29 14.05 0.05 33.33 6.66 6.66 36.67 19.08 11.23 33.33 6.66 6.66
34.04 5.74 8.2 0.1 20 6.67 0 30.74 14.63 15.34 20 6.67 0 25.21 6.42 5.72 Iris
0.5 15.18 4.32 1.85 14.45 3.56 6.67 15.18 4.32 1.85 15.01 1.58 1.12 1.0 13.33
6.66 6.66 12.53 5.59 8.09 13.33 6.66 6.66 13.08 6.22 5.1 1.5 6.67 6.67 6.67
6.83 7.05 6.55 6.67 6.67 6.67 7.56 7.63 7.07 2.0 0 6.67 6.67 0 2.22 0.22 0
6.67 6.67 0.04 0.81 0.11 0.01 36.51 4.76 6.35 31.72 7.42 13.05 38.51 4.76 6.35
35.61 7.94 7.5 0.05 33.33 4.76 6.35 22.65 0.57 9.09 35.33 4.76 6.35 29.26 4.71
10.11 0.1 28.57 3.17 6.35 20.95 1.39 10.13 30.57 2.99 6.35 28.76 7.77 11.38
Balance 0.5 22.22 1.58 3.17 14.42 2.21 4.39 24.22 1.58 3.17 16.98 0.18 2.33
1.0 19.05 3.18 3.18 12.41 1.92 7.08 21.05 3.18 3.18 15.15 2.54 5.46 1.5 17.46
4.76 4.76 8.74 0.89 4.57 19.46 4.86 4.76 12.63 2.88 4.31 2.0 12.7 6.35 7.94
5.33 2.11 3.83 14.7 6.35 7.94 8.23 4.92 5.78 0.01 42.28 1.08 1.31 58.75 2.68
4.3 42.28 1.08 1.31 51.45 4.63 3.28 0.05 41.97 3.01 4.09 57.11 4.96 4.59 41.97
3.01 4.09 50.95 6.09 8.16 0.1 40.89 3.62 4.86 56.22 3.57 6.03 40.89 3.62 4.86
49.2 6.58 8.13 Nursery 0.5 37.81 6.02 13.35 53.37 6.07 17.16 37.81 6.02 13.35
42.51 6.03 8.65 1.0 25.61 9.25 11.96 41.22 8.91 12.78 25.61 9.25 11.96 31.2
9.64 13.11 1.5 15.58 8.18 5.71 31.21 10.42 5.33 15.58 8.18 5.71 20.29 7.92
4.97 2.0 4.94 3.17 1.08 20.12 4.59 1.17 4.94 3.17 1.08 9.66 3.03 1.19

Table 6: Improvement in the values of CA, P, R, and FS of DA-PMLM vs PDLM [18] Dataset | Epsilon | Increments in the value of parameters
---|---|---
|  | CA | P | R | FS
| 0.01 | -9.71 | -8.33 | -9.71 | -8.34
| 0.05 | -6.48 | -5.58 | -6.48 | -5.86
Heart | 0.1 | -0.03 | -1.17 | -0.03 | -0.61
Disease | 0.5 | 3.19 | 2.87 | 3.19 | 3.53
| 1.0 | 6.42 | 6.49 | 6.42 | 7.37
| 1.5 | 9.64 | 9.77 | 9.64 | 9.44
| 2.0 | 12.87 | 13.72 | 12.87 | 12.97
| 0.01 | -23.05 | -30.95 | -23.05 | -23.34
| 0.05 | -16.38 | -16.51 | -16.38 | -10.67
| 0.1 | -3.05 | -10.58 | -3.05 | -1.84
Iris | 0.5 | 1.77 | 5.71 | 1.77 | 8.36
| 1.0 | 3.62 | 7.63 | 3.62 | 10.29
| 1.5 | 10.28 | 13.33 | 10.28 | 15.81
| 2.0 | 16.95 | 20.16 | 16.95 | 23.33
| 0.01 | -23.73 | -23.45 | -23.73 | -24.76
| 0.05 | -20.55 | -14.95 | -20.55 | -18.41
| 0.1 | -15.79 | -12.68 | -15.79 | -17.91
Balance | 0.5 | -9.44 | -6.15 | -9.44 | -6.13
| 1.0 | -6.27 | -4.14 | -6.27 | -4.3
| 1.5 | -4.68 | -0.47 | -4.68 | -1.78
| 2.0 | 0.08 | 2.94 | 0.08 | 2.62
| 0.01 | -33.16 | -35.09 | -33.16 | -34.79
| 0.05 | -32.85 | -33.45 | -32.85 | -34.29
| 0.1 | -31.77 | -32.56 | -31.77 | -32.54
Nursery | 0.5 | -28.69 | -29.71 | -28.69 | -25.85
| 1.0 | -16.49 | -17.56 | -16.49 | -14.54
| 1.5 | -6.46 | -7.55 | -6.46 | -3.63
| 2.0 | 4.18 | 3.54 | 4.18 | 7.00

Likewise, DA-PMLM outperforms PDLM [18] because the proposed model’s
efficiency is increased by performing fewer calculations on synthetic data
instead of ciphertext. Table 6 shows that the improvement ranges for CA, P, R,
and FS are 0.08% to 16.95%, 2.87% to 20.16%, 0.08% to 16.95%, and 2.62% to
23.33% over all datasets.

Moreover, the parameters (CA, P, R, and FS) results for DA-PMLM are less than
the results of clean data in all the cases due to noise addition. The maximum
gap for CA is 42.28% on the Nursey dataset with 0.01 privacy budget, but the
lowest gap is found 0.0% on the Iris dataset with 2.0 privacy budget.
Similarly, the highest difference for P is 58.75% on the Nursey dataset with
0.01 privacy budget, while the lowest difference is found 0.0% on the Iris
Disease with 2.0 privacy budget. The maximum decrement for R of DA-PMLM is
42.28% from clean data on the Nursey dataset with 0.01 privacy budget, whereas
the smallest decrement is 0.0% on the Iris dataset with 2.0 privacy budget.
The highest difference for FS is 51.45% on the Nursey dataset with 0.01
privacy budget, but the lowest difference for FS is 0.04% on the Iris dataset
with 2.0 privacy budget. However, The parameters (CA, P, R, and FS) results
for DA-PMLM are less or equal and provide more protection than the clean data,
PMLM, MLPAM, DPEL, and PDLM. The proposed model acquires a trade-off between
accuracy loss and privacy improvement when the noise is added using individual
privacy constraints.

Furthermore, a comprehensive feature analysis along with a comparison of DA-
PMLM against the state-of-the-art works [16], [23], [33], [18] is performed.
From Table 7, it is observed that DA-PMLM is the only model that preserves the
privacy of data and classification model. It also permits several untrusted
entities, data owners, request users, and classifier owners to participate.
DA-PMLM works on the dynamic privacy budget. In DA-PMLM, the use of the
privacy budget is according to the level of protection required. Therefore,
the performance of DA-PMLM outperforms than the existing schemes [16], [23],
[33], [18].

Table 7: Comparative analysis of relevant schemes

Schemes | UE | DO | RU | CO | Privacy | Data | Model
---|---|---|---|---|---|---|---
|  |  |  |  | Budget | Privacy | Privacy
PMLM [16] | multiple | multiple | no | $\times$ | static | $\checkmark$ | $\times$
MLPAM [23] | multiple | multiple | multiple | $\times$ | static | $\checkmark$ | $\times$
DPEL [33] | single | single | no | $\times$ | dynamic | $\times$ | $\checkmark$
PDLM [18] | single | multiple | no | $\times$ | no | $\checkmark$ | $\times$
DA-PMLM | multiple | multiple | multiple | $\checkmark$ | dynamic | $\checkmark$ | $\checkmark$

###  9.5 Statistical Analysis

Statistical analysis is used to validate the CA, P, R, and FS of the model. In
this context, the non-parametric test is applied to the dataset that is not
normally distributed. The null hypothesis states that the acquired results
from different methods are statistically identical in the Wilcoxon signed-rank
test. This test compares the performance of DA-PMLM to that of the existing
DPEL [33], PMLM [16], MLPAM [23], and PDLM [18] models. The test is run on the
dataset with a significance level [38] (p-value) of 0.05 to determine the
importance of classifying parameters. Table 8 demonstrates the results of the
test statistics.

Table 8: Wilcoxon test statistics (p-value is 0.05)

Dataset | Classification | Comparison of | Comparison of DA-PMLM, | Comparison of
---|---|---|---|---
| Parameters | DA-PMLM & DPEL [33] | PMLM [16] & MLPAM [23] | DA-PMLM & PDLM [18]
|  | p-value | Result | p-value | Result | p-value | Result
| Accuracy | 0.180 | Accepted | 0.018 | Rejected | 0.612 | Accepted
Heart | Precision | 0.018 | Rejected | 0.018 | Rejected | 0.398 | Accepted
Disease | Recall | 0.180 | Accepted | 0.018 | Rejected | 0.612 | Accepted
| F1-score | 0.018 | Rejected | 0.018 | Rejected | 0.398 | Accepted
| Accuracy | 0.026 | Rejected | 0.016 | Rejected | 1.000 | Accepted
Iris | Precision | 0.018 | Rejected | 0.018 | Rejected | 0.866 | Accepted
| Recall | 0.026 | Rejected | 0.016 | Rejected | 1.000 | Accepted
| F1-score | 0.018 | Rejected | 0.018 | Rejected | 0.735 | Accepted
| Accuracy | 0.017 | Rejected | 0.018 | Rejected | 0.028 | Rejected
Balance | Precision | 0.018 | Rejected | 0.018 | Rejected | 0.043 | Rejected
Scale | Recall | 0.017 | Rejected | 0.017 | Rejected | 0.028 | Rejected
| F1-score | 0.018 | Rejected | 0.018 | Rejected | 0.043 | Rejected
| Accuracy | 0.018 | Rejected | 0.018 | Rejected | 0.028 | Rejected
Nursery | Precision | 0.018 | Rejected | 0.018 | Rejected | 0.028 | Rejected
| Recall | 0.018 | Rejected | 0.018 | Rejected | 0.028 | Rejected
| F1-score | 0.018 | Rejected | 0.018 | Rejected | 0.043 | Rejected

While comparing DA-PMLM and DPEL, it is observed that the null hypothesis for
P, and FS is rejected but accepted for CA, and R on the Heart Disease dataset.
The null hypothesis is rejected for CA, P, R, and FS on the Iris, Balance
Scale, and Nursery dataset, because their p-values are less than 0.05.
Similarly, comparing DA-PMLM, PMLM, and MLPAM, the null hypothesis is rejected
for CA, P, R, and FS on the Heart Disease, Iris, Balance Scale, and Nursery
datasets. Moreover, comparing DA-PMLM and PDLM, the null hypothesis is
accepted for CA, P, R, and FS on the Heart Disease, Iris datasets, but
rejected for Balance Scale, and Nursery datasets. Based on the achieved
statistics results, DA-PMLM improves CA, P, R, FS, which demonstrates the
superiority of the proposed model.

##  10 Conclusion

This paper proposed a novel model named DA-PMLM that protects the sensitive
data and classification model outsourced by multiple owners in a real cloud
environment. DA-PMLM permits the various data owners to store and analyze
their data and the classifier owner to share the model to handle the multiple
requests in the cloud. In this work, data owners inject varying statistical
noise to sensitive data according to their queries as well as classifier owner
adds the statistical noise into the model to preserve privacy. The experiments
have been performed, and the results demonstrate that the parameter’s values
of classification of DA-PMLM (accuracy, precision, recall, and f1-score) are
degraded by increased privacy (i.e., lower $\epsilon$). However, the results
also illustrate that more elevated $\epsilon$ can enhance the parameter’s
classification values of DA-PMLM. The performance of DA-PMLM on the prominent
data sets is more secure, efficient, and optimal than existing works in this
regard. As part of our future work, we will devise a more efficient privacy-
preserving mechanism that reduces performance degradation.

## References

  * [1] Daiji Iwasa, Teruaki Hayashi, and Yukio Ohsawa.  Development and evaluation of a new platform for accelerating cross-domain data exchange and cooperation.  New Generation Computing, 38(1):65–96, 2020.
  * [2] Shengmin Xu, Guomin Yang, Yi Mu, and Robert H Deng.  Secure fine-grained access control and data sharing for dynamic groups in the cloud.  IEEE Transactions on Information Forensics and Security, 13(8):2101–2113, 2018.
  * [3] Christos Stergiou and Kostas E Psannis.  Efficient and secure big data delivery in cloud computing.  Multimedia Tools and Applications, 76(21):22803–22822, 2017.
  * [4] Iqbal H Sarker.  Machine learning: Algorithms, real-world applications and research directions.  SN Computer Science, 2(3):1–21, 2021.
  * [5] Ali Batouche and Hamid Jahankhani.  A comprehensive approach to android malware detection using machine learning.  In Information Security Technologies for Controlling Pandemics, pages 171–212. Springer, 2021.
  * [6] Amal Ghorbel, Mahmoud Ghorbel, and Mohamed Jmaiel.  Privacy in cloud computing environments: a survey and research challenges.  The Journal of Supercomputing, 73(6):2763–2800, 2017.
  * [7] Mazhar Ali, Samee U Khan, and Athanasios V Vasilakos.  Security in cloud computing: Opportunities and challenges.  Information sciences, 305:357–383, 2015.
  * [8] Wenting Shen, Jing Qin, Jia Yu, Rong Hao, and Jiankun Hu.  Enabling identity-based integrity auditing and data sharing with sensitive information hiding for secure cloud storage.  IEEE Transactions on Information Forensics and Security, 14(2):331–346, 2018.
  * [9] Cisco.  Cisco Secure.  https://www.cisco.com/c/dam/en_us/about/doing_business/trust-center/docs/cisco-cybersecurity-series-2021-cps.pdf, 2021\.  [Online; accessed 2021].
  * [10] Lifei Wei, Haojin Zhu, Zhenfu Cao, Xiaolei Dong, Weiwei Jia, Yunlu Chen, and Athanasios V Vasilakos.  Security and privacy for storage and computation in cloud computing.  Information sciences, 258:371–386, 2014.
  * [11] Amirhossein Peyvandi, Babak Majidi, Soodeh Peyvandi, and Jagdish Patra.  Computer-aided-diagnosis as a service on decentralized medical cloud for efficient and rapid emergency response intelligence.  New Generation Computing, pages 1–24, 2021.
  * [12] Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith.  Calibrating noise to sensitivity in private data analysis.  In Theory of cryptography conference, pages 265–284. Springer, 2006\.
  * [13] Xiaodong Zhao, Yulan Dong, and Dechang Pi.  Novel trajectory data publishing method under differential privacy.  Expert Systems with Applications, 138:112791, 2019.
  * [14] Xianjin Fang, Qingkui Zeng, and Gaoming Yang.  Local differential privacy for data streams.  In International Conference on Security and Privacy in Digital Economy, pages 143–160. Springer, 2020.
  * [15] Danan Thilakanathan, Shiping Chen, Surya Nepal, and Rafael A Calvo.  Secure data sharing in the cloud.  In Security, privacy and trust in cloud systems, pages 45–72. Springer, 2014.
  * [16] Ping Li, Tong Li, Heng Ye, Jin Li, Xiaofeng Chen, and Yang Xiang.  Privacy-preserving machine learning with multiple data providers.  Future Generation Computer Systems, 87:341–350, 2018.
  * [17] Zhibo Wang, Xiaoyi Pang, Yahong Chen, Huajie Shao, Qian Wang, Libing Wu, Honglong Chen, and Hairong Qi.  Privacy-preserving crowd-sourced statistical data publishing with an untrusted server.  IEEE Transactions on Mobile Computing, 18(6):1356–1367, 2018.
  * [18] Xindi Ma, Jianfeng Ma, Hui Li, Qi Jiang, and Sheng Gao.  Pdlm: Privacy-preserving deep learning model on cloud with multiple keys.  IEEE Transactions on Services Computing, 14(4):1251–1263, 2018\.
  * [19] Alzubair Hassan, Rafik Hamza, Hongyang Yan, and Ping Li.  An efficient outsourced privacy preserving machine learning scheme with public verifiability.  IEEE Access, 7:146322–146330, 2019.
  * [20] Weibei Fan, Jing He, Mengjiao Guo, Peng Li, Zhijie Han, and Ruchuan Wang.  Privacy preserving classification on local differential privacy in data centers.  Journal of Parallel and Distributed Computing, 135:70–82, 2020\.
  * [21] Kang Wei, Jun Li, Ming Ding, Chuan Ma, Howard H Yang, Farhad Farokhi, Shi Jin, Tony QS Quek, and H Vincent Poor.  Federated learning with differential privacy: Algorithms and performance analysis.  IEEE Transactions on Information Forensics and Security, 15:3454–3469, 2020.
  * [22] Xiaoyuan Liu, Hongwei Li, Guowen Xu, Rongxing Lu, and Miao He.  Adaptive privacy-preserving federated learning.  Peer-to-Peer Networking and Applications, 13(6):2356–2366, 2020\.
  * [23] Ishu Gupta, Rishabh Gupta, Ashutosh Kumar Singh, and Rajkumar Buyya.  Mlpam: A machine learning and probabilistic analysis based model for preserving security and privacy in cloud environment.  IEEE Systems Journal, 2020.
  * [24] Xingxing Xiong, Shubo Liu, Dan Li, Zhaohui Cai, and Xiaoguang Niu.  Real-time and private spatio-temporal data aggregation with local differential privacy.  Journal of Information Security and Applications, 55:102633, 2020\.
  * [25] Peng Liu, YuanXin Xu, Quan Jiang, Yuwei Tang, Yameng Guo, Li-e Wang, and Xianxian Li.  Local differential privacy for social network publishing.  Neurocomputing, 391:273–279, 2020.
  * [26] Jhilakshi Sharma, Donghyun Kim, Ahyoung Lee, and Daehee Seo.  On differential privacy-based framework for enhancing user data privacy in mobile edge computing environment.  IEEE Access, 9:38107–38118, 2021.
  * [27] Tong Li, Jin Li, Zheli Liu, Ping Li, and Chunfu Jia.  Differentially private naive bayes learning over multiple data sources.  Information Sciences, 444:89–104, 2018.
  * [28] Chong zhi Gao, Qiong Cheng, Pei He, Willy Susilo, and Jin Li.  Privacy-preserving naive bayes classifiers secure against the substitution-then-comparison attack.  Information Sciences, 444:72–88, 2018.
  * [29] Ping Li, Jin Li, Zhengan Huang, Chong-Zhi Gao, Wen-Bin Chen, and Kai Chen.  Privacy-preserving outsourced classification in cloud computing.  Cluster Computing, 21(1):277–286, 2018.
  * [30] Xiaoqian Liu, Qianmu Li, Tao Li, and Dong Chen.  Differentially private classification with decision tree ensemble.  Applied Soft Computing, 62:807–816, 2018.
  * [31] Zhili Xiong, Longyuan Li, Junchi Yan, Haiyang Wang, Hao He, and Yaohui Jin.  Differential privacy with variant-noise for gaussian processes classification.  In Pacific Rim International Conference on Artificial Intelligence, pages 107–119. Springer, 2019.
  * [32] Puyu Wang and Hai Zhang.  Differential privacy for sparse classification learning.  Neurocomputing, 375:91–101, 2020.
  * [33] Xianxian Li, Jing Liu, Songfeng Liu, and Jinyan Wang.  Differentially private ensemble learning for classification.  Neurocomputing, 430:34–46, 2021.
  * [34] Rui Chen, Benjamin CM Fung, S Yu Philip, and Bipin C Desai.  Correlated network data publication via differential privacy.  The VLDB Journal, 23(4):653–676, 2014.
  * [35] Frank D McSherry.  Privacy integrated queries: an extensible platform for privacy-preserving data analysis.  In Proceedings of the 2009 ACM SIGMOD International Conference on Management of data, pages 19–30, 2009.
  * [36] Daniel Berrar.  Bayes’ theorem and naive bayes classifier.  Encyclopedia of Bioinformatics and Computational Biology: ABC of Bioinformatics; Elsevier Science Publisher: Amsterdam, The Netherlands, pages 403–412, 2018.
  * [37] Qinbao Song, Guangtao Wang, and Chao Wang.  Automatic recommendation of classification algorithms based on data set characteristics.  Pattern recognition, 45(7):2672–2689, 2012.
  * [38] Jitendra Kumar, Deepika Saxena, Ashutosh Kumar Singh, and Anand Mohan.  Biphase adaptive learning-based neural network model for cloud datacenter workload forecasting.  Soft Computing, pages 1–18, 2020.