# SoK: Privacy Preserving Machine Learning using Functional Encryption:
Opportunities and Challenges

Prajwal Panzade  Georgia State UniversityAtlantaGAUSA
<EMAIL_ADDRESS>and  Daniel Takabi  Georgia State
UniversityAtlantaGAUSA<EMAIL_ADDRESS>
###### Abstract.

With the advent of functional encryption, new possibilities for computation on
encrypted data have arisen. Functional Encryption enables data owners to grant
third-party access to perform specified computations without disclosing their
inputs. It also provides computation results in plain, unlike Fully
Homomorphic Encryption.
The ubiquitousness of machine learning has led to the collection of massive
private data in the cloud computing environment. This raises potential privacy
issues and the need for more private and secure computing solutions. Numerous
efforts have been made in privacy-preserving machine learning (PPML) to
address security and privacy concerns. There are approaches based on fully
homomorphic encryption (FHE), secure multiparty computation (SMC), and, more
recently, functional encryption (FE). However, FE-based PPML is still in its
infancy and has not yet gotten much attention compared to FHE-based PPML
approaches.
In this paper, we provide a systematization of PPML works based on FE
summarizing state-of-the-art in the literature. We focus on Inner-product-FE
and Quadratic-FE-based machine learning models for the PPML applications. We
analyze the performance and usability of the available FE libraries and their
applications to PPML. We also discuss potential directions for FE-based PPML
approaches. To the best of our knowledge, this is the first work to
systematize FE-based PPML approaches.

Privacy-preserving Machine Learning, Functional Encryption, Computation on
Encrypted Data

††conference: ; ; ††ccs: Computer systems organization Embedded systems††ccs:
Computer systems organization Redundancy††ccs: Computer systems organization
Robotics††ccs: Networks Network reliability

##  1\. Introduction

Machine learning’s expansion into domains such as computer vision, natural
language processing, and speech processing has resulted in a plethora of
incredible applications that have become an inseparable part of people’s
lives. More real-world machine learning applications today rely on a cloud
computing environment following the concept of machine learning as a service
(MLaaS) (ribeiro2015mlaas, ). More and more highly regulated enterprises and
organizations, for example, banks, governments, insurances, and health, are
migrating their data and machine learning services to the cloud. As a result
of this advancement, there arises an increase in demand for secure and
confidential computing solutions that preserve data and model privacy in ML
applications relying on cloud-based machine learning. In light of this,
researchers are paying close attention to privacy-preserving machine learning
(PPML) (hesamifard2018privacy, ; securenn, ; deepsecure, ). The PPML’s purpose
is to deal with the problems related to data and model privacy during the
training to deployment stages of machine learning.

Figure 1. Overview of PPML using Functional Encryption

In the literature, fully homomorphic encryption (FHE) (gentry2009fully, ),
secure multi-party computation (SMC) (bogetoft2009secure, ), federated
learning (mcmahan2017communication, ), differential privacy
(dwork2006calibrating, ), and trusted execution environments (sabt2015trusted,
) are existing approaches to achieve PPML. In addition to this, functional
encryption (boneh2011functional, ) is also advancing day by day. Fully
homomorphic encryption is an encryption standard that allows users to compute
over encrypted data without decrypting the ciphertext itself. Secure multi-
party computation allows many parties to jointly compute a function over their
inputs while keeping their inputs private. On the other hand, federated
learning is a machine learning strategy that allows multiple clients to train
a model owned by a server while keeping the input data of each participant
private. Differential privacy enables the quantification of an algorithm’s
level of privacy protection to the underlying sensitive dataset and
facilitates machine learning model training on private data. A secure enclave,
also known as a trusted execution environment (TEE), is a combination of
software and hardware elements that creates an isolated environment in a
system that ensures the security of applications running inside the TEE.
Functional encryption is a type of encryption comparable to fully homomorphic
encryption in which computations can be performed on encrypted data. The key
distinction between fully homomorphic and functional encryption is that fully
homomorphic encryption produces ciphertext results, whereas functional
encryption produces plaintext results.

There are primarily two ways to achieve PPML: Fully Homomorphic Encryption
(gentry2009fully, ) and Secure Multiparty Computation (bogetoft2009secure, ).
Dowlin et al. proposed a method for transforming a trained neural network to a
CryptoNet (cryptonet, ), that enables data owners to send homomorphically
encrypted data to the server and receive an encrypted inference. In another
approach called CryptoDL, Hesamifard et al. (cryptodl, ) proposed an FHE-based
method for privacy-preserving inference on pre-trained convolutional neural
networks. Al-Badawi et al. developed Private Fast Text (PrivFT)(privft, ), a
method that utilizes FHE for privacy-preserving text classification. Li et al.
introduced HomoPAI (homopai, ), an FHE-based machine learning platform built
on top of PAI (Platform of Artificial Intelligence, an Alibaba Cloud product),
enabling data owners from different organizations to store their encrypted
data and perform collaborative machine learning. Jäschke et al. in (unsup, )
proposed an approach to perform unsupervised machine learning where they
implemented a K-means clustering algorithm using FHE.

Graepel et al. in ML Confidential (graepel2012ml, ) proposed a binary
classification method using polynomial approximations and FHE. In SecureML
(secureml, ), Zhang et al. proposed an efficient two-party protocol for
training linear regression, logistic regression, and neural networks models.
Rouhani et al. in DeepSecure (deepsecure, ) proposed an approach for the
scalable execution of deep learning models in the PPML setting that uses an
optimized version of Yao’s Garbled Circuit (GC) protocol. Wagh et al. in
SecureNN (securenn, ) proposed a three-party computation protocol for privacy-
preserving training and inference on convolutional neural networks.

In this paper, we consider PPML approaches using functional encryption. When
machine learning is offered as a service, the model is stored on the server,
and one or more clients are responsible for training. Sometimes there might be
pre-trained models possessed by the server. In general, FHE-based machine
learning models are trained on unencrypted data, and then their inference is
obtained on encrypted data. In the FHE-based machine learning setting, a
client sends encrypted data to the server; the server then performs tasks like
classification using the pre-trained ML model on the data received from the
client and produces the prediction results in the ciphertext. Even though the
server performs computations on encrypted data, it doesn’t learn anything and
generates ciphertext predictions. So, only the owner of the data can see the
actual result. However, in the case of functional encryption, the server
generates the computation results in plaintext using a special key that
enables partial decryption needed for computation (see Fig. 1). The rest of
the steps are identical to those of fully homomorphic encryption. In the realm
of PPML, we found the idea of producing plaintext results over encrypted data
without fully decrypting it to be intriguing for some of the applications.
Note that both FHE and FE-based PPML works need high computation costs.

Consider a hospital that outsources the prediction of cardiovascular diseases
to a machine learning as a service (MLaaS) company. Assume that this MLaaS
company is responsible for performing machine learning operations on sensitive
data like patient records. In such a scenario, a good ML model should predict
the correct results and should also preserve the privacy of the patients’
data. Earlier approaches to such cases can perform training of a machine
learning model on the plain data and inference on homomorphically encrypted
data. However, when an MLaaS is expected to produce the result in plain FHE-
based approaches needs an additional key to decrypt the results predicted by
the server. Functional encryption can be used to avoid sharing an additional
key for the decryption of results inferred by the ML model.

This SoK paper aims to study several research works in the field of functional
encryption-based privacy-preserving machine learning. We investigate and
examine the impact of functional encryption in the machine learning domain,
i.e., machine learning computations on encrypted data. This is intended to
give the research community a clear snapshot of FE-based PPML approaches. We
only focus on FE-based PPML works out of many other existing PPML approaches.

Initiatives have been taken in recent years to develop FE-based PPML systems.
FE-based PPML approaches are brodly categorized as 1) Inner-product FE-based
(IPFE) approaches and 2) Quadratic FE-based (QFE) approaches. Ligier et al.
(ligier2017privacy, ) proposed an approach to perform privacy-preserving
classification on data encrypted using IPFE. An IPFE-based deep neural network
approach on MNIST dataset proposed by Xu et al. in (xu2019cryptonn, ). Panzade
et al. in (panzade2021towards, ) proposed a new approach to compute faster
secure activation function based on function-hiding inner product encryption
for PPML. In (ryffel2019partially, ), Ryffel et al. introduced a system that
used QFE with adversarial training to perform privacy-preserving predictions.
Marc et al. (marc2019privacy, ) presented the first fully-fledged FE libraries
as well as a variety of applications for developing privacy-enhanced machine
learning models.

The following are the primary contributions of this paper:

  1. (1)

We present a basic but substantial theoretical foundation to help researchers
understand current approaches to FE-based PPML.

  2. (2)

We provide a thorough review of the literature on FE-based PPML, emphasizing
the strengths and shortcomings of the various approaches to assessing how they
supplement one another.

  3. (3)

We examine the current constraints that prohibit the implementation of
existing FE-based PPML solutions in real-world settings, mostly due to issues
with efficiency and usability.

  4. (4)

We provide research directions to intensify existing works in terms of time
performance and security that the research community may pursue in the coming
years.

##  2\. Background Knowledge

Functional encryption is a generalization of public-key encryption that allows
a key holder to compute a particular function of encrypted data using
constrained secret keys (boneh2011functional, ). Here this function is called
functionality. e.g., an FE scheme may be particularly designed to compute
inner-products; in this case, the functionality becomes inner-product. In the
FE scheme, a key management authority with a master secret key generates a
secret key skfe; a decryptor can use that to compute a function on an
encrypted message x. This section summarizes the two major functional
encryption schemes, inner-product functional encryption, and quadratic
functional encryption, used by privacy-preserving machine learning approaches.

###  2.1. Inner-product Functional Encryption

The Decisional Diffie-Hellman assumption (DDH) underpins the method outlined
by Abdalla et al. in (abdalla2015simple, ). Let GroupGenerator be a
probabilistic polynomial-time (PPT) algorithm with input security parameter
1λ, which produces a triplet ($\mathbb{G}$, $p$, $g$), where $\mathbb{G}$ is a
group of order p created by g $in$ $\mathbb{G}$. The tuples (g, ga, gb, gab)
and (g, ga, gb, gc) are computationally indistinguishable, according to DDH,
where ($\mathbb{G}$, $p$, $g$) $\leftarrow$ GroupGenerator(1λ), and a, b, c
$\in$ Zp are chosen uniformly and independently at random.

The $\Pi_{\text{ipfe }}=$ ( Setup, Encrypt, KeyDerivation, Decrypt) FE scheme
for IPFE in DDH is as follows:
Setup(1λ, 1l): This algorithm samples ($\mathbb{G}$,$p$,$g$)$\leftarrow$
GroupGenerator(1λ) and s = (s1,……sl)$\leftarrow$ Z${}_{p}^{l}$, sets mpk =
$\left(h_{i}=g^{s_{i}}\right)_{i\in[\ell]}$ and msk = s and finally returns a
pair of (mpk, msk).
Encrypt(mpk, x): This algorithm takes mpk and message x = (x1,……xl) $\in$
Z${}_{p}^{l}$ as input, chooses random number r $\leftarrow$ Zp, computes Ct0
= gr and, for each i $\in$ [l], Cti = hir. g${}^{{x}_{i}}$ and returns
ciphertext Ct.
KeyDerivation(msk, y): This algorithm takes msk and vector y=(y1,……yl) $\in$
Zp as input and outputs key skfe.
Decrypt(mpk, Ct, skfe): This algorithm takes the master public key, ciphertext
and skfe for vector y as input and outputs the discrete logarithm in basis g
of

$\prod_{i\in[l]}Ct_{i}^{y_{i}}/Ct_{0}^{sk_{fe}}$
Correctness: The method’s correctness is demonstrated as follows
(abdalla2015simple, ):
$\forall$ (mpk,msk) $\leftarrow$ Setup(1λ, 1l), all y $\in$ Z${}_{p}^{l}$ and
x $\in$ Z${}_{p}^{l}$
for skfe $\leftarrow$ KeyDerivation(msk, y) and Ct $\leftarrow$
Encrypt(mpk,x).
Decrypt(mpk, $C_{t},sk_{fe}$)
= $\frac{\prod_{i\in[l]}Ct_{i}^{y_{i}}}{Ct_{0}^{sk_{fe}}}$
$=\frac{\prod_{i\in[l]}(g^{s_{i}r+x_{i})^{y_{i}}}}{g^{r(\sum{i\in[l]}{}y_{i}s_{i})}}$
$=g^{\sum{i\in[l]}y_{i}s_{i}r+\sum{i\in[l]}{}y_{i}x_{i}-r(\sum{i\in[l]}{}y_{i}s_{i})}$
= $g^{\sum{i\in[l]}y_{i}x_{i}}$
$=g^{<x,y>}$

Table 1. Symbols and acronyms used in the paper Acronym / symbol | Description
---|---
mpk | Master Public Key
msk | Master Secret Key
skfe | FE Key for IPFE
$\mathrm{sk}_{qe}$ | FE key for QFE
Ct | Ciphertext
IND-CPA | security against chosen-plaintext attacks
ERT | Extremely Randomized Trees
IPFE | Inner-product Functional Encryption
QFE | Quadratic Functional Encryption

###  2.2. Quadratic Functional Encryption

Quadratic functional encryption scheme uses bilinear groups (also known as
pairing groups), has been proposed by (boneh2003identity, ; joux2004one, ). In
the case of QFE-based PPML, we refer to schemes proposed by Ryffel et al. in
(ryffel2019partially, ). Here, GroupGenerator is a PPT algorithm on inputting
$1^{\lambda}$ returns
$\mathcal{PG}=\left(\mathbb{G}_{1},\mathbb{G}_{2},p,g_{1},g_{2},e\right)$ of
an asymmetric bilinear group, where $\mathbb{G}_{1}$ and $\mathbb{G}_{2}$ are
cyclic groups of prime order $p$ (for a $2\lambda$-bit prime $p$ ) and $g_{1}$
and $g_{2}$ are generators of $\mathrm{G}_{1}$ and $\mathrm{G}_{2}$,
respectively. The application
$e:\mathbb{G}_{1}\times\mathbb{G}_{2}\rightarrow\mathbb{G}_{T}$ is an
admissible pairing i.e. it can be efficiently computable, non-degenerated, and
bilinear:
$e\left(g_{1}^{\alpha},g_{2}^{\beta}\right)=e\left(g_{1},g_{2}\right)^{\alpha\beta}$
for any scalars $\alpha,\beta\in\mathbb{Z}_{p}$. Therefore,
$g_{T}:=e\left(g_{1},g_{2}\right)$ which makes the group $\mathbb{G}_{T}$ of
order $p$, where $p$ is prime. For any $s\in\\{1,2,T\\},n\in\mathbb{N},$ and
vector $u:=\left(\begin{array}[]{c}u_{1}\\\ \vdots\\\
u_{n}\end{array}\right)\in\mathbb{Z}_{p}^{n},$ it is denoted by
$g_{s}^{u}:=\left(\begin{array}[]{c}g_{s}^{u_{1}}\\\ \vdots\\\
g_{s}^{u_{n}}\end{array}\right)\in\mathbb{G}_{s}^{n}.$
Similarly, for any vectors $u\in\mathbb{Z}_{p}^{n},v\in\mathbb{Z}_{p}^{n},$ It
is denoted by $e\left(g_{1}^{u},g_{2}^{v}\right)=$
$\prod_{1-1}e\left(g_{1},g_{2}\right)^{\mathbf{u}_{t}\cdot\bar{v}_{1}}=e\left(g_{1},g_{2}\right)^{\mathbf{u}\cdot\boldsymbol{v}}\in\mathbb{G}_{T},$
since $\boldsymbol{u}\cdot\boldsymbol{v}$ denotes the inner-product between
the vectors $\boldsymbol{u}$ and $v,$ that is:
$\boldsymbol{u}\cdot\boldsymbol{v}:=\sum_{\mathrm{i}=1}^{n}u_{1}v_{i}$.

Ryffel et al. in (ryffel2019partially, ) build an efficient FE scheme shown
below for the set of functions defined, for all
$n,B_{x},B_{y},B_{f}\in\mathbb{N}^{*},$ as
$\mathcal{F}_{n,B_{x},B_{y},B_{f}}=\left\\{f:\left[-B_{x},B_{x}\right]^{n}\times\left[-B_{y},B_{y}\right]^{n}\rightarrow\mathbb{Z}\right\\}$
where the functions $f$ $\in\mathcal{F}_{n,B_{x},B_{y},B_{f}}$ are expressed
as a set of bounded coefficients
$\left\\{f_{i,j}\in\left[-B_{f},B_{f}\right]\right\\}_{i,j\in[n]},$ and for
all vectors
$\boldsymbol{x}\in\left[-B_{x},B_{x}\right]^{n},\boldsymbol{y}\in\left[-B_{y},B_{y}\right],$
:

$f(\boldsymbol{x},\boldsymbol{y})=\sum_{i,j\in[n]}f_{i,j}x_{i}y_{j}$

FE scheme is explained as follows:
Setup(1${}^{\lambda},\mathcal{F}_{n,B_{x},B_{y},B_{f}})$:
$\mathcal{PG}:=\left(\mathbb{G}_{1},\mathbb{G}_{2},p,g_{1},g_{2},e\right)\leftarrow\operatorname{GroupGenenerator}\left(1^{\lambda}\right),\\\
\boldsymbol{s},\boldsymbol{t}\leftarrow\mathbb{\mathbb{Z}}_{p}^{n},\mathbf{msk}:=(\boldsymbol{s},\boldsymbol{t}),\\\
\mathbf{mpk}:=\left(\mathcal{P}\mathcal{G},g_{1}^{\boldsymbol{s}},g_{2}^{t}\right)$
Return (mpk, msk).

Encrypt(mpk,(x,y)):
$\gamma\leftarrow\mathbb{Z}_{p},\mathbf{W}\leftarrow\mathrm{GL}_{2},\text{ for
all }i\in[n],\\\
\boldsymbol{a}_{i}:=\left(\mathbf{W}^{-1}\right)^{\top}\left(\begin{array}[]{c}x_{i}\\\
\gamma
s_{i}\end{array}\right),\boldsymbol{b}_{i}:=\mathbf{W}\left(\begin{array}[]{c}y_{i}\\\
-t_{i}\end{array}\right)$
Return
Ct:=$\left(g_{1}^{\gamma},\left\\{g_{1}^{a_{i}},g_{2}^{b_{i}}\right\\}_{i\in[n]}\right)\in\mathbb{G}_{1}\times\left(\mathbb{G}_{1}^{2}\times\mathbb{G}_{2}^{2}\right)^{n}$
KeyDerivation(msk,f):
Return
$\mathrm{sk}_{qe}:=\left(g_{2}^{f(s,t)},f\right)\in\mathbb{G}_{2}\times\mathcal{F}_{n,B_{x},B_{y},B_{f}}$
Decrypt( mpk,
Ct:=(g${}_{1}^{\gamma},\\{g_{1}^{a_{i}},g_{2}^{b_{i}})\\}_{i\in[n]}$),
$\mathrm{sk}_{qe}:=(g_{2}^{f(s,t)},f)\textbf{ ):}\vspace{1em}\\\
out:=e\left(g_{1}^{\gamma},g_{2}^{f(s,t)}\right)\cdot\prod_{i,j\in[n]}e\left(g_{1}^{a_{i}},g_{2}^{b_{i}}\right)^{f_{i,j}}$
Return $\log(out)\in\mathbb{Z}$
Correctness:
For all $i,j\in[n]$:

$e\left(g_{1}^{d_{i}},g_{2}^{b_{j}}\right)=g_{T}^{d_{i}\cdot
b_{j}}=g_{T}^{x_{i}y_{j}-\gamma s_{i}t_{j}}$

since

$\displaystyle\vec{a}_{i}\cdot\vec{b}_{j}$
$\displaystyle=\left(\left(\mathbf{W}^{-1}\right)^{\top}\left(\begin{array}[]{c}x_{i}\\\
\gamma
s_{i}\end{array}\right)\right)^{\top}\cdot\left(\mathbf{W}\left(\begin{array}[]{c}y_{j}\\\
-t_{j}\end{array}\right)\right)$
$\displaystyle=\left(\begin{array}[]{c}x_{i}\\\ \gamma
s_{i}\end{array}\right)^{\top}\mathbf{W}^{-1}\mathbf{W}\left(\begin{array}[]{c}y_{j}\\\
-t_{j}\end{array}\right)=x_{i}y_{j}-\gamma s_{i}t_{j}$

Therefore,

$\text{out}=e\left(g_{1}^{\gamma},g_{2}^{q(\vec{s},\vec{t})}\right)\cdot\prod_{i,j}e\left(g_{1}^{\vec{a}_{i}},g_{2}^{\vec{b}_{i}}\right)^{q_{i,j}}$

$=g_{T}^{\gamma q(\vec{s},\vec{t})}\cdot
g_{T}^{\sum_{i,j}q_{i,j}x_{i}y_{j}-\gamma q_{i,j}s_{i}t_{j}}$

$=g_{T}^{\gamma q(\vec{s},\vec{t})}\cdot g_{T}^{q(\vec{x},\vec{y})-\gamma
q(\vec{s},\vec{t})}=g_{T}^{q(\vec{x},\vec{y})}$

We refer the readers to (abdalla2015simple, ) and (baltico2017practical, ;
ryffel2019partially, ) for more cryptographic details on IPFE and QFE schemes,
respectively.

###  2.3. Neural networks

The artificial neural network, often known as a neural network, is a machine
learning model that is hierarchical and non-linear, with several layers and
several neurons in each layer. Each layer of a neural network processes the
input provided by the previous layer before passing it on to the next.

  * •

Input layer: The preprocessed raw data or features extracted from raw data in
a particular format make up the first layer of the neural network.

  * •

Hidden Layer: A neural network can have one or more hidden layers. The First
Hidden layer’s neurons are linked to the input layer and followed by an
activation function. Further hidden layers are fed with the previous layer’s
output. Weight values are associated with layers, and they are updated during
the forward and backpropagation processes until convergence.

  * •

Activation function: The activation function of a neuron in a neural network
determines the output of that neuron given a single or group of inputs. In
machine learning, there are several activation functions such as sigmoid,
Rectified Linear Unit (ReLU), and tanh. The ReLU activation function is an
example of one of the most frequent activation functions. If the input value
is less than zero, the ReLU activation function returns zero, and if it is
larger than zero, it returns the same input.

  * •

Output layer: The output layer of a neural network is the final layer of
neurons that provides the network’s output.

###  2.4. Polynomial Neural Network

Polynomial neural networks are the kinds of neural networks that primarily
facilitate linear components like fully connected layers, convolutions with
average pooling, and activation functions are approximated using polynomials.
They have demonstrated fairly high accuracy for the relatively simple
benchmarks in image recognition tasks (cryptonet, ; badawi2018alexnet, ). They
have also been used to propose the applicability of novel machine learning
protocols in various early-stage implementations, as presented in
(bourse2018fast, ; chillotti2016faster, ). The simplicity of the operations
upon which polynomial neural networks are built ensures high efficiency,
particularly for gradient computations. Research works such as
(livni2014computational, ) have demonstrated that they can reach convergence
rates comparable to those of networks with non-linear activation functions.

##  3\. Scope and Methodology

Our analysis of the FE-based PPML methodologies is divided into two parts. To
begin, we conduct a thorough review of existing methodologies and emphasize
their salient features and functions. Second, we evaluate these tools in
practice by comparing their usability, complexity, and performance across
various case study scenarios. We integrate quantitative performance analysis
with a qualitative review, highlighting the difficulties inherent in designing
various applications. The secure computation ecosystem encompasses a wide
variety of tools. On the low level, there exist math libraries that facilitate
the construction of FE implementations, for example, by efficiently
implementing algorithms useful for generic lattice-based cryptography. Then
there are FE libraries that implement certain schemes and provide slightly
more advanced APIs, such as setup, encrypt, key generation, and decrypt. These
libraries abstract away computation features like parameter selection,
encryption, and decryption by providing a higher-level language in which
developers can implement their computation.

The primary objective of this work is to comprehend the landscape of MLaaS in
data-sensitive scenarios via privacy-preserving machine learning computing,
for example, when data supplied to third parties for processing is encrypted.
PPML ensures the privacy and confidentiality of input data. Additionally, they
alleviate excessive pressures on the client endpoint in computing as cloud
server does most of the computing part. Finally, they may be used in machine
learning situations where clients can contribute data toward a training or the
inference goal. PPML is frequently used in conjunction with other approaches.
We discuss the application of all PPML approaches based on FE just at these
crossing places. Although fully homomorphic encryption is a widely used
technique in PPML, we only include FE-based approaches because of fewer
available articles to the research community on this topic. Also, this work
does not cover federated learning, differential privacy, and secure multi-
party computation due to their different computational infrastructure
requirements and implementation.

PPML that is based on FE has efficiency and usability concerns. FE-based PPML
is limited in its computing capabilities due to performance concerns
associated with extensive computations. Even though they have been frequently
used in other contexts, like private data aggregation and statistics, their
application to machine learning is not simple. Moreover, the works focusing on
FE-based PPML are significantly lower in number. So, we consider a solution
efficient if it offers enhancements to previous methods that reduce their
runtime on machine learning applications.

The second problem is associated with the techniques’ deployability. Numerous
frameworks and technologies make advanced machine learning accessible to data
scientists who are not necessarily professionals in computer sciences.
However, adopting these frameworks for use with PPML is challenging. We
consider usability enhancements in this area if the proposal fits one of the
following two requirements. To start, if it makes the solution more adaptable
to existing machine learning frameworks (for example,by giving tools that
reduce total programming effort). As such, our study incorporates works that
propose Application Programming Interfaces (APIs), compilers, or other
significant practical tools that aid in the implementation and deployment of
theoretical ideas into real applications, hence increasing their usability.
Second, when an open-source implementation supplements the concept. Apart from
facilitating future revisions to the approach, open-source implementations
ensure that the results are reproducible. Additionally, we verify whether the
work contains references to open-source implementations or is released
independently (for example, by visiting the authors’ websites or GitHub
repositories).

Furthermore, we investigate and analyze two parameters of such open-source
repositories: i) the code’s adherence with the theoretical claims (for
example, if it employs the security methods and features stated in the paper);
and ii) how well the source code is maintained or integrated with other
current frameworks (for example, the version updates since it was released).

In summary, we examine the following for each of the proposals utilizing FE-
based PPML: (i) the problem being addressed (training, inference, or both),
(ii) the machine learning model used, (iii) the specific FE techniques
involved (IPFE or QFE), and (iv) the efficiency and usability considerations
examined.

We execute searches in multiple scholarly repositories and databases, mainly
Google scholar, IEEE Xplore and ACM digital library for relevant publications
using specified keywords such as privacy-preserving machine learning,
functional encryption, secure computation, and computation over encrypted
data. Secondly, we select those focused on FE-based PPML and have been
published in renowned venues like NeurIPS, PETS, IEEE S&P, CRYPTO, EUROCRYPT,
TCC, CCS. We then read their abstracts and methodologies to determine if they
fall within the topic of our research. This provides us with an initial
collection of works, which we meticulously studied. Finally, we use snowball
sampling to add additional suitable works by using references from the first
collection of papers. We made every effort to include all significant works
implementing PPML using FE.

##  4\. Overview of FE-based PPML models and libraries

###  4.1. What has been done?

There are two variations of the FE-based PPML methodologies available in the
literature. The first one uses inner-product functional encryption, whereas
the other one uses quadratic functional encryption. The IPFE-based
methodologies involve training and inference, whereas the QFE methodologies
involve simply the inference stage of machine learning.

####  4.1.1. Inner-product FE-based Machine Learning

In this type of methodology, the inputs are encrypted using inner-product
functional encryption. Then during the activation, inner products between
encrypted inputs and weight matrices are unfolded based on the special
property of FE. Later, the neural network operations are done similarly to
regular neural networks. Here both forward propagation and backpropagation can
be made secure using FE. This methodology supports both training and
prediction over encrypted data, unlike the quadratic FE-based approaches.

####  4.1.2. Quadratic FE-based Machine Learning

In quadratic FE-based methodologies, the training phase happens similar to the
regular neural networks. It is also worthwhile to note that they train the
neural networks with plain data. In the prediction phase, encrypted data is
fed to the neural network. It undergoes the process of polynomial
approximation, and then the other steps are applied similar to regular neural
networks. These approaches are found to be faster than inner-product FE-based
approaches.

###  4.2. Cryptography libraries for Functional Encryption implementation

Presently, there are only two dedicated libraries that focus on implementing
the state-of-the-art FE schemes. The first one is called CiFEr, and the other
is GoFE. Three entities are involved in functional encryption and decryption:
an encryptor, a decryptor, and a key management authority. An encryptor
encrypts the data and obtains ciphertext. The decryptor decrypts the
ciphertext received from the encryptor. The key management authority handles
the responsibility of generating a variety of cryptographic keys. In the FE
scheme, based on the involvement of encryptors, it can be either single input
or multi-input. We detailed both of these libraries and supporting
cryptography libraries used in the FE-based PPML works as follows:

####  4.2.1. CiFEr

CiFEr (marc2019privacy, ) is a functional encryption library developed by the
FENTEC group that is written in the C language. This library provides
developers to use functions to perform various FE operations like encrypt,
decrypt, and key generation. These libraries are built in such a way that the
predefined functions can be directly called without setting many parameters.
Here, the cryptographic key generation is abstract to the user, and users set
the security parameters in terms of bits. It provides both single-input and
multi-input FE implementations commonly observed in FE-based PPML approaches.

####  4.2.2. GoFE

GoFE (marc2019privacy, ) is another library provided by the FENTEC group that
is written in Golang. Like CiFEr, it also provides an option to use FE
functions. Both of CiFEr and GoFE have the same set of state-of-the-art
functional encryption implementations. Their performance is based on the
underlying programming languages.

####  4.2.3. FLINT

FLINT (hart2013flint, ) is a cryptography library that is used for performing
number theory-related operations. It is written in C. Unlike GoFE and CiFEr,
it does not dedicatedly provide FE functionality. In some of the FE-based
works, this library is used for the implementation of FE schemes.

####  4.2.4. Charm

Charm (akinyele2013charm, ) is a framework designed for implementing various
advanced cryptosystems. It is built using Python to decrease development time
and code complexity while fostering component reuse.

####  4.2.5. PBC

Pairing-based Cryptography (PBC) (lynn2006pbc, ) is a C library that enables
rapid development of cryptosystems based on pairings. It implements a bilinear
cyclic group abstractly, hiding the programmer from mathematical details. Both
PBC and Charm are used in some of the FE-based PPML implementations.

[b]  Research works FE type Training1 Prediction2 ML model Security Ligier et
al. (ligier2017privacy, ) IPFE3 ERT4 Selective IND-CPA5 Xu et al.
(xu2019cryptonn, ) IPFE 5 layer NN6 Selective IND-CPA Sans et al.
(sans2018reading, ) QFE7 2 layer NN Adaptive IND-CPA8 Ryffel et al.
(ryffel2019partially, ) QFE 2 layer NN Adaptive IND-CPA Carpov et al.
(carpov2020illuminating, ) IPFE and QFE 2 layer NN Adaptive IND-CPA

Table 2. Comparison of FE-based PPML models in terms of functionality

  * 1

Training on encrypted data

  * 2

Prediction on encrypted data

  * 3

Inner-product Functional Encryption

  * 4

Extremely Randomized Trees

  * 5

Selective security against chosen-plaintext attack

  * 6

Neural Network

  * 7

Quadratic Functional Encryption

  * 8

Adaptive security against chosen-plaintext attack

##  5\. Insights

Recall that we consider the FE-based PPML works focused on training and/or
inference over encrypted data.

###  5.1. Threat model

Assumption: In FE-based PPML, it is assumed that there is a trusted
independent key management authority that is responsible for the generation of
the required keys. Such assumptions are common in all functional encryption
schemes.

FE-based PPML techniques adhere to an honest but curious security model, in
which both parties comply with the protocol while attempting to gain as much
information as possible. In this case, the server follows the protocol but
tries to learn additional information. Typically, the approach includes three
primary components: a key management authority, a server, and a client. The
key management authority generates encryption and decryption keys. FE requires
three distinct keys, namely master public key (mpk), master secret key (msk),
and functional encryption key(skfe), that operate slightly differently than
public-key cryptography techniques as discussed in section 2.

The research works in this field assume an MLaaS setting. The server holds the
trained model, and the client provides the data to the server and receives the
prediction results using the learned model. Additionally, in some
circumstances, the capability of training models over encrypted data may be
available.

The client firstly encrypts the data using the master public key before
sending it to the server. In some scenarios, the client may require to
preprocess the data before encryption; this varies by application. For
example, computer vision applications may require scaling or transforming
images prior to encryption. Normalization and standardization may be necessary
in the case of structured data.

The server holds the model. The model is developed by training the neural
network using client-supplied data. Because the server cannot see the data
sent by the client, it must obtain the functional encryption secret key skfe
in order to execute functionality-based computations on the data. Either
polynomial approximation uses such computations for QFE-based methods or
inner-product computation in IPFE-based methods discussed in the later
sections.

Figure 2. IPFE-based PPML

###  5.2. IPFE-based PPML

Fig. 2 depicts IPFE-based PPML. Out of the available works in this field
dedicated to FE-based PPML (ligier2017privacy, ) and (xu2019cryptonn, ) use
inner-product FE schemes.

####  5.2.1. Functional encryption scheme

The work proposed in (ligier2017privacy, ) and (xu2019cryptonn, ) use the
inner product schemes proposed in (agrawal2016fully, ) and (abdalla2015simple,
), respectively.

####  5.2.2. Training phase

In the approach proposed by (ligier2017privacy, ), Extremely Randomized Trees
are used as a machine learning model. These trees are nothing but an ensemble
learning model that uses decision trees. In their work, they perform the
training on plain data that is similar to regular machine learning models. For
the inference, they used IPFE.

The methodology proposed in [48] uses a LeNet like 5-layer neural network. In
their work based on IPFE, inputs are first encrypted by the client and then
sent to a server. The server obtains the FE key and runs the IPFE decrypt
function in order to compute the activation results of the first hidden layer.
The computation between the input vector and weight matrix is done as follows:

A = ReLU(skfe(W) * Encrypt(X) + b)
This output obtained after the first hidden layer is then fed through the
remaining layers of the neural network. So it can be said that these
methodologies can work entirely on FE encrypted data. The training method over
encrypted data proposed by (xu2019cryptonn, ) requires 57 hours to train the
5-layer neural network model. Regular neural network with the same settings on
non-encrypted data takes 4 hours. This shows a massive gap between computation
on encrypted data over non-encrypted data considering neural networks.

####  5.2.3. Inference phase

The methodology proposed in (ligier2017privacy, ) can perform predictions on
encryption data with a prediction time of fewer than 0.1 seconds. Also, it
produces a validation accuracy of 95.64 %. The work proposed in
(xu2019cryptonn, ) can run the inference on encrypted data by following the
similar use of IPFE decryption functions for the inference. Their model
achieves a validation accuracy of 95.49 %.

###  5.3. QFE-based PPML

Fig. 3 depicts QFE-based PPML. Three works use quadratic functional encryption
schemes, which are defined and shown utilization towards machine learning
problems in (sans2018reading, ; ryffel2019partially, ; carpov2020illuminating,
).

Figure 3. QFE-based PPML

####  5.3.1. Functional encryption scheme

This type of research works either use the similar or modified approach
proposed by Baltico et al. (baltico2017practical, ). Works proposed by
(sans2018reading, ) and (ryffel2019partially, ) are related to each other in
terms of the FE scheme and neural networks used. They proposed a novel
functional encryption scheme that can be used on multivariate quadratic
polynomials.

Table 3. Comparison of FE-based PPML models in terms of performance Research works | Year | Training time | Prediction time | FE key generation | Encryption time | Decryption | Accuracy
---|---|---|---|---|---|---|---
Ligier et al. (ligier2017privacy, ) | 2017 | NA |  $\leq$ 0.1s | 12 ms | 150 ms | 69s | 95.64%
Xu et al. (xu2019cryptonn, ) | 2018 | 57 hrs | not specified | not specified | not specified | not specified | 95.49%
Sans et al. (sans2018reading, ) | 2019 | NA | a few seconds | 8ms | 8.1s | 3.3s | 97.54%
Ryffel et al. (ryffel2019partially, ) | 2019 | NA |  $\leq$ 3 ms | 94 $\pm$ 5ms | 12.1 $\pm$ 0.3s | 24 $\pm$ 9ms | 97.7%
Carpov et al. (carpov2020illuminating, ) | 2020 | NA | not specified | not specified | not specified | not specified | 90 %
Table 4. Implementation details

Research works | Dataset | FE Libraries | Python Libraries | System specifications
---|---|---|---|---
Ligier et al. (ligier2017privacy, ) | MNIST1 | FLINT2 | sklearn (pedregosa2011scikit, ) | Intel Core i7-4650U, 8GB RAM
Xu et al. (xu2019cryptonn, ) | MNIST | Charm3 | Numpy (van2011numpy, ) | Intel Core i7, 8GB RAM
Sans et al. (sans2018reading, ) | MNIST | PBC4 and Charm | Tensorflow (abadi2015tensorflow, ) | Intel Core i5-6440HQ, 8GB RAM
Ryffel et al. (ryffel2019partially, ) | MNIST | PBC and Charm | Tensorflow | Intel Core i7, 16GB RAM
Carpov et al. (carpov2020illuminating, ) | MNIST, Census Income | not specified | Keras (chollet2018keras, ) | not specified

  * 1

MNIST handwritten digits dataset (lecun2010mnist, )

  * 2

Fast Library For Number Theory (hart2013flint, )

  * 3

A framework for rapidly prototyping cryptosystems (akinyele2013charm, )

  * 4

Pairing-based Cryptograph library (lynn2006pbc, )

  * 5

1994 Census database (kohavi1996scaling, )

####  5.3.2. Training phase

The training phase in this type of methodology is done on plain data similar
to regular neural networks (sans2018reading, ) and (carpov2020illuminating, ).
In work proposed by (ryffel2019partially, ), the adversarial training approach
is used in order to avoid information leakage after functional decryption.
They optimized both the primary classification objective and the opposite of
the collateral objective of a particular simulated adversary simultaneously
using adversarial training.

####  5.3.3. Inference phase

In the inference phase of all the works proposed in (sans2018reading, ;
ryffel2019partially, ; carpov2020illuminating, ), polynomial neural networks
are used. Here they use two-layer neural networks. Wherever quadratic
functional encryption is used, a special type of neural network called the
polynomial neural network is used discussed in section 2.4. The approaches
based on QFE have not yet been used for training the model over encrypted data
because of the complexities involved in the cryptographical aspects. Also, the
present functional encryption is defined for the usage for degree 2
polynomials. The methodology proposed by (sans2018reading, ) and
(ryffel2019partially, ) requires fewer than 3 seconds to produce a prediction
result. Moreover, in the case of QFE-based works, accuracy is shown a little
higher. (sans2018reading, ) and (ryffel2019partially, ) obtain the accuracy of
97.54% and 97.7%, respectively.

###  5.4. Security

Theorem 1 Under the decisional diffie hellman (DDH) assumption, IPFE scheme
given in section 2.1 is selectively secure against chosen-plaintext attacks
(IND-IPFE-CPA).

Theorem 2 Under the matrix decisional diffie hellman (MDDH) assumption, QFE
scheme given in section 2.2 is adaptively secure against chosen-plaintext
attacks (IND-QFE-CPA).

We refer the readers to (abdalla2015simple, ) and (baltico2017practical, ;
ryffel2019partially, ) for the security proofs of the theorems.

The security of the FE-based PPML approaches is dependent on the cryptographic
security of the underlying FE schemes. They offer two types of security:
adaptive security and selective security against chosen-plaintext attacks. If
the security guarantee happens only for the messages known ahead of time, it
is known as selective security. However, adaptive security is a concept where
a security guarantee happens even for randomly chosen messages at any point in
time. The approaches proposed in (ligier2017privacy, ) and (xu2019cryptonn, )
are selectively secure whereas the approaches proposed in (sans2018reading, ),
(ryffel2019partially, ) and (modic2019privacy, ) show adaptive security based
on the underlying crypto schemes.

###  5.5. Information leakage and attacks

Although functional encryption schemes provide cryptographic security to the
machine learning models, there is a risk of information leakage. The outputs
obtained in both the IPFE and QFE based models are in plain after the FE
decryptions are done. So, if the server tries to retrieve part of the
information from the obtained plain data after FE computations, there is a
possibility of information leakage. Carpov et al. in (carpov2020illuminating,
) discusses the type of information leakage. Also, an adversarial training-
based approach proposed in (ryffel2019partially, ) can avoid information
leakage to some extent. We believe there is a scope of attacks like model
inversion attacks (fredrikson2015model, ) and direct inference attacks that
machine learning security researchers can study. In order to make the FE
models deployable, this potential threat should be taken into consideration.

###  5.6. Implementation Details

Almost all of the methodologies use MNIST dataset (lecun2010mnist, ) for the
experiments. MNIST contains 70,000 grayscale images of size 28x28 pixels of
handwritten single digits from 0 to 9. Out of 70K images, the training set
consists of 60K images, and the test set consists of 10K images. In
(carpov2020illuminating, ), Census Income Dataset (kohavi1996scaling, ) is
used in addition to MNIST dataset. Census Income data was extracted from the
1994 Census bureau database. It is a multivariate dataset with 48842
instances. There are 14 attributes like age, work class, education with
categorical and integer data. The task here is to predict whether a person’s
income exceeds $50K/yr.

FE Libraries. Ligier et al. in (ligier2017privacy, ) implemented IPFE scheme
using FLINT library (hart2013flint, ) whereas (xu2019cryptonn, ) used Charm
libary (akinyele2013charm, ) for implementation of IPFE scheme. Sans et al.
and Ryffel et al. (sans2018reading, ), (ryffel2019partially, ) used PBC
(pairing-based cryptography) (lynn2006pbc, ) and (akinyele2013charm, ) library
for implementation of QFE scheme. The work proposed in (modic2019privacy, )
used GoFE libary proposed for the implementation of QFE scheme. We have
already detailed these libraries in section 5.2.

Python Libraries. (ligier2017privacy, ) used sklearn (pedregosa2011scikit, )
in python for implementation of ML model. (sans2018reading, ) and
(ryffel2019partially, ) implemented the ML model in Tensorflow
(abadi2015tensorflow, ) in python. (xu2019cryptonn, ) uses just a Numpy
(van2011numpy, ) for the implementation of ML model whereas
(carpov2020illuminating, ) used Keras (chollet2018keras, ) in python for the
implementation of ML model.

System specifications. Table 4 summarizes the machine specifications used for
performing experiments by the various authors. The experiments done by all the
research works are on intel CPUs. As CPUs are involved, there is an issue of
slow computation involved during training and prediction. This is far away
from the faster computations done by today’s GPU-based machine learning
models.

##  6\. Discussion

###  6.1. Pros and Cons

To ensure the fully privacy-preserving machine learning system, both training
and prediction phases over encrypted data should be integrated into the
system.

Our studies observed that for a perfect PPML system, both training and
prediction should be undergone over encrypted data. By doing so, the system
ensures the privacy of the user’s data for both tasks. As suggested in the
example of privacy-preserving cardiovascular disease prediction service in
section 1, such data of the user is highly confidential. In such cases, it
would be great if the cloud server can train the possessed model without
seeing the user’s private data. Similarly, if predictions are to be done, it
would also use the encrypted inputs. So, this interesting concept of training
and prediction without seeing helps us develop a fully privacy-preserving
machine learning system. Presently, only (xu2019cryptonn, ) focuses on both of
these tasks, but it is not efficient in terms of time.

Considering information leakage and adaptive security simultaneously is highly
important.

Security of the FE-based PPML schemes is entirely dependant on the underlying
FE scheme. IPFE is protected by selective security, whereas QFE is protected
by selective as well as adaptive security against chosen-plaintext attack.
There is a theoretical advancement by (cryptoeprint:2020:209, ) towards making
IPFE adaptively secured, but FE-based PPML works do not yet implement it.
Also, information leakage proposed by (carpov2020illuminating, ) is the only
work that discusses this concept. This work also follows the traditional
selective IPFE and adaptive QFE schemes. This raises a high need to consider
both information leakage and adaptive security while building an FE-based PPML
scheme.

There is no fit-for-all methodology.

None of the existing solutions considers all the criteria for a perfect PPML
system. However, we can rank the methodologies from each group i.e one from
the IPFE-based scheme and another from the QFE-based scheme. In the IPFE-based
scheme, (xu2019cryptonn, ) satisfies most of the criteria, whereas
(ryffel2019partially, ) performs the best in QFE-based methodologies. If
prediction accuracy and privacy is considered, then (ryffel2019partially, )
leads, whereas the overall performance of the system in terms of security,
privacy, and accuracy is concerned, then (xu2019cryptonn, ) leads over others.

FE Vs FHE.

Both FHE and FE cryptosystems are based on post-quantum lattice-based
cryptography. Apart from the key distinction, i.e., the additional key
requirement of FHE, FE has some pros and cons. The FE can be the best choice
when the computation results on the encrypted data are expected to be plain.
Although this is an advantage, present FE schemes support only inner-product
and quadratic computation on encrypted data. In this regard, FHE wins over FE
as it gives computation capability for more complex polynomials. Both
cryptosystems have the overhead of time and space complexity required for the
encryption, decryption, and key generation processes. FHE is more developed in
engineering aspects as leading companies like Microsoft (chen2017simple, ) and
IBM (halevi2014algorithms, ) have been working on it. FE has not yet received
attention and lacks engineering aspects. Due to this, even though FE and FHE
have a set of complex operations, FHE uses available modern hardware
accelerator devices and engineering solutions and wins over FE.

###  6.2. Challenges and future research directions

####  6.2.1. Functionality enhancement in FE scheme

The current research done on FE is limited to two functionality on integers:
inner product and bilinear maps. Recall that functionality is a function
computed over encrypted data to obtain decrypted results. As discussed, inner
product functionality is used in the IPFE scheme, whereas bilinear maps are
used in the QFE scheme. Because of this limited availability of
functionalities, FE-based PPML methodologies still require enhancement. For
example, QFE-based machine learning methods can use only 2-degree polynomial
networks. Also, FE cannot perform Min/max and comparison operations. If
enhancement in FE functionality is done, it will be helpful for PPML
researchers to come up with practical solutions for supervised and
unsupervised machine learning problems. However, these systems are not at a
level to be used in real-world applications.

####  6.2.2. Improving efficiency

Though FE-based PPML solutions have shown promising results on partially
encrypted machine learning, there is a need to improve the efficiency.
Ciphertexts and keys generated with larger security parameters are slower
compared to other cryptographic approaches. This becomes a threat when
computations are performed on large datasets. Therefore this efficiency stands
as a big challenge in FE-based PPML methodologies.

####  6.2.3. Improvement in Security and Privacy of FE scheme

As discussed in the previous subsection, the FE-based PPML methodologies’
security depends on the underlying FE scheme. IPFE schemes (abdalla2015simple,
; agrawal2016fully, ) are selectively secured against chosen-plaintext attacks
under the decisional-diffie hellman (DDH) assumption. QFE schemes
(ryffel2019partially, ; baltico2017practical, ) are adaptively secured against
chosen plaintext attacks under Matrix Decisional Diffie-Hellman assumption.
These FE schemes will lag in stricter security applications such as the
defense domain. Information leakage in PPML based FE schemes needs to be
explored and improved. In addition to this, machine learning attacks are also
interesting to consider in this area.

####  6.2.4. Enhancing Privacy-preserving Neural Networks

Today’s FE-based works have shown their demonstration only up to 5-layer
neural network. There is a scope to enhance the structure of neural networks.
As discussed above, QFE-based methods are limited to using only degree-2
polynomial networks. So, the enhancement of these networks is somehow
dependent on the underlying FE scheme. Implementation of various activation
functions is also dependent on the underlying FE scheme. Apart from this,
complex deep learning convolutional neural networks like VGGNet, AlexNet, and
GoogleNet are far from implementing FE. Also, the methods like transfer
learning can be applied if there exists a functionality that can decrypt the
encrypted model and retrieve the saved parameters like general CNNs.

####  6.2.5. Training-centric PPML systems

As the PPML is growing day by day, training over encrypted data is gaining the
attention of researchers (securenn, ). Presently, FE-based PPML schemes
proposed in the literature are more inference-centric. Although the work
proposed in (xu2019cryptonn, ) has successfully proposed a way for training a
neural network using functional encryption, it lacks computational efficiency.
There is a high need for PPML systems that can perform both training and
inference over encrypted data.

####  6.2.6. Training from multiple data sources

As far as machine learning is concerned, there are methodologies that focus on
training models from multiple data sources in the literature (securenn, ). The
enhanced versions of the FE scheme, like multi-input functional encryption
(abdalla2017multi, ; brakerski2018multi, ), can be used to serve this purpose.
These schemes facilitate using multiple vectors to perform computation based
on inner-product functionality. Such schemes could leverage the problem of
training the model possessed by a cloud server for training from multiple data
sources.

####  6.2.7. Using Multi-authority and Decentralized extensions of FE

The FE scheme used in PPML services requires a trusted third party or trusted
authority to be involved in generating the keys. The scheme proposed by
(ambrona2021controlled, ) can be used to avoid the involvement of trusted
authorities. In addition to this, there is a great scope for making the FE
schemes used in PPML decentralized by using the approaches proposed by
(chotard2018decentralized, ) and (chotard2020dynamic, ).

####  6.2.8. Need for open source library support

Functional encryption has gained widespread attention from researchers as far
as the theoretical aspects are concerned. However, it still lacks practical
library implementations compared to its predecessor, fully homomorphic
encryption. For example, HElib (halevi2014algorithms, ) by IBM research group,
and SEAL (chen2017simple, ) by Microsoft research group are great tools for
performing the fully homomorphic encryption-related task. Presently,
commendable efforts have been taken by FENTEC group (modic2019privacy, ) for
providing C and Go language versions of FE libraries, namely CiFEr and GoFE.
These libraries are in the development stage and are not yet ready to be
deployed in production. So, there is a considerable need for practically
applicable functional encryption libraries.

####  6.2.9. Need for hardware acceleration support

In all the implementations of FE-based PPML methodologies studied,
computations are done on the CPU. Until now, there has been no GPU support
available for FE implementation. Because of these limitations, we lack the
opportunity to utilize the highest GPU compute available today. Real-world
applications in machine learning are trained on massive data. So, performing
computations in such applications is very time-consuming if done on the CPU.
The work proposed by (fpga, ) uses SoC implementation to provide acceleration
support, but it needs special hardware. If easy-to-use hardware acceleration
support using widespread GPU or FPGA is provided in such applications, it will
speed up the computations faster. Such acceleration will eventually help in
making FE deployable in real-world applications.

####  6.2.10. Improving scalability and application to non-image datasets

Present works focused on FE-based PPML are limited to using MNIST like small
datasets for their experiments, and there are no implementations for larger
datasets. In addition to this, the applications to non-image datasets like
text corpus may also be explored. Therefore, this stands as a future research
direction in improving FE utilization in real-world applications.

##  7\. Conclusion

Privacy-preserving machine learning has gained widespread attention among
industry and academic researchers in recent years. Although approaches based
on fully homomorphic encryption, secure multi-party computation, and
differential privacy have been extensively studied, functional encryption-
based solutions are still less investigated. In this paper, we provide a
summary and systematization of privacy-preserving machine learning approaches
based on functional encryption. Our analysis assessed the extent to which
previous work has addressed the PPML using FE and identified key weaknesses in
this area. Additionally, our analysis demonstrates that FE-based approaches
could significantly contribute to the PPML issues, but there exist challenges
that should be addressed to achieve practical solutions. We hope that this
effort paves the path for the research community to investigate this emerging
yet important topic.

## References

  * (1) Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. Tensorflow: Large-scale machine learning on heterogeneous systems, 2015\.
  * (2) Abdalla, M., Bourse, F., De Caro, A., and Pointcheval, D. Simple functional encryption schemes for inner products.  In IACR International Workshop on Public Key Cryptography (2015), Springer, pp. 733–751.
  * (3) Abdalla, M., Gay, R., Raykova, M., and Wee, H. Multi-input inner-product functional encryption from pairings.  In Annual International Conference on the Theory and Applications of Cryptographic Techniques (2017), Springer, pp. 601–626.
  * (4) Agrawal, S., Libert, B., Maitra, M., and Titiu, R. Adaptive simulation security for inner product functional encryption.  Cryptology ePrint Archive, Report 2020/209, 2020.  https://eprint.iacr.org/2020/209.
  * (5) Agrawal, S., Libert, B., and Stehlé, D. Fully secure functional encryption for inner products, from standard assumptions.  In Annual International Cryptology Conference (2016), Springer, pp. 333–362.
  * (6) Akinyele, J. A., Garman, C., Miers, I., Pagano, M. W., Rushanan, M., Green, M., and Rubin, A. D. Charm: a framework for rapidly prototyping cryptosystems.  Journal of Cryptographic Engineering 3, 2 (2013), 111–128.
  * (7) Ambrona, M., Fiore, D., and Soriente, C. Controlled functional encryption revisited: Multi-authority extensions and efficient schemes for quadratic functions.  Proceedings on Privacy Enhancing Technologies 2021, 1 (2021), 21–42.
  * (8) Badawi, A. A., Chao, J., Lin, J., Mun, C. F., Jie, S. J., Tan, B. H. M., Nan, X., Aung, K. M. M., and Chandrasekhar, V. R. The alexnet moment for homomorphic encryption: Hcnn, the first homomorphic cnn on encrypted data with gpus.  arXiv preprint arXiv:1811.00778 31 (2018).
  * (9) Badawi, A. A., Hoang, L., Mun, C. F., Laine, K., and Aung, K. M. M. Privft: Private and fast text classification with homomorphic encryption.  arXiv preprint arXiv:1908.06972 (2019).
  * (10) Bahadori, M., and Järvinen, K. A programmable soc-based accelerator for privacy-enhancing technologies and functional encryption.  IEEE Transactions on Very Large Scale Integration (VLSI) Systems 28, 10 (2020), 2182–2195.
  * (11) Baltico, C. E. Z., Catalano, D., Fiore, D., and Gay, R. Practical functional encryption for quadratic functions with applications to predicate encryption.  In Annual International Cryptology Conference (2017), Springer, pp. 67–98.
  * (12) Bogetoft, P., Christensen, D. L., Damgård, I., Geisler, M., Jakobsen, T., Krøigaard, M., Nielsen, J. D., Nielsen, J. B., Nielsen, K., Pagter, J., et al. Secure multiparty computation goes live.  In International Conference on Financial Cryptography and Data Security (2009), Springer, pp. 325–343.
  * (13) Boneh, D., and Franklin, M. Identity-based encryption from the weil pairing.  SIAM journal on computing 32, 3 (2003), 586–615.
  * (14) Boneh, D., Sahai, A., and Waters, B. Functional encryption: Definitions and challenges.  In Theory of Cryptography Conference (2011), Springer, pp. 253–273.
  * (15) Bourse, F., Minelli, M., Minihold, M., and Paillier, P. Fast homomorphic evaluation of deep discretized neural networks.  In Annual International Cryptology Conference (2018), Springer, pp. 483–512.
  * (16) Brakerski, Z., Komargodski, I., and Segev, G. Multi-input functional encryption in the private-key setting: Stronger security from weaker assumptions.  Journal of Cryptology 31, 2 (2018), 434–520.
  * (17) Carpov, S., Fontaine, C., Ligier, D., and Sirdey, R. Illuminating the dark or how to recover what should not be seen in fe-based classifiers.  Proceedings on Privacy Enhancing Technologies 2020, 2 (2020), 5–23.
  * (18) Chen, H., Laine, K., and Player, R. Simple encrypted arithmetic library-seal v2. 1.  In International Conference on Financial Cryptography and Data Security (2017), Springer, pp. 3–18.
  * (19) Chillotti, I., Gama, N., Georgieva, M., and Izabachene, M. Faster fully homomorphic encryption: Bootstrapping in less than 0.1 seconds.  In international conference on the theory and application of cryptology and information security (2016), Springer, pp. 3–33.
  * (20) Chollet, F., et al. Keras: The python deep learning library.  Astrophysics Source Code Library (2018), ascl–1806.
  * (21) Chotard, J., Dufour-Sans, E., Gay, R., Phan, D. H., and Pointcheval, D. Dynamic decentralized functional encryption.  In Annual International Cryptology Conference (2020), Springer, pp. 747–775.
  * (22) Chotard, J., Sans, E. D., Gay, R., Phan, D. H., and Pointcheval, D. Decentralized multi-client functional encryption for inner product.  In International Conference on the Theory and Application of Cryptology and Information Security (2018), Springer, pp. 703–732.
  * (23) Dwork, C., McSherry, F., Nissim, K., and Smith, A. Calibrating noise to sensitivity in private data analysis.  In Theory of cryptography conference (2006), Springer, pp. 265–284.
  * (24) Fredrikson, M., Jha, S., and Ristenpart, T. Model inversion attacks that exploit confidence information and basic countermeasures.  In Proceedings of the 22nd ACM SIGSAC conference on computer and communications security (2015), pp. 1322–1333.
  * (25) Gentry, C. Fully homomorphic encryption using ideal lattices.  In Proceedings of the forty-first annual ACM symposium on Theory of computing (2009), pp. 169–178.
  * (26) Gilad-Bachrach, R., Dowlin, N., Laine, K., Lauter, K., Naehrig, M., and Wernsing, J. Cryptonets: Applying neural networks to encrypted data with high throughput and accuracy.  In International Conference on Machine Learning (2016), pp. 201–210.
  * (27) Graepel, T., Lauter, K., and Naehrig, M. Ml confidential: Machine learning on encrypted data.  In International Conference on Information Security and Cryptology (2012), Springer, pp. 1–21.
  * (28) Halevi, S., and Shoup, V. Algorithms in helib.  In Annual Cryptology Conference (2014), Springer, pp. 554–571.
  * (29) Hart, W., Johansson, F., and Pancratz, S. Flint: Fast library for number theory. v. 2.4. 3.  See http://flintlib. org (2013).
  * (30) Hesamifard, E., Takabi, H., and Ghasemi, M. Cryptodl: Deep neural networks over encrypted data.  arXiv preprint arXiv:1711.05189 (2017).
  * (31) Hesamifard, E., Takabi, H., Ghasemi, M., and Wright, R. N. Privacy-preserving machine learning as a service.  Proceedings on Privacy Enhancing Technologies 2018, 3 (2018), 123–142.
  * (32) Joux, A. A one round protocol for tripartite diffie–hellman.  Journal of cryptology 17, 4 (2004), 263–276.
  * (33) Kohavi, R. Scaling up the accuracy of naive-bayes classifiers: A decision-tree hybrid.  In Kdd (1996), vol. 96, pp. 202–207.
  * (34) LeCun, Y., Cortes, C., and Burges, C. Mnist handwritten digit database.
  * (35) Li, Q., Huang, Z., Lu, W.-j., Hong, C., Qu, H., He, H., and Zhang, W. Homopai: A secure collaborative machine learning platform based on homomorphic encryption.  In 2020 IEEE 36th International Conference on Data Engineering (ICDE) (2020), IEEE, pp. 1713–1717.
  * (36) Ligier, D., Carpov, S., Fontaine, C., and Sirdey, R. Privacy preserving data classification using inner-product functional encryption.  In ICISSP (2017), pp. 423–430.
  * (37) Livni, R., Shalev-Shwartz, S., and Shamir, O. On the computational efficiency of training neural networks.  Advances in neural information processing systems 27 (2014).
  * (38) Lynn, B. Pbc library manual 0.5. 11, 2006.
  * (39) Marc, T., Stopar, M., Hartman, J., Bizjak, M., and Modic, J. Privacy-enhanced machine learning with functional encryption.  In European Symposium on Research in Computer Security (2019), Springer, pp. 3–21.
  * (40) McMahan, B., Moore, E., Ramage, D., Hampson, S., and y Arcas, B. A. Communication-efficient learning of deep networks from decentralized data.  In Artificial Intelligence and Statistics (2017), PMLR, pp. 1273–1282.
  * (41) Modic, J. Privacy-enhanced machine learning with functional encryption.  In Computer Security–ESORICS 2019: 24th European Symposium on Research in Computer Security, Luxembourg, September 23–27, 2019, Proceedings, Part I (2019), vol. 11735, Springer Nature, p. 3.
  * (42) Mohassel, P., and Zhang, Y. Secureml: A system for scalable privacy-preserving machine learning.  In 2017 IEEE Symposium on Security and Privacy (SP) (2017), IEEE, pp. 19–38.
  * (43) Panzade, P., and Takabi, D. Towards faster functional encryption for privacy-preserving machine learning.  In 2021 Third IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA) (2021), IEEE, pp. 21–30.
  * (44) Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., et al. Scikit-learn: Machine learning in python.  the Journal of machine Learning research 12 (2011), 2825–2830.
  * (45) Ribeiro, M., Grolinger, K., and Capretz, M. A. Mlaas: Machine learning as a service.  In 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA) (2015), IEEE, pp. 896–902.
  * (46) Rouhani, B. D., Riazi, M. S., and Koushanfar, F. Deepsecure: Scalable provably-secure deep learning.  In Proceedings of the 55th Annual Design Automation Conference (2018), pp. 1–6.
  * (47) Ryffel, T., Dufour-Sans, E., Gay, R., Bach, F., and Pointcheval, D. Partially encrypted machine learning using functional encryption.  arXiv preprint arXiv:1905.10214 (2019).
  * (48) Sabt, M., Achemlal, M., and Bouabdallah, A. Trusted execution environment: what it is, and what it is not.  In 2015 IEEE Trustcom/BigDataSE/ISPA (2015), vol. 1, IEEE, pp. 57–64.
  * (49) Sans, E. D., Gay, R., and Pointcheval, D. Reading in the dark: Classifying encrypted digits with functional encryption.  IACR Cryptol. ePrint Arch. 2018 (2018), 206.
  * (50) Van Der Walt, S., Colbert, S. C., and Varoquaux, G. The numpy array: a structure for efficient numerical computation.  Computing in science & engineering 13, 2 (2011), 22–30.
  * (51) Wagh, S., Gupta, D., and Chandran, N. Securenn: 3-party secure computation for neural network training.  Proceedings on Privacy Enhancing Technologies 2019, 3 (2019), 26–49.
  * (52) Wiesberg, A., and Armknecht, F. Unsupervised machine learning on encrypted data.  Cryptology ePrint archive 2018 (2018), Report–411.
  * (53) Xu, R., Joshi, J. B., and Li, C. Cryptonn: Training neural networks over encrypted data.  In 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS) (2019), IEEE, pp. 1199–1209.