heshort = HE, long = Homomorphic Encryption fheshort = FHE, long = fully
Homomorphic Encryption mlshort = ML, long = Machine Learning mpcshort = MPC,
long = Multi-Party Computation idashshort = iDASH, long = integrating Data for
Analysis, Anonymization, and Sharing nnshort = NN, long = Neural Network,
short-plural = s, long-plural = s rlweshort = RLWE, long = Ring Learning with
Error teeshort = TEE, long = Trusted Execution Environements gcshort = GC,
long = garbled circuits ssshort = SS, long = secret sharing otshort = OT, long
= oblicious transfer dhshort = DH, long = data holder, short-plural = s, long-
plural = s dpshort = DP, long = data processor, short-plural = s, long-plural
= s

# Report: State of the Art Solutions for Privacy Preserving Machine Learning
in the Medical Context

Jasmin Zalonis111University of Mannheim, Mannheim, Germany first.last@uni-
mannheim.de Frederik Armknecht11footnotemark: 1 Björn Grohmann222gematik GmbH,
Berlin, Germany<EMAIL_ADDRESS>Manuel Koch22footnotemark: 2

##  1 Introduction

ml on Big Data gets more and more attention in various fields. Over the last
years a lot of improvement happened in the field of ml. Originally, the
assumed scenario was as follows: One party has a big amount of data in the
clear and wants to learn something on the data. But this has changed in the
last years. Privacy-preserving techniques become more important, even
necessary due to legal regulations such as the General Data Protection
Regulation (GDPR) and on the other hand data is often distributed among
various parties. Especially in the medical context there are several dh, e.g.
hospitals or the patient itself and we need to deal with highly sensitive
values. Whenever sensitive data is distributed among parties the standard
scenario with one data processor that holds the data in clear is not
applicable any more. A real world scenario would be data that is held in an
electronic patient record that is available in many countries by now. In
Germany, for example, the "elektronische Patientenakte" (ePA) is available
since 2021. It provides their users with the functionality of storing medical
data that can then be used for further diagnosis or research. The medical data
is encrypted. Users (e.g. physicians, hospitals) can only decrypt the data
after patient authorization. One of the main questions concerning this
scenario is whether it is possible to process the data for research purposes
without violating the privacy of the data owner. We want to evaluate which
cryptographic mechanism can be used in the scenario stated in Figure 1.

Figure 1: Scenario of the infrastructure for training a ML algorithm.

On the left-hand side we can see that the patient holds his own data and wants
to protect his sensitive information, on the right-hand side we have the
research institute that wants to gain new information from patient data by
applying a ml algorithm on a data set. In the following, we want to describe
an approach where these two interests can be united. Each dh, i.e. each
patient, can choose which research institute can learn from his data, but it
is crucial that nobody has access to the clear text data except the dh itself.
In the middle we have an infrastructure provider processing the patient data
for model training. It is crucial, that the infrastructure provider does not
learn anything about the patient. The patient sends his data either encrypted
to one dp or distributed to several dp. After the training phase of the ml
algorithm is done the research institute gets the trained model in a secure
form and can use it to gain new information or even to offer his algorithm as
a service, again in a privacy-preserving way, see Figure 2.

Figure 2: Scenario of the infrastructure for evaluating a ML algorithm.

In this context we want to explore mainly three different mechanisms: he, mpc
and tee. The above scenario changes a little with each technique. For he we
assume that each dh encrypts his data with the public key of the research
institute but instead of sending it directly to the research institute, which
would allow them to access the data in clear, they send it to one of the dp.
We assume there are at least two dp and the final data set is distributed
among them, so no dp has the whole encrypted data set. They both train
individual a local machine learning model, eventually exchanging some
intermediate data in encrypted form and therefore generating a global model,
which is send back to the research institute. The institute now can decrypt
the model and has only access to the learned model. In contrast the scenario
in mpc does not include any encryption. Rather we assume that each dh splits
his data into so called shares, pieces that alone do not reveal something
about the data record. Each dp gets one piece, one share of the data and then
compute together with the other dp the algorithm for training the ml model.
The research institute gets the model as final result as before, for example
in form of all the shares which can then be recombined. In tee we assume that
each server (dp) has a secure enclave. We assume full trust in this enclave.
The patients as dh send their data to a server in encrypted form. The data can
only be decrypted inside the secure enclave, which trains a local ml
algorithm. Training of the global model requires communication between the
enclaves, either in encrypted form, so that only the enclaves can decrypt it
or in a masked form, so that the intermediate results that are exchanged do no
reveal anything about the data. Especially in the context of mpc the scenario
where we have several clients that do not want to be involved in the
computation but also do not want to provide their data in clear, is called the
client-server model. Overall we assume a semi-honest security model, that is
the dp merely try to gather information out of the protocol, but do not
deviate from the protocol specification itself. In the mpc scenario we
additionally assume that we have an honest majority.

It should be noted, that in the above scenario we do not want to clarify the
question which functions, i.e. ml algorithms are save to compute. We do not
consider indirect leakage through the model parameters or the predictions of
the model. It is out of the scope of this work to investigate which models are
save in a way that the research institute does not learn something about the
data from the generated model. Rather we want to investigate the direct
leakage, e.g. how can we compute the model while minimizing avoiding the
damage to privacy by addressing following questions: Which cryptographic
technique makes the most sense for the above scenario, he, mpc, or tee? Which
ml algorithm is suited the best for the distributed scenario? We want to
address the questions first by giving a quick overview of the standard ml
algorithms and the three cryptographic mechanism. Following we will look at
the state-of-the-art solutions in the recent research and with this
information conclude to give a recommendation. Since the training of the
algorithms are the more challenging task we focus on paper that consider the
training phase and won’t discuss paper who consider only the evaluation of the
previous trained model.

##  2 Cryptographic Techniques

###  2.1 Homomorphic Encryption

he is an encryption technique that allows computation on ciphertext while
preserving the possibility to decrypt to the corresponding computations on the
plaintexts. A distinction is made between partially he, where only some
operations can be computed in a homomorphic way, leveled or somewhat he, where
only a limited number of computations can be correctly evaluated and fhe where
arbitrary unlimited computations on encrypted data can be evaluated. The
security of the most fhe schemes is based on the hardness of the rlwe problem.
Where a standard encryption scheme has three algorithms key-generation $Gen$,
encryption $Enc$ and decryption $Dec$, a homomorphic encryption scheme has an
additional algorithm evaluation $Eval$ that takes an supported function f, a
set of ciphertexts and computes the corresponding function on the ciphertexts.
Additionally the scheme needs to expand the correctness to the evaluation,
i.e. $Dec(Eval(f,Enc(m_{1}),...,Enc(m_{n})))=f(m_{1},...,m_{n})$ and needs to
fulfill a compactness property, meaning that the size of the output of $Eval$
is independent of the size of the evaluated function. The standard scenario
for he is out-sourced computation, meaning a party has a big amount of data
but not the resources to process these data. So in order to get to the result
it sends the data in encrypted form to another party, which then can process
the data in encrypted form and returns the encrypted result. Even when the
service provider’s system is compromised, the data would remain secure. The
most popular schemes today are BGV [12], CKKS [17], BFV [23] and TFHE [18],
supported by various libraries, e.g HElib, PALISADE, Microsoft SEAL and TFHE.
fhe is a promising technique to enable a untrusted server to perform analyses
on data. Because all input and output data as well as all intermediate results
are encrypted privacy is preserved. With solutions that use only fhe we need
to trust that the server performs the algorithm as intended.

###  2.2 Multi-Party Computation

In contrast the goal of mpc is for two or more parties to jointly compute a
function over their inputs while keeping those inputs private and hidden from
the other party. Informally the basic properties that a mpc protocol must
fulfill are: input privacy, no information about the private data can be
inferred from the sent messages than the information that can be inferred from
seeing the output of the computed function alone; Correctness, the parties
obtain the correct output, even when some parties misbehave. Sometimes also
fairness is seen as a property of mpc protocols, meaning that if one party
learns the output, then all of the parties learn the output. Especially mpc
protocols with more than two parties often rely on ss, where a secret input is
shared among a number of parties by distributing a share of the input to each
party, as described in Section 1. Shamir ss and additive ss are the two
commonly used protocols. Other protocols like gc are based on two or more
server, which have their own private data. Since this is not fit for our use
case we will only look at paper, that accomplish our requirement of a
distributed data set among the servers. There are mainly two security models
in mpc.

  * -

Semi-Honest or Passive Security: Corrupted parties only cooperate to gather
information but do not deviate from the protocol specification. Also called
honest but curious adversary.

  * -

Malicious or Active Security: In this scenario the adversary parties are also
allowed to deviate from the protocol. Therefore protocols that achieve
security in an active setting yield in a higher security guarantee by
detecting malicious behavior. When it comes to handling the malicious behavior
the protocol can either abort and no one receives the output, or can guarantee
robustness/fairness where everybody receives the correct output. The last
variant is the most expensive variant.

###  2.3 Trusted Execution Environments

tee also called secure enclaves, as for example Intel SGX [1] and ARM
TrustZone [2], enable execution of programs in secure hardware enclaves in an
untrusted system. In our scenario we assume we have an application that wants
to train or evaluate a ml algorithm. The clients send their sensitive data in
encrypted form to the application, which can do computations with the help of
the enclave on these secrets, without ever having access to the data in the
clear. The confidentiality and integrity of the executed code and data are
protected inside the enclaves. The server on which the enclave runs can seen
as untrusted but we need to have trust in the enclave.

##  3 Machine Learning Algorithm

The field of ml algorithm is very wide and there are lots of different
algorithms and their advancements. We want to give a quick overview of the
basic algorithms. In general ml algorithm are divided into supervised and
unsupervised algorithms. In supervised ml there are both training data with an
explicit label and test or evaluation data whereas in unsupervised machine
there are no labeled data records. In supervised ml the training data is used
to train a model and predict the outcomes of the evaluation data on the basis
of the learned model. Supervised learning is additional divided into
regression models, e.g. linear regression and classification/categorization
models, e.g. decision trees (classification trees) or logistic regression.
Neural Networks (NN) are also supervised ml algorithms that can be used both
for regression and categorization. Unsupervised ml algorithms do not use
training data and therefore do not predict based on a trained model. Rather
they aim to find structure in given data. Popular unsupervised algorithms are
for example clustering algorithms. Since our use case is to train a model with
some ground data and later on provide the model for further prediction only
supervised ml algorithms are practicable. We will shortly describe these
algorithms and how they can be trained in the following. We assume a labeled
training set $X=(x^{i},y^{i})^{n}_{i=1}$ with $n$ data records
$x^{i}=(x^{i}_{1},....,X^{i}_{m})$, $m$ features and $y$ as the corresponding
labeled output.

###  3.1 Linear Regression

The idea of Linear Regression is to fit a linear function to the input data,
so that the function passes as close as possible to all of the data points.
The trained model will consists of a weight-vector
$\theta=(\theta_{1},\theta_{2},...,\theta_{m})$ and to predict the label of an
evaluation record $x$ we simply compute the linear function
$h_{\theta}(x)=\sum_{j=1}^{m}\theta_{j}\cdot x_{j}$ where
$h_{\theta}\in\mathbb{R}$ is a continuous value. To train the model, i.e. to
receive the weights that best fit the training data, we want to minimize the
costs of the predicted outcome versus the real outcome (labels). The cost
function can be defined with different metrics. For example the mean squared
error: $J(\theta)=\frac{1}{2n}\sum_{i=1}^{n}(h_{\theta}(x^{i})-y^{i})^{2}$ or
the L2-norm: $J(\theta)=\sqrt{\sum_{i=1}^{n}(h_{\theta}(x^{i})-y^{i})^{2}}$.
The weights can theoretically be computed as $\theta=(X^{T}\cdot X)^{-1}\cdot
X^{T}\cdot Y$ but for big data sets this has a high computational effort.
There also exists iterative methods to compute $\theta$ with less effort.
Because most of the paper listed in Section 4 use a variant of the gradient
descent we will present the main idea of this iterative algorithm.

#### Gradient Descent

is an optimization algorithm to find a minimum of a function, in the case of
linear regression the cost function $J_{\theta}$. The main idea is to start
with some initial arbitrary values for the weights $\theta_{1},...,\theta_{m}$
and keep changing them according to the derivative to reduce $J_{\theta}$, see
Algorithm 1. There are three big variants of the gradient descent algorithm in
ml. In the stochastic gradient descent we will go through the data set using
one record per iteration and if the function did not converge after one round
in the data set we walk through the data set again. In (mini-)batch gradient
descent a batch of data is used in each iteration until convergence. When the
batch is smaller than the whole data set, it is sometimes called mini-batch
gradient descent, which combines the advantages of stochastic and batch
gradient descent.

repeat simultaneously until convergence {

$\theta_{j}=\theta_{j}-\alpha\frac{\partial}{\partial\theta_{j}}J_{\theta}$
for $j=1,...m$

// with $\alpha$ as learning rate

}

Algorithm 1 Gradient Descent

###  3.2 Logistic Regression

Logistic Regression is a classification algorithm, where the predicted values
take on only a small number of discrete values, e.g. in the binary
classification problem the values $0$ and $1$ ore sometimes also $-1$ and $1$.
The model again consists of a weight vector $\theta$ and the model uses the
sigmoid function or sometimes called logistic function to predict a value in
the range of $0$ and $1$: $h_{\theta}(x)=\sigma(\theta^{T}x)$ with
$\sigma(z)=\dfrac{1}{1+e^{-z}}$. The sigmoid function $\sigma$ maps any real
number to the $(0,1)$ interval. $h_{\theta}(x)$ gives the probability, that
our data record belongs to the class 1. Often the classification is done based
on the threshold $\tau=0.5$, meaning that every record with $h_{\theta}(x)\geq
0.5$ belongs to the class 1. The cost function for logistic regression looks
like
$J(\theta)=-\dfrac{1}{n}\sum_{i=1}^{n}\left(y^{i}log\left(h_{\theta}(x^{i})\right)+(1-y^{i})log\left(1-h_{\theta}(x^{i})\right)\right)$
and is called logistic loss or binary cross entropy. To build a model with
more than two categories we basically train a logistic regression classifier
for each class (throwing all the other classes into a single second class) and
for a prediction pick the class that has the highest probability. The model
can be trained using the gradient descent algorithm (Algorithm 1). Another
popular algorithm is Newton’s Method to train a logistic regression model.

#### Newton’s Method

is an iterative method that either can be used to find the root of a
differentiable function or in our context to find the root of the derivative
of a function in order to find the critical points (minima, maxima or saddle
points) of $f$: Given a twice differentiable function we want to solve the
optimization problem $min_{x\in\mathbb{R}^{n}}f(x)$, see Algorithm 2. In the
context of the logistic regression we want to minimize the cost function,
therefore $f(\theta)=J(\theta)$

repeat until convergence {

$\theta^{t+1}=\theta^{t}-H_{J}(\theta^{t})^{-1}\triangledown J(\theta^{t})$

}

// with $H_{J}(\theta)$ is the Hessian-Matrix with
$H_{ij}=\dfrac{\partial^{2}J(\theta))}{\partial\theta_{i}\partial\theta_{j}}$

// and $\triangledown J(\theta)=\begin{pmatrix}\dfrac{\partial
J(\theta)}{\partial\theta_{0}}\\\ ...\\\ \dfrac{\partial
J(\theta)}{\partial\theta_{n}}\end{pmatrix}$ the gradient

Algorithm 2 Newton’s Method for Optimization

Where gradient descent is a simpler algorithm, newton’s method does not need
parameters and typically needs fewer iterations and therefore converges much
faster. One downside is that newton’s method has more expensive iterations and
therefore is not fit for all use cases.

###  3.3 Decision Trees

Decision trees can either be part of regression (regression tree) or a part of
classification (classification tree), yet the main idea is the same. Therefore
we will only describe the classification trees. The model consists of a binary
tree with several thresholds $\tau_{i}$ at the inner nodes. Each leaf node
corresponds to one class. A new data record $x$ is classified by walking down
the tree from the root to the leafs. At each inner node $i$ there is a
decision criteria $\tau_{i}$ for one of the features of $x$ to decide on which
branch to take next. There exists non-parametric learning techniques that can
be used to produce either classification or regression trees, for example ID3,
C4.5, C5.0. All of them share the same idea. At the root node start with all
training examples, i.e. the hole data set $X$ and select an attribute on the
basis of an splitting criteria. We then partition the instances according to
the selected attribute recursively. The partitioning stops when there are no
examples left or all examples for a given node belong to the same class (=
purity) or there are no remaining attributes for further partitioning. When it
comes to select an attribute for partitioning, we want to select the
attribute, that gives the most information. A common used metrics is the Gini-
Index: $Gini(D)=1-\sum_{i=1}^{c}p_{i}^{2}$ with $c$ the number of classes and
$p_{i}$ the relative frequency of class $j$ in the data set $D$. If the data
set $D$ is split on attribute $k$ into $l$ subsets the Gini-Index is defined
as $Gini_{k}(D)=\sum_{i=1}^{l}\dfrac{|D_{i}|}{|D|}\cdot Gini(D_{i})$. We aim
to minimize the Gini-index. Another popular metric is the information gain,
based on entropy $H(D)=-\sum_{j=1}^{c}p_{j}\cdot log_{2}(p_{j})$. The
information gain is the measure of the difference in entropy from before to
after the set $D$ is split on an attribute $k$:
$IG_{k}(D)=H(D)-\sum_{i=1}^{l}p(D_{i})H(D_{i})$.

#### Random Forest

are algorithm that can be used both for classification and regression, the
algorithm builds multiple decision trees and merges the result back together
by either computing the mean or average for a regression problem or returning
the majority decision of all decision trees in the forest.

###  3.4 Neural Networks

A nn is a ml algorithm that consists of different nodes, where each node
basically is a own ml algorithm, e.g. each node could be a logistic
regression. In general a nn is built of different layers, an input and and
output layer and often in between several hidden layers. Each layer has some
nodes, called neurons. The number of neurons of the output layer is defined by
the desired output, e.g. for a simple binary classification problem the output
layer has two nodes (see Figure 3).

$\vdots$$\vdots$InputlayerHiddenlayerOuputlayer Figure 3: Exemplary setup of a
nn

The model consists of several weight vectors
$\theta^{k}=(\theta^{k}_{i},j)_{i\in\\{1,...,l_{k}\\},j\in\\{l_{k=1}\\}}$
connecting the $l_{k}$ neurons of layer $k$ to the $l_{k+1}$ neurons of layer
$k+1$. In each neuron an activation function is applied on the weighted sum of
the inputs to compute the output.There are different activation functions, we
want to mention some popular ones:

  * •

Linear function: $f(x)=ax$

  * •

Sigmoid function: $\sigma(x)=\dfrac{1}{1+e^{-x}}$

  * •

Tanh function: $tanh(x)=\dfrac{e^{2x}-1}{e^{2x}+1}$

  * •

ReLu333Rectified linear unit function: $f(x)=max(0,x)$ gives an output $x$ if
$x$ is positive and $0$ otherwise

  * •

PReLu444Parametric ReLu function: $f(x)=max(x,ax),a\leq 1$ with $a$ as
parameter that will change depending on the model.

  * •

Softmax function: $\sigma(x)_{j}=\dfrac{e^{z_{j}}}{\sum_{k=1}^{K}e^{z_{k}}}$,
usually used when we want to address the multiple class scenario with $k$
classes

It is out of the scope of this work to further explain when to use which
activation function or how to build the architecture of the network before
training. In this context we assume only nn in supervised ml. A common
learning algorithm for nn is again the gradient descent algorithm, with help
of back propagation. Again, the aim is to minimize the cost-function. But
instead of modifying just one weight vector, in a nn we need to adapt all
weights of each node of each layer. So in each iteration the algorithm first
calculates the overall costs of the training data set, called forward pass or
forward propagation, afterwards the algorithm, starting at the output layer,
adapts the weights of the previous layer, called backward pass or backward
propagation. The state-of-the-art learning algorithms in the privacy-
preserving context are mostly optimizations of the gradient descent method.

###  3.5 k-Nearest-Neighbor ($k$NN)

Although the $k$NN algorithm belongs to the supervised algorithms, it does not
directly learn a model like the previous ones and therefore does not need a
training phase before the evaluation phase. Rather it stores all the training
data and classifies new data records based on similarity measures. $k$ is the
previous defined constant and for evaluating the data record is assigned to
the class which is most frequent among the $k$ nearest training samples.
Because this algorithm requires to store the training data at the party that
later wants to evaluate new data records, the $k$NN is not fit for our use
case and we will not go into further detail.

###  3.6 Distributed Machine Learning

Since in the case of he and tee our use case requires to have the data
partitioned so that no server has access to the whole data set, we need to
consider distributed learning, also called data parallelism. Since the
upcoming of big data the parallelism of ml is a focus of research. A widely
examined and adopted way is data parallelism in gradient descent based
learning algorithms (e.g. linear regression, logistic regression, Neural
Networks). The data is divided so that each server has one part of the data
set and the model. Each server operates on its subset of the data and predicts
the errors between the training samples and the labeled outputs. After that
the server update their model based on the errors and need to communicate the
changes to the other nodes. So in each iteration the servers need to
synchronize the model parameters, or gradients, at the end of the batch
computation to ensure they are training a consistent model. The training of
decision trees in a distributed way is not quite as obvious as in gradient
descent, but again there are several algorithms that can be used, e.g. [48],
[10], [44], [51]. Especially when dealing with big data without the privacy-
preserving approach there exists frameworks that handle distributed ml like
Apache Spark [43].

##  4 Existing Solutions

###  4.1 Surveys on privacy-preserving ML

Several paper give an overview of existing solutions in privacy-preserving ml.
We want to present some of them. [19] consider different types of
heterogeneous data and privacy-preserving mechanism that can be used for these
data types. They consider not only cryptographic methods but their main focus
lies on other methods like anonymization and obfuscation. With this work they
aim to show which privacy-preserving mechanism are applicable for the
characteristics of each data type. [32] review privacy-aware anonymity-based
techniques for electronic health records for heterogeneous data types but do
not consider cryptographic methods. [55] provide a technical review and
comparison of existing solutions for outsourced computation. They look at
secure mpc, pseudorandom functions, software guard extensions, perturbation
approaches but mainly aim to provide an overview of the existing secure
outsourcing solutions based on he algorithms. A wide range of applications
like regression training or biometric authentication is considered. They
compare the security and efficiency performances. [9] provide a overview of
existing he and mpc solutions for the problem of privacy preserving nn,
whereas [30] focuses on clustering, a popular unsupervised machine learning
algorithm and considers he as well as mpc. They claim to be the first survey
that is concerned with an unsupervised ml algorithm. [14] performs a cross-
field systematization of knowledge of privacy-preserving tree-based models.
They consider differential privacy based solutions as well as cryptographic
solutions like mpc and he. [52] investigate the state-of-the-art works to
resolve training of ml in a privacy-preserving way. They consider secure mpc
and federated learning solutions. They categorize secure mpc frameworks into
he-based,gc-based and ss-based and mixed learning frameworks. As far as we
know, we are the first that consider several ml algorithms in the context of
cryptographic solutions focusing on the client-server model where several
clients provide their data in a privacy-preserving way and do not want to be
involved with the computation. Next we want to consider some recent state-of-
the-art solutions using the explained cryptographic solutions, he, mpc and to
some extend also tee. We will categorize them by the used technique and the
trained algorithm, considering only solutions that consider privacy-preserving
ml training in the medical context.

###  4.2 he based Solutions

In this section we want to introduce some state-of-the-art paper considering
homomorphic ml. Although our use case in the he scenario designs a distributed
learning, we will see only classic one server setups, since we could not find
any paper regarding distributed homomorphic ml. Table 1 gives an overview over
the paper considering he.

|  linear Regression |  logistic Regression |  Decision Trees |  NN |  CKKS |  FV |  BGV |  TFHE |  |
---|---|---|---|---|---|---|---|---|---|---
Paper | Algorithm | Scheme | Data Set⋆ | Time
[28]∗ | ● | ○ | ○ | ○ | ○ | ○ | ○ | ○ |  $10^{4}$ r, 20 f | 10.28 s
[11] | ○ | ● | ○ | ○ | ○ | ● | ○ | ○ | (7) 245 r, 3 f, 10643 SNPs | 44 min
[15] | ○ | ● | ○ | ○ | ○ | ● | ○ | ○ | (5) 1579 r, 108 f | 14.9/115.33 h
1500 r, 196 f | 27.1/48.76 h
[35] | ○ | ● | ○ | ○ | ● | ○ | ○ | ○ | (5) 1579 r, 108 f | 8 min
(1) 1253 r, 10 f | 3.6 min
(2) 189 r, 10 f | 3.3 min
(3) 15,649 r, 16 f | 7.3 min
(4) 379 r, 10 f | 3.5 min
(6) 575 r, 9 f | 3.5 min
[49]∗ | ● | ○ | ○ | ○ | ○ | ○ | ○ | ○ |  $10^{4}$ r, 20 f | 64.35 s
[36] | ○ | ● | ○ | ○ | ● | ○ | ○ | ○ | (1) 1253 r, 10 f | 131 min
(2) 189 r, 10 f | 101 min
(3) 15,649 r, 16 f | 265 min
(4) 379 r, 10 f | 119 min
(6) 575 r, 9 f | 109 min
[7] | ○ | ○ | ● | ○ | ● | ○ | ○ | ○ | - | 19 - 1220 min
[13] | ○ | ● | ○ | ○ | ● | ○ | ○ | ● | (7) 245 r, 3 f, 10643 SNPs | 186 min
[42] | ○ | ○ | ○ | ● | ○ | ○ | ● | ● | (10) 70000 r, 784 f | 8 days
Table 1: State of the art homomorphic encryption based solutions.
∗Using Pailier as partial he scheme and data masking.
⋆More information about the data sets can be found in the Appendix A.

#### Linear Regression

[28] addresses privacy-preserving linear regression via a partial homomorphic
encryption scheme (e.g. Pailier). They assume several data owners, that want
to share their information but only if it is encrypted, a Machine-Learning
Engine (MLE) and a Crypto Service Provider (CSP). The MLE wants to run a
linear regression algorithm on the merged data. The CSP generates a
public/private key pair and sends the public key to data holders and the MLE.
The data owner send their encrypted records to the MLE which performs
homomorphically a masking algorithm and sends the encrypted masked data to the
CSP. The CSP decrypts the masked data and performs the linear equation of the
regression problem and sends the result back to the MLE which then demasks the
result and obtains the real model. [49] is very similar to [28], but they use
a different masking technique. The papers presented above both assume a
similar use case as we do and the combination of homomorphic encryption and
data masking realizes a protocol ideally suited for practical applications.
One drawback is, that the use of the CSP introduces communication costs, that
are prevented in the standard he scenario.

#### Logistic Regression

[36] train the logistic regression with the HEAAN scheme [17] using gradient
descent. Since the sigmoid function cannot be directly evaluated with an he
scheme it needs to be approximated. They use a global approximation method for
the sigmoid function of the logistic regression minimizing the mean squared
error. They test their implementation with degree 3 and 7 least square
approximation while degree 3 requires a smaller depth for evaluation and
degree 7 polynomial has better precision. To speed up the computation they use
a packing mechanism to perform several evaluations in parallel. Instead of
running the gradient descent until convergence, they use a fixed number of
iterations. The runtime of their implementation needs between 265 and 94
minutes depending on the used data set. [15] are participants of the
idash555Since 2014 the idash competition is yearly hosted. The competition in
the biomedical sector is concerned with privacy enhancing technologies around
the theme of genomic and biomedical privacy. They aim to bring experts in
security, privacy and bioinformatics together to test the limits of secure
computations. 2017 competition. In the year 2017 one of the tasks was to train
a logistic regression model over encrypted genomic data to predict disease
based on a patients genome. They used the FV-scheme [24] with batching and
modified the gradient decent method to minimize the increase in the size of
the numbers. For this they designed two different variants. The first variant
named 1-Bit Gradient Descent updates in each iteration the weight by a
learning rate multiplied by the sign of the current gradient, in the second
they use the original gradient descent with an approximated sigmoid function.
Additionally they modified the bootstrapping method to combine boot with
scaling to prevent plaintext size expansion. They tested the both algorithms
on the idash dataset with 36 iterations of the Gradient Descent and also on a
modified MNIST dataset, where they only used 1500 images, containing the
handwritten numbers 3" or "8" and reduced every image to 196 features. They
did 10 iterations on the MNIST dataset. [11] also participated in the idash
2017 competition and also used the FV-scheme. They used in contrast to [15]
not the gradient descent but rather applied a new iterative method that uses a
simplified fixed hessian method to reduce the multiplicative depth. In the
practical training they only did one iteration with a fixed number of
covariates (=20) and needed 44min for training on 1000 records. [35] are the
winners of the task of the idash competition 2017, participating with a
implementation based on the HEAAN-scheme [17]. They used a new encoding method
which reduces the required storage space and optimizes the computational time.
They adapted the Nesterov’s accelerated gradient descent algorithm to reduce
the needed iterations and therefore increase the speed. [13] are one of the
finalist of the idash 2018 competition, where the task was to find all
features in an encrypted data base that will improve the quality of an already
trained regression model. Concretely the data analyst builds a logistic
regression model using phenotype features and updates this model afterwards
with genotype features. The training of the logistic regression in Step 1 uses
TFHE [18], the updating process is build with HEAAN [17].

#### Decision Trees

[7] propose a new privacy-preserving solution for training and prediction of
tree-based algorithms. As a central technique they developed a soft-step
function yielding in a low-degree approximation (degree) and a lightweight
interactive protocol, that is independent of the tree or data set size. They
evaluated several UCI data sets on a single decision tree using [17] with the
SEAL-library and achieved a runtime between 19 minutes (100 examples, 4
features, 3 labels) to 1220 min (10000 examples, 54 features, 7 labels)
depending of the data set size.

#### Neural Networks

[47] are the first to address the training of a nn in encrypted form using the
BGV scheme. The architecture of the nn used in this work is a 3-layer fully
connected network with sigmoid activation function that are realized with look
up tables. They need for one mini-batch containing 60 training samples about
40 min. Because they do not provide the full training costs this work is not
presented in Table 3. [42] propose Glyph, a fhe-based technique for nn. They
use TFHE for implementing activation functions such as Relu and softmax and
switching to vectorial-arithmetic-friendly BGV when processing fully-connected
and convolutional layers. They test their implementation on various data sets
for example the MNIST data set.

###  4.3 mpc based Solutions

Since mpc is an older technique than he it is already widely used in practice
and there are more paper regarding mpc. Additionally the complexity of
algorithms considered in the state-of-the-art solutions has increased. While
in he barely one paper considers nn, in mpc deep nn are the main focus of
recent research. Therefore we will mainly consider nn in this Chapter and will
only present a few paper to other machine learning algorithms. As stated in
Section 2 we will only consider papers that accomplish our requirement of a
distributed data set among the servers via secret sharing. Table 2 gives an
overview over the paper considering mpc.

|  |  linear Regression |  logistic Regression |  Decision Trees |  NN |  2PC |  3PC |  4PC |  passive |  active |  |  |
---|---|---|---|---|---|---|---|---|---|---|---|---|---
Paper | Framework | Algorithm | Parties | sec. model | Data Set | Time | Comm. Cost
$\cite[cite]{[\@@bibref{}{mohassel2017}{}{}]}^{\ast}$ | SecureML | ● | ○ | ○ | ○ | ● | ○ | ○ | ● | ○ | $10^{6}$ r, 784 f | 50 s - 100 s | -
○ | ● | ○ | ○ | 100 s - 150 s
○ | ○ | ○ | ● | 653 s - 4239.7 s
[50] | SMAC-GLORE | ○ | ● | ○ | ○ | ● | ○ | ○ | ● | ○ | 60 r, 3 f | 7290.67 s | -
○ | ● | ○ | 17862.47 s |
[16] | SPDZ | ○ | ● | ○ | ○ |  |  |  | ● | ○ | 11 | 16.45 s | -
[21]⋆ | - | ○ | ● | ○ | ○ | ○ | ◐ | ○ | ● | ○ | (8) 470 r, 17814 f | 2.52 s | -
(9) 225 r, 12634 f | 26.9 s | -
[22] | VIFF | ○ | ○ | ● | ○ | ○ | ● | ○ | ● | ○ | 3196 r | 73 s | -
[5] | - | ○ | ○ | ● | ○ | ● | ○ | ○ | ● | ○ | (9) 225 r, 12634 f | 12.6 s | -
[4] | MP-SPDZ | ○ | ○ | ● | ○ | ○ | ● | ○ | ● | ○ | 8192 r | 34 s | 3.783 GB
○ | ● | 182.1 s | 16.552 GB
[53] | SecureNN | ○ | ○ | ○ | ● | ○ | ● | ○ | ● | ○ | (10) 70000 r, 784 f | 0.88 h | 113 GB
[45] | Aby3 | ○ | ○ | ○ | ● | ○ | ● | ○ | ● | ○ | (10) 70000 r, 784 f | 0.75 h | 31.744 GB
[6] | QOUTIENT | ○ | ○ | ○ | ● | ● | ○ | ○ | ● | ○ | (10) 70000 r, 784 f | 50.25 h | -
[54] | FALCON | ○ | ○ | ○ | ● | ○ | ● | ○ | ● | ○ | (10) 70000 r, 784 f | 0.17 h | 16.384 GB
○ | ● | 0.56 h | 90.112 GB
[8] | - | ○ | ○ | ○ | ● | ○ | ● | ○ | ● | ○ | (10) 70000 r, 784 f | 117 s | -
○ | ● | 570 s
[27] | - | ○ | ○ | ○ | ● | ● | ○ | ○ | ● | ○ | (10) 70000 r, 784 f | 0.238 h | 48.128 GB
Table 2: State of the art multi-party computation based solution. The runtimes
are always in the LAN-Setting.
∗Considering only the online phase for the running time.
⋆ One party who pre-computes multiplication triplets, two parties that are
actively computing the final result.

#### Linear Regression

[46] present a new and efficient protocol for privacy-preserving machine
learning not only for linear regression but also for logistic regression and
neural networks. They were the first to present a privacy-preserving protocol
for the latter two. The setup is the two-server model where data owners
distribute their private data among two non colluding servers that train via
secure two-party computation. To improve the runtime they propose mpc friendly
alternatives to the non-linear functions (e.g. sigmoid function) and develop
new techniques to support secure arithmetic operations on shared decimal
numbers. Their protocol has an offline and a online phase. In the offline
phase they generate shared multiplication triplets and present two protocols,
one with linearly homomorphic encryption (LHE) and the other with ot.

#### Logistic Regression

[50] propose a ss , circuit-based secure mpc framework (SMAC-GLORE)
considering the client-server setup, that not only protects patient-level data
but also all intermediary information. They implement the Newton-Raphson
method for learning on the data. [16] investigates the efficiency of the SPDZ
framework [20] that provides malicious security. The compare their results to
applications implemented with semi-honest mpc techniques and show that they
easily outperform the previous implementations while providing stronger
security. [21] train a logistic regression model via a secure two-party
protocol using a trusted initializer in an offline phase. The initializer
distributes correlated randomness (multiplication triples) and can operate in
an offline phase before the data is known. The other two parties compute the
final result in an online phase using additive ss. They implement a new
protocol for the activation function, that does neither require secure
comparison protocol nor Yao’s garbled circuits. This paper is an improvement
of one of the winners in track 4 of the idash 2019 competition. Unfortunately
they do not present the communication costs. As above described [46] also
present a protocol for logistic regression.

#### Decision Trees

[22] consider the ID3 algorithm for decision tree learning via mpc using ss.
Their evaluation needs only a few seconds to minutes, depending on the used
data set. [5] address the problem of mpc unfriendly learning algorithms. They
observed that the standard algorithm for training decision trees with
continuous features (C4.5 algorithm) that is used for cleartext data requires
sorting of training examples, which is very expensive in mpc. Therefore they
propose three different, more efficient alternatives. Additionally they
propose several improvements and optimizations to important building blocks of
privacy-preserving ml protocols. Training with the alternatives needs a few
minutes and achieves accuracy that is the same with those obtained in the
clear. With the new algorithms they present the first protocols that can
handle continuous data without relying on a full sorting of the data set. In
contrast [4] too consider training of decision trees on continuous data but
stick with the sorting of the data using a modified and stripped down version
of C4.5. They achieve a protocol that can handle both discrete and continuous
data and also both semi-honest (passive) and malicious (active) adversary.
They implement their protocols using the MP-SPDZ framework [33].

#### Neural Networks

Although MNIST is not a medical data set, but rather contains handwritten
digits we will present the runtime of the considered paper with the MNIST
dataset because it is typically used as a benchmark in the context of nn.
As above described [46] also present a protocol for nn. [45] design and
implement a new framework for privacy-preserving ml in a three-server model
with a single corrupted server, called ABY3. Beside a new approximate fixed-
point multiplication protocol they also develop a new protocol for efficiently
converting between binary sharing, arithmetic sharing and Yao sharing.
Unfortunately they do not state their evaluation run time for the training of
the nn in their maliciously setting but only for the semi-honest threat model.
[53] developed new information-theoretically secure protocols for various
building blocks of neural networks and therefore achieve very efficient
protocols for training of different nn architectures. They test their system
called SecureNN on the MNIST Data Set and need for a single image inference
0.04s and 2.08MB communication. In contrast to [46] they achieve a speed up
98x and 8x for the 2 and 3-party setting. QUOTIENT[6] is a new framework for
discretized training of deep nn in the 2 party setting. They do not need an
offline phase and test their implementation on various data sets yielding in a
performance for the MNIST data set with a 3 x (128FC) nn and 5 training epochs
of 50.25 h. [54] supports a new protocol in a honest-majority 3-party setting
called FALCON. They improve the round and communication complexity and provide
security against maliciously corrupt adversaries with an hones majority.
Additionally they are the first that demonstrate efficient protocols for
batch-normalization. [8] propose ADAM a new secure and efficient protocol for
a set of elementary functions that are useful for depp nn, including secure
division, exponentiation, inversion and square root extraction. They aim to
use state of the art mpc-unfriendly algorithms without approximation using
mpc-friendly functions, for example the Adam algorithm. [27] implement a 2PC
protocol to train a nn. They consider only the passive - semi-honest security
model. They build a new preprocessing protocol for mask generation, support
and realize secret sharing comparison and aim to further reduce the
communication costs.

###  4.4 TEE based Solutions

Since neither the computational effort nor the communication costs play a big
role in tee, the task of training or evaluating a machine learning algorithm
is not necessarily a challenge. Therefore there exists only few paper in the
recent research concerning simple machine learning tasks [38], [37]. Rather
there is a lot of work to prevent solutions with tee for the standard attacks
on tee like side channel attacks [37]. Another area is the connection of tee
and he or mpc [25], [31], which we will not explore further in this context.

##  5 Conclusion

We discussed a lot of implemented solutions that are possible in our unique
use case. Although we have presented runtimes for most of the implementations,
these can only be seen as a rough guideline, as the implementations differ in
several important details. On the one hand, the computing power of the devices
used is often very different, on the other hand, several data sets are used
for bench marking, and finally, the number of iterations of the training
algorithm is not handled uniformly. Most of the ml algorithms are possible in
a distributed way, for example in the he scenario, especially those that are
trained via the gradient descent method. The logistic regression has various
advantages in the medical field. It is easy to train and already widely used
and it is easy to interpret the trained model, which is useful in the medical
context. nn on the other hand can detect more complex relations but are
costlier to train and not really interpretable. Decision Trees are easy to
understand and are therefore also often used but they are not as suitable for
our distributed context and again costlier to train as the logistic
regression. But the choice of the algorithms naturally depends on the research
question and the existing data. If we look at the cryptographic methods that
can be used in our use case we can see the standard advantages and
disadvantages. tee are efficient and also common but they bring along a trust
issue, since the data will be decrypted in the enclave. he usually is used
with one dh and one dp as outsourced computation. A big advantage of he over
mpc is that there are no communication costs. On the other hand the
computations are still slower than in mpc. If we assume a distributed training
we need to introduce additional communication costs in both scenarios, tee and
he, which takes away the advantage. In he we do not need to protect the
communication between the servers, because all data and intermediate results
are encrypted anyways, in tee there should be secure communication channels.
Then again, a disadvantage of mpc in our scenario with shared data is that we
need robust computing server. If one computer fails the computation cannot be
done in the most frameworks. In the scenario with tee or he the data is
distributed in a horizontal way (ore sometimes also an vertical way) where
failing of a server often only means a smaller data set for training.

## References

  * [1] Intel® software guard extensions.  https://software.intel.com/content/www/us/en/develop/topics/software-guard-extensions.html.
  * [2] ARM trustzone developer.  https://developer.arm.com/ip-products/security-ip/trustzone.
  * [3] MNSIT dataset.  https://deepai.org/dataset/mnist.
  * [4] Mark Abspoel, Daniel Escudero, and Nikolaj Volgushev.  Secure training of decision trees with continuous attributes.  Proc. Priv. Enhancing Technol., 2021(1):167–187, 2021.
  * [5] Samuel Adams, Chaitali Choudhary, Martine De Cock, Rafael Dowsley, David Melanson, Anderson CA Nascimento, Davis Railsback, and Jianwei Shen.  Privacy-preserving training of tree ensembles over continuous data.  arXiv preprint arXiv:2106.02769, 2021.
  * [6] Nitin Agrawal, Ali Shahin Shamsabadi, Matt J Kusner, and Adrià Gascón.  Quotient: two-party secure neural network training and prediction.  In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, pages 1231–1247, 2019.
  * [7] Adi Akavia, Max Leibovich, Yehezkel S Resheff, Roey Ron, Moni Shahar, and Margarita Vald.  Privacy-preserving decision trees training and prediction.  In ECML/PKDD (1), pages 145–161, 2020.
  * [8] Nuttapong Attrapadung, Koki Hamada, Dai Ikarashi, Ryo Kikuchi, Takahiro Matsuda, Ibuki Mishina, Hiraku Morita, and Jacob CN Schuldt.  Adam in private: Secure and fast training of deep neural networks with adaptive moment estimation.  arXiv preprint arXiv:2106.02203, 2021.
  * [9] Monir Azraoui, Muhammad Bahram, Beyza Bozdemir, Sébastien Canard, Eleonora Ciceri, Orhan Ermis, Ramy Masalha, Marco Mosconi, Melek Önen, Marie Paindavoine, Boris Rozenberg, Bastien Vialla, and Sauro Vicini.  SoK: Cryptography for Neural Networks, pages 63–81.  Springer International Publishing, 2020.
  * [10] Yael Ben-Haim and Elad Tom-Tov.  A streaming parallel decision tree algorithm.  Journal of Machine Learning Research, 11(2), 2010.
  * [11] Charlotte Bonte and Frederik Vercauteren.  Privacy-preserving logistic regression training.  BMC Medical Genomics, 11, 2018.
  * [12] Zvika Brakerski, Craig Gentry, and Vinod Vaikuntanathan.  (leveled) fully homomorphic encryption without bootstrapping.  ACM Transactions on Computation Theory (TOCT), 6(3):1–36, 2014\.
  * [13] Sergiu Carpov, Nicolas Gama, Mariya Georgieva, and Juan Ramón Troncoso-Pastoriza.  Privacy-preserving semi-parallel logistic regression training with fully homomorphic encryption.  BMC Medical Genomics, 13, 07 2020.
  * [14] Sylvain Chatel, Apostolos Pyrgelis, Juan Ramón Troncoso-Pastoriza, and Jean-Pierre Hubaux.  Sok: Privacy-preserving collaborative tree-based model learning.  Proceedings on Privacy Enhancing Technologies, 2021(3):182–203, 2021.
  * [15] Hao Chen, Ran Gilad-Bachrach, Kyoohyung Han, Zhicong Huang, Amir Jalali, Kim Laine, and Kristin Lauter.  Logistic regression over encrypted data from fully homomorphic encryption.  BMC Medical Genomics, 11(4), 2018.
  * [16] Valerie Chen, Valerio Pastro, and Mariana Raykova.  Secure computation for machine learning with spdz.  arXiv preprint arXiv:1901.00329, 2019.
  * [17] Jung Hee Cheon, Andrey Kim, Miran Kim, and Yongsoo Song.  Homomorphic encryption for arithmetic of approximate numbers.  In Tsuyoshi Takagi and Thomas Peyrin, editors, Advances in Cryptology – ASIACRYPT 2017, pages 409–437. Springer International Publishing, 2017.
  * [18] Ilaria Chillotti, Nicolas Gama, Mariya Georgieva, and Malika Izabachène.  Tfhe: Fast fully homomorphic encryption over the torus.  Journal of Cryptology, 2019.
  * [19] Mariana Cunha, Ricardo Mendes, and João P. Vilela.  A survey of privacy-preserving mechanisms for heterogeneous data types.  Computer Science Review, 41:100403, 2021.
  * [20] Ivan Damgård, Valerio Pastro, Nigel Smart, and Sarah Zakarias.  Multiparty computation from somewhat homomorphic encryption.  In Annual Cryptology Conference, pages 643–662. Springer, 2012\.
  * [21] Martine De Cock, Rafael Dowsley, Anderson CA Nascimento, Davis Railsback, Jianwei Shen, and Ariel Todoki.  High performance logistic regression for privacy-preserving genome analysis.  BMC Medical Genomics, 14(1):1–18, 2021.
  * [22] Sebastiaan De Hoogh, Berry Schoenmakers, Ping Chen, and Harm op den Akker.  Practical secure decision tree learning in a teletreatment application.  In International Conference on Financial Cryptography and Data Security, pages 179–194. Springer, 2014.
  * [23] Junfeng Fan and Frederik Vercauteren.  Somewhat practical fully homomorphic encryption.  IACR Cryptol. ePrint Arch., 2012:144, 2012.
  * [24] Junfeng Fan and Frederik Vercauteren.  Somewhat practical fully homomorphic encryption.  Cryptology ePrint Archive, Report 2012/144, 2012.  https://eprint.iacr.org/2012/144.
  * [25] Andreas Fischer, Benny Fuhry, Florian Kerschbaum, and Eric Bodden.  Computation on encrypted data using data flow authentication.  arXiv preprint arXiv:1710.00390, 2017.
  * [26] LogisticDx: Diagnostic Tests for Models with a Binomial Response.  NHANES iii data.  https://rdrr.io/rforge/LogisticDx/man/nhanes3.html.
  * [27] Zhengqiang Ge, Zhipeng Zhou, Dong Guo, and Qiang Li.  Practical two-party privacy-preserving neural network based on secret sharing.  arXiv preprint arXiv:2104.04709, 2021.
  * [28] Irene Giacomelli, Somesh Jha, Marc Joye, C David Page, and Kyonghwan Yoon.  Privacy-preserving ridge regression with only linearly-homomorphic encryption.  In International Conference on Applied Cryptography and Network Security, pages 243–261. Springer, 2018.
  * [29] F. Graf, H.-P. Kriegel, M. Schubert, S. Poelsterl, and A. Cavallaro.  Relative location of CT slices on axial axis.  UCI Machine Learning Repository, 2011.  https://archive.ics.uci.edu/ml/datasets/Relative+location+of+CT+slices+on+axial+axis#.
  * [30] Aditya Hegde, Helen Möllering, Thomas Schneider, and Hossein Yalame.  Sok: Efficient privacy-preserving clustering.  Cryptology ePrint Archive, Report 2021/809, 2021.  https://eprint.iacr.org/2021/809.
  * [31] Yichen Jiang, Jenny Hamer, Chenghong Wang, Xiaoqian Jiang, Miran Kim, Yongsoo Song, Yuhou Xia, Noman Mohammed, Md Nazmus Sadat, and Shuang Wang.  Securelr: Secure logistic regression model via a hybrid cryptographic protocol.  IEEE/ACM transactions on computational biology and bioinformatics, 16(1):113–123, 2018.
  * [32] Tehsin Kanwal, Adeel Anjum, and Abid Khan.  Privacy preservation in e-health cloud: taxonomy, privacy requirements, feasibility analysis, and opportunities.  Cluster Computing, 24:293–317, 2021.
  * [33] Marcel Keller.  MP-SPDZ: A versatile framework for multi-party computation.  Cryptology ePrint Archive, Report 2020/521, 2020.  https://eprint.iacr.org/2020/521.
  * [34] RL Kennedy, HS Fraser, LN McStay, and RF Harrison.  Early diagnosis of acute myocardial infarction using clinical and electrocardiographic data at presentation: derivation and evaluation of logistic regression models.  European heart journal, 17(8):1181–1191, 1996.
  * [35] Andrey Kim, Yongsoo Song, Miran Kim, Keewoo Lee, and Jung Hee Cheon.  Logistic regression model training based on the approximate homomorphic encryption.  BMC Medical Genomics, 11(4), 2018.
  * [36] Miran Kim, Yongsoo Song, Shuang Wang, Yuhou Xia, and Xiaoqian Jiang.  Secure logistic regression based on homomorphic encryption: Design and evaluation.  Cryptology ePrint Archive, Report 2018/074, 2018.  https://eprint.iacr.org/2018/074.
  * [37] Andrew Law, Chester Leung, Rishabh Poddar, Raluca Ada Popa, Chenyu Shi, Octavian Sima, Chaofan Yu, Xingmeng Zhang, and Wenting Zheng.  Secure collaborative training and inference for xgboost.  In Proceedings of the 2020 Workshop on Privacy-Preserving Machine Learning in Practice, pages 21–26, 2020.
  * [38] Chester Leung, Andrew Law, and Octavian Sima.  Towards privacypreserving collaborative gradient boosted decision trees.  Technical report, UC Berkeley, Tech. Rep, 2019.
  * [39] LogisticDx: Diagnostic Tests for Models with a Binomial Response.  LBW: Low birth weigth study data.  https://rdrr.io/rforge/LogisticDx/man/lbw.html.
  * [40] LogisticDx: Diagnostic Tests for Models with a Binomial Response.  PCS : Prostate cancer study data.  https://rdrr.io/rforge/LogisticDx/man/pcs.html.
  * [41] LogisticDx: Diagnostic Tests for Models with a Binomial Response.  UIS: Umaru impact study data.  https://rdrr.io/rforge/LogisticDx/man/uis.html.
  * [42] Qian Lou, Bo Feng, Geoffrey C Fox, and Lei Jiang.  Glyph: Fast and accurately training deep neural networks on encrypted data.  arXiv preprint arXiv:1911.07101, 2019.
  * [43] Qi Meng, Guolin Ke, Taifeng Wang, Wei Chen, Qiwei Ye, Zhi-Ming Ma, and Tie-Yan Liu.  A communication-efficient parallel algorithm for decision tree.  11 2016.
  * [44] Xiangrui Meng, Joseph Bradley, Burak Yavuz, Evan Sparks, Shivaram Venkataraman, Davies Liu, Jeremy Freeman, DB Tsai, Manish Amde, Sean Owen, et al.  Mllib: Machine learning in apache spark.  The Journal of Machine Learning Research, 17(1):1235–1241, 2016\.
  * [45] Payman Mohassel and Peter Rindal.  Aby3: A mixed protocol framework for machine learning.  In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, pages 35–52, 2018.
  * [46] Payman Mohassel and Yupeng Zhang.  Secureml: A system for scalable privacy-preserving machine learning.  In 2017 IEEE symposium on security and privacy (SP), pages 19–38. IEEE, 2017.
  * [47] Karthik Nandakumar, Nalini Ratha, Sharath Pankanti, and Shai Halevi.  Towards deep neural network training on encrypted data.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 0–0, 2019.
  * [48] Biswanath Panda, Joshua S Herbach, Sugato Basu, and Roberto J Bayardo.  Planet: massively parallel learning of tree ensembles with mapreduce.  2009\.
  * [49] Guowei Qiu, Xiaolin Gui, and Yingliang Zhao.  Privacy-preserving linear regression on distributed data by homomorphic encryption and data masking.  IEEE Access, 8:107601–107613, 2020.
  * [50] Haoyi Shi, Chao Jiang, Wenrui Dai, Xiaoqian Jiang, Yuzhe Tang, Lucila Ohno-Machado, and Shuang Wang.  Secure multi-party computation grid logistic regression (smac-glore).  BMC medical informatics and decision making, 16(3):175–187, 2016\.
  * [51] Si Si, Huan Zhang, S Sathiya Keerthi, Dhruv Mahajan, Inderjit S Dhillon, and Cho-Jui Hsieh.  Gradient boosted decision trees for high dimensional sparse output.  In International conference on machine learning, pages 3182–3190. PMLR, 2017.
  * [52] Lushan Song, Haoqi Wu, Wenqiang Ruan, and Weili Han.  Sok: Training machine learning models over multiple sources with privacy preservation.  CoRR, abs/2012.03386, 2020.
  * [53] Sameer Wagh, Divya Gupta, and Nishanth Chandran.  Securenn: Efficient and private neural network training.  IACR Cryptol. ePrint Arch., 2018:442, 2018.
  * [54] Sameer Wagh, Shruti Tople, Fabrice Benhamouda, Eyal Kushilevitz, Prateek Mittal, and Tal Rabin.  Falcon: Honest-majority maliciously secure framework for private deep learning.  arXiv preprint arXiv:2004.02229, 2020.
  * [55] Yang Yang, Xindi Huang, Ximeng Liu, Hongju Cheng, Jian Weng, Xiangyang Luo, and Victor Chang.  A comprehensive survey on secure outsourced computation and its applications.  IEEE Access, 7:159426–159465, 2019.

##  Appendix A Data Sets

There are a variety of data sets from fully synthetic to partially synthetic
to real data. Sometimes there is a reference data set for certain algorithms,
such as the MNIST for nn, but often the research paper use some synthetic data
to test the performance of their solutions with a set number of records and
features. Whenever they use a special and published data set, we listed it in
Table 3.

| Data Set | records | features
---|---|---|---
1 | Edinburgh Myocardial Infarction [34] | 1253 | 10
2 | Low Birth Weight Study [39] | 189 | 10
3 | Nhanes III [26] | 15,649 | 16
4 | Prostate cancer Study [40] | 379 | 10
5 |  idash 2017 | 1579 | 108
6 | Umaru Impact Study [41] | 575 | 9
7 |  idash 2018 | 245 | 3
|  |  | 10643 SNPs
8 | BC-TCGA (idash 2019-1) | 470 | 17814
9 | GSE2034 (idash 2019-2) | 225 | 12634
10 | MNIST [3] | 70000 | 28x28 pixel = 784
11 | CT slices [29] | 53 500 | 384
Table 3: Overview of data sets


# Local gauge versus non-local gauge in QFT

I.V. Anikin<EMAIL_ADDRESS>Bogoliubov Laboratory of Theoretical
Physics, JINR, 141980 Dubna, Russia

###### Abstract

In Quantum Field Theory, we discuss the main features of the non-local contour
gauge which extends the local axial-type gauge used in most approaches. Based
on the gluon geometry, we demonstrate that the contour gauge does not suffer
from the residual gauge in the finite domain of space. Matching between the
local and non-local gauge has been presented.

Contour Gauge, Path Group, Gauge symmetry.

###### pacs:

13.40.-f,12.38.Bx,12.38.Lg

##  I Introduction

Since any gauge theories can be considered as the systems with dynamical
constraints, it is difficult to overestimate the role of gauge conditions
which are nothing but the additional constraints appeared in the Hamiltonian
formalism. As well-known, the gauge theory can be consistently quantized if it
is possible to resolve uniquely all constraint conditions eliminating the
unphysical degrees of freedom. However, in the Hamiltonian (Lagrangian)
approach, the constraint conditions with respect to the generalized momenta
and coordinates have, as a rule, very nontrivial forms and, hence, finding
unique solutions of these equations is not a simple task. It sometimes becomes
even impossible due to the presence of the Gribov ambiguities.

Fortunately, it turns out that the infinite group orbit volume, which leads to
the problem with quantization, can be factorized out to the insubstantial
normalization factor thanks to the gauge invariance property of the
corresponding Lagrangian (or Hamiltonian) of a theory Faddeev:1980be . The
mentioned infinite volume that has been included in the factorized prefactor
of the functional integration is now not the principle obstruction for
quantization of the gauge theory.

Nowadays, only the local type of gauges can traditionally be called as the
most popular gauges used for the practical applications. At the same time,
some of useful local gauges might suffer from the residual gauge freedom
which, in its turn, can ultimately give rise to the spurious singularity.

In contrast to the local gauge condition, the use of non-local gauge allow us
to fix completely the gauge freedom and there is no the residual gauge freedom
at least in the finite region of space Anikin:2021oht . The contour gauge
advocated in the paper belongs to the class of non-local gauges. In
particular, within the contour gauge conception the gauge orbit representative
should be first fixed and, then, a certain local gauge condition which is
correlated with the given (gauge) orbit representative should be searched. In
some sense, the opposite logic compared to the use of usual local gauges takes
place.

In the paper, in order to clarify the mathematical features of the contour
gauge, we consider the Hamiltonian formalism as the most suitable language.
Within this frame, it is more convenient to understand how the contour gauge
condition defines the manifold surface crossing over a group orbit of the
fiber uniquely. However, the Lagrangian formalism is assumed to be better
designed for the practical applications. We combine both formalisms depending
on the current goals. We also implement a comparative analysis of the local
and non-local gauges of axial type. The useful instrument to do that is the
geometrical interpretation of gluon fields as a connection on the principle
fiber bundle. We concentrate our main efforts on the clarification of
essential features of the contour gauge use.

##  II The Lagrangian and Hamiltonian systems with the dynamical constraints

The contour gauge refers to the non-local class of axial-type gauges and
generates the standard axial gauge in Quantum Field Theory (QFT). To clarify
the main features of the contour gauge, we begin with a comparison of the
Lagrangian (L-system) and Hamiltonian (H-system) approaches to the
quantization of gauge fields (see, for example, Konopleva:1981ew ).

Before going further, we remind the basic items of the method the use of which
makes a theory with dynamical constraints quantized. Let us consider the
H-system, defined by $H(p_{i},q_{i})$, where the phase space $\Gamma$ has been
formed by the generalized momenta $p_{i}$ and coordinates $q_{i}$, and $2m$
constraints, $\left(\varphi_{a}(p_{i},q_{i});\chi_{a}(p_{i},q_{i})\right)$, on
$p_{i}$ and $q_{i}$ have been imposed. We suppose that the H-system we
consider has an equivalence orbit which is nothing but the gauge group orbit
in the gauge theory. Since the phase space $\Gamma$ is overfilled by
unphysical degrees, in the most ideal case, we have to resolve all
constraints. The additional constraints $\chi_{a}(p_{i},q_{i})$ are necessary
to fix uniquely the orbit representative in order to quantize the H-system.
After resolving all constraints, we deal with the quantized H-system where the
physical phase space $\Gamma^{*}$ of dimension $2(n-m)$ is a subspace of the
initial space $\Gamma$ of dimension $2n$ and is a fundament for H-system in
terms of physical configurations, $H^{*}(p_{i}^{*},q^{*}_{i})$.

Hence, in terms of the functional integration, the amplitude between the
initial $|q_{1}^{in},...,q^{in}_{n};t^{in}\rangle$ and the final $\langle
q_{1}^{f},...,q^{f}_{n};t^{f}|$ states takes the following form (modulo the
unimportant normalization factors) Konopleva:1981ew ; Faddeev:1980be

$\displaystyle\langle
q_{1}^{f},...,q^{f}_{n};t^{f}|q_{1}^{in},...,q^{in}_{n};t^{in}\rangle=$
$\displaystyle N\,\int{\cal D}p_{i}(t){\cal
D}q_{i}(t)\,\delta(\varphi_{a})\delta(\chi_{a})\,\text{det}\big{\\{}\varphi_{a},\chi_{a}\big{\\}}\times$
$\displaystyle\text{exp}\Big{\\{}i\int_{t_{in}}^{t_{f}}dt\big{[}p_{i}\partial_{0}q_{i}-H(p_{i},q_{i})\big{]}\Big{\\}},$
(1)

where $\\{..\,,\,..\\}$ denotes the Poisson brackets. After resolving the
constraint conditions, presented by the $\delta$-functions, the amplitude is
now given by the following functional integration

$\displaystyle\langle
q_{1}^{f},...,q^{f}_{n};t^{f}|q_{1}^{in},...,q^{in}_{n};t^{in}\rangle=N\,\int{\cal
D}p^{*}_{i}(t){\cal D}q^{*}_{i}(t)\times$ (2)
$\displaystyle\text{exp}\Big{\\{}i\int_{t_{in}}^{t_{f}}dt\big{[}p^{*}_{i}\partial_{0}q^{*}_{i}-H^{*}(p^{*}_{i},q^{*}_{i})\big{]}\Big{\\}},$

where only the physical generalized momenta and coordinates are forming the
integration measure and the Hamiltonian.

It is instructive to illustrate the difference between the H-system with
unresolved and resolved constraint conditions with the help of a trivial
mechanical example, see Fig. 1. Let us consider the homogenous ball which
moves from the point $A$ to the point $B$. The ball has a spherical symmetry
under the rotation around the inertia center. For the simplicity, we focus on
the rotation of ball in some plane. Since the ball surface is homogenous, we
have no a chance to observe the rotation of the moving ball unless we mark
some point on the surface. The invisible ball rotation around its center of
inertia corresponds, roughly speaking, to the inner (gauge) transformations of
the H-system and does not affect much the trajectory provided the angle
velocity is constant, see the left panel of Fig. 1. In this case, the inertia
center plays a role of “physical” configurations of the H-system, while the
moving of different sites on the ball surface is invisible and relates to the
“unphysical” configurations.

If we mark the site on the ball surface by a dash, we break down the rotation
symmetry in the meaning of that we choose the preferable site of the ball
surface and the ball rotation becomes visible. Therefore, in this case, we can
describe the moving dash together with the inertia center as “physical”
configurations of the H-system when the ball position varies from $A$ to $B$,
see the right panel of Fig. 1.

The Abelian and/or non-Abelian gauge theory can be treated as the H-system
where the amplitudes of the state transitions take the forms which are similar
to Eqn. (II) provided the following correspondences between the canonical
variables and the dynamical constraints (see below)

$\displaystyle\big{(}p_{i};\,q_{i}\,;t\big{)}$ $\displaystyle\Longrightarrow$
$\displaystyle\big{(}(E_{0},E_{i});\,(A_{0},A_{i})\,;x\big{)},$
$\displaystyle\delta(\varphi_{a})$ $\displaystyle\Longrightarrow$
$\displaystyle\delta\left(\partial_{i}E_{i}\right)\,\delta\left(E_{0}\right),$
$\displaystyle\delta(\chi_{a})$ $\displaystyle\Longrightarrow$
$\displaystyle\delta\left(\partial_{i}A_{i}\right)\,\delta\left(A_{0}\right),$
$\displaystyle\text{det}\big{\\{}\varphi_{a},\chi_{a}\big{\\}}$
$\displaystyle\Longrightarrow$ $\displaystyle\Phi(A).$ (3)

In the ideal case of resolved constraints, we would merely deal with the
photon (or gluon) as a massless vector field (transforming as a spinor of
$2$-rank under the Lorentz group) which is described by the Hamiltonian
without the gauge transforms.

However, in the most typical gauge theories, resolving the additional (gauge)
conditions is not a simple task (even it might be practically impossible at
all). The gauge conditions have to lead to the equation system for the gauge
function $\theta(x)$ which would have an unique solution $\theta_{0}$. To
avoid the needs of finding the unique solution $\theta_{0}$, Faddeev and Popov
have proposed the approach (FP-method) where the infinite group orbit volume
can be factorized out to the insubstantial normalization factor thanks to the
gauge invariance property of the corresponding Lagrangian (or Hamiltonian) of
theory Faddeev:1980be .

From pedagogical reasons, we are now going to dwell on $U(1)$ gauge theory.
The H-system can be readily stemmed from the Lagrangian formalism of first
order where the Lagrangian is defined as

$\displaystyle{\cal
L}_{(1)}=-\frac{1}{2}\Big{[}\partial_{\mu}A_{\nu}(x)-\partial_{\nu}A_{\mu}-\frac{1}{2}F_{\mu\nu}\Big{]}F_{\mu\nu},$
(4)

with $A_{\mu}$ and $F_{\mu\nu}$ being independent field configurations. The
Lagrangian of Eqn. (4) can be written in $3$-dimensional form as

$\displaystyle{\cal
L}_{(1)}=E_{i}\,\partial_{0}A_{i}+A_{0}\,\partial_{i}E_{i}+\frac{1}{2}E^{2}+\frac{1}{4}B_{i}\,\epsilon_{ik\ell}\partial_{k}A_{\ell}+\frac{1}{2}B^{2},$
(5)

where $E_{i}=F_{i0}$ and $A_{i}$ imply the generalized momenta and
coordinates, respectively; $B_{i}=1/2\,\epsilon_{ijk}F_{jk}$ are the implicit
variables in the integration measure, see below. From the Euler-Lagrange
equations we can get the equations which do not include $\partial_{0}$ and
give the constraint conditions written in the forms of

$\displaystyle E_{0}=0\text{ -- the primary condition},$
$\displaystyle\partial_{i}E_{i}=0\text{ -- the secondary condition}.$ (6)

As well-known, any gauge theory can be considered as the theory with the
constraints which are applied on the field configurations. Moreover, the
constraint conditions of Eqn. (II) have to be completed by the additional
(gauge) conditions, for example

$\displaystyle A_{0}=0\,\,\,\text{and}\,\,\,\partial_{i}A_{i}=0.$ (7)

From the theoretical point of view, the full set of conditions defined by
Eqns, (II) and (7) should eliminate all unphysical degrees of freedom for the
correct quantization of H- (or L-) system. We stress again that it is possible
in the most ideal case.

Using the FP method, we adhere the functional integration approach to
quantization of gauge fields and start from the functional integration written
for the L-system as Konopleva:1981ew

$\displaystyle\int{\cal D}\theta(x)\,\int{\cal
D}A_{\mu}\,\delta\left(F[A]\right)\Phi(A)\,e^{iS[A]},$ (8)

where the infinite group orbit volume given by ${\cal D}\theta(x)$ has been
factorized out in the integration measure due to the gauge invariant action
$S[A]$ and functional $\Phi(A)$. Hence, the exact magnitude of the group
volume prefactor, i.e.

$\displaystyle v_{g}\equiv\int{\cal
D}\theta(x)=\int\prod_{x}[d\theta(x)]=\big{\\{}\infty,\,0\big{\\}},$ (9)

is irrelevant because the prefactor should be cancelled by the corresponding
normalization of Green functions. The integration measure $[d\theta(x)]$ can
be defined on the group manifold as an invariant measure,

$\displaystyle\int[d\theta]f(\theta)\sim\sum_{\theta\in G}f(\theta).$ (10)

The factorized ${\cal D}\theta(x)$ can be omitted, unless it leads to
misunderstanding.

In Eqn. (9), $\prod_{x}[d\theta(x)]$ denotes the product of invariant measures
defined on the structure group $G$ of fiber over each point of the Minkowski
space. If the gauge function $\theta$ is not fixed, we have the infinite
integration over the group invariant measure, otherwise (that is, the gauge
function $\theta$ is somehow fixed) the integration is equal zero for each
point of the Minkowski space but the value of fixed $\theta_{i}$ can vary from
one point to another.

The combination $\delta\left(F[A]\right)\Phi(A)$ has been appeared due to the
insertion of the functional identity presented in the form of

$\displaystyle{\bf 1}=\Phi(A)\int{\cal
D}\theta(x)\,\delta\left(F[A^{\theta}]\right).$ (11)

Eqn. (8) can be identically rewritten as

$\displaystyle\int{\cal D}A_{\mu}{\cal
D}F_{\mu\nu}\,\delta\left(F[A]\right)\Phi(A)\,e^{iS[A,F]},$ (12)

where the functional of action $S[A,F]$ is given the Lagrangian of Eqn. (4).
Focusing on the three-dimensional forms, we obtain that the functional
integration is given by

$\displaystyle\int{\cal D}A_{i}{\cal D}A_{0}\,{\cal D}B_{i}{\cal
D}E_{i}\,\delta\left(F[A]\right)\Phi(A)\,e^{iS[A,E,B]},$ (13)

where $S[A,E,B]$ is now defined by Eqn. (5). Integrations over $B_{i}$ and
$A_{0}$ in Eqn. (13) lead to the following functional integral:

$\displaystyle\int{\cal D}A_{i}\,{\cal
D}E_{i}\,\delta\left(F[A]\right)\Phi(A)\delta\left(\partial_{i}E_{i}\right)\times$
$\displaystyle\text{exp}\Big{\\{}i\int
dz\Big{[}E_{i}\,\partial_{0}A_{i}+H(E_{i},A_{i})\Big{]}\Big{\\}},$ (14)

where the Hamiltonian is given by

$\displaystyle
H(E_{i},A_{i})=\frac{1}{2}E^{2}-\frac{1}{2}\left(\epsilon_{ijk}\partial_{j}A_{k}\right)^{2}$
(15)

and the gauge condition can be written as $F(A)=\partial_{i}A_{i}$. On the
other hand, Eqn. (II) can be presented in the equivalent form as

$\displaystyle\int{\cal D}A_{i}{\cal D}A_{0}\,{\cal D}E_{i}{\cal
D}E_{0}\,\delta\left(\partial_{i}A_{i}\right)\Phi(A)\delta\left(\partial_{i}E_{i}\right)\delta\left(A_{0}\right)\delta\left(E_{0}\right)\times$
$\displaystyle\text{exp}\Big{\\{}i\int
dz\Big{[}E_{i}\,\partial_{0}A_{i}+E_{0}\partial
A_{0}+H(E_{i},A_{i};E_{0},A_{0})\Big{]}\Big{\\}},$ (16)

where the primary and secondary constraints together with the gauge conditions
of Eqn. (7) have been explicitly shown in the functional integrand. This
representation of H-system resembles the functional integration presented by
Eqn. (II).

In conclusion of this section, we emphasize that within the FP-method the
additional (gauge) conditions have been left unresolved with respect to the
gauge function $\theta(x)$. At the same time, thanks for the gauge invariance
of Lagrangian (Hamiltonian) and functional $\Phi(A)$, the infinite group orbit
volume defined by $[d\theta]$ has been factorized out in the corresponding
functional integration measure giving a possibility to quantize the gauge
theory. On the other hand, if we resolve the gauge condition with respect to
$\theta$, the group representative on each of orbit can be fixed uniquely and
the factorized group integral defined through the Riemann summations is equal
to zero. However, it does not mean that the functional integration of Eqn. (8)
disappears. Indeed, as above mentioned, the denominator of Green function
normalization includes the same zero prefactor and cancels this zero
prefactor.

Figure 1: The H-system with unresolved and resolved constrains. The left panel
represents the system with the inner (gauge) symmetry; the right panel
corresponds to the system with the fixed (preferable) site $\xi$ on the ball
surface.

##  III The use of contour gauge

In this section, we concentrate on the description of the contour gauge
conception which implies that the corresponding gauge condition can be
formally resolved in order to find the unique gauge orbit representative. A
few decades ago, the contour gauge was a subject of intense studies and had
been attracted much attentions due to the fact that the quantum gauge theory
should not suffer from the Gribov ambiguities (see, for example, ContourG1 ;
ContourG2 ).

It states that the gauge function can be completely fix (in the H-system, see
Eqn. (II)) or the unphysical gluons can be eliminated (in the L-system, see
Eqn. (8)) if we demand the path dependent functional (Wilson path functional)
to be equal to unity, i.e.

$\displaystyle{\bf
g}(x|A)\equiv\mathbb{P}\text{exp}\Big{\\{}ig\int_{P(x_{0},x)}d\omega_{\mu}A_{\mu}(\omega)\Big{\\}}={\bf
1}$ (17)

where the path $P(x_{0},x)$ between the points $x_{0}$ and $x$ is fixed. The
axial (light-cone) gauge, $A^{+}=0$, is a particular case of the non-local
contour gauge determined by Eqn. (17) provided the fixed path is given by the
straightforward line connecting $\pm\infty$ with $x$. By construction, the
contour gauge does not possess the residual gauge freedom in the finite space
and gives, from the technical point of view, the simplest way to fix totally
the gauge function.

The contour gauge conception is closely related to the path group formalism
Mensky:2012iy  and can be traced from the Mandelstam approach
Mandelstam:1962mi . The geometrical interpretation of gluon fields as a
connection on the principle fiber bundle ${\cal P}({\cal X},\pi\,|\,G)$ is the
underlying basis for the use of non-local gauges. In this connection, let us
define two directions: one direction is determined in the base ${\cal X}$ and
is the tangent vector of a curve going through the point $x\in{\cal X}$; the
other direction is defined in the fiber and can be uniquely determined as the
tangent subspace related to the parallel transport Mensky:2012iy . These two
directions allow to introduce the horizontal vector defined as

$\displaystyle H_{\mu}=\frac{\partial}{\partial
x^{\mu}}-igA^{a}_{\mu}(x)\,D^{a},\quad D^{a}={\bf
g}\frac{\partial}{\partial{\bf g}},$ (18)

where $D^{a}$ denotes the corresponding shift generator along the group fiber
written in the differential form and, together with the vector coefficients
(connection of the principle fiber bundle) $A^{a}_{\mu}(x)$, defines the
algebraic vertical (tangent) vector field on the fiber Mensky:2012iy . The
horizontal vector $H_{\mu}$ is invariant under the structure group $G$ acting
on the given representation of the fiber by construction.

In ${\cal P}({\cal X},\pi\,|\,G)$, the functional ${\bf g}(x|A)$ of Eqn. (17)
is a solution of the parallel transport equation given by

$\displaystyle\frac{dx_{\mu}(s)}{ds}H_{\mu}(A){\bf g}(x(s)|A)=0,$ (19)

where the fiber point $p(s)=\left(x(s),{\bf g}(x(s))\right)$ with the curve
$x(s)\in{\cal X}$ that has been parametrized by $s$. Eqn. (19) being an
algebraic differential equation takes place even if ${\bf g}$ is fixed on the
group because, in this case, $D^{a}{\bf g}=0$ while $A^{a}_{\mu}(x)\not=0$. In
this connection, the condition presented by Eqn. (17) implies that the full
curve-linear integration goes to zero rather then the integrand itself.

It can be shown Mensky:2012iy  that every of points belonging to the fiber
bundle, ${\cal P}({\cal X},\pi\,|\,G)$, has one and only one horizontal vector
corresponding to the given tangent vector at $x\in{\cal X}$. We remind that
the tangent vector at the point $x$ is uniquely determined by the given path
passing through $x$. In the frame of H-system based on the geometry of gluons
the condition of Eqn. (17) corresponds to the determining of the surface on
${\cal P}({\cal X},\pi\,|\,G)$ that is parallel to the base plane with the
path. Moreover, it singles out the identity element, ${\bf g}=1$, in every
fibers of ${\cal P}({\cal X},\pi\,|\,G)$, see Fig. 2.

Notice that since the gauge condition given by Eqn. (17) selects on the each
fiber only the identity element of group $G$, it means that the gauge
transforms have been reduced to the “global” gauge transforms. That is, the
gauge function $\theta(x)$ becomes the coordinate independent and is fixed
$\theta_{0}$.

Since the functional ${\bf g}(x|A)$ depends on the whole path in ${\cal X}$,
the contour gauge refers to the non-local class of gauges and generalises
naturally the familiar local axial-type of gauges. It is also worth to notice
that two different contour gauges can correspond to the same local (axial)
gauge with the different fixing of residual gauges Anikin:2010wz ;
Anikin:2015xka . If we would consider the local gauge without the connection
with the non-local gauge, the residual gauge freedom would require the extra
conditions to fix the given freedom. This statement reflects the fact that, in
contrast to the local axial gauge, the contour gauge does not possess the
residual gauge freedom in the finite region of a space Anikin:2021oht .

Figure 2: The contour gauge: the plane of ${\bf g}=1$ in the principle fiber
bundle ${\cal P}({\cal X},\pi|G)$.

##  IV Local and non-local gauge matching

Since the contour gauge as a non-local kind of gauges generalizes the standard
local gauge of axial type, it is worth to discuss shortly the correspondence
between the local and non-local gauge transforms.

Let us focus on the axial local gauge defined by $A^{+}(x)=0$. The non-local
analog of this gauge is given by Eqn. (17) provided the starting point is
$x_{0}=-\infty$. As mentioned, the local gauge suffers from the residual gauge
transformations (which can lead to the spurious pole uncertainties) whilst the
non-local contour gauge fixes all the gauge freedom in the finite space (see
for instance Anikin:2021oht ). Indeed, we should consider the local axial
gauge as an equation on the gauge function $\theta(x)$, i.e.

$\displaystyle
A^{+\,\theta}(x)=\omega(x)A^{+}\omega^{-1}(x)+\frac{i}{g}\omega(x)\partial^{+}\omega^{-1}(x)=0,$
(20)

where $x=(x^{-},\tilde{x})$ with $\tilde{x}=(x^{+},{\bf x}_{\perp})$. We can
readily find a solution of this equation which takes the form of the
undetermined integration as

$\displaystyle\omega_{0}(x)=C\,\mathbb{P}\text{exp}\Big{\\{}-ig\int
dx^{-}A^{+}(x)\Big{\\}},$ (21)

or the form of determined integration as

$\displaystyle\omega_{0}(x^{-},\tilde{x})=C(\tilde{x})\,\overline{\omega}(x^{-},\tilde{x}),$
$\displaystyle\overline{\omega}(x^{-},\tilde{x})=\mathbb{P}\text{exp}\Big{\\{}-ig\int_{x_{0}^{-}}^{x^{-}}dz^{-}A^{+}(z^{-},\tilde{x})\Big{\\}}.$
(22)

Here, $x^{-}_{0}$ is fixed and $C(\tilde{x})$ is an arbitrary function which
does not depend on $x^{-}$ and is given by

$\displaystyle\omega_{0}(x^{-}_{0},\tilde{x})=C(\tilde{x}).$ (23)

The arbitrariness of $C$-function also reflects the fact that we here deal
with an arbitrary fixed starting point $x_{0}$. We then come to the study of
the residual gauge freedom requiring both $A^{+,\,\theta}(x)=0$ and
$A^{+}(x)=0$, we have

$\displaystyle\omega_{0}(x^{-},\tilde{x})\Big{|}_{\overline{\omega}=1}\equiv\omega^{\text{res}}(\tilde{x})=C(\tilde{x})\equiv
e^{i\,\tilde{\theta}(\tilde{x})}.$ (24)

One can see that the function $C$ determines the residual gauge transforms.

The non-local contour gauge extends the local axial-type gauge and demands
that the full (curve-linear) integral in the exponential of Eqn. (17) has to
go to zero (in contrast to the local gauge where the corresponding exponential
disappears thanks to that the integrand $A^{+}$ goes to zero).

Moreover, in the contour gauge the residual gauge function
$\tilde{\theta}(\tilde{x})$ can be related to the configurations $A^{-}$ and
$A^{i}_{\perp}$ which are also disappeared eliminating the whole gauge freedom
(see Anikin:2016bor ; Belitsky:2002sm  for details). That is, if we restore
the full path in the Wilson line for a given process, we can derive that

$\displaystyle C(\tilde{x})=\tilde{C}(x^{+}_{0},x^{-}_{0},{\bf
x}^{\perp}_{0})$ (25)
$\displaystyle\times\mathbb{P}\text{exp}\Big{\\{}ig\int_{{\bf
x}_{0}^{\perp}}^{{\bf
x}^{\perp}}d\omega_{\perp}^{i}A^{i}_{\perp}(x^{+}_{0},x^{-}_{0},\omega_{\perp})\Big{\\}}$
$\displaystyle\times\mathbb{P}\text{exp}\Big{\\{}-ig\int_{x_{0}^{+}}^{x^{+}}d\omega^{+}A^{-}(\omega^{+},x^{-}_{0},{\bf
x}_{\perp})\Big{\\}}$

and, then, making use of the corresponding contour gauge we get that

$\displaystyle
C(\tilde{x})\Big{|}_{\text{c.g.}}=\tilde{C}(x^{+}_{0},x^{-}_{0},{\bf
x}^{\perp}_{0}).$ (26)

Eqn. (26) means that there is no the gauge freedom at all. We emphasize that
the exact value of the fixed starting point $x_{0}$ depends on the process
under our consideration Belitsky:2002sm .

However, working with L-system, it is not always easy to restore all the
Wilson lines by the corresponding summation of gluon (photon) radiations. As a
rule, it is not even necessary to do that. Indeed, considering the deeply
virtual Compton scattering (DVCS) amplitude in the frame of the factorization
procedure, we obtain that the standard diagram contributions give the gauge
invariant quark string operator which reads

$\displaystyle\bar{\psi}(0^{+},0^{-},{\bf 0}_{\perp})[0^{+},0^{-},{\bf
0}_{\perp};\,0^{+},+\infty^{-},{\bf 0}_{\perp}]_{A^{+}}\Gamma\times$
$\displaystyle[0^{+},+\infty^{-},{\bf 0}_{\perp};\,0^{+},z^{-},{\bf
0}_{\perp}]_{A^{+}}\psi(0^{+},z^{-},{\bf 0}_{\perp}),$ (27)

where following the traditional notations we introduce

$\displaystyle[z_{2};\,z_{1}]_{A}={\bf
g}(z_{2}|A)\Big{|}_{P(z_{1},z_{2})}={\bf g}(P(z_{1},z_{2})|A),$ (28)

and $\Gamma$ stands for $\gamma$-matrices the exact form of which is now
irrelevant.

Here and in what follows the gluons radiated from the internal quark of the
hard subprocess generate the standard diagrams, while the non-standard
diagrams are formed by the gluon radiations from the external quark of the
hard subprocess (see Anikin:2020ipg  for details).

The non-standard diagram contributions result in the string operator defined
as

$\displaystyle\bar{\psi}(0^{+},0^{-},{\bf 0}_{\perp})[-\infty^{+},0^{-},{\bf
0}_{\perp};\,0^{+},0^{-},{\bf 0}_{\perp}]_{A^{-}}\Gamma\times$
$\displaystyle[0^{+},z^{-},{\bf 0}_{\perp};\,-\infty^{+},z^{-},{\bf
0}_{\perp}]_{A^{-}}\psi(0^{+},z^{-},{\bf 0}_{\perp}).$ (29)

Hence, using the contour gauge conception, we eliminate all the Wilson lines
with the longitudinal gluon fields $A^{+}$ and $A^{-}$ demanding that

$\displaystyle[0^{+},0^{-},{\bf 0}_{\perp};\,0^{+},+\infty^{-},{\bf
0}_{\perp}]_{A^{+}}={\bf 1},$ $\displaystyle[0^{+},+\infty^{-},{\bf
0}_{\perp};\,0^{+},z^{-},{\bf 0}_{\perp}]_{A^{+}}={\bf 1}$ (30)

and

$\displaystyle[-\infty^{+},0^{-},{\bf 0}_{\perp};\,0^{+},0^{-},{\bf
0}_{\perp}]_{A^{-}}={\bf 1},$ $\displaystyle[0^{+},z^{-},{\bf
0}_{\perp};\,-\infty^{+},z^{-},{\bf 0}_{\perp}]_{A^{-}}={\bf 1}.$ (31)

Eqns. (IV) and (IV) give rise to the local gauge conditions given by $A^{+}=0$
and $A^{-}=0$.

With respect to the Wilson line with the transverse gluons $A^{i}_{\perp}$, we
remind that we work here not only with L-system but also with the
factorization procedure applied for DVCS-amplitude. In this case, the Wilson
lines with the transverse gluon fields are considered in the form of an
expansion due to the fixed twist-order, and the transverse gluons correspond
to the physical configurations of L-system.

##  V The practical use of contour gauge

In the preceding sections, the different mathematical aspects of the contour
gauge have been presented in the general, even formal, manner. In this
section, we dwell on the certain examples which relate to the practical use of
the contour gauge. The Drell-Yan process with the polarized hadrons gives the
unique example where the contour gauge use manifests from the very practical
point of view. In particular, the contour gauge use allows to find the new
contributions to the Drell-Yan-like hadron tensor which restore and ensure the
gauge invariance of the hadron tensors Anikin:2010wz ; Anikin:2015xka ;
Anikin:2016bor . Besides, in the similar manner, the $\xi$-process of DVCS-
amplitude which clarifies the gauge invariance of the non-forward processes
takes the closed form again due to the contour gauge conception Anikin:2020ipg
.

To begin with, we consider the Drell-Yan (DY) process with one transversely
polarized hadron Anikin:2010wz

$\displaystyle
N^{(\uparrow\downarrow)}(p_{1})+N(p_{2})\to\gamma^{*}(q)+X(P_{X})\to$
$\displaystyle\ell(l_{1})+\bar{\ell}(l_{2})+X(P_{X})\,,$ (32)

where the virtual photon producing the lepton pair ($l_{1}+l_{2}=q$) has a
large mass squared ($q^{2}=Q^{2}$) while the transverse momenta are small and
integrated out. As usual, the dominant light-cone directions for the DY
process are defined as ($x_{B}$ and $y_{B}$ denote the Bjorken fractions)

$\displaystyle p_{1}\approx\frac{n^{*}\,Q}{x_{B}\sqrt{2}},\quad
p_{2}\approx\frac{n\,Q}{y_{B}\sqrt{2}},$ $\displaystyle
n^{*}_{\mu}=(n^{*\,+},\,0^{-},\,{\bf 0}_{\perp}),\quad
n_{\mu}=(0^{+},\,n^{-},\,{\bf 0}_{\perp}).$ (33)

In the process, see Eqn. (V), under our consideration, the invariant $Q^{2}$
is a very large. This asymptotical regime gives a possibility to apply the
factorization theorem. As a result, the DY hadron tensor takes a form of
convolution as

$\displaystyle\text{Hadron tensor}=$ $\displaystyle\\{\text{Hard part
(pQCD)}\\}\otimes\\{\text{Soft part (npQCD)}\\}\,,$ (34)

where both the hard and soft parts should be independent of each other and are
in agreement with the ultraviolet and infrared renormalizations. Moreover, the
relevant single spin asymmetries (SSAs), which is a subject of experimental
studies, can be presented as

$\displaystyle{\cal A}\sim
d\sigma^{(\uparrow)}-d\sigma^{(\downarrow)}\sim{\cal
L}_{\mu\nu}\,H_{\mu\nu}\,,$ (35)

where ${\cal L}_{\mu\nu}$ and $H_{\mu\nu}$ are the lepton and hadron tensors,
respectively. The hadron tensor $H_{\mu\nu}$ includes the the polarized hadron
matrix element which takes a form of

$\displaystyle\langle
p_{1},S^{\perp}|\bar{\psi}(\lambda_{1}\tilde{n})\,\gamma^{+}\,gA^{\alpha}_{\perp}(\lambda_{2}\tilde{n})\,\psi(0)|p_{1},S^{\perp}\rangle\stackrel{{\scriptstyle{\cal
F}}}{{=}}$ $\displaystyle
i\varepsilon^{\alpha+S^{\perp}-}(p_{1}p_{2})\,B(x_{1},x_{2}),$ (36)

where $\stackrel{{\scriptstyle{\cal F}}}{{=}}$ stands for the Fourier
transform between the coordinate space, formed by positions
$\lambda_{i}\tilde{n}$, and the momentum space, realized by fractions $x_{i}$;
the light-cone vector $\tilde{n}$ is a dimensionful analog of $n$. In Eqn.
(V), the parametrizing function $B$ describes the corresponding parton
distribution.

###  V.1 The case of local gauge

In the studies, see for example Boer:2003cm ; BQ ; Teryaev ; Boer , where the
local light-cone gauge $A^{+}=0$ has been used, $B$-function is given by a
purely real function and is defined as

$\displaystyle B(x_{1},x_{2})=\frac{{\cal P}}{x_{1}-x_{2}}T(x_{1},x_{2})\,$
(37)

where the function $T(x_{1},x_{2})\in\Re\text{e}$ parametrizes the
corresponding projection of $\langle\bar{\psi}\,G_{\alpha\beta}\,\psi\rangle$
and obeys $T(x,x)\not=0$.

In Eqn. (37), the pole at $x_{1}=x_{2}$ is treated as a principle value and it
leads to $B(x_{1},x_{2})\in\Re\text{e}$. Indeed, within the local gauge
$A^{+}=0$, the statement on that $B$ is the real function stems actually from
the ambiguity in the solutions of the following differential equation (which
is equivalent to the definition of $G_{\mu\nu}$)

$\displaystyle\partial^{+}\,A^{\alpha}_{\perp}=G^{+\,\alpha}_{\perp}\,.$ (38)

The formal resolving of Eqn. (38) leads to two representations written as

$\displaystyle
A^{\mu}_{(1)}(z)=\int_{-\infty}^{z}d\omega^{-}G^{+\mu}(\omega^{-})+A^{\mu}(-\infty),$
(39) $\displaystyle
A^{\mu}_{(2)}(z)=-\int_{z}^{\infty}d\omega^{-}G^{+\mu}(\omega^{-})+A^{\mu}(\infty).$
(40)

We stress that within the approaches backed on the local axial gauge use,
there are no evidences to think that Eqns. (39) and (40) are not equivalent
each other. That is, the local gauge $A^{+}=0$ inevitably leads to (see
Anikin:2015xka  for details)

$\displaystyle\Big{\\{}A^{\mu}_{(1)}\stackrel{{\scriptstyle
loc.g.}}{{=}}A^{\mu}_{(2)}\Big{\\}}\Rightarrow\Big{\\{}B(x_{1},x_{2})={\cal
P}\frac{T(x_{1},x_{2})}{x_{1}-x_{2}}\in\Re\text{e}\Big{\\}}.$ (41)

This equation demonstrates that the equivalence of Eqns. (39) and (40) causes
the representation of $B$-function as in Eqn. (37). The discussion on the
boundary configurations can be found in Anikin:2015xka .

Regarding the DY process, the physical consequences of the use of $B$
presented by Eqn. (37) are the problem with the photon gauge invariance of DY-
like hadron tensors and the losing of significant contributions to SSAs
Anikin:2010wz ; Anikin:2015xka . Besides, based on the local gauge $A^{+}=0$,
the representation of gluon field as a linear combination of Eqns. (39) and
(40) has been used in the different studies, see Belitsky:2005qn ;
Hatta:2011zs ; Lorce:2013gxa .

###  V.2 The case of non-local gauge

In contrast to the local gauge $A^{+}=0$, the path dependent non-local gauge
(see Eqn. (17)) fixes unambiguously the representation of gluon field which is
given either by Eqn. (39) or by Eqn. (40). Indeed, fixing the path
$P(x_{0},x)$ results in a solution of Eqn. (17) presented in the form of (see
Anikin:2010wz ; Anikin:2015xka ; Durand:1979sw  for details)

$\displaystyle A^{c.g.}_{\mu}(x)=\int_{P(x_{0},x)}dz_{\alpha}\frac{\partial
z_{\beta}}{\partial x_{\mu}}\,G_{\alpha\beta}(z|A),$ (42)

where the boundary configuration $A^{c.g.}_{\mu}(x_{0})$ has been neglected.
In other words, we can show that the non-local gauge ${\bf
g}(P(-\infty,x)|A)=1$ leads to the gluon filed representation of Eqn. (39),
while the non-local gauge ${\bf g}(P(x,+\infty)|A)=1$ corresponds to the gluon
field representation of Eqn. (40). Moreover, we can readily check that
Anikin:2010wz ; Anikin:2015xka

$\displaystyle\Big{\\{}\text{Eq.~{}(\ref{Ag1})}\Rightarrow
B_{+}(x_{1},x_{2})\Big{\\}}\not=\Big{\\{}B_{-}(x_{1},x_{2})\Leftarrow\text{Eq.~{}(\ref{Ag2})}\Big{\\}},$
(43)

where

$\displaystyle
B_{+}(x_{1},x_{2})=\frac{T(x_{1},x_{2})}{x_{1}-x_{2}+i\epsilon},\quad
B_{-}(x_{1},x_{2})=\frac{T(x_{1},x_{2})}{x_{1}-x_{2}-i\epsilon}.$ (44)

Notice that both the non-local gauges, i.e. ${\bf g}(P(-\infty,x)|A)=1$ and
${\bf g}(P(x,+\infty)|A)=1$, can be projected into the same local gauge
$A^{+}=0$. However, staying on the projection given by $A^{+}=0$ does not give
a possibility to understand which of the non-local gauges generates the local
gauge. This situation resembles a very trivial mathematical statement that if
two (different) vectors have the same projection on the certain direction, it
does not mean that two vectors are equal to each other.

In the studies of DY-like processes, the corresponding non-local gauge gives
rise to the correct representation of $B$-functions, see Eqn. (44), with the
non-zero imaginary part. This enables to find the new significant
contributions to the hadron tensors which ensure ultimately the gauge
invariance Anikin:2010wz ; Anikin:2015xka . In a similar manner, with the help
of Eqn. (44) we can fix the prescriptions for the spurious singularities in
the gluon propagators Anikin:2016bor .

##  VI Conclusions

In the paper, we have presented the comprehensive analysis to show how the
local and non-local gauges match each other. The combination of the Hamilton
and Lagrangian approaches to the gauge theory has been exploited in our
consideration. The Hamiltonian formalism is supposed to be convenient in
connection with the gluon geometry, while the Lagrangian formalism is very
well designed for the practical uses. The presented analysis should help to
clarify the main advantages of the use of non-local contour gauges. Since the
contour gauge is mainly backed on the geometrical interpretation of gluon
fields as a connection on the principle fiber bundle, we have provided the
illustrative demonstration of geometry of the contour gauge. For the practical
application, we have shown that the local axial-type gauge can lead to the
certain ambiguities in the gluon field representation. These ambiguities may
cause the wrong final results. Meanwhile, as demonstrated in the paper, the
non-local contour gauge can fix this kind of problems and, for example, can
provide not only the correct (gauge invariant) final result but also find the
new contributions to the hadron tensors of DY-like processes Anikin:2010wz ;
Anikin:2015xka .

## Acknowledgements

We thank Y. Hatta, C. Lorce, A.V. Pimikov, L. Szymanowski and A.S. Zhevlakov
for useful discussions.

## References

  * (1) L. D. Faddeev and A. A. Slavnov, Front. Phys. 50, 1-232 (1980)
  * (2) I. V. Anikin and A. S. Zhevlakov, [arXiv:2105.02719 [hep-ph]].
  * (3) N. P. Konopleva and V. N. Popov, “Gauge Fields,” Translated from the Second Russian Edition and edited by N.M. Queen, Harwood Academic Publishers, (1981)
  * (4) S. V. Ivanov, G. P. Korchemsky and A. V. Radyushkin, Yad. Fiz. 44, 230 (1986) [Sov. J. Nucl. Phys. 44, 145 (1986)]
  * (5) S. V. Ivanov and G. P. Korchemsky, Phys. Lett. B 154, 197 (1985)
  * (6) M. B. Mensky, Theor. Math. Phys. 173, 1668 (2012)
  * (7) S. Mandelstam, Annals Phys. 19, 1 (1962)
  * (8) A. V. Belitsky, X. Ji and F. Yuan, Nucl. Phys. B 656, 165 (2003)
  * (9) I. V. Anikin and O. V. Teryaev, Phys. Lett. B 690, 519 (2010)
  * (10) I. V. Anikin and O. V. Teryaev, Eur. Phys. J. C 75, no. 5, 184 (2015)
  * (11) I. V. Anikin, I. O. Cherednikov and O. V. Teryaev, Phys. Rev. D 95, no. 3, 034032 (2017)
  * (12) I. V. Anikin, Symmetry 12, no.12, 1996 (2020)
  * (13) D. Boer, P. J. Mulders and F. Pijlman, Nucl. Phys. B 667, 201 (2003)
  * (14) D. Boer and J. w. Qiu, Phys. Rev. D 65, 034008 (2002)
  * (15) N. Hammon, O. Teryaev and A. Schafer, Phys. Lett. B 390, 409 (1997)
  * (16) D. Boer, P. J. Mulders and O. V. Teryaev, Phys. Rev. D 57, 3057 (1998)
  * (17) A. V. Belitsky and A. V. Radyushkin, Phys. Rept. 418, 1-387 (2005)
  * (18) Y. Hatta, Phys. Rev. D 84, 041701 (2011)
  * (19) C. Lorcé, Phys. Rev. D 88, 044037 (2013)
  * (20) L. Durand and E. Mendel, Phys. Lett. 85B, 241 (1979).