11affiliationtext: School of Informatics, Aristotle University of
Thessaloniki, Greece22affiliationtext: Khoury College of Computer Sciences,
Northeastern University, Boston, US affiliationtext:
e-mails:11affiliationtext:
<EMAIL_ADDRESS><EMAIL_ADDRESS>
# On Neural Network Equivalence Checking using SMT Solvers††thanks:
Acknowledgment: This work has received funding from the European Union’s
Horizon 2020 research and innovation programme under grant agreement No
956123.

Charis Eleftheriadis  Nikolaos Kekatos  Panagiotis Katsaros  Stavros Tripakis

###### Abstract

Two pretrained neural networks are deemed equivalent if they yield similar
outputs for the same inputs. Equivalence checking of neural networks is of
great importance, due to its utility in replacing learning-enabled components
with equivalent ones, when there is need to fulfill additional requirements or
to address security threats, as is the case for example when using knowledge
distillation, adversarial training etc. SMT solvers can potentially provide
solutions to the problem of neural network equivalence checking that will be
sound and complete, but as it is expected any such solution is associated with
significant limitations with respect to the size of neural networks to be
checked. This work presents a first SMT-based encoding of the equivalence
checking problem, explores its utility and limitations and proposes avenues
for future research and improvements towards more scalable and practically
applicable solutions. We present experimental results that shed light to the
aforementioned issues, for diverse types of neural network models (classifiers
and regression networks) and equivalence criteria, towards a general and
application-independent equivalence checking approach.

##  1 Introduction

For two pretrained neural networks of the same or different architectures, the
problem of equivalence checking concerns with checking whether the networks
yield similar outputs (fit for purpose) for the same inputs. The exact
definition of the problem can refer to various “equivalence” criteria,
depending on the specific neural network application (e.g. classifier,
regression, etc.), while the motivation behind the quest for a solution is
fundamental, for a series of recent developments in machine learning
technology.

More specifically, we refer to knowledge distillation
[Hinton2015DistillingTK], i.e. the process of transferring knowledge from a
large neural network to a smaller one that may be appropriate for deployment
on a device with limited computational resources. Another area of interest
includes the techniques, widely known under the term regularization
[Kukacka2017RegularizationFD], which aim to lower the complexity of neural
networks, in order to show better performance during inference time, when they
process data that are not in the training data set (avoid data overfitting).
Moreover, neural network models in systems or programs with learning-enabled
components [Christakis2021] may have to be updated for a number of reasons
[Paulsen_2020]; for example, security concerns such as the need to withstand
data perturbations (e.g. adversarial examples), or possibly incomplete
coverage of the neural network’s input domain.

In all aforementioned cases, we usually require a different neural network
model than the original one, which is expected to comply with respect to some
given criterion of “equivalence”, depending on the specific neural network
application.

This work presents the experience from our attempt to address the
aforementioned problem based on the use of Satisfiability Modulo Theory (SMT)
solvers, which provide certain advantages, as well as limitations that justify
the need for further research efforts. Among their advantages, we stress their
potential to deliver sound and complete verification procedures for the
equivalence between two neural networks. Regarding the limitations, we focus
on their inability to scale towards solving the problem for real-size neural
networks like the ones referred in well-known benchmarks, such as the neural
network architectures for the MNIST dataset. We study the scalability bounds
of our initial SMT-based encoding for our “equivalence” criteria, with respect
to the neural network model parameters and the number of derived SMT
variables.

More concretely, this article introduces:

  * •

the problem definition of equivalence checking based on various criteria that
may be appropriate, for different neural network applications

  * •

an approach to reduce the equivalence checking problem to a logical
satisfiability problem based on our SMT-based encoding

  * •

experimental results including (i) sanity checks of our SMT-based encoding, as
well as (ii) equivalence checks for three diverse neural network applications
covering the cases of classifiers and regression models

Section 2 lays the background of our work by providing a formal definition of
neural network models. In Section 3, we propose diverse “equivalence” criteria
for the wide range of common neural network applications and we formally
define the problem of equivalence checking. Section 4 presents our SMT-based
encoding for reducing the problem of equivalence checking to a logical
satisfiability problem. Section 5 includes the experimental results and their
interpretation. In Section LABEL:sec:6 we review the related work and the
paper concludes with a summary on our contributions and the future research
prospects.

##  2 Preliminaries: Neural Networks

###  2.1 Notation

The set of real numbers is denoted by $\mathbb{R}$. The set of natural numbers
is denoted by $\mathbb{N}$. Given some $x\in\mathbb{R}^{n}$ and some
$i\in\\{1,...,n\\}$, $x(i)$ denotes the $i$-th element of $x$.

###  2.2 Neural Networks

In general, a neural network (NN) can be defined as a function:

$f:I\to O$ (1)

where $I\subseteq\mathbb{R}^{n}$ is some input domain with $n$ _features_ and
$O\subseteq\mathbb{R}^{m}$ is some output domain.

For a neural network image classifier, we typically have
$I=[0,255]^{n}\subseteq\mathbb{N}^{n}$ and a labeling function
$L:\mathbb{R}^{m}\to\mathbb{N}$ that maps each $y\in O$ to some label
$l\in\mathbb{N}$. For neural networks solving regression problems, we will
have $I\subseteq\mathbb{R}^{n}$ and no labelling function.

The above definition of neural networks is purely semantic. Concretely, a
neural network is structured into layers of _nodes_ (neurons), which may
include one _hidden layer_ ($H$) or more, beyond the layers of input ($I$) and
output nodes ($O$). Nodes denote a combination of affine value transformation
with a piecewise linear or non-linear _activation function_. Value
transformations are _weighted_ based on how nodes of different layers are
connected with each other, whereas an extra term called _bias_ is added per
node. Weights ($W$) and biases ($b$) for all nodes of a neural network are the
network’s parameters and their values are determined via training.

Since every layer is multidimensional we use vectors and/or matrices to
represent all involved operations. Let $\mathbf{x}\in\mathbb{R}^{1\times n}$
be the matrix denoting some $x\in I$. For a hidden layer with $r$ nodes,
$\mathbf{H}^{1\times r}$ represents the output of the hidden layer. Assuming
that the hidden and output layers are fully connected, we denote with
$\mathbf{W}^{{(1)}}\in\mathbb{R}^{n\times r}$ the hidden layer weights and
with $\mathbf{b}^{{(1)}}\in\mathbb{R}^{1\times r}$ the biases associated with
its nodes. Similarly, the output layer weights are denoted by
$\mathbf{W}^{{(2)}}\in\mathbb{R}^{r\times m}$, where $m$ refers to the number
of output layer nodes, and $\mathbf{b}^{{(2)}}\in\mathbb{R}^{1\times m}$
denotes the corresponding biases. Then, the output $y=f(x)$ of the neural
network is given by $\mathbf{y}\in\mathbb{R}^{m}$ where $\mathbf{y}$ is
computed as:

$\displaystyle\mathbf{H}=\alpha(\mathbf{x}\,\mathbf{W}^{{(1)}}+\mathbf{b}^{{(1)}})$
(2)
$\displaystyle\mathbf{y}=\alpha^{\prime}(\mathbf{H}\,\mathbf{W}^{{(2)}}+\mathbf{b}^{{(2)}})$
(3)

where $\alpha(\cdot),\alpha^{\prime}(\cdot)$ denote the activation functions
(e.g. $sigmoid$, $tanh$, $ReLU$ etc.) that are applied to the vectors of the
hidden and the output layers element-wise. For example, a commonly used
activation function is the ReLU, which is defined, for $\chi\in\mathbb{R}$,
as:

$\displaystyle ReLU(\chi)=\begin{cases}\chi,&\text{if }\chi\geq 0\\\
0,&\text{otherwise}\end{cases}$ (4)

Another example is the hard $tanh$ function [collobert:2004b] that serves as
the output layer activation function of the neural networks that we have
trained for a regression problem that we addessed in the experiments of
Section 5. For $\chi\in\mathbb{R}$, hard $tanh$ is defined as:

$\displaystyle HardTanh(\chi)=\begin{cases}1,&\text{if }\chi>1\\\ -1,&\text{if
}\chi<-1\\\ \chi,&\text{otherwise}\end{cases}$ (5)

##### Generalization to multiple hidden layers

The neural network definition provided above can be easily generalised to any
number of hidden layers $H,H^{\prime},H^{\prime\prime},\cdots$. We consider
that weights and biases for all layers and nodes are fixed, since we focus on
equivalence checking of neural networks after training.

##### Example

Consider a simple feedforward NN with two inputs, two outputs and one hidden
layer with two nodes. The selection of the weights and biases is done
randomly, the activation function of the hidden layer is ReLU while there is
no activation function for the output layer. A drawing of the NN is shown in
Figure 1. For this example, equation (2) takes the form:

$\displaystyle\begin{bmatrix}x_{1}&x_{2}\end{bmatrix}\cdot\begin{bmatrix}W_{11}&W_{12}\\\
W_{21}&W_{22}\end{bmatrix}+\begin{bmatrix}b^{(1)}_{1}&b^{(1)}_{2}\end{bmatrix}$
$\displaystyle=\begin{bmatrix}x_{1}&x_{2}\end{bmatrix}\cdot\begin{bmatrix}-2&1\\\
1&2\end{bmatrix}+\begin{bmatrix}1&1\end{bmatrix}$
$\displaystyle=\begin{bmatrix}-2\cdot x_{1}+x_{2}+1&x_{1}+2\cdot
x_{2}+1\end{bmatrix}$

Denote the result of the affine transformation of the neural network’s hidden
layer by:

$\displaystyle\begin{bmatrix}z_{1}&z_{2}\end{bmatrix}$
$\displaystyle=\begin{bmatrix}-2\cdot x_{1}+x_{2}+1&x_{1}+2\cdot
x_{2}+1\end{bmatrix}$

Then, we get:

$\displaystyle\mathbf{H}$
$\displaystyle=\begin{bmatrix}h_{1}&h_{2}\end{bmatrix}=\begin{bmatrix}ReLU(z_{1})&ReLU(z_{2})\end{bmatrix}$

The output $\mathbf{y}$ of the neural network from equation (3) is:

$\displaystyle\mathbf{y}=\begin{bmatrix}h_{1}&h_{2}\end{bmatrix}\cdot\begin{bmatrix}2&-1\\\
-1&-2\end{bmatrix}+\begin{bmatrix}b^{(2)}_{1}&b^{(2)}_{2}\end{bmatrix}=\begin{bmatrix}2\cdot
h_{1}-h_{2}+2&-h_{1}-2\cdot h_{2}+2\end{bmatrix}$

$x_{1}^{{\color[rgb]{1,0.85,0.7}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.85,0.7}[1]}}$
ptInputLayer
$x_{2}^{{\color[rgb]{1,0.85,0.7}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.85,0.7}[1]}}$
$z_{1}^{[1]}$ ptHiddenLayer $z_{2}^{[1]}$
$h_{1}^{{\color[rgb]{0.5,0.5,1}\definecolor[named]{pgfstrokecolor}{rgb}{0.5,0.5,1}[1]}}$
$h_{2}^{{\color[rgb]{0.5,0.5,1}\definecolor[named]{pgfstrokecolor}{rgb}{0.5,0.5,1}[1]}}$
$y_{1}^{[2]}$ ptOutputLayer $y_{2}^{[2]}$ $-2$$2$$1$$1$ReLUReLU2-2-1-1 Figure
1: Feedforward neural network with an input layer of 2 inputs ($x_{1}$,
$x_{2}$), an output layer with 2 outputs ($y_{1}$, $y_{2}$) and no activation
function, and 1 hidden layer with 2 neurons and a ReLU activation function.
The values on the transitions correspond to the _weights_ and the upperscript
values to the biases.

##  3 Strict and Approximate Equivalences for Neural Networks

In this section, we propose different equivalence relations and we formulate
the equivalence checking problem.

###  3.1 Strict Neural Network Equivalence

Strict NN equivalence is essentially functional equivalence:

######  Definition 1 (Strict NN Equivalence)

Consider two neural networks $f:I\to O$ and $f^{\prime}:I\to O$. We say that
$f$ and $f^{\prime}$ are _strictly equivalent_ , denoted $f\equiv f^{\prime}$,
if and only if the following holds:

$\forall x\in I,f(x)=f^{\prime}(x)$ (6)

Strict NN equivalence is a true equivalence relation, i.e., it is reflexive
($f\equiv f$ for any NN $f$), symmetric ($f\equiv f^{\prime}$ iff
$f^{\prime}\equiv f$), and transitive ($f\equiv f^{\prime}$ and
$f^{\prime}\equiv f^{\prime\prime}$ implies $f\equiv f^{\prime\prime}$).

However, strict NN equivalence can often be a too strong requirement. For
example, if we have two classifiers we may want to consider them equivalent if
they always select the same top output class, even though they may not order
the remaining output classes in the same way. This motivates us to consider
the following approximate notions of equivalence. We remark that these
approximate “equivalences” need not be true equivalences, i.e., they may not
satisfy the transitivity property (although they are always reflexive and
symmetric).

###  3.2 Approximate Neural Network Equivalences based on Lp Norms

As usual, we assume that $O\subseteq\mathbb{R}^{m}$. Let $\lVert
y\rVert_{p}=norm_{p}(y)$ denoting the $Lp$-norm of vector $y\in O$, for
$norm_{p}:O\to\mathbb{R}$ with $p=1,2,\infty$. For any two vectors
$y,y^{\prime}\in O$, if $p=1$ we have the widely used Manhattan norm,
$L_{1}(y,y^{\prime})=\lVert y-y^{\prime}\rVert_{1}=\sum_{i=1}^{m}\lvert
y(i)-y^{\prime}(i)\rvert$, which measures the sum of (differences) between two
vectors. For $p=2$, we refer to the so-called Euclidean distance
$L_{2}(y,y^{\prime})=\lVert y-y^{\prime}\rVert_{2}=(\sum_{i=1}^{m}\lvert
y(i)-y^{\prime}(i)\rvert^{2}\large)^{\frac{1}{2}}$. Finally, for $p=\infty$ we
have the $L_{\infty}$ distance, which measures the maximum change to any
coordinate, i.e. it is given as:

$L_{\infty}(y,y^{\prime})=\lVert\hskip 2.84526pty-y^{\prime}\hskip
2.84526pt\rVert_{\infty}=\max(\lvert y(1)-y^{\prime}(1)\rvert,\dots,\lvert
y(m)-y^{\prime}(m)\rvert)$ (7)

Then, we define the following notion of approximate equivalence:

######  Definition 2 ($(p,\epsilon)$-approximate equivalence)

Consider two neural networks $f:I\to O$ and $f^{\prime}:I\to O$,
$norm_{p}:O\to\mathbb{R}$, and some $\epsilon>0$. We say that $f$ and
$f^{\prime}$ are $(p,\epsilon)$-_approximately equivalent_ , denoted
$f\sim_{p,\epsilon}f^{\prime}$, if and only if the following holds:

$\forall x\in I,\hskip 5.69054pt\lVert f(x)-f^{\prime}(x)\rVert_{p}<\epsilon$
(8)

It can be seen that the relation $\sim_{p,\epsilon}$ is reflexive and
symmetric.

###  3.3 Approximate Neural Network Equivalences based on Order of Outputs

NN classifiers work essentially by computing output values and then mapping
these outputs to specific classes. For such networks, we may want to consider
them equivalent if they always produce the same order of outputs, even though
the output values might not be the same. For example, consider two classifiers
$f$ and $f^{\prime}$ over three possible output classes. Suppose that, for a
given input, $f$ produces $(0.3,0.5,0.2)$ whereas $f^{\prime}$ produces
$(0.25,0.6,0.15)$. We may then consider that for this input the outputs of $f$
and $f^{\prime}$ are equivalent, since they have the same order, namely,
$2,1,3$ (assuming vector indices start at $1$). If this happens for all
inputs, we may want to consider $f$ and $f^{\prime}$ (approximately)
equivalent.

To capture the above notion of approximate equivalence, we introduce the
$\texttt{argsort}_{m}$ function, where $m\in\mathbb{N}$:

$\displaystyle\mathtt{argsort}_{m}:\mathbb{R}^{m}\to\mathcal{Z}_{m}$

where $\mathcal{Z}_{m}\subseteq\\{1,2,3,\mathellipsis,m\\}^{m}$ is the set of
permutations of indices of the $m$ elements. Then, for a given
$s\in\mathbb{R}^{m}$, $\texttt{argsort}_{m}(s)$ returns the permutation that
sorts $s$ in decreasing order. For example,
$\mathtt{argsort}_{3}(0.3,0.5,0.2)=\mathtt{argsort}_{3}(0.25,0.6,0.15)=(2,1,3)$.
In the case where two vector values are equal, $\mathtt{argsort}$ orders them
from lower to higher index. This ensures determinism of the $\mathtt{argsort}$
function. For example, $\mathtt{argsort}_{3}(0.3,0.4,0.3)=(1,3,2)$.

######  Definition 3 (Top-$k$ $\mathtt{argsort}$ equivalence)

Suppose $O\subseteq\mathbb{R}^{m}$. Consider two neural networks $f:I\to O$
and $f^{\prime}:I\to O$, and some $k\in\\{1,...,m\\}$. We say that $f$ and
$f^{\prime}$ are top-$k$ $\mathtt{argsort}$ equivalent, denoted
$f\approx_{k}f^{\prime}$, if and only if

$\forall x\in I,\forall
i\in\\{1,...,k\\},\Big{(}\mathtt{argsort}_{m}\big{(}f(x)\big{)}\Big{)}(i)=\Big{(}\mathtt{argsort}_{m}\big{(}f^{\prime}(x)\big{)}\Big{)}(i)$
(9)

Top-$k$ $\mathtt{argsort}$ equivalence requires the first $k$ indices of the
argsort of the outputs of $f$ and $f^{\prime}$ to be equal. Top-$k$
$\mathtt{argsort}$ equivalence is reflexive, symmetric, and transitive, i.e.,
it is a true equivalence relation.

A special case of top-$k$ $\mathtt{argsort}$ equivalence is when $k=1$. We
call this special case the $\mathtt{argmax}$ equivalence, with reference to
the $\mathtt{argmax}$ function which returns the index of the maximum value of
a given vector. For example,
$\mathtt{argmax}(0.3,0.5,0.2)=\mathtt{argmax}(0.25,0.6,0.15)=2$.

######  Definition 4 ($\mathtt{argmax}$ equivalence)

Consider the same setting as in Definition 3. We say that $f$ and $f^{\prime}$
are $\mathtt{argmax}$ equivalent iff $f\approx_{1}f^{\prime}$.

###  3.4 The Neural Network Equivalence Checking Problem

######  Definition 5 (NN equivalence checking problem)

Given two (trained) neural networks $f$ and $f^{\prime}$, and given a certain
NN equivalence relation
$\simeq\;\in\\{\equiv,\sim_{p,\epsilon},\approx_{k}\\}$, and parameters
$p,\epsilon,k$ as required, the neural network equivalence checking problem
(NNECP) is to check whether $f\simeq f^{\prime}$.

##  4 Neural Network Equivalence Checking using SMT Solvers

Our approach to solving the NNECP is to reduce it to a logical satisfiability
problem. The basic idea is the following. Suppose we want to check whether
$f\simeq f^{\prime}$, for two NNs $f:I\to O$ and $f^{\prime}:I\to O$ and a
given NN equivalence relation $\simeq$. We proceed as follows: (1) encode $f$
into an SMT formula $\phi$; (2) encode $f^{\prime}$ into an SMT formula
$\phi^{\prime}$; (3) encode the equivalence relation $f\simeq f^{\prime}$ into
an SMT formula $\Phi$ such that $f\simeq f^{\prime}$ iff $\Phi$ is
unsatisfiable; (4) check, using an SMT solver, whether $\Phi$ is satisfiable:
if not, then $f\simeq f^{\prime}$; if $\Phi$ is satisfiable, then $f$ and
$f^{\prime}$ are not equivalent, and the SMT solver also typically provides a
counterexample, i.e., an input which violates the equivalence of $f$ and
$f^{\prime}$.

This idea is based on the fact that the negation of $f\simeq f^{\prime}$ can
be encoded as a formula which asserts that there exist an input $x\in I$ and
two outputs $y,y^{\prime}\in O$, such that $y=f(x)$,
$y^{\prime}=f^{\prime}(x)$, and $y$ and $y^{\prime}$ do not satisfy the
equivalence conditions imposed by $\simeq$. For example, for the case of
strict NN equivalence, checking whether $f\equiv f^{\prime}$ amounts to
checking:

$\neg\Big{(}\exists x\in I,y\in O,y^{\prime}\in O,\;\;y=f(x)\land
y^{\prime}=f^{\prime}(x)\land y\neq y^{\prime}\Big{)}$

This in turn amounts to checking that the formula $y=f(x)\land
y^{\prime}=f^{\prime}(x)\land y\neq y^{\prime}$ is unsatisfiable. In this
case, we have $\phi:=y=f(x)$, $\phi^{\prime}:=y^{\prime}=f^{\prime}(x)$, and
$\Phi:=\phi\land\phi^{\prime}\land y\neq y^{\prime}$.

We proceed to provide the details of building $\phi$ and $\phi^{\prime}$ for
given neural networks, as well as $\Phi$ for the NN equivalence relations
defined earlier.

###  4.1 Encoding Neural Networks as SMT Formulas

####  4.1.1 Input variables

From (2), the input of a neural network $f$ is a vector
$\mathbf{x}=[x_{1},...,x_{n}]\in\mathbb{R}^{1\times n}$. The SMT formula
$\phi$ encoding $f$ will have $n$ input variables, which we will denote
$x_{1},...,x_{n}$.

####  4.1.2 Encoding input constraints

Sometimes the inputs are constrained to belong in a certain region. For
example, we might assume that the input lies between given lower and upper
bounds. In such cases, we can add input constraints as follows:

$\displaystyle\bigwedge_{j=1}^{n}l_{j}\leq x_{j}\leq u_{j}$ (10)

with $l_{j},u_{j}\in\mathbb{R}$ denoting the lower and upper bounds for the
domain of input feature $x_{j}$.

####  4.1.3 Internal variables

For each hidden layer, we associate a set of internal variables $z_{i}$
corresponding to the affine transformation, and a set of internal variables
$h_{i}$ corresponding to the activation function.

####  4.1.4 Constraints encoding the affine transformations

Consider a single hidden layer of $f$ with $r$ nodes. Then, from the affine
transformation of (2), we derive the constraints:

$\displaystyle\bigwedge_{j=1}^{r}\left(z_{j}=\sum_{k=1}^{n}x_{k}W^{(1)}_{kj}+b^{(1)}_{j}\right)$
(11)

####  4.1.5 Constraints encoding the ReLU activation function

If the activation function is $ReLU$, then its effect is encoded with the
following constraints:

$\displaystyle\bigwedge_{j=1}^{r}(z_{j}\geq 0\land
h_{j}=z_{j})\lor(z_{j}<0\land h_{j}=0)$ (12)

####  4.1.6 Constraints encoding the hard $tanh$ activation function

If the activation function is the hard $tanh$, then its effect is encoded with
the following constraints:

$\displaystyle\bigwedge_{j=1}^{r}\large(z_{j}\geq 1\land
h_{j}=1)\lor(z_{j}\leq-1\land h_{j}=-1\large)\lor(-1<z_{j}<1\land
z_{j}=h_{j})$ (13)

####  4.1.7 Other activation functions

The constraints described so far include atoms of the linear real arithmetic
theory [Kroe08] that most SMT-solvers can check for satisfiability through
decision procedures of various degrees of efficiency, for the problem at hand.

Other activation functions than the ones described above are often problematic
as there are limitations regarding the expressions and relations that SMT
solvers can handle. For example, most SAT/SMT solvers cannot handle formulas
with exponential terms, which is the case of activation functions such as (not
hard) Tanh, Sigmoid, and Softmax. This fact raises the need to find
alternative ways to represent such functions, through e.g. using encodings for
“hard” versions of these functions found in related works [PGL-051],
[collobert:2004b]. This is the reason we opted for the hard $tanh$ encoding
described above.

####  4.1.8 Generalization to multiple hidden layers and output layer

The constraints of equations (11 \- 13) are generalized to any number of
hidden layers, say $H,H^{\prime},H^{\prime\prime},\mathellipsis$ (that consist
respectively of $r,r^{\prime},r^{\prime\prime},\mathellipsis$ nodes).

The neural network encoding is completed with the constraints for the output
layer that are derived, for $\mathbf{y}\in\mathbb{R}^{m}$ from equation (3),
as previously.

##### Example (continued)

This example contains a single NN and we herein show how the SMT constraints
are derived and encoded in case $0\leq x_{1}\leq 1$ and $0\leq x_{2}\leq 1$.
For the transformation, we introduce the variables $z_{1}$ and $z_{2}$ with
$z_{1}=-2\cdot x_{1}+x_{2}+1$ and $z_{2}=x_{1}+2\cdot x_{2}+1$. Moving to the
activation functions, we have $h_{1}=ReLU(z_{1})=\max(0,z_{1})$. We add the
constraint $\\{(z_{1}\geq 0\wedge h_{1}=z_{1})\vee(z_{1}<0\wedge h_{1}=0)\\}$.
Similarly, $h_{2}=ReLU(z_{2})=\max(0,z_{2})$. We add $\\{(z_{2}\geq 0\wedge
h_{2}=z_{2})\vee(z_{2}<0\wedge h_{2}=0)\\}$. Finally, we add the output
constraint $y_{1}=2\cdot h_{1}-h_{2}+2$ and $y_{2}=-h_{1}-2\cdot h_{2}+2$.

The resulting SMT formula consists of the following constraints

$\displaystyle\phi:=\\{$ $\displaystyle 0\leq x_{1}\wedge x_{1}\leq 1\wedge
0\leq x_{2}\wedge x_{2}\leq 1\wedge z_{1}=-2\cdot
x_{1}+x_{2}+1\wedge((z_{1}\geq 0\wedge h_{1}=z_{1})\vee(z_{1}<0\wedge
h_{1}=0))$ $\displaystyle\wedge z_{2}=x_{1}+2\cdot x_{2}+1\wedge((z_{2}\geq
0\wedge h_{2}=z_{2})\vee(z_{2}<0\wedge h_{2}=0))\wedge y_{1}^{\prime}=2\cdot
h_{1}-h_{2}+2$ $\displaystyle\wedge y_{2}=-h_{1}-2\cdot h_{2}+2\\}$

###  4.2  Encoding of the equivalence relation

In Subsection 4.1 we described how to encode a given neural network $f$ as a
formula $\phi$. As mentioned at the beginning of this section, in order to
check equivalence of two given neural networks $f$ and $f^{\prime}$, we need
to generate, first, their encodings $\phi$ and $\phi^{\prime}$ as described in
Subsection 4.1, and second, the encoding of the (negation of the) equivalence
relation. The latter encoding is described next.

We assume that the two neural networks $f$ and $f^{\prime}$ to be compared
have the same number of outputs $m$, and we let $\mathbf{y}=(y_{1},...,y_{m})$
and $\mathbf{y^{\prime}}=(y_{1}^{\prime},...,y_{m}^{\prime})$ denote their
respective output variables.

####  4.2.1 Strict equivalence checking

Strict equivalence (c.f., Definition 1) requires that
$\mathbf{y}=\mathbf{y}^{\prime}$. As explained in the beginning of this
section, the reduction to a satisfiability problem means that we must encode
the negation of the above constraint, namely:

$\displaystyle\bigvee_{i=1}^{m}y_{i}\neq y^{\prime}_{i}$ (14)

####  4.2.2 $(p,\epsilon)$-approximate equivalence checking

$(p,\epsilon)$-approximate equivalence (c.f., Definition 2) requires that
$\mathbf{\lVert y-y^{\prime}\rVert_{p}<\epsilon}$. Again, we encode the
negation:

  * •

for $p=1$,

$\displaystyle\sum_{i=1}^{m}\lvert y_{i}-y^{\prime}_{i}\rvert\geq\epsilon$
(15)

  * •

for $p=2$,

$\displaystyle(\sum_{i=1}^{m}\lvert
y_{i}-y^{\prime}_{i}\rvert^{2}\large)^{\frac{1}{2}}\geq\epsilon$ (16)

  * •

whereas for $p=\infty$,

$\displaystyle\bigvee_{i=1}^{m}\lvert y_{i}-y^{\prime}_{i}\rvert\geq\epsilon$
(17)

The $L_{2}$ norm is currently not supported in our implementation.

####  4.2.3 argmax equivalence checking

argmax equivalence (c.f., Definition 4) requires that
$\textbf{argmax(y)}=\textbf{argmax}\mathbf{(y^{\prime})}$. Again, we wish to
encode the negation, i.e.,
$\textbf{argmax(y)}\neq\textbf{argmax}\mathbf{(y^{\prime})}$. This can be done
by introducing the macro ${\mathsf{argmaxis}}(y,i,m)$ which represents the
constraint $\mathtt{argmax}(y)=i$, assuming the vector $y$ has length $m$.
Then, $\textbf{argmax(y)}\neq\textbf{argmax}\mathbf{(y^{\prime})}$ can be
encoded by adding the constraints below:

$\displaystyle\mathop{\bigvee_{i,i^{\prime}\in\\{1,...,m\\}}}_{i\neq
i^{\prime}}{\mathsf{argmaxis}}(y,i,m)\land{\mathsf{argmaxis}}(y^{\prime},i^{\prime},m)$
(18)

where ${\mathsf{argmaxis}}$ is defined as follows:

$\displaystyle{\mathsf{argmaxis}}(y,i,m):=\big{(}\bigwedge_{j=1}^{i-1}y_{i}>y_{j}\big{)}\land\big{(}\bigwedge_{j=i+1}^{m}y_{i}\geq
y_{j}\big{)}$ (19)

For example, for $m=2$, we have:

$\displaystyle{\mathsf{argmaxis}}(y,1,2)=y_{1}\geq y_{2}$
$\displaystyle{\mathsf{argmaxis}}(y,2,2)=y_{2}>y_{1}$
$\displaystyle{\mathsf{argmaxis}}(y^{\prime},1,2)=y^{\prime}_{1}\geq
y^{\prime}_{2}$
$\displaystyle{\mathsf{argmaxis}}(y^{\prime},2,2)=y^{\prime}_{2}>y^{\prime}_{1}$

and the overall constraint encoding
$\textbf{argmax(y)}\neq\textbf{argmax}\mathbf{(y^{\prime})}$ becomes:

$\displaystyle(y_{1}\geq y_{2}\land
y^{\prime}_{2}>y^{\prime}_{1})\lor(y_{2}>y_{1}\land y^{\prime}_{1}\geq
y^{\prime}_{2})$

##### Example (continued)

Let’s assume that there is an additional neural network $f^{\prime}$ with the
same number of inputs and outputs as $f$ that can be encoded via constraints
$\phi^{\prime}$. Below, we show how the complete SMT formula $\Phi$ would be
encoded for different equivalence relations:

$\displaystyle\Phi_{strict}:$
$\displaystyle=\\{\phi\wedge\phi^{\prime}\wedge\bigvee_{i=1}^{m}y_{i}\neq
y^{\prime}_{i}\\}$ $\displaystyle\Phi_{(1,\epsilon)-\text{approx}}:$
$\displaystyle=\\{\phi\wedge\phi^{\prime}\wedge\sum_{i=1}^{m}\lvert
y_{i}-y^{\prime}_{i}\rvert\geq\epsilon\\}$
$\displaystyle\Phi_{(\infty,\epsilon)-\text{approx}}:$
$\displaystyle=\\{\phi\wedge\phi^{\prime}\wedge\bigvee_{i=1}^{m}\lvert
y_{i}-y^{\prime}_{i}\rvert\geq\epsilon\\}$ $\displaystyle\Phi_{argmax}:$
$\displaystyle=\\{\phi\wedge\phi^{\prime}\wedge\mathop{\bigvee_{i,i^{\prime}\in\\{1,...,m\\}}}_{i\neq
i^{\prime}}{\mathsf{argmaxis}}(y,i,m)\land{\mathsf{argmaxis}}(y^{\prime},i^{\prime},m)\\}$

##  5 Experimental results

In this section, we report on a set of experiments on verifying equivalence
relations between two NNs. In our experiments we have used the SMT solver Z3
111https://z3prover.github.io/api/html/ to check satisfiability of all
formulas used to encode NN equivalence. All experiments were conducted on a
laptop with a 4-core $2.8$GHz processor and $12$ GB RAM. We experiment with
different sizes of NNs, while we conduct equivalence checking for the various
equivalence relations defined in Section 3. We focus on the two main
categories of supervised learning problems, i) classification, and ii)
regression. We examine two case studies for classification and one for
regression.

Bit-Vec case study – Classification

A bit vector (Bit-Vec) is a mapping from an integer domain to values in the
set 0, 1. For this case study, we consider that the inputs are 10-bit vectors
and the targets (labels) are binary: either True (1) or False (0). The models
we check for equivalence are Feed-Forward Multi-Layer Perceptrons and we make
use of two different architectures. In the first, there is a single hidden
layer, while in the second one there are two hidden layers; the networks have
the same number of nodes per layer. We experiment with $8$ different models
per architecture and for each model we incrementally increase the number of
nodes per layer. We train neural networks with the objective to approximate
(“learn”) that for 3 or more consecutive 1s in the vector the output label is
True, otherwise the output label is False.

MNIST case study – Classification

The second case study uses the _MNIST_ dataset, a popular dataset on image
classification. The dataset contains 70,000 grayscale images, from which
60,000 are used for training and the rest 10,000 for testing the models’
performance. Every image’s size is 28x28 (pixels), while every pixel value is
a real number in the range $[0,1]$. We experiment with the same two
architectures used in the Bit-Vec case study and $5$ models per architecture.

Automotive Control – Regression

For the regression case study, the goal is to use neural networks to
approximate the behaviour of a Model Predictive Controller (MPC) that has been
designed for an automotive lane keeping assist system. The dataset contains
10,000 instances with six features representing different system
characteristics obtained by the sensors and the resulting steering angle that
the automotive car should follow (target). In Table 1 the features’ and
target’s details and valid value ranges are presented.

Type/Parameter | Answer/Value | Remarks
---|---|---
output/target | $[-1.04,1.04]$ | steering angle $[-60,60]$
input range $x_{1}$ | [-2,2] |  $v_{x}$ (m/s)
input range $x_{2}$ | [-1.04, 1.04] | rad/s
input range $x_{3}$ | [-1,1] | m
input range $x_{4}$ | [-0.8, 0.8] | rad
input range $x_{5}$ | [-1.04, 1.04] |  $u_{0}$ (steering angle)
input range $x_{6}$ | [-0.01,0.01] | $\rho$
Table 1: Regression Problem Input Characteristics – Constraints

###  5.1 Sanity checks

In this first set of experiments, we have two primary goals to achieve. First,
we want to sanity-check our prototype implementation, in order to ensure that
it does not contain any bugs and that it is able to provide concrete
solutions. Second, we want to perform an empirical scalability study focusing
on the computational demands required for NNs of increasing complexity to be
equivalence-checked according to the relations described in Section 3.

More specifically, we verify two identical neural networks under the various
equivalence relations. We apply these checks in the BitVec and MNIST case
studies and for two neural network architectures for each of them. Tables
LABEL:tab:Table_2 and LABEL:tab:Table_3 summarize the results for the BitVec
case study, while Tables LABEL:tab:Table_4 and LABEL:tab:Table_5 summarize the
results for the MNIST case study. In each table, the first column shows the
number of nodes per hidden layer per model, the second column the number of
trainable parameters per model and the third the total number of variables
(“unknowns”) in the formula given to the SMT solver. The tables then report
the time in seconds (s) that the SMT solver took to check each equivalence
relation for each pair of identical neural networks. In the BitVec tables this
is the average over 10 runs, with the worst and best case not presented since
the standard deviation between observations is below $3\%$ in all occasions.
In the MNIST tables, each sanity check was conducted once as the neural
networks have much bigger size and a longer time is needed for every check to
be completed.


# A Game Theoretic Analysis for
Cooperative Smart Farming

Deepti Gupta14, Paras Bhatt24, Smriti Bhatt3<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
4 These authors contributed equally to this work. 1Dept. of Computer Science,
University of Texas at San Antonio, San Antonio, Texas 78249, USA
2Dept. of Information Systems and Cyber Security, University of Texas at San
Antonio, San Antonio, Texas 78249, USA
3Dept. of Computing and Cyber Security, Texas A & M University-San Antonio,
San Antonio, Texas 78224, USA

###### Abstract

The application of Internet of Things (IoT) and Machine Learning (ML) to the
agricultural industry has enabled the development and creation of smart farms
and precision agriculture. The growth in the number of smart farms and
potential cooperation between these farms has given rise to the Cooperative
Smart Farming (CSF) where different connected farms collaborate with each
other and share data for their mutual benefit. This data sharing through CSF
has various advantages where individual data from separate farms can be
aggregated by ML models and be used to produce actionable outputs which then
can be utilized by all the farms in CSFs. This enables farms to gain better
insights for enhancing desired outputs, such as crop yield, managing water
resources and irrigation schedules, as well as better seed applications.
However, complications may arise in CSF when some of the farms do not transfer
high-quality data and rather rely on other farms to feed ML models. Another
possibility is the presence of rogue farms in CSFs that want to snoop on other
farms without actually contributing any data. In this paper, we analyze the
behavior of farms participating in CSFs using game theory approach, where each
farm is motivated to maximize its profit. We first present the problem of
defective farms in CSFs due to lack of better data, and then propose a ML
framework that segregates farms and automatically assign them to an
appropriate CSF cluster based on the quality of data they provide. Our
proposed model rewards the farms supplying better data and penalize the ones
that do not provide required data or are malicious in nature, thus, ensuring
the model integrity and better performance all over while solving the
defective farms problem.

###### Index Terms:

Cooperative smart farming, IoT device, Edge computing, Cloud computing, Game
theory, Cooperative Game Model

##  I Introduction

Farming is one the earliest activities undertaken by humans to build
civilizations. Feeding the ever-increasing world population is a huge
challenge and the role farming in meeting this challenge cannot be overstated.
The recent recognition of the World Food Program
(WFP)111https://www.nobelprize.org/prizes/peace/2020/summary/ as the Nobel
Peace Prize recipient underlines the importance food security plays in our
lives. Consequently, it is equally important to ensure that sustainable
farming practices are followed so that land is not overused and exploited.
Internet of Things (IoT) technology can serve as a critical asset to enhance
agricultural practices and enable efficient and sustainable farming. With the
application of IoT and Machine Learning (ML) algorithms in the farming sector,
a new domain of smart agriculture has emerged. Using IoT to monitor and
measure different aspects of soil, land and crops can lead to development of
better and more sophisticated farms that use data-driven models and
applications to enhance their production, which is also known as smart farming
and/or precision agriculture today.

With the rapid diffusion of smart devices and IoT technology in the
agriculture domain, farms have grown more advanced, technically competent and
production efficient [1]. The use of IoT has been abundant and in turn led to
the creation of a technology based smart ecosystem. Such a connected system
enabled by smart devices and key technologies as cloud and edge computing
leads to the creation of smart farms which generates large amounts of data.
This large amount of data can be analyzed using ML and Artificial Intelligence
(AI) technologies to extract crucial insights and implications. The need for
such smart farms has been stressed in research and is equally important for
the sustenance of smart cities [2, 3]. As smart cities develop so should the
infrastructure around farming, as it is not only an extension but a necessary
derivative of the fully connected IoT architecture. However, the increase in
number of smart connected farms has not been without issues. There is a
possibility of adversaries sabotaging farms, malicious agents compromising the
functioning of such farms or even cyber-attacks against smart farming adopters
[4, 5]. It is therefore important to have strong security in place that
thwarts such attacks and better data management practices so that informed
decisions can be made that do not jeopardize smart farm outputs. Another
reason to ensure data security and management is to increase the farm owners’
confidence in the ability of technology to actually provide them with visible
benefits. The benefits accruing from data analysis of smart farms can either
be related to greater production output, or better land use, or even the
discovery of efficient seed application techniques.

The use of cloud computing as well as web-based services has ushered in a new
era where smart farms plan better yield management, adopt responsible farming
practices, and indulge in sustainable techniques. It is for this reason that
IT has been so readily deployed and used in the agriculture sector, giving
rise to the smart farms concept [6]. Smart farming involves the adoption of
IoT devices to automate processes on regular farms, such as irrigation and
seed application. It also involves collecting data on the soil quality,
moisture and crop yield metrics. Such data can drive new knowledge discovery
which benefits and informs the entire agricultural industry of new and
efficient ways to farm and grow crops. Also, as new developments take place in
the IoT industry and advances are made in data usage techniques, it is equally
important that these improvements rapidly percolate to the agriculture domain
primarily to the associated smart farms.

The increasing number of smart farms demands cooperation between each other
which allows these farms to learn efficient farming techniques by applying
data-driven models to smart farm data and share resources among each other.
This type of cooperation between several smart farms is known as Cooperative
Smart Farming (CSF). While CSF is an attractive framework to benefit multiple
farms together, it is important to understand that there are also security and
privacy issues that could arise in this framework. Data privacy could be a
major concern for some smart farms, thus, it is essential to address security
and privacy in CSFs. There may be different types of farms participating in a
CSF. A farm, which has low-quality agriculture data generated from IoT
devices, always wants to be a part of CSF to take benefit from other farms. A
farm, which has huge number of resources and high-quality agriculture data,
does not receive any benefit in CSF. Therefore, there is an issue for smart
farms to become a member of CSF and contribute effectively as member of the
CSF. In this paper, we address the research problem of unfair collaboration
among smart farms in a CSF by proposing a CSF game model, and a novel fair
strategy which enforces each smart farm to cooperate in CSF based on the
clusters formed to achieve benefit through AI assisted application. The main
contributions of this paper are as follows.

  1. 1.

We identify the issue of unfair collaboration among farms in cooperative smart
farming. For instance, some smart farms which generate low-quality data to
build machine learning model can take advantage from other farms which have
high-quality data.

  2. 2.

We present two different use case scenarios including different types of smart
farms in a CSF along with rationality assumption.

  3. 3.

We propose a fair strategy to enforce farms to cooperate in CSF to build ML
model.

  4. 4.

We present a proposed implementation framework that can be employed and
extended to enable CSF use case scenarios utilizing a real-world cloud-enabled
IoT platform, Amazon Web Services (AWS).

The rest of paper is organized as follows. Section II presents relevant work
on CSF, security and privacy issues in CSF, and game theoretic models used in
smart farming. Cooperative smart farming use case scenarios along with
rational assumption are discussed in Section III. Section IV shows cooperative
smart farming game and also shows the game analysis in same section. Section V
presents proposed fair strategy. Section VI discuss proposed implementation
framework for enabling CSF using real-world cloud-enabled IoT platform.
Section VII concludes the paper with future research directions.

##  II Related Work

In this section, we discuss the relevant work in perspective of three specific
areas - cooperative smart farming (CSF), security and privacy issues in CSFs,
and game theory application in the context of smart farming.

###  II-A Cooperative Smart Farming

Farming has always been a community activity and in its truest sense a
communal one that includes a participatory process. Therefore, even though
individual farms operating alone may be efficient revenue maximizers, it is
still an evident fact that significant benefits can accrue to those
participating in a cooperative bloc [7]. Similar to cooperation in the
traditional sense of farming, where farmers share resources such as seeds and
equipment; a CSF scenario in the technical sense involves sharing of data
collected by numerous sensors, devices, and other resources. The reasons to
indulge in CSF are not only economic ones but also ones of security. For
instance, the participating farms share data which can lead to better crop
output. Meanwhile, the data shared can also be analyzed to figure out threats
to crops and protect them from damage [8].

Figure 1: Cooperative Smart Farming Architecture

Such critical insights are only possible with the help of IoT devices that
relay data from the farms in the cooperative hosted on cloud or edge servers.
These farms reinforced with IoT devices come under the purview of smart farms
which have been researched in many studies [9, 10]. Smart farms also employ
remote sensing techniques to keep track of the covered land, monitor crop
health and even classify weeds [11].

Moreover, the use of technology in smart farms has been so far advanced as
distinguishing between different plowing techniques and classifying them
according to plowing depth [12]. The use of Unmanned Aerial Vehicles (UAV) has
been consistent in smart farms along with the use of IoT devices. An emerging
setting for smart and cooperative farming is indoor farming, especially
greenhouses, where IoT devices that share data with the cloud have been
utilized to replicate the results achieved in outdoor farms [13].

In terms of implementation, there have been several instantiations of smart
and connected farming ecosystems. These smart farms have been fitted with a
variety of IoT devices that can interact with each other and also with the
users to run different tasks. Similarly, REST-APIs have been used to encourage
the cooperative development of applications and solutions to speed up adoption
of the concept of smart and cooperative farming [14]. Data Modeling has been
another important tool used in smart farming to perform a range of tasks, such
as the very basic ones like feed modeling to more environmentally responsible
ones as reducing methane emissions [15]. With many IoT devices in place, there
is a sizable generation of continuous data streams in smart and cooperative
farms. Big data technologies play an important role in the collection, storage
and analysis of data and its conversion to meaningful information, which can
in turn help in overall farm management and streamlining of farm processes
[16].

With regard to the advances in smart farms, AI techniques are increasingly
being used in the smart farming domain today. ML models and techniques too
have begun to play a prominent role in enhancing the capabilities of smart
farms. Image processing using artificial neural networks have been used to
monitor plants both during growth and harvest periods for detecting diseases
as well as for grading them [13]. In addition to plants, ML has also been used
to predict soil and land properties even the propensity of droughts to occur
[17]. Also, with respect to autonomous operations in smart farms, models have
been developed to minimize the instances of human intervention required for
monitoring crops [18]. Another interesting application of AI capabilities in
the smart farming context has been for weed classification where spectral
images collected by small Micro Aerial Vehicles (MAVs) have been used to
identify the weeds that may pose a danger to the surrounding plants or crops
[19].

###  II-B Security and Privacy issues on Cooperative Smart Farming

The use of IoT devices in the smart farms has surely transformed the
cooperative farming landscape, however it has also ushered in the risks
attached to such devices. These risks translate to the various security and
privacy risks that target IoT devices in general. Since a major part of the
infrastructure of a cooperative smart farm is built around participating farms
sharing data with each other, if a particular data source becomes corrupted,
then it can have far reaching implications for all the farms in the bloc.
Privacy by design is critical in such scenarios to protect the privacy
sensitive information of any member farm from being subject to unwanted
disclosure [20]. Also, security of the data being circulated in CSF bloc is
equally important, essentially with respect to the three triad -
confidentiality, integrity, and availability. If this data is tampered with,
then it can lead to devastating consequences such as wiping out entire crop
harvest or withering of plants because of inefficient application of
fertilizers. Therefore, it is crucial to ensure security and privacy be robust
in smart farms.

Security and privacy issues in CSF architecture relate to authorization and
trust, authentication and sever communication as well as compliance and
regulations [4, 21]. Access control plays an important part in regulating the
level of access to the many different IoT devices that may be present within
CSF architecture. Federated access control reference model [22] is used to
enhance the privacy of security of big data platforms. Similarly, there have
been prior work on access control models for developing flexible access
control models for securing access and data communication in IoT [23, 24, 25,
26], and cloud-enabled IoT platforms [27, 28, 29]. Data is transferred quite a
bit among the cooperative smart farms as well as shared with the cloud servers
responsible for aggregating the data for analysis. To combat these security
threats there can be a number of solutions. A potential one solution is the
application of Blockchain, with which the data transferred among the farms can
be verified for its validity [30], and another solution is based on
trustworthy evaluation framework [31]. Also, with the increasing popularity of
smart farms, another security challenge that arises is that these farms
inadvertently play host to a plethora of security challenges [32]. However,
enforcement of data transfer agreements and verifying the approved learning
received from the cloud servers is a very important task in the connected farm
system.

Figure 2: Cooperative Smart Farming Model

Further, the issue of security and privacy in smart farming is not only
restricted to data alone but encompasses both the human and technology
components of CSF architecture. It is important that the farm owners are aware
and trained on the latest threats that may pose a danger to their farms. On
the other hand, the technology providers and vendors should provide end users
with applications and software that is free from flaws and secure by design
[33]. To enhance the security in smart farms, researchers have also looked at
using a private cloud infrastructure that combines data from heterogeneous
sources to provide privacy secure data analytic capabilities to individual
farms [34]. Such a solution can be ported into a collaborative mechanism that
can effectively serve participating smart farms in a cooperative bloc.
Ensuring proper device authentication of smart devices with low computing
power, that are embedded in connected farm systems, can also act as an
effective deterrent against cyber-attacks and minimize security violations
[35].

###  II-C Game Theory in Smart Farming

This paper [36] investigates the utilization of game theory models for
automated analysis of hyperspectral imagery data. The authors discuss two
models on feature-level fusion and decision fusion for hypertemporal-
hyperspectral datasets, and both models are implemented under the assumption
where all players are rational. In agriculture, this research [37] presents
game theory model for decision making in risky situations. In beginning, a
farmer makes plans for obtaining feasible goals and then simply carry out the
plans. Later, he/she can face various uncertain issues in agriculture
production. This study [38] provides possible solutions to actual decision
problems of Iowa farmers. This paper [39] deals with the application of game
theory with solution to handle an agricultural economics problem. This study
claims to establish Nash Equilibrium for an agricultural company that is
considered together with its three sub-units. Gupta et al. [40] presents a
novel fair strategy for collaborative deep learning game, where all mobile
edge devices [41] choose any strategy to make own profit. The past studies
show that no research has analyzed the farm’s rational behavior in CSF.
Therefore, we develop a game model for farms in CSF and analyze the game.

##  III Cooperative Smart Farming

Figure 3: Cooperative Smart Farm Model - Threat Scenario 1

Cooperatives are formal corporation which are financed, controlled, and owned
by members for correlative advantage [42, 43, 44]. These cooperative (co-op)
work is based on membership agreement, which has operational rules along with
conditions and provide the benefit of shared resources. Such cooperatives
operate in different sectors, such as grocery suppliers, water supplies,
credit unions, utilities, farm suppliers, transportation and childcare.
According to United States Department of Agriculture (USDA) [45], the
importance of co-ops is more meaningful in the rural communities. Figure 1
presents the multi-layer architecture of CSF, where physical layer has
Internet of Agriculture Things (IoAT) devices, the edge layer provides local
real-time computation and analysis needed for smart resources, and cloud layer
comprises of cloud services and co-op level.

###  III-A A Use Case

We present a use case of multiple participating smart farms in a CSF bloc
which is based on a game theory-based model. Participating smart farms are
rational choice makers and will only become members of CSF bloc if they get
tangible benefits from it. These benefits have to be more beneficial than
would normally accrue to the farms if they operated alone based on their own
capabilities. In this setting as shown in Figure 2, CSF bloc provides its
members, Farm A and Farm B, the benefits from a shared ML model. This model
can provide numerous insights related to various factors, such as plant and
soil health, soil and water quality, and crop output and yield management
information, based on data collected from Farm A and Farm B. Therefore, if a
smart farm wishes to use the information provided by the model, then it has to
register and join CSF bloc so that it can avail of the learning from the
model. This ML model is trained on the cloud or edge server with data that is
transferred to it from the participating farms. Generally, we assume that the
larger the dataset collected from different farms the better ML model will be;
thus, there will be an inherent incentive for the participating farms to
become a member of CSF bloc and share their data in order to get better
insights about their farming practices and in turn get more profits in the
long run.

If a smart farm does not need the information being provided by the model,
then it will not choose to join CSF bloc. For example, if a smart farm needs
information about soil moisture or soil pH levels and the model only provides
insights about water quality and yield management, then the farm will not
participate in the bloc. Similarly, if earlier ML model in a bloc used to
provide the soil moisture information, but going forward, due to a lack of
good quality data the model stops providing that specific information, then a
smart farm can choose to defect from CSF bloc. Also, if a smart farm is
technically superior and has abundant data and ML capabilities, then it will
not join a CSF bloc. Instead it will operate alone, relying on its own
capabilities. Similarly, if a participating smart farm acquires such superior
capabilities while being a member of a CSF bloc then, it can choose to defect
from the bloc. However, we envision that these scenarios are only fringe
options that depends on different types of participating farms and may or may
not be seen often in the context of our smart farms based CSF architecture.

Figure 4: Cooperative Smart Farm - Threat Scenario 2

###  III-B Threat Model

####  III-B1 Scenario 1

In a CSF bloc, there are two farms – Farm A and Farm B. Farm A has various
sensors, smart devices, robust data collection procedures in place,
sophisticated instruments that collects high-quality data with regard to crop
yield per hectare, soil moisture, water quality, irrigation schedules etc.
Farm B does not have such advanced sensors, devices, and methods to collect
good quality data. Both farms share data with a cloud ML model with the
instruments they have installed on their farms. Figure 3 shows this scenario.

The data collection (what metrics are collected), storage (where are they
storing and where – IoT Hub or Edge computing servers, or cloud) and sharing
schedule (data is sent hourly, or daily or weekly) may be different or
standardized for different participating farms. The cloud ML model uses data
across different farms and trains model to make predictions regarding
pesticide application, seeds needed per hectare, minimum irrigation levels
needed to ensure good crop yield, predict scarcity of soil nutrients etc. The
ML model shares the data with all the farms irrespective of their size or
quality of the data contribution. In this scenario, Farm A has good devices,
data collected from those devices, and resources; however, Farm B does not
have good farm devices, data, and resources. Therefore, the ML model should
process the data from Farms A and Farm B and then provide outputs/insights
based on the quality of data that each farm shared rather than providing both
with the same insights, as is the general case with the ML model learning.
Figure 3 shows both current learning and insights sharing mechanism with solid
green lines, and our proposed mechanism with dashed lines.

####  III-B2 Scenario 2

Figure 4 depicts the second threat scenario where one of the farms could have
malicious intents. Here, the scenario also consists of two farms, Farm A and
Farm B, where Farm A is a technologically advanced farm compared to Farm B and
Farm A provides better quality data to the ML model, similar to scenario 1.
Now, Farm B decides to act rogue and stop sharing the entire data it collects,
or does not replace outdated or broken IoT devices, or Farm B has malicious
intentions and does not collect good quality data. For instance, the malicious
intent can either be competitors trying to sabotage a CSF and smart farms
participating in that CSF, or data injection attacks from hackers for ruining
crops for other farms and their ML model outputs. In scenario 1, Farm B still
gets the ML model’s prediction without actually cooperating and participating
in the smart farming architecture. In scenario 2, Farm B actively tries to
sabotage the whole CSF infrastructure by compromising the data source from
which the ML model is trained. With our game theory-based fair strategy
approach (described in later section) proposed in this paper, we try to solve
this problem and ensure that the participating smart farms are equal and
honest in all respects. As a consequence of providing high-quality and true
data, the smart farms receive models and their outputs/insights that
accurately represent the aggregate learning from CSF bloc.

###  III-C Rationality Assumption

Prior research in smart farming [46] have presented cyber-attacks on smart
farming infrastructure where farmers or IoT devices are controlled by
adversary. Malicious participants could arbitrarily deviate from suggested
protocol in CSF or could arbitrarily drop communication between edge gateways
and cloud. However, here we assume that farms are honest in a selfish
environment. In the context of this paper, rationality means that a rational
farm earns maximum profit by collaboration or individual modeling in CSF.

##  IV Cooperative Smart Farming Game

In this section, we present a game model of CSF with multiple member farms in
selfish environment. This game model involves with $N$-players and is
addressing the threat scenario 1 as discussed in III. This game model refers
as a cooperative smart farming game G. In this game, IoT devices send their
generated data to centralized cloud where data supports all the member farms
through ML models. Co-op allows to aggregate only that data, which is
generated from members of CSF and builds ML model for AI-assisted
applications. This updated global ML model provide insight to all
participating member farms via AI-assisted application, where exists a social
dilemma for all defection behavior.

Table 1 shows game approach between two players, one has high-quality data and
other has low-quality data. List of symbols are shown in Table 2.

###  IV-A Game Theoretic Model

Game theory allows for modeling situations of conflict and for predicting the
behavior of participants when they interact with each other. In CSF game G,
farms which sign a co-op agreement to be part of CSF are participants. They
interact with each other through co-op cloud level without having any
knowledge about each other. The Game G is a static game, because all
participants must choose their strategy. The Game G is a tuple $(P,S,U)$,
where $P$ is the set of players, $S$ is the set of strategies and $U$ is the
set of payoff values.

  * •

Players ($P$): The set of players $P=\sum_{i=1}^{N}P_{i}$ corresponds to the
set of member farms which signed co-op agreement and allows to send generated
data from their IoT devices through edge gateways to build ML model in CSF
game G.

  * •

Strategy ($S$): Each participant $P_{i}$ can choose between two actions
$s_{i}$ (i) Cooperative ($CP$) or (ii) Defective ($DF$). Hence the set of
strategies in this game is $S$ = {$CP$,$DF$}. Strategy of each participant
$P_{i}$ determines whether $P_{i}$ participates in CSF. In particular, if a
participant $P_{i}$ plays $CP$ strategy, i.e., the member farm allows IoT
devices to send the generated data to centralized cloud and extract model
results via AI-assisted applications. In cooperative strategy, participant
pays full cost. In contrast, if a participant $P_{i}$ neither sends the
generated data through IoT devices to the cloud nor utilize the global ML
model, i.e., the member farm $P_{i}$ plays $DF$ strategy. Thus, participants
saves various costs including membership costs $c^{o}$, communication costs
$c^{m}$, $c^{m}{}^{{}^{\prime}}$ and storage cost $c^{s}$. Here, this
participant is not part of CSF and builds ML model individually.

  * •

Payoff ($U$): The goal of each participants in CSF game G is to maximize their
utility, which is a function of the accuracy and its costs. In this game, we
do not consider any malicious or threat activities from players; hence, the
accuracy based on AI assisted application is benefit for each player. The
players pay various costs including communication costs $c^{m}$,
$c^{m^{\prime}}$, storage cost $c^{s}$, and membership cost $c^{o}$, and also
pay penalty $c^{p}$ if any player breaks co-op membership agreement. Total
cost $c_{i}^{t}$ to participate in CSF can be characterized as

$c_{i}^{t}=c^{o}+c^{p}+c^{m}+c^{m^{\prime}}+c^{s}$ (1)

TABLE I: Game Table Between Two Players Game Options |  Low Data Farm |  High Data Farm
---|---|---
Low Data Farm |  Less robust model - Train model on (Low-Low) farms. |  Cancel model training and move to find next pair of (High-High) farms.
High Data Farm |  Cancel model training and move to find next pair of (High-High) farms |  Robust model - Trained on matched sets on (High-High) farms.
TABLE II: List of Symbols Symbol | Definition
---|---
$N$ | Number of member farms
$n$ | Total number of IoT devices at farm
$D\textsubscript{i}$ | Generated data from IoT device i
$P$ | Players
$S$ | Set of strategies
$U$ | Total payoff
$B$ | Coefficient
$a\textsuperscript{co-op}$ | Accuracy of model based on co-op level
$a$ | Accuracy of model without co-op
$c\textsuperscript{o}$ | Membership cost for co-op agreement
$c\textsuperscript{p}$ | Penalty to break co-op agreement
$c\textsuperscript{m}$ | Communication cost to send data to cloud
$c\textsuperscript{m'}$ | Communication cost to download updated model from cloud
$c\textsuperscript{s}$ | Storage cost to store data on cloud
$c^{plocal}$ | Computation cost to build model locally
$c_{i}\textsuperscript{t}$ | Total cost for participating in cooperative smart farming
$C_{i}$ | Number of cooperative participants
$N-C_{i}$ | Number of defective participants
Figure 5: A Sample of Model Attributes Figure 6: Proposed Implementation of
Cooperative Smart Farming (CSFs) in AWS cloud

Here, the benefit and the cost are not on the same scale as the first depends
on the accuracy of ML model while the latter on different types of associated
costs. To make them comparable, we introduce a coefficient: the benefit is
multiplied with B.

Now, we compute the payoff of player $P_{i}$ in this game. If we assume that
the player $P_{i}$ is cooperative, i.e., $P_{i}$ $\in$ $CP$. Similarly, if
$P_{i}$ is defective, i.e., $P_{i}$ $\in$ $DF$, and these payoffs can be
defined as follows.

$u_{i}(CP)=B(a\textsuperscript{co-
op})-(c^{o}+c^{p}+c^{m}+c^{m^{\prime}}+c^{s})$ (2)

$u_{i}(DF)=B(a)-(c^{plocal})$ (3)

Based on the above calculated utilities, we analyze the game G as discussed in
the game analysis.

The most fundamental game-theoretic concept, Nash Equilibrium, which is
introduced by John Nash [47] is used to solve strategic behavior of
participants.

Definition 1. A Nash Equilibrium is a concept of game theory where none of the
players can unilaterally change their strategy to increase their payoff.

In a non-cooperative game, players do not have any motivation to deviate
unilaterally from the given strategy. The prisoner’s dilemma is a standard
example to analyze a game and shows that two rational players cooperate, come
out with best outcome, and the players do not cooperate with one another, then
they choose defecting strategy in the hope of attaining individual gain at the
rival’s expense. In prisoners’ dilemma defecting strategy strictly dominates
the cooperation strategy. Hence, the only Nash Equilibrium in prisoners’
dilemma, is a mutual defection.

Based on the cost and benefit of player to build ML model, we build a CSF game
model G. In the following theorems, we show that the CSF game G is a public
good game.

Theorem 1. In cooperative smart farming game G, if each participant builds its
local machine learning model individually, then we can establish All-Defective
strategy profile as a Nash Equilibrium.

Proof. Let us consider all N participants follow defective-$DF$ strategy where
all participants neither send their generated data from IoT devices through
gateways to cloud, nor upload any updated ML model/extract any results from
cloud, i.e., no membership fees $c^{o}$, no communication between IoT devices
and cloud. So, participants do not pay any communication costs $c^{m}$,
$c^{m^{\prime}}$, and storage cost $c^{s}$. Now each participant $P_{i}$
trains local data sets $D_{i}$ to build its ML model individually and pays
only computation cost $c^{plocal}$. None of participants cannot change his
strategy profile unilaterally. Let us assume if a participant deviates from
defect-$DF$ strategy to cooperate-$CP$ strategy unilaterally, then participant
will pay some costs ($c^{o}$\+ $c^{m}$ \+ $c^{m^{\prime}}$ +$c^{s}$ \+
$c^{p}$), which is greater than $c^{plocal}$. The payoff of cooperate-$CP$
strategy is less than defect-$DF$ strategy, so All-$DF$ is a Nash equilibrium
profile and G is a public good game.

Theorem 2 shows that we can never enforce an All-Cooperative $CP$ strategy in
game G, and therefore, we could not establish a Nash Equilibrium.

Theorem 2. In cooperative smart farming game G, if each participant builds its
local ML model, then we cannot establish All-Cooperative strategy profile as a
Nash Equilibrium.

Proof. We first assume that all N participants are members of CSF (i.e., All-
cooperative $CP$ strategy profile) and pays mandatory costs (membership cost,
communication costs, and storage cost) and optional cost (penalty cost). We
can compute the payoff of each participant $P_{i}$ by Equation (2). Hence, if
a participant will deviate from the cooperation and play defection
unilaterally, its payoff would be equal to Equation (3), which is always
greater than cooperative payoff. Hence, each participant has incentive to
deviate unilaterally and increases its payoff. Then, the All cooperate-$CP$
strategy profile is never a Nash Equilibrium.

##  V Proposed Approach - A Fair Strategy

We present the proposed model to ensure member farms in CSF can be maintained
with a fair-game strategy. Figure 3 and Figure 4 present CSF models of use
case threat scenario 1 and scenario 2, which we discussed in section III. Our
goal is to design a mechanism for eliciting cooperation in CSF and solves the
issue discussed in scenario 1 where one farm (Farm A) has advanced and good
quality devices and provide good quality data. The proposed fair strategy
enforces member farms for cooperation in CSF; however, Theorem 1 and 2 proves
that member farm defect from CSF game, i.e., after some number of iterations,
member farm will not be part a of CSF if they will not get any benefit from
other member farms. The quality of each dataset of each participant is
calculated based on data attributes at co-op level. The generated data is
classified using Support Vector Machine (SVM) algorithm and a result of sample
data attributes is shown in Figure 5. This shows different categories of data
items and hierarchy of attributes - soil data, irrigation data, yield data,
and sharing schedule for data and resources. Each of these categories are
further broken down into other attributes that provide detailed information
about these attributes. For instance, soil data include moisture and nutrient
content, irrigation data include water quality and watering schedule, etc.
Based on such identified attributes for a ML model, we can define the data
being collected and transmitted by smart farms to the CSF is quality data or
not.

As shown in Table 1, the best strategy is to classify the smart farms based on
the quality of data they provide to the model and co-op, such as high-quality
data farms should participate in high-quality CSF, and low-quality data farms
can be grouped together and should participate in low-quality CSF. We believe
that this strategy will allow the smart farms to co-operate in CSFs and take
maximum benefit from it, and can reduce the problem of defective farms.
However, further research on data attributes for ML models to efficiently
classify the smart farms is necessary. We plan to focus on this aspect in our
future work. In the context of the CSF game G, before the start of the game,
each participant has to choose his strategy between $CP$ and $DG$ to play the
CSF game G. However, in the beginning of this game, each participant is in
dilemma to choose strategy, which can depend on other participant’s strategy.
In CSF, each member farm does not know about the quality and quantity of other
participant’s data. Here, we propose a novel fair strategy employing K-means
clustering at co-op cloud level. K-means clustering is an unsupervised ML
technique, whose purpose is to segment a data set into K clusters. The co-op
cloud will create clusters based on quality and quantity of their dataset to
enforce in CSF game G. Algorithm 1 shows our proposed strategy using data
attributes (which would be defined for qualitative and quantitative data by
co-ops) and K-means clustering. Using this algorithm and classified dataset,
we can generate sustainable CSFs without the issue defective farms.

Algorithm 1 Proposed Fair Strategy

1: Each participants send data $D_{i}$ to co-op cloud level.

2: The data is classified based on data attributes using Support Vector
Machine (SVM) algorithm.

3: The classified data is tested on build-in ML model to calculate accuracy of
each data set.

4: Apply k-means clustering algorithm on accuracy value of each participant i.

5: create the clusters at co-op level.

6: if participant i belongs a cluster with at least one other participant j
then

7: $P_{i}$ , $P_{j}$ $\in$ $CP$

8: else

9: $P_{i}$ $\in$ $DF$

10: end if

##  VI Proposed Implementation

In this section, we present a proposed implementation for CSF using proposed
fair strategy utilizing AWS cloud and IoT platform, shown in Figure 6. In CSF,
various smart member farms collaborate with each other. As shown in Figure 6,
the data is collected from different smart member farms, which need to be
classified using SVM algorithm. This proposed implementation utilizes AWS IoT
core222https://docs.aws.amazon.com/iot/latest/developerguide/what-is-aws-
iot.html in cloud, AWS Greengrass333https://aws.amazon.com/greengrass/ running
on edge gateways (e.g., Raspberry Pi enabled gateway) that can enable edge
computing as required, and IoT devices/things corresponding to specific use
cases requirements. The edge gateway can be enabled using Raspberry
Pi444https://www.raspberrypi.org/ that host the AWS Greengrass deployment, and
customized lambda555https://aws.amazon.com/lambda/ functions that can run on
the gateway for edge communication and computation, sending notifications, or
enforcing access control and privacy policies. AWS Greengrass allows farmers
to securely connect their smart devices at the edge of the network to gather
data. When connectivity is re-established, the data will synchronize with AWS
cloud and its IoT services using MQTT/HTTP protocols.

For enabling a secure architecture, AWS IoT core provides mutual
authentication based on certificates and encryption for secure data transfer
and also enable different access control levels, which have been discussed in
[27, 28]. The data can be stored at AWS cloud storage and be analyzed at the
co-op level for classified member farms based on the quality and quantity of
their data attributes. The member farms cluster will be created using proposed
fair strategy at co-op cloud level, then the member farms will be part of CSF.
AWS IoT analytic service can be used to analyze the data using specific ML
models and provide meaningful insights to the member smart farms through the
smart farm applications as shown in the top layer of the proposed
implementation architecture.

##  VII Conclusion

In this paper, we present a system model of cooperative smart farm (CSF) and
introduce the problem of strategic behavior of farmers in CSF. We analyze
rational behavior of member farms in CSF using game theory model. We also
analyze the Nash Equilibrium strategy profile for each scenario, where member
farms are enforced to cooperate based on some data attributes identified by
CSFs and using our proposed fair strategy in CSF. This work presents an
extensive understanding of non-cooperative behavior of member farms in CSF.
For future work, we plan to do further research on data attributes
identification for ML models that can be used by CSFs to better classify the
smart farms based on the quality and quantity of data. We also plan to
implement the model with farming data set and evaluate the accuracy of ML
model using proposed fair strategy.

## References

  * [1] J.-c. Zhao, J.-f. Zhang, Y. Feng, and J.-x. Guo, “The study and application of the iot technology in agriculture,” in _2010 3rd International Conference on Computer Science and Information Technology_ , vol. 2. IEEE, 2010, pp. 462–465.
  * [2] S. H. Hallett, “Smart cities need smart farms,” 2017.
  * [3] D. Gupta, S. Bhatt, M. Gupta, and A. S. Tosun, “Future smart connected communities to fight covid-19 outbreak,” _arXiv preprint arXiv:2007.10477_ , 2020.
  * [4] M. Gupta, M. Abdelsalam, S. Khorsandroo, and S. Mittal, “Security and privacy in smart farming: Challenges and opportunities,” _IEEE Access_ , vol. 8, pp. 34 564–34 584, 2020.
  * [5] S. S. L. Chukkapalli, A. Piplai, S. Mittal, M. Gupta, A. Joshi _et al._ , “A Smart-Farming Ontology for Attribute Based Access Control,” in _6th IEEE International Conference on Big Data Security on Cloud (BigDataSecurity 2020)_ , 2020.
  * [6] M. Colezea, G. Musat, F. Pop, C. Negru, A. Dumitrascu, and M. Mocanu, “Cluefarm: Integrated web-service platform for smart farms,” _Computers and Electronics in Agriculture_ , vol. 154, pp. 134–154, 2018\.
  * [7] D. L. Chinn, “Cooperative farming in north china,” _The Quarterly Journal of Economics_ , vol. 94, no. 2, pp. 279–297, 1980.
  * [8] K. P. Satamraju, K. Shaik, and N. Vellanki, “Rural bridge: A novel system for smart and co-operative farming using iot architecture,” in _2017 International Conference on Multimedia, Signal Processing and Communication Technologies (IMPACT)_. IEEE, 2017, pp. 22–26.
  * [9] A. Kamilaris, F. Gao, F. X. Prenafeta-Boldu, and M. I. Ali, “Agri-iot: A semantic framework for internet of things-enabled smart farming applications,” in _2016 IEEE 3rd World Forum on Internet of Things (WF-IoT)_. IEEE, 2016, pp. 442–447.
  * [10] M. Bacco, A. Berton, E. Ferro, C. Gennaro, A. Gotta, S. Matteoli, F. Paonessa, M. Ruggeri, G. Virone, and A. Zanella, “Smart farming: Opportunities, challenges and technology enablers,” in _2018 IoT Vertical and Topical Summit on Agriculture-Tuscany (IOT Tuscany)_. IEEE, 2018, pp. 1–6.
  * [11] P. Lottes, R. Khanna, J. Pfeifer, R. Siegwart, and C. Stachniss, “Uav-based crop and weed classification for smart farming,” in _2017 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2017, pp. 3024–3031.
  * [12] P. Tripicchio, M. Satler, G. Dabisias, E. Ruffaldi, and C. A. Avizzano, “Towards smart farming and sustainable agriculture with drones,” in _2015 International Conference on Intelligent Environments_. IEEE, 2015, pp. 140–143.
  * [13] M. Jhuria, A. Kumar, and R. Borse, “Image processing for smart farming: Detection of disease and fruit grading,” in _2013 IEEE Second International Conference on Image Information Processing (ICIIP-2013)_. IEEE, 2013, pp. 521–526.
  * [14] M. Ryu, J. Yun, T. Miao, I.-Y. Ahn, S.-C. Choi, and J. Kim, “Design and implementation of a connected farm for smart farming system,” in _2015 IEEE SENSORS_. IEEE, 2015, pp. 1–4.
  * [15] M. J. O’Grady and G. M. O’Hare, “Modelling the smart farm,” _Information processing in agriculture_ , vol. 4, no. 3, pp. 179–187, 2017.
  * [16] S. Wolfert, L. Ge, C. Verdouw, and M.-J. Bogaardt, “Big data in smart farming–a review,” _Agricultural Systems_ , vol. 153, pp. 69–80, 2017.
  * [17] N. G. Rezk, E. E.-D. Hemdan, A.-F. Attia, A. El-Sayed, and M. A. El-Rashidy, “An efficient iot based smart farming system using machine learning algorithms,” _Multimedia Tools and Applications_ , pp. 1–25, 2020.
  * [18] R. Varghese and S. Sharma, “Affordable smart farming using iot and machine learning,” in _2018 Second International Conference on Intelligent Computing and Control Systems (ICICCS)_. IEEE, 2018, pp. 645–650.
  * [19] I. Sa, Z. Chen, M. Popović, R. Khanna, F. Liebisch, J. Nieto, and R. Siegwart, “weednet: Dense semantic weed classification using multispectral images and mav for smart farming,” _IEEE Robotics and Automation Letters_ , vol. 3, no. 1, pp. 588–595, 2017.
  * [20] P. Porambage, M. Ylianttila, C. Schmitt, P. Kumar, A. Gurtov, and A. V. Vasilakos, “The quest for privacy in the internet of things,” _IEEE Cloud Computing_ , vol. 3, no. 2, pp. 36–45, 2016.
  * [21] M. Gupta, F. M. Awaysheh, J. Benson, M. Al Azab, F. Patwa, and R. Sandhu, “An attribute-based access control for cloud-enabled industrial smart vehicles,” _IEEE Transactions on Industrial Informatics_ , 2020.
  * [22] F. M. Awaysheh, M. Alazab, M. Gupta, T. F. Pena, and J. C. Cabaleiro, “Next-generation big data federation access control: A reference model,” _Future Generation Computer Systems_ , 2020.
  * [23] A. Ouaddah, H. Mousannif, A. Abou Elkalam, and A. A. Ouahman, “Access control in the Internet of Things: Big challenges and new opportunities,” _Computer Networks_ , vol. 112, pp. 237–262, 2017.
  * [24] A. Alshehri and R. Sandhu, “Access control models for virtual object communication in cloud-enabled IoT,” in _2017 IEEE International Conference on Information Reuse and Integration (IRI)_. IEEE, 2017, pp. 16–25.
  * [25] S. Bhatt, F. Patwa, and R. Sandhu, “An access control framework for cloud-enabled wearable internet of things,” in _2017 IEEE 3rd International Conference on Collaboration and Internet Computing (CIC)_. IEEE, 2017, pp. 328–338.
  * [26] S. Bhatt and R. Sandhu, “Abac-cc: Attribute-based access control and communication control for internet of things,” in _Proceedings of the 25th ACM Symposium on Access Control Models and Technologies_ , 2020, pp. 203–212.
  * [27] S. Bhatt, F. Patwa, and R. Sandhu, “Access control model for aws internet of things,” in _International Conference on Network and System Security_. Springer, 2017, pp. 721–736.
  * [28] D. Gupta, S. Bhatt, M. Gupta, O. Kayode, and A. S. Tosun, “Access control model for google cloud iot,” in _2020 IEEE 6th Intl Conference on Big Data Security on Cloud (BigDataSecurity), IEEE Intl Conference on High Performance and Smart Computing,(HPSC) and IEEE Intl Conference on Intelligent Data and Security (IDS)_. IEEE, 2020, pp. 198–208.
  * [29] S. Bhatt, A. T. Lo’ai, P. Chhetri, and P. Bhatt, “Authorizations in cloud-based internet of things: Current trends and use cases,” in _2019 Fourth International Conference on Fog and Mobile Edge Computing (FMEC)_. IEEE, 2019, pp. 241–246.
  * [30] A. S. Patil, B. A. Tama, Y. Park, and K.-H. Rhee, “A framework for blockchain based secure smart green house farming,” in _Advances in Computer Science and Ubiquitous Computing_. Springer, 2017, pp. 1162–1167.
  * [31] M. N. Aladwan, F. M. Awaysheh, S. Alawadi, M. Alazab, T. F. Pena, and J. C. Cabaleiro, “Truste-vc: Trustworthy evaluation framework for industrial connected vehicles in the cloud,” _IEEE Transactions on Industrial Informatics_ , vol. 16, no. 9, pp. 6203–6213, 2020.
  * [32] L. Barreto and A. Amaral, “Smart farming: Cyber security challenges,” in _2018 International Conference on Intelligent Systems (IS)_. IEEE, 2018, pp. 870–876.
  * [33] T. Gundu and V. Maronga, “Iot security and privacy: Turning on the human firewall in smart farming,” in _Proceedings of 4th International Conference on the_ , vol. 12, 2019, pp. 95–104.
  * [34] C. Krintz, R. Wolski, N. Golubovic, B. Lampel, V. Kulkarni, B. Sethuramasamyraja, B. Roberts, and B. Liu, “Smartfarm: Improving agriculture sustainability using modern information technology,” in _KDD Workshop on Data Science for Food, Energy, and Water_ , 2016.
  * [35] C.-J. Chae and H.-J. Cho, “Enhanced secure device authentication algorithm in p2p-based smart farm system,” _Peer-to-Peer Networking and Applications_ , vol. 11, no. 6, pp. 1230–1239, 2018.
  * [36] L. M. Bruce and D. Reynolds, “Game theory based data fusion for precision agriculture applications,” in _2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS)_. IEEE, 2016, pp. 3563–3566.
  * [37] J. L. Dillon, “Applications of game theory in agricultural economics: Review and requiem,” _Australian Journal of Agricultural Economics_ , vol. 6, no. 429-2016-29355, pp. 20–35, 1962.
  * [38] O. L. Walker, “Game theory applications in agricultural decisions,” 1959.
  * [39] S. Cabrera García, J. E. Imbert Tamayo, J. Carbonell Olivares, and Y. Pacheco Cabrera, “Application of the game theory with perfect information to an agricultural company,” _Agricultural Economics (AGRICECON)_ , vol. 59, no. 1, pp. 1–7, 2013.
  * [40] D. Gupta, O. Kayode, S. Bhatt, M. Gupta, and A. S. Tosun, “Learner’s Dilemma: IoT Devices Training Strategies in Collaborative Deep Learning,” in _2020 IEEE 6th World Forum on Internet of Things (WF-IoT)_. IEEE, 2020, pp. 1–6.
  * [41] M. Gupta, J. Benson, F. Patwa, and R. Sandhu, “Secure V2V and V2I communication in intelligent transportation using cloudlets,” _arXiv preprint arXiv:2001.04041_ , 2020.
  * [42] D. Barton, “What is a cooperative,” _Cooperatives in agriculture_ , pp. 1–20, 1989.
  * [43] C. Berner, M. Holmes, A. Reynolds, and J. Rinehart, “Successful cooperative ownership transitions,” _Center Cooperatives, Univ. Wisconsin-Madison, Madison, WI, USA, Tech. Rep_ , vol. 1, 2014.
  * [44] S. S. L. Chukkapalli, S. Mittal, M. Gupta, M. Abdelsalam, A. Joshi, R. Sandhu, and K. Joshi, “Ontologies and Artificial Intelligence Systems for the Cooperative Smart Farming Ecosystem,” _IEEE Access_ , vol. 8, pp. 164 045–164 064, 2020.
  * [45] “USDA,” https://www.rd.usda.gov/programs-services/all-programs/cooperative-programs., [Online; accessed 21-February-2020].
  * [46] S. Sontowski, M. Gupta, S. S. L. Chukkapalli, M. Abdelsalam, S. Mittal, A. Joshi, and R. Sandhu, “Cyber Attacks on Smart Farming Infrastructure,” _UMBC Student Collection_ , 2020.
  * [47] J. Nash, “Non-cooperative games,” _Annals of mathematics_ , pp. 286–295, 1951\.