# ScissionLite: Accelerating Distributed Deep Neural Networks Using Transfer
Layer

Hyunho Ahn, Munkyu Lee, Cheol-Ho Hong, and Blesson Varghese Hyunho Ahn, Munkyu
Lee, and Cheol-Ho Hong (corresponding author) are with the School of
Electrical and Electronics Engineering, Chung-Ang University, Seoul, Korea.
e-mail<EMAIL_ADDRESS>Blesson Varghese is with with the School of
Electronics, Electrical Engineering and Computer Science, Queen’s University
Belfast, United Kingdom.

###### Abstract

Industrial Internet of Things (IIoT) applications can benefit from leveraging
edge computing. For example, applications underpinned by deep neural networks
(DNN) models can be sliced and distributed across the IIoT device and the edge
of the network for improving the overall performance of inference and for
enhancing privacy of the input data, such as industrial product images.
However, low network performance between IIoT devices and the edge is often a
bottleneck. In this study, we develop ScissionLite, a holistic framework for
accelerating distributed DNN inference using the Transfer Layer (TL). The TL
is a traffic-aware layer inserted between the optimal slicing point of a DNN
model slice in order to decrease the outbound network traffic without a
significant accuracy drop. For the TL, we implement a new lightweight
down/upsampling network for performance-limited IIoT devices. In ScissionLite,
we develop ScissionTL, the Preprocessor, and the Offloader for end-to-end
activities for deploying DNN slices with the TL. They decide the optimal
slicing point of the DNN, prepare pre-trained DNN slices including the TL, and
execute the DNN slices on an IIoT device and the edge. Employing the TL for
the sliced DNN models has a negligible overhead. ScissionLite improves the
inference latency by up to 16 and 2.8 times when compared to execution on the
local device and an existing state-of-the-art model slicing approach
respectively.

###### Index Terms:

Edge Computing; IIoT; Deep Neural Networks; Model Slicing; Inference

##  1 Introduction

Edge computing is a comparatively new computing paradigm that leverages
computation resources at the edge of a network. A local device that offloads
its workload to the edge is then able to achieve lower latency than cloud
computing [1, 2, 3]. Edge computing is currently gaining growing interest in
Industrial Internet of Things (IIoT) applications. IIoT applications usually
employ different types of industrial resources through the network to
construct a service-oriented industrial ecosystem [4]. Edge computing in IIoT
allows data generated from industrial machines, devices, and sensors to be
processed with low latency and thus enables time-critical decision-making.

Recent advances in edge computing for IIoT are best exemplified by deep neural
network (DNN) model slicing for product surface inspection [5]. This research
employs a surface inspection camera on an automated assembly line and captures
the surface of products one by one for DNN inference. However, as IIoT devices
have limited computation capability, the inspection time can be continuously
delayed, resulting in a significant decline of outcomes. To address this
issue, the research splits the entire DNN model and distributes each slice
into the device and the edge. Leveraging both device and edge resources for
DNN inference has the benefit of preserving privacy for input data and also
reducing inference time.

_Scission_ [6], our previous work, generates a list of slicing points of a DNN
model against different user-defined objectives for minimizing inference time.
Scission automatically benchmarks the DNN model on the device, edge, and cloud
resource. It then analyzes the benchmark data including the execution time of
respective DNN layers and the data transfer time. Scission decides the optimal
point for model slicing based on the following criteria: (1) Computationally
intensive layers should be placed on a capable resource, and (2) minimum data
transfer is required between the device, edge, and cloud. Scission discovers
the intersection point of the two policies that is best suited for the lowest
inference time.

Scission can achieve optimal performance for distributed deep learning
inference. However, the overhead of data transfer between the local and the
edge is still problematic in IIoT applications. IIoT now endeavors to exploit
multi-access edge computing (MEC) that enables cloud computing at the edge of
the cellular network [4, 7, 8]. 5G cellular networks provide high download
bandwidth ranging from 600 to 1,700 Mbps [9]. However, the upload speed is
comparatively low (30 – 60 Mbps) owing to weak cellular radio power on
devices. Therefore, when the data of an IIoT device is uploaded to the multi-
access edge, the limited network bandwidth can be a significant bottleneck.

Although existing research presents methods to reduce the data transfer amount
between the device and the edge by applying a data encoding method to the
split point of the DNN [10, 11, 12, 13], they are limited in the following
ways: (1) The methods are usually optimized for a very slow network
environment (e.g., 50 Kbps to 20 Mbps) [10, 13]. Therefore, it is not clear
whether the proposed methods would suit faster 5G networks. (2) The studies
adopt compression techniques that are not integrated into the original DNN
[10] or that incur high computational complexity [11, 12]. In these cases, the
compression techniques would be a further bottleneck for performance-limited
IIoT devices. (3) The methods do not find the optimal slicing point when the
traffic is reduced since they are based on estimations rather than empirical
data. (4) The studies are validated on lightweight DNN models such as
MobileNetV2 and VGG16 with relatively few layers and small datasets [10, 11,
12] such as CIFAR111https://www.cs.toronto.edu/~kriz/cifar.html. Small models
and input images may not be suitable for IIoT applications that use high-
quality images as input and demand high inference accuracy.

In this paper, we propose _ScissionLite_ , a framework to accelerate edge-
based distributed deep learning inference for IIoT. ScissionLite inserts an
additional traffic-aware layer called the Transfer Layer (TL) between the
optimal slicing point of the DNN model in order to decrease the amount of data
transferred between the device and the edge without significantly sacrificing
the accuracy of inference. The TL embeds a small neural network between the
split point that compresses the output data of the IIoT device and expands the
compressed data on the edge. For this purpose, we develop a new lightweight TL
using a down/upsampling network that is integrated into the original DNN for
performance-limited IIoT devices.

ScissionLite is a holistic framework for edge-based distributed inference that
consists of ScissionTL, the Preprocessor, and the Offloader. ScissionTL is a
Scission-based tool for automated benchmarking that can decide the optimal
slicing point when the traffic is reduced by the TL. The Preprocessor creates
and retrains a new DNN model that employs the TL. Finally, the Offloader is a
runtime platform that distributes deep learning inference into the IIoT device
and the edge. For evaluation, we utilize five large-scale pre-trained
production DNNs, namely DenseNet169, DenseNet201, ResNet101,
InceptionResNetV2, and InceptionV3222https://keras.io/api/applications/, with
the ImageNet dataset333https://image-net.org in an emulated 5G environment. It
is noted that using TL introduces a negligible overhead. ScissionLite can
increase the performance of deep learning inference by up to 16 and 2.8 times
compared to the local device execution and original Scission respectively with
0.9 – 1.4% of an accuracy drop.

The contributions of this article are summarized as follows:

  * •

We design ScissionLite and perform its evaluation using large-scale production
DNNs with a high-precision data set in an emulated 5G environment. To the best
of our knowledge, this is the first work that shows the feasibility of
transfer layer-based distributed DNNs in a realistic edge-based IIoT
environment.

  * •

We implement a new lightweight TL using a down/upsampling neural network
suitable for low power processors in IIoT devices. This simple network is
seamlessly integrated into the target DNN, so that it neither requires
specialized hardware for processing nor shows high computational complexity.

  * •

We develop ScissionTL, which performs automated benchmarking for determining
the optimal slicing point of the DNN model when the traffic is decreased by
the TL. ScissionTL accurately finds the slicing point based on empirical data
whereas other studies find an approximate solution with estimations.

The rest of this paper is organized as follows: Section 2 presents the
background and related work. Section 3 presents the design of ScissionLite.
Section 4 shows the performance evaluation results. Finally, we present our
conclusions in Section 5.

##  2 Background and Related Work

Figure 1: Profiling, slicing, benchmarking and ranking in Scission

In this section, we present the background research on which ScissionLite is
developed and then present an overview of the related research.

###  2.1 Scission

In ScissionLite, the transfer layer based benchmarking tool, namely ScissonTL,
is based on Scission444Scission is available at https://github.com/qub-
blesson/Scission [6] that determines the optimal slicing point for distributed
deep learning inference. The following design choices drove the development of
Scission: (1) DNN slicing should be possible across different resource tiers
in the device-edge-cloud continuum, (2) DNN slicing must identify optimally
performing slices that can be distributed across resources, (3) DNN slicing
must be rapid and based on empirical data rather than estimations because
layers are extremely performance sensitive that cannot be easily predicted
with estimations, and (4) DNN slicing should account for user-defined
constraints, such as the target overall latency.

Scission is underpinned by a methodology involving profiling, slicing,
benchmarking, and ranking as shown in Figure 1 and described below:

Profiling: The DNN is profiled to find suitable slicing points. All layers in
the sequential DNN will be a slicing point. However, for a DNN model in which
there are parallel branches, the parallel paths will need to be identified.
The parallel branches are not sliced, but the layers within the branch are
considered as a block.

Slicing: This step ensures that the DNN is sliced into distinct sub-models
with individual layers or blocks for the purposes of benchmarking.

Benchmarking: Each layer or block is benchmarked against hardware resources
for obtaining the average execution time and the time taken to transfer data
between the layers and blocks.

Ranking: The DNN slices are ranked based on user-defined constraints, and a
suitable DNN slice configuration can be chosen for deployment across the edge-
cloud environment.

Although Scission supports different resource tiers including the cloud, the
focus in this paper is on the device and edge tiers. Current processors offer
powerful computation resources such as GPUs, and therefore they are leveraged
at the edge without sending data to the cloud for better inference performance
[14].

###  2.2 Related Work

Slicing and distributing DNNs across a combination of the device, edge, or
cloud resources have performance benefits [15, 14]. Slicing approaches rely on
identifying a sequence of layers and mapping them onto resources so that the
distributed DNN is optimized against the overall latency, ingress bandwidth,
or a combination of these.

Slicing requires the identification of the optimal slicing point, which in the
literature is based on four different approaches. The first is an estimation-
based approach in which the slicing point is determined based on an estimation
of the performance of the layers on a target hardware platform [16, 17, 18].
The second is based on integer linear programming, which is usually time
consuming with the aim of minimizing inference latency and maximizing
inference accuracy [19, 20]. The third is based on a structural modification
to the DNN. Although this approach is effective, they can be intrusive methods
requiring modification of the underlying libraries [21, 22]. However, more
fine-grain control over DNN slices can be obtained using this approach.
Finally, benchmarking based approaches for slicing DNNs are employed. These
measurement-based approaches gather empirical data on the performance of each
layer from target hardware resources. Scission, an underlying approach in
ScissionTL presented in this paper, adopts a benchmarking-based approach [6].

Given that ScissionLite uses the TL that is inserted in the DNN split point
and is based on Scission, ScissionLite is a hybrid of structural modification-
based and benchmarking-based DNN slicing approaches.

Existing research presents methods to reduce the traffic amount in distributed
deep learning inference. BottleNet [10] compresses the output layer of a
mobile device by using conventional compressors such as JPEG before sending
the data to the cloud. The research was evaluated with ResNet50 and VGG19 in
3G, 4G, and WiFi environments that offer 1.1, 5.85, and 18.88 Mbps
respectively for the upload bandwidth. Data compressors based on an
autoencoder is presented [11]. An autoencoder is an artificial neural network
duplicating its inputs to the outputs with several convolutional layers and
incurs additional computational complexity to the original DNN. This study
utilized MobileNetV2 with CIFAR.

Compared to existing research, ScissionLite offers a lightweight compression
technique using a down/upsampling network and a method to obtain the optimal
slicing point in response to the network traffic change. In addition, it is
evaluated by large-scale DNN models such as DenseNet201 and ResNet101 with the
ImageNet dataset in a 5G environment.

##  3 ScissionLite

Figure 2: Overall design of ScissionLite

In this section, we present the architecture of ScissionLite, which is a
framework to accelerate edge-based distributed deep learning inference for
IIoT.

###  3.1 Overall Design

ScissionLite inserts an additional traffic-aware layer called the Transfer
Layer (TL) between the split point of the DNN. The TL decreases the amount of
data transfer between the two entities by adopting a raw data encoding method.
ScissionLite decides the optimal split point of the target DNN model
considering the transfer amount reduced by the TL and injects the TL into the
split point. For this purpose, ScissionLite develops three components
including ScissionTL, the Preprocessor, and the Offloader as shown in Figure
2. ScissionTL is a Scission-based tool for automated benchmarking that can
decide the model’s optimal slicing point when the TL is applied. The
Preprocessor generates a new DNN model called the _TLModel_ that embeds the TL
in the optimal slice point. The Offloader is a runtime platform that can
deploy the TLModel and enable distributed inference utilizing both device and
edge resources.

###  3.2 Transfer Layer

The Transfer Layer (TL) is a small neural network layer that is embedded
between the split point of the DNN for reducing the amount of data exchanged
during communication. The TL is composed of the _DeviceTL_ and _EdgeTL_ layers
respectively for the device and edge resources as shown in Figure 2. The
DeviceTL layer compresses the feature maps of the end layer of the sliced DNN
on the IIoT device. The compressed data is then transferred through the
cellular network connection. The EdgeTL layer expands the received data and
passes the data to the starting layer of the remaining DNN on the edge. In
this manner, the TL can decrease the amount of data transferred without
significantly sacrificing inference accuracy.

The TL can be implemented by using an existing raw data encoding method such
as a traditional image compression technique [23] (e.g., JPEG compression) or
an autoencoder-based coding scheme [24]. These methods are a good candidate
for realizing communication-efficient inference in edge-based machine learning
systems. However, we identify that the former generally requires specialized
hardware based on an application-specific integrated circuit (ASIC) to boost
the compression and expansion process [25], and the latter demands significant
training time for adapting the autoencoder and computation resources during
inference due to the complexity of the autoencoder [24]. Since IIoT devices
may have relatively performance-limited resources, a more efficient method is
required for the TL.

To address this issue, we develop a new lightweight TL for IIoT using a
down/upsampling neural network [26], which can be performed on low power
processors. With this thin TL, we can achieve acceptable training speed,
inference time, and top-5 accuracy, which will be explained in Section 4.
Although we present the design of the TL in the context of our own
down/upsampling network, the TL is not specific to it and can be applied to
other data encoding methods. Our implementations of the DeviceTL and EdgeTL
layers are as follows:

Figure 3: Max pooling and upsampling of a neural network

The DeviceTL layer compresses the output of the local IIoT device to a quarter
by adopting a downsampling network. We implement the DeviceTL as a max pooling
layer [26], which sub-samples (or shrinks) each feature map of the last layer
of the IIoT device. Each neuron in the DeviceTL is connected to a fewer number
of neurons inside a small rectangular kernel in the last DNN layer as shown in
Figure 3. We configure the kernel size as $2\times$2 and select the maximum
value in the kernel as an output value in the DeviceTL. Other neurons lower
than the max value are dropped. We apply a stride of 2 with no padding to move
the kernel back and forth across the feature map. The DeviceTL thus obtains
sub-sampled feature maps whose size is a quarter of the last layer. A larger
kernel size can improve the compression ratio but may incur a significant
accuracy drop owing to information loss.

The EdgeTL layer expands the received data from the IIoT device by an
upsampling neural network. The lost information caused by max pooling is
replenished with a nearest-neighbor interpolation method, which selects the
value from the nearest pixel and copies the selected value into the dropped
neurons. This algorithm is computationally efficient. We have also considered
other interpolation methods including bilinear and bicubic interpolation, but
they resulted in an accuracy degradation than the nearest-neighbor
interpolation.

###  3.3 ScissionTL

As described in Section 2.1, Scission automatically benchmarks DNN models on a
target set of the device, edge, and cloud resources for deciding the optimal
slicing point for maximizing inference performance. In this study, we develop
_ScissionTL_ , which is a Scission-based tool for automated benchmarking that
can acquire the optimal slicing point when the TL is used. This section
presents a comparative analysis on Scission and ScissionTL and how ScissionTL
finds the optimal slicing point.

The TL is an additional neural network layer that demands computation
resources such as CPUs and GPUs during inference. The execution time for
processing the TL is denoted by $E_{TL}$. As the TL is composed of the
DeviceTL and the EdgeTL, $E_{TL}$ is calculated as follows:

$\displaystyle E_{TL}$ $\displaystyle=T(DeviceTL(Output_{i}))$ (1)
$\displaystyle\quad+T(EdgeTL(InputTL_{i}))$

where $T(x)$ denotes the execution time of running $x$. $Output_{i}$ is the
output of the sliced DNN at splitting point $i$ in the device and becomes the
parameter for the DeviceTL functionality. $InputTL_{i}$ is the downsampled
result of $Output_{i}$ and is provided to the EdgeTL functionality.

As explained in Section 3.2, we apply a lightweight max pooling and upsampling
mechanism for the TL. Then, the computation cost, which is $E_{TL}$, is
expected to be low. However, the computation burden would be high when a
compute-intensive data encoding method such as an autoencoder is adopted.

Before network transmission, data or neurons compressed by the DeviceTL should
be serialized to an appropriate data format such as Protocol Buffers
(Protobuf) or JavaScript Object Notation (JSON). At the edge, the received
data will be deserialized. This requires also computation resources. The
execution time regarding (de)serialization, $S_{TL}$, is calculated as
follows:

$\displaystyle S_{TL}$ $\displaystyle=T(Serial(OutputDown_{i}))$ (2)
$\displaystyle\quad+T(DeSerial(InputDownTL_{i}))$

where $Serial(OutputDown_{i})$ serializes $OutputDown_{i}$, the down sampled
result of $Output_{i}$, in the device, and $DeSerial(InputDownTL_{i})$
deserializes $InputDownTL_{i}$, the serialized result of $OutputDown_{i}$, in
the edge.

Original Scission’s (de)serialization time, $S_{Orig}$, is calculated in the
same way as follows:

$\displaystyle S_{Orig}$ $\displaystyle=T(Serial(Output_{j}))$ (3)
$\displaystyle\quad+T(DeSerial(InputOrig_{j}))$

where $Output_{j}$ is the output of the sliced DNN at splitting point $j$ in
the device, and $InputOrig_{j}$ is the serialized result of $Output_{j}$.

The communication time between the device and the edge is denoted by $C_{TL}$,
when the TL is applied. $C_{TL}$ is then calculated as follows:

$C_{TL}=Latency+\frac{Size(OutputDown_{i})}{Bandwidth}$ (4)

where $Size(x)$ denotes the size of $x$, the network latency is $Latency$, and
the network bandwidth is $Bandwidth$.

When the TL is not applied in original Scission, the communication time,
$C_{Orig}$, is calculated as follows:

$C_{Orig}=Latency+\frac{Size(Output_{j})}{Bandwidth}$ (5)

Both communication times are affected by inherent network latency and the data
transfer time, which can be obtained by dividing the size of the data by the
network bandwidth. In the 5G cellular network, the upload bandwidth is as low
as between 30 and 60 Mbps. Therefore, to decrease communication time, it is
essential to reduce the amount of data transfer.

Consequently, $\Delta t$, which is the benefit of adopting the TL, is
calculated as follows:

$\Delta t=(S_{Orig}+C_{orig})-(E_{TL}+S_{TL}+C_{TL})$ (6)

The performance improvement that will be achieved is shown in Section 4.

As presented in Figure 1, original Scission executes all possible split points
individually for deciding the optimal slicing point. In addition to this,
ScissionTL embraces the above-mentioned parameters including $E_{TL}$,
$S_{TL}$, and $C_{TL}$ for each slicing point. In the benchmarking and ranking
phases, ScissionTL calculates the three parameters and uses these values to
find the optimal slicing point.

###  3.4 Preprocessor

The Preprocessor in ScissionLite prepares a pre-trained DNN model that employs
the TL using the following three components: the Inserter, the Trainer, and
the Splitter.

Inserter: The Inserter injects the TL in the split point of the DNN model. We
have utilized pre-trained models from Keras
Applications555https://keras.io/api/applications/, which provide various DNN
models with pre-trained weights. The Inserter loads the target model and
divides the model into two for putting the TL between them. Afterward, it
joins the sub-models together for retraining. The new model created by the
Inserter is called the _TLModel_.

Trainer: The Trainer retrains the TLModel for preventing an accuracy drop. We
use a pre-trained model from Keras Applications, but the pre-trained weights
are not aware of the TL, which can lead to a significant accuracy drop.
Existing neurons then need to learn how to cooperate with new neurons in the
TL. As the TL is lightweight, existing layers can be adapted to the TL fast
during retraining. The top-5 accuracy and retraining time of the TLModel will
be presented in Section 4.3.

Splitter: The Splitter divides the TLModel into two for the IIoT device and
the edge. For the device-side model, the Splitter piles up the layers until
the $DeviceTL$ layer and exports them as a single Keras model. Similarly, the
edge-side model starts from the $EdgeTL$ layer and ends at the last fully-
connected layer.

###  3.5 Offloader

The Offloader in ScissionLite is an inference platform that deploys the
TLModel on the device and the edge and establishes the connection between
them. It consists of the Runtime and the Communicator.

Runtime: In the IIoT device, we use the TensorFlow runtime to execute the
sliced TLModel. On the edge side, we utilize the NVIDIA Triton inference
server666https://developer.nvidia.com/nvidia-triton-inference-server, which is
open-source serving software and optimized for edge deployment, for executing
the remaining TLModel. The output data of $DeviceTL$ is transferred to the
$EdgeTL$ layer in Triton through the Communicator. After the inference task is
completed in the edge, the result is returned to the IIoT device again using
the Communicator.

Communicator: The Communicator connects each Runtime in the IIoT device and
the edge. Both REST and gRPC protocols can be utilized as a communication
method, but we adopted gRPC because it is based on HTTP/2 and is faster than
REST with HTTP/1.1. gRPC uses Protocol Buffers (Protobuf) as the message
interchange format. The Communicator implements the Device and Edge converters
in order to serialize the output of $DeviceTL$ to Protobuf and deserialize the
Protobuf to the input data of $EdgeTL$.

##  4 Evaluation

We present the performance evaluation results of ScissionLite in this section.

###  4.1 Experimental Environment

TABLE I: Test bed configurations Figure 4: Best performance of various DNN
models obtained from the CPUDevice, GPUDevice, and GPUDevice-GPUEdge
configurations Figure 5: Slice-by-slice analysis of DenseNet169 in Scission,
ScissionTL, and ScissionLite with four test bed configurations. Figure 6:
Slice-by-slice analysis of ResNet101 in Scission, ScissionTL, and ScissionLite
with four test bed configurations.

We implemented ScissionLite with an IIoT device and an edge server for
distributed inference. As the IIoT device, we used an NVIDIA Jetson TX2, which
is an embedded AI computing device that equips two dual-core and quad-core ARM
CPUs and an NVIDIA Pascal-family GPU having 8 GB of memory. The edge server is
an Intel Xeon E5-2698 CPU platform with twenty 2.2 GHz cores and an NVIDIA
GeForce RTX 3090 GPU having 24 GB of memory. The IIoT device is directly
connected to the edge server via 1 GbE link. We adjusted the upload bandwidth
and latency of the network connection as 30 – 60 Mbps and 30 ms respectively
by exploiting the Linux Traffic Control [27] tool. These values emulate the
commercial 5G network environment based on the latest research [9].

We employed the TensorFlow runtime version 2.3 in the IIoT device and NVIDIA
Triton as an inference server in the edge. We obtained deep learning models
with pre-trained weights from Keras
Applications777https://keras.io/api/applications/. We used five pre-trained
production DNNs, namely DenseNet169, DenseNet201, ResNet101,
InceptionResNetV2, and InceptionV3 for evaluation. We also employed ImageNet
[28] as an image database. The test bed configurations reflecting an
operational IIoT environment are presented in Table I. We combined possible
hardware resource types in the IIoT device and the edge server.

###  4.2 Inference Latency

Figure 7: Transferred data amount of DenseNet169 and ResNet101 at each split
point in Scission and ScissionLite. Figure 8: Intermediate representations in
front layers of ResNet101 Figure 9: Inference latency of DenseNet201 when the
network bandwidth is given as 60 and 30 Mbps respectively.

ScissionLite accelerates distributed deep learning inference by inserting the
TL between the split point. ScissionLite first obtains the optimal split point
from ScissionTL for minimizing inference latency and runs the sliced DNN model
with the Offloader.

Feasibility of distributed inference: Figure 4 shows the best inference
latency of the five DNN models respectively executed in the CPUDevice,
GPUDevice, and GPUDevice-GPUEdge configurations in Table I. The first two
configurations do not employ DNN model slicing so that the DNN models were run
in the local IIoT device using its embedded CPU and GPU. The last
configuration utilizes DNN model slicing with ScissionLite. As shown in the
figure, ScissionLite improves the performance of the DNN models up to 16 and
5.7 times compared to the CPUDevice and GPUDevice configurations respectively.
This result shows that DNN model slicing is effective in edge-based IIoT.

Transfer Layer (TL) computation overhead: ScissionLite adopts the TL to
decrease the volume transferred to a quarter of the size. The TL compresses
the output of the IIoT device by using a downsampling network and expands the
compressed data with an upsampling network at the edge. The scheme is
computationally lightweight for an IIoT device. The DeviceTL takes at most 300
$\mu$s on the device GPU and 2,500 $\mu$s on the device CPU. The EdgeTL takes
at most 200 $\mu$s on the edge server GPU. Therefore, the overhead of adopting
the TL is low.

Slice-by-slice analysis: In this experiment, we provide a slice-by-slice
analysis of DenseNet169 in Figure 5 and ResNet101 in Figure 6 in Scission,
ScissionTL, and ScissionLite with four test bed configurations. We represent
the Offloader in ScissionLite, which is an actual runtime platform, as
ScissionLite in the following explanations. The upload bandwidth and network
latency are configured as 57 Mbps and 28 ms respectively, which are the peak
performance of commercial 5G [9].

Each sub-figure presents the total end-to-end latency when the split point or
layer changes from the first to the last layer; less important split points
are not shown in the figure. In the case of ScissionLite, the profiling result
in each split point is shown as a stacked column chart. The optimal split
points decided by Scission are denoted by a red circle and square.
ScissionTL’s decisions are shown by the blue symbols.

As shown in Figure 5 and 6, the latency values of ScissionTL and ScissionLite
are converged across the split points in each test bed configuration. This
means that both ScissionTL and ScissionLite are synchronized. Therefore, the
optimal point suggested by ScissionTL without testing all split points in the
production platform is trustworthy. This is useful when the network
environment or a user-defined constraint is changed in the IIoT system. For
example, when the network bandwidth of 5G is significantly increased,
ScissionTL can decide on a new slicing point rapidly with existing benchmarked
results. In this case, the new DNN can be deployed to the IIoT production
platform without further testing.

Based on the optimal points suggested by Scission and ScissionTL in
DenseNet169 and ResNet101, ScissionLite performs faster than Scission, because
ScissionLite adopts the traffic-aware layer between the split point.
ScissionLite can improve the performance of distributed inference by up to
2.8, 2.6, 2.8, and 2.5 times in DenseNet169 respectively with the CPUDevice-
CPUEdge, CPUDevice-GPUEdge, GPUDevice-CPUEdge, and GPUDevice-GPUEdge
configurations, and 2.31, 2.42, 2.24, and 2.25 times in ResNet101 with the
same configurations, compared to Scission.

In all configurations, both Scission and ScissionTL suggest Layer 1 as the
optimal point in DenseNet169 and ResNet101. This is because the amount of data
transfer at Layer 1 is the smallest among early and middle layers as shown in
Figure 7. Scission and ScissionTL then decide to process most of DNN layers at
the edge server to exploit more powerful CPUs and GPUs.

Transferring the output of Layer 1 can benefit from privacy preservation than
sending the original image to the edge [6]. However, when the output of a
later layer is sent to the edge, security and privacy can be enhanced. As the
split layer is higher, the intermediate representations contain less
information on the original image as shown in Figure 8.

To enhance privacy, we impose a user constraint by configuring the split point
as five or more; this value can be changed according to the DNN. In most
cases, Scission suggests a later layer as the optimal point (red square
symbol) compared with ScissionTL (blue square) in DenseNet169 and ResNet101.
Scission considers the significantly increased data amount right after Layer 1
(Figure 7), and investigates later points that can meet both low traffic
overhead and computation offloading benefit. However, ScissionTL is less
affected by the data traffic amount due to the TL, and therefore it suggests
an earlier layer than Scission. For example, ScissionTL recommends Layer 51,
whereas Scission proposes Layer 139 in the GPUDevice-GPUEdge configuration in
DenseNet169.

Different network bandwidth: Figure 9 shows the inference latency of
DenseNet201 with the GPUDevice-GPUEdge configuration when the network
bandwidth is given as 60 and 30 Mbps respectively. Scission shows increased
latency between Layer 2 and 50 at 30 Mbps owing to large data transfer (Figure
7). However, ScissionLite shows stable performance even with limited bandwidth
owing to the TL.

###  4.3 Inference Accuracy

TABLE II: Top-5 accuracy of the original Keras model and the retrained TLModel
in ScissionLite

In ScissionLite, the TL implements a down/upsampling neural network, which is
computationally efficient for IIoT devices. However, as the feature maps of
the last DNN layer are subsampled in the DeviceTL, the TL causes some
information loss. In order to prevent an accuracy drop, the Trainer retrains
the split model embedding the TL (TLModel), as explained in Section 3.4. The
Trainer employs the stochastic gradient descent (SGD) optimizer with 0.001 of
a learning rate for retraining. The TLModel is adapted to the new neurons in
the TL during retraining and converges to minimum loss rapidly. This takes
about between 5 and 10 hours with a single NVIDIA GeForce RTX 3090 GPU. The
training speed can be boosted by using multiple GPUs in a cloud data center.

Table II compares the top-5 accuracy of the original Keras model and the
retrained TLModel in ScissionLite. Both accuracy values were measured with the
ImageNet validation dataset888https://keras.io/api/applications/. The result
shows that the retrained TLModel provides acceptable inference accuracy with
just 0.9 – 1.4% of a drop. This highlights that the TL relieves the
communication overhead without deteriorating the overall accuracy
significantly.

##  5 Conclusions

Edge computing offers the distinct advantage of bringing compute resources
closer to devices and sensors where data is generated for reducing the overall
application latency. The Industrial Internet of Things (IIoT) is one area that
benefits from using the edge. Given that IIoT devices may have relative
resource limitations when compared to the edge, workloads can be offloaded
from the device to the edge to accelerate data processing. For example, Deep
Neural Network (DNN) applications used in IIoT can be sliced and distributed
across the IIoT device and the edge for reducing the total inference time.

One of the key challenges when slicing DNNs and distributing them is finding
the optimal slicing point while accounting for the inter-layer data transfer
sizes. An optimal slicing point may require a large amount of data to be
transferred between the DNN slices on the device and the edge. This is
problematic when faced with variable network conditions between the device and
the edge.

This paper presents an end-to-end traffic-aware framework, namely ScissionLite
that finds the optimal DNN slicing point but at the same time reduces the
amount of data transferred between the slices across the device and the edge.
A novel method that uses a lightweight transfer layer using a down/upsampling
network for performance-limited IIoT devices is developed.

ScissionLite is tested on five major production DNNs in a lab-based physical
test bed comprising GPUs. The experimental results obtained highlight that DNN
inference can improve performance by up to 16 times compared with the local
execution while only incurring a 0.9% – 1.4% accuracy drop. The use of the
transfer layer is shown to have a negligible overhead. ScissionLite
demonstrates the feasibility of the transfer layer approach for accelerating
DNN inference for IIoT applications.

## Acknowledgments

The authors are grateful to the anonymous reviewers for their valuable
comments and suggestions. This work was supported by the National Research
Foundation of Korea (NRF) grant funded by the Korean government (MSIP) (No.
NRF-2019R1C1C1011068). The last author was supported by a Royal Society Short
Industry Fellowship.

## References

  * [1] C.-H. Hong and B. Varghese, “Resource management in fog/edge computing: a survey on architectures, infrastructure, and algorithms,” _ACM Computing Surveys (CSUR)_ , vol. 52, no. 5, pp. 1–37, 2019.
  * [2] C.-H. Hong, K. Lee, M. Kang, and C. Yoo, “qcon: Qos-aware network resource management for fog computing,” _Sensors_ , vol. 18, no. 10, p. 3444, 2018\.
  * [3] N. Wang, B. Varghese, M. Matthaiou, and D. S. Nikolopoulos, “Enorm: A framework for edge node resource management,” _IEEE Transactions on Services Computing_ , 2017.
  * [4] T. Qiu, J. Chi, X. Zhou, Z. Ning, M. Atiquzzaman, and D. O. Wu, “Edge computing in industrial internet of things: Architecture, advances and challenges,” _IEEE Communications Surveys & Tutorials_, vol. 22, no. 4, pp. 2462–2488, 2020.
  * [5] L. Zeng, E. Li, Z. Zhou, and X. Chen, “Boomerang: On-demand cooperative deep neural network inference for edge intelligence on the industrial internet of things,” _IEEE Network_ , vol. 33, no. 5, pp. 96–103, 2019.
  * [6] L. Lockhart, P. Harvey, P. Imai, P. Willis, and B. Varghese, “Scission: Performance-driven and context-aware cloud-edge distribution of deep neural networks,” in _Proceedings of the 13th IEEE/ACM 13th International Conference on Utility and Cloud Computing_ , 2020, pp. 257–268.
  * [7] X. Hou, Z. Ren, K. Yang, C. Chen, H. Zhang, and Y. Xiao, “Iiot-mec: A novel mobile edge computing framework for 5g-enabled iiot,” in _Proceedings of the IEEE Wireless Communications and Networking Conference_ , 2019, pp. 1–7.
  * [8] W. Sun, J. Liu, Y. Yue, and H. Zhang, “Double auction-based resource allocation for mobile edge computing in industrial internet of things,” _IEEE Transactions on Industrial Informatics_ , vol. 14, no. 10, pp. 4692–4701, 2018.
  * [9] A. Narayanan, E. Ramadan, J. Carpenter, Q. Liu, Y. Liu, F. Qian, and Z.-L. Zhang, “A first look at commercial 5g performance on smartphones,” in _Proceedings of the Web Conference 2020_ , 2020, pp. 894–905.
  * [10] A. E. Eshratifar, A. Esmaili, and M. Pedram, “Bottlenet: A deep learning architecture for intelligent mobile cloud computing services,” in _Proceedings of the IEEE/ACM International Symposium on Low Power Electronics and Design_ , 2019, pp. 1–6.
  * [11] D. Hu and B. Krishnamachari, “Fast and accurate streaming cnn inference via communication compression on the edge,” in _Proceedings of the 5th IEEE/ACM International Conference on Internet-of-Things Design and Implementation_ , 2020, pp. 157–163.
  * [12] J. Shao and J. Zhang, “Bottlenet++: An end-to-end approach for feature compression in device-edge co-inference systems,” in _Proceedings of the IEEE International Conference on Communications Workshops_ , 2020, pp. 1–6.
  * [13] Y. Matsubara, M. Levorato, and F. Restuccia, “Split computing and early exiting for deep learning applications: Survey and research challenges,” _arXiv preprint arXiv:2103.04505_ , 2021.
  * [14] X. Wang, Y. Han, V. C. Leung, D. Niyato, X. Yan, and X. Chen, “Convergence of edge computing and deep learning: A comprehensive survey,” _IEEE Communications Surveys & Tutorials_, vol. 22, no. 2, pp. 869–904, 2020.
  * [15] Z. Zhou, X. Chen, E. Li, L. Zeng, K. Luo, and J. Zhang, “Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing,” _Proceedings of the IEEE_ , vol. 107, no. 8, pp. 1738–1762, 2019.
  * [16] Y. Kang, J. Hauswald, C. Gao, A. Rovinski, T. Mudge, J. Mars, and L. Tang, “Neurosurgeon: Collaborative Intelligence Between the Cloud and Mobile Edge,” in _Proceedings of the 22nd International Conference on Architectural Support for Programming Languages and Operating Systems_ , 2017, pp. 615–629.
  * [17] M. Xu, F. Qian, M. Zhu, F. Huang, S. Pushp, and X. Liu, “DeepWear: Adaptive Local Offloading for On-Wearable Deep Learning,” _IEEE Transactions on Mobile Computing_ , vol. 19, no. 2, pp. 314–330, 2020\.
  * [18] R. Hadidi, J. Cao, M. Woodward, M. S. Ryoo, and H. Kim, “Musical chair: Efficient real-time recognition using collaborative iot devices,” _arXiv preprint arXiv:1802.02138_ , 2018.
  * [19] H. Li, C. Hu, J. Jiang, Z. Wang, Y. Wen, and W. Zhu, “JALAD: Joint Accuracy and Latency-aware Deep Structure Decoupling for Edge-Cloud Execution,” in _Proceedings of the 24th IEEE International Conference on Parallel and Distributed Systems_ , 2018, pp. 671–678.
  * [20] A. E. Eshratifar, M. S. Abrishami, and M. Pedram, “JointDNN: An Efficient Training and Inference Engine for Intelligent Mobile Cloud Computing Services,” _IEEE Transactions on Mobile Computing_ , 2019.
  * [21] Z. Zhao, K. M. Barijough, and A. Gerstlauer, “DeepThings: Distributed Adaptive Deep Learning Inference on Resource-Constrained IoT Edge Clusters,” _IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems_ , vol. 37, no. 11, pp. 2348–2359, 2018.
  * [22] J. Mao, X. Chen, K. W. Nixon, C. Krieger, and Y. Chen, “MoDNN: Local Distributed Mobile Computing System for Deep Neural Network,” in _Proceedings of the Design, Automation Test in Europe Conference Exhibition_ , 2017, pp. 1396–1401.
  * [23] Z. Liu, T. Liu, W. Wen, L. Jiang, J. Xu, Y. Wang, and G. Quan, “Deepn-jpeg: A deep neural network favorable jpeg-based image compression framework,” in _Proceedings of the 55th Annual Design Automation Conference_ , 2018, pp. 1–6.
  * [24] D. B. Kurka and D. Gündüz, “Deep joint source-channel coding of images with feedback,” in _Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing_ , 2020, pp. 5235–5239.
  * [25] J. H. Ko, T. Na, M. F. Amir, and S. Mukhopadhyay, “Edge-host partitioning of deep neural networks with feature space encoding for resource-constrained internet-of-things platforms,” in _Proceedings of the 15th IEEE International Conference on Advanced Video and Signal Based Surveillance_ , 2018, pp. 1–6.
  * [26] A. Giusti, D. C. Cireşan, J. Masci, L. M. Gambardella, and J. Schmidhuber, “Fast image scanning with deep max-pooling convolutional neural networks,” in _Proceedings of the IEEE International Conference on Image Processing_ , 2013, pp. 4034–4038.
  * [27] B. Hubert, T. Graf, G. Maxwell, R. van Mook, M. van Oosterhout, P. Schroeder, J. Spaans, and P. Larroy, “Linux advanced routing & traffic control,” in _Ottawa Linux Symposium_ , vol. 213, 2002.
  * [28] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2009, pp. 248–255.

|  Hyunho Ahn received the B.E. degree in Electrical and Electronics
Engineering from Chung-Ang University, Seoul. He is now a master student in
the School of Electrical and Electronics Engineering, Chung-Ang University,
Seoul, Korea. His current research interests include edge computing and deep
learning.
---|---
|  Munkyu Lee received the B.E. and M.E. degrees in Electrical and Electronics
Engineering from Chung-Ang University, Seoul. He is now a Ph.D. candidate in
the School of Electrical and Electronics Engineering, Chung-Ang University,
Seoul, Korea. His current research interests include system virtualization,
deep learning, and GPU architectures.
---|---
|  Cheol-Ho Hong received the BS, MS, and PhD degrees in Computer Science from
Korea University, Seoul. He worked as a Research Professor in Korea University
from 2013 to 2015 and as a Research Fellow in Queen’s University Belfast from
2015 to 2018. He is now an Associate Professor in the School of Electrical and
Electronics Engineering, Chung-Ang University, Seoul, Korea. His current
research interests include system virtualization, edge computing, deep
learning, and GPU architectures.
---|---
|  Blesson Varghese received the PhD degree in Computer Science from the
University of Reading, UK. He worked as a researcher at the Dalhousie
University, Canada, and the University of St Andrews previously. He is now a
Senior Lecturer (Associate Professor) in the School of Electronics, Electrical
Engineering and Computer Science, Queen’s University Belfast, UK and a Royal
Society Short Industry Fellow. His current research interests include
developing and analyzing state-of-the-art cloud-edge systems and distributed
machine learning. More information at www.blessonv.com.
---|---


# Equilibrium thermodynamic properties of binary hard-sphere mixtures from
integral equation theory

Banzragch Tsednee  Tsogbayar Tsednee  Tsookhuu Khinayat  Institute of Physics
and Technology, Mongolian Academy of Sciences
Peace Ave 54B, 13330 Ulaanbaatar, Mongolia

###### Abstract

We present an equilibrium thermodynamic properties of binary hard-sphere
mixtures from integral equation approach combined with the Percus-Yevick (PY)
and the Martynov-Sarkisov (MS) approximations. We use the virial, the
compressibility and the Boublík-Mansoori-Carnahan-Starling-Leland (BMCSL)
equations of state in the PY approximation, while the virial equation of state
is only employed in the MS approximation. We employ a closed-form expression
for evaluating the excess chemical potential. The excess Helmholtz free energy
is obtained using the Euler relation of thermodynamics. For a number of binary
sets of the mixtures we compare our findings for thermodynamic properties with
previously obtained results in the literature. Generally, the findings from
the MS approximation show better agreement with the results than those from
the PY approximation.

##  I Introduction

The hard-sphere (HS) model plays a major role in the development of the modern
theory of liquids Hansen_06 . This system is often used as a standard
reference system in the perturbation approach of the liquids for studying the
liquid-state properties McQuarre_76 ; Hansen_06 . Mixtures of the HSs of
different size distributions can be employed to model the colloidal
suspensions which play an important role in chemical and bioengineering fields
Heno91 ; Pago01 .

The binary HS mixture is the simplest model of the multicomponent systems.
Computer simulations for its thermodynamic and structural properties began in
the mid-1960s. Adler Adler64  used the molecular dynamic (MD) simulation to
solve the equation of state (EOS) for the binary mixture. Rotenberg
Rotenberg65  performed the Monte-Carlo (MS) EOS calculation for a mixture of
HSs. An alternative approach to study a liquid is the integral equation (IE)
method in which the IE combined with approximate closures is solved to predict
the structure and to obtain thermodynamic properties. Mansoori et al.
Mansoori71  investigated the EOS for the mixture of the HSs using the MC and
MD simulations, and proposed an analytical expression for the EOS using the
solution the PY integral equation PY58 . Barošová et al. Barosova96  applied a
test particle insertion method to obtain the excess chemical potential of the
binary HS mixtures. Santos and his co-workers have investigated extensively
thermodynamic properties and structural nature for multicomponent HS fluids in
terms of the analytical approaches from the IE method Santos13 ; Heyes18  and
MD simulation Santos20 . Moreover, Ballone et al. Ballone86  tested the PY
approximation for the HS mixtures. Schmidt Schmidt92  and Malijevský et al.
Malijevsky97  applied the Martynov-Sarkisov (MS) Martynov83  closure for the
binary mixtures as well.

In the present article our goal is to report on thermodynamic properties for
binary HS mixtures at equilibrium using the IE approach combined with the PY
and the MS approximations. We employ the virial and the compressibility
routes, and the Boublík-Mansoori-Carnahan-Starling-Leland (BMCSL) Boublik70 ;
Mansoori71  approach to calculate pressure in the PY approximation. The virial
EOS is used to obtain pressure in the MS approximation. In terms of the IE
methodology, the excess chemical potential can be obtained with an analytical
expression based on the correlation functions Tsog19 ; Banz22 . Such a closed-
form expression is employed for computing the excess chemical potential. The
excess Helmholtz free energy is obtained from thermodynamic relation. Then we
compare our numerical results for thermodynamic properties with those obtained
with the MC Barosova96  and the MD Heyes18  simulations.

The organization of this paper is as follows. In Section II we discuss
theoretical formulation of the IE method for multicomponent mixture and
thermodynamic quantities which we will compute. In Section III we present our
results and discussion, which is followed by conclusions.

##  II Integral equation theory

For multicomponent mixtures we consider the site-site Ornstein-Zernike (SSOZ)
equation which establishes a relation between the total correlation function
and the direct correlation function. For binary systems with the total number
density $\rho$, the SSOZ equation has a form

$\displaystyle
h_{ij}(\mathbf{r})=c_{ij}(\mathbf{r})+\rho\sum^{2}_{k=1}x_{k}\int
d\mathbf{r}^{\prime}c_{ik}(\mathbf{r}-\mathbf{r}^{\prime})h_{kj}(\mathbf{r}^{\prime})$
(1)

where $h_{ij}(r)$ and $c_{ij}(r)$ are the total and direct correlation
functions, respectively, and $x_{i}=\rho_{i}/\rho$ is the mole fraction for
the component $i$ with $\sum_{i=1}^{2}x_{i}=1$.

Since the SSOZ equation (1) contains two unknown correlation functions, it
cannot be solved directly. In order to solve for these functions, an another
equation, called a closure relation must be introduced, which couples the
total and direct correlation functions with the pair interaction potential. A
general closure equation for the mixture may be written in the form

$h_{ij}(r)=\exp[-\beta u_{ij}(r)+\gamma_{ij}(r)+B_{ij}(r)]-1\qquad(i,j=1,2).$
(2)

Here $u_{ij}(r)$ is an interaction potential between particles in the system,
$\gamma_{ij}\equiv h_{ij}-c_{ij}$ is an indirect correlation function,
$B_{ij}(r)$ is the bridge function, $\beta=1/k_{\mathrm{B}}T$,
$k_{\mathrm{B}}$ is the Boltzmann constant and $T$ is the temperature for the
system.

The total and direct correlation functions can be found by numerical solutions
of equation (1) and equation (2) provided that $u_{ij}(r)$, $T$ and
$B_{ij}(r)$ are known.

For HS mixture, a form of an interaction potential in this work is given by

$u_{ij}(r)=\begin{cases}\infty,&\quad r<\sigma_{ij},\\\ 0,&\quad
r\geq\sigma_{ij},\end{cases}$ (3)

with $\sigma_{ij}=\frac{1}{2}(\sigma_{i}+\sigma_{j})$ and
$\sigma_{ii}=\sigma_{i}$ is the hard sphere diameter for the component $i$.

Since there is not an exact known expression for the bridge function, an
approximated functions can be used for it, such the PY and MS approximations.
In the PY approximation PY58  the bridge function is approximated by

$B_{ij}(r)=\ln(1+\gamma_{ij})-\gamma_{ij},$ (4)

and in the MS approximation Martynov83  by

$B_{ij}(r)=(1+2\gamma_{ij})^{1/2}-\gamma_{ij}-1.$ (5)

###  II.1 Thermodynamic quantities

####  II.1.1 Pressure

For binary HS mixture, we use a following virial (v) EOS

$\frac{\beta
p_{v}}{\rho}=1+\frac{2\pi}{3}\rho\sum^{2}_{i,j=1}x_{i}x_{j}\sigma^{3}_{ij}g_{ij}(\sigma_{ij}),$
(6)

where $g_{ij}(\sigma_{ij})$ are the contact values of the radial distribution
functions. In the PY approximation the pressure from the compressibility (c)
route can be computed with an analytical expression Hiroike69

$\displaystyle\frac{\beta p^{\mathrm{PY}}_{c}}{\rho}$ $\displaystyle=$
$\displaystyle 1+\frac{1}{2\rho}\sum^{2}_{i,j=1}\rho_{i}\rho_{j}\int
d\mathbf{r}c_{ij}(g_{ij}-2)$
$\displaystyle+\frac{1}{8\pi^{3}}\frac{1}{\rho}\int
d\mathbf{k}[\mathrm{Tr}(R\hat{C})+\ln\mathrm{det}|I-R\hat{C}|],$

where $R$ is a diagonal matrix based on density of each component and
$\hat{C}$ is $2\times 2$ matrix whose element is $\hat{c}_{ij}(k)$. Here
$\hat{c}_{ij}(k)$ is the Fourier transform of the direct correlation function
$c_{ij}(r)$ and $I$ is an identity matrix. A $\mathrm{det}$ means the
determinant of the matrix.

In the PY approximation one may also employ a rather accurate BMCSL EOS
Boublik70 ; Mansoori71  which is basically an interpolation between above two
expressions:

$\frac{\beta p^{\mathrm{BMCSL}}}{\rho}=\frac{1}{3}\frac{\beta
p^{\mathrm{PY}}_{v}}{\rho}+\frac{2}{3}\frac{\beta p^{\mathrm{PY}}_{c}}{\rho}.$
(8)

####  II.1.2 An excess chemical potential

For the excess chemical potential for the component $i$, we use a closed-form
expression Tsog19 ; Banz22

$\beta\mu^{e}_{i}\approx\beta\mu^{\mathrm{HNC}}_{i}+\sum^{2}_{j=1}\rho_{j}\int
d\mathbf{r}\Big{(}B_{ij}+\frac{2h_{ij}}{3}B_{ij}\Big{)},$ (9)

where $\beta\mu^{\mathrm{HNC}}_{i}$ is the hypernetted chain (HNC)-type
expression for the excess chemical potential with appropriate bridge function
$B_{ij}$,

$\beta\mu^{\mathrm{HNC}}_{i}=\sum_{j=1}^{2}\rho_{j}\int
d\mathbf{r}\Big{(}\frac{1}{2}h^{2}_{ij}-c_{ij}-\frac{1}{2}h_{ij}c_{ij}\Big{)}.$
(10)

Note that an analytical expression (9) for the excess chemical potential can
be used for any bridge functions since it does not require the explicit forms
of them.

####  II.1.3 An excess free energy

Once we have the values of the compressibility factor $Z\equiv\beta p/\rho$
and the excess chemical potential $\beta\mu^{e}_{i}$, we can compute the
excess Helmholtz free energy per particle using a following Euler equation of
thermodynamics

$\beta a^{e}=\sum_{i=1}^{2}x_{i}\beta\mu^{e}_{i}-Z+1.$ (11)

##  III Results and Discussion

In our calculation we chose a component 1 with a diameter $\sigma_{1}$ as a
reference particle, which is a larger component of binary HS mixture, that is,
$(\sigma_{1}>\sigma_{2})$, and a packing fraction $\eta$ is given as
$\eta=(\pi/6)\rho(x_{1}\sigma^{3}_{1}+x_{2}\sigma^{3}_{2})$. A simple Picard
iterative method for solving the SSOZ equation (1) was employed, and the
numerical tolerance for the root-mean-squared residual of the indirect
correlation functions during successive was set at $10^{-8}\sigma_{1}$. All
calculations were performed with the same number of grid points, $N=16384$,
and a length parameter, $L=32\sigma_{1}$. We note that since the IE approach
is an implicit method of the statistical mechanics, we do not consider the
number of particles involved in the system.

We first performed numerical calculations for binary mixtures at values of
$\eta=0.15,0.25,0.35$ and $0.45$, and for $\sigma_{2}/\sigma_{1=0.5}$ and
$x_{1}=0.5$ values. In Table I shows numerical result for the EOSs and their
comparisons with an accurate MD values Heyes18 . Looking at the PY values in
columns 2 and 3 the virial EOS underestimates the MD values (last column),
while the compressibility EOS overestimates them. However, the BMCSL
interpolation technique presents very good values (column 4). Column 5 shows
the virial EOS values from the MS approximation, which are quite comparable
against the BMCSL EOS and MD values.

Table 1: Values of virial, compressibility, and BMCSL EOSs from the PY and MS approximations at various values of a packing fraction $\eta$, and $x_{1}=0.5$, and $\sigma_{2}/\sigma_{1}=0.5$ and MD results Heyes18 .  $\eta$ | $\beta p^{\mathrm{PY}}_{v}/\rho$ | $\beta p^{\mathrm{PY}}_{c}/\rho$ | $\beta p^{\mathrm{BMCSL}}/\rho$ | $\beta p^{\mathrm{MS}}_{v}/\rho$ |  MD Heyes18
---|---|---|---|---|---
0.15 | 1.77 | 1.78 | 1.78 | 1.77 | 1.7773
0.25 | 2.70 | 2.80 | 2.77 | 2.74 | 2.7642
0.35 | 4.23 | 4.67 | 4.54 | 4.47 | 4.5362
0.45 | 7.09 | 8.43 | 7.99 | 7.77 | 7.9623

In Table II we have compared our numerical values of the excess chemical
potential for two components with the MD values Heyes18 . In both
approximations our values were obtained with the analytical expression (9).
The MS values (columns 3 and 6) are better than the PY ones (columns 2 and 5),
and are close to the MD values (columns 4 and 7) with an increase of $\eta$.

Table 2:  The same as shown in Table 1, but for the excess chemical potential. $\eta$ |  $\beta\mu^{e}_{1}$ (PY) |  $\beta\mu^{e}_{1}$ (MS) |  MD Heyes18  |  |  |  $\beta\mu^{e}_{2}$ (PY) |  $\beta\mu^{e}_{2}$ (MS) |  MD Heyes18
---|---|---|---|---|---|---|---|---
0.15 | 2.07 | 2.08 | 2.1070 |  |  | 0.73 | 0.74 | 0.7378
0.25 | 4.37 | 4.48 | 4.5322 |  |  | 1.46 | 1.48 | 1.4684
0.35 | 8.19 | 8.57 | 8.6696 |  |  | 2.55 | 2.62 | 2.5924
0.45 | 15.6 | 16.3 | 16.1404 |  |  | 4.37 | 4.54 | 4.4099

Table III demonstrates numerical values of the excess Helmholtz free energy
calculated in the PY and MS approximations using the thermodynamic expression
(11) against the MD values Heyes18 . In evaluating $\beta a^{e}$ in the PY
approximation, we used values of $\beta p^{\mathrm{BMCSL}}/\rho$. Our results
are in good agreement with the MD values up to $\eta=0.35$. At $\eta=0.45$ our
PY value is lower than the MD value, while the MS one is slightly higher than
it. Note that our all results shown in tables are independent on computational
parameters $N$ and $L$.

Table 3:  The same as shown in Table 1, but for the excess Helmholtz free energy. $\eta$ |  $\beta a^{e}$ (PY) |  $\beta a^{e}$ (MS) |  MD Heyes18
---|---|---|---
0.15 | 0.62 | 0.64 | 0.6451
0.25 | 1.15 | 1.24 | 1.2361
0.35 | 1.83 | 2.13 | 2.0939
0.45 | 3.00 | 3.67 | 3.3129

After above calculations, we continued our numerical experiments of
thermodynamic quantities for a number of binary sets of mixtures in which one
of the three parameters (diameter ratio $\sigma_{2}/\sigma_{1}$, mole fraction
of a larger sphere $x_{1}$, and a packing fraction $\eta$) was varied while
the remaining two were kept constant. Our findings as a function of
$\sigma_{2}/\sigma_{1}$ at $\eta=0.3$ and $x_{1}=0.5$ are presented in Figure
1, while those as functions of $x_{1}$ at $\eta=0.3$ and
$\sigma_{2}/\sigma_{1}=0.5$, and $\eta$ at $\sigma_{2}/\sigma_{1}=0.5$ and
$x_{1}=0.5$ have been exhibited in Figures 2 and 3 respectively. In all plots
a dashed and solid curves are obtained from the PY and MS respectively. Black
cross in the plots denotes the MC result Barosova96 , while green ones
presents the MD data Heyes18 .

Figure 1:  Results from the PY and MS approximations are shown with a solid
and dashed curves, respectively. Plots of the excess chemical potential
$\beta\mu_{i}$ (a), and the excess Helmholtz free energy (b) and the
compressibility factor $Z-1$ (b) as a function of a diameter ratio
$\sigma_{2}/\sigma_{1}$ at $\eta=0.3$ and $x_{1}=0.5$. Black and green crosses
show the MC Barosova96  and MD Heyes18  simulations, respectively.

The plots of $\beta\mu^{e}_{1}$ for component 1 from the PY approximation
shown as dashed curves in all panels (1a-3a) show noticeable differences which
look like an expected pattern shown in Table 2. However, the PY values of
$\beta\mu^{e}_{2}$ follow the MD and MC simulations. The MS values for excess
chemical potential for two components are almost indistinguishable from the MD
and MC simulations. The compressibility factor $Z-1$ obtained from the PY and
MS approximations closely follow the MD values (panels 1b-3b). Note that the
plots of the excess Helmholtz free energy $\beta a^{e}$ from the PY
approximation show very visible differences which are related mostly to the
underestimated values of $\beta\mu^{e}_{1}$. However the MS values for $\beta
a^{e}$ are generally in good agreement with the MD data, except for the case
of a larger $\eta$ (panel 3b).

Figure 2: The same plots as shown in Figure 1, but for as a function of the
mole fraction $x_{1}$ at $\eta=0.30$ and $\sigma_{2}/\sigma_{1}=0.5$. Green
crosses show the MD Heyes18  simulations.

Figure 3:  The same plots as shown in Figure 1, but for as a function of the
packing fraction $\eta$ at $x_{1}=0.5$ and $\sigma_{2}/\sigma_{1}=0.5$. Green
crosses show the MD Heyes18  simulations.

We say that a reason why the MS values for the excess chemical potential and
compressibility factor are in good agreement with those from the MD data can
be connected to a fact that the correlation functions obtained in this
approximation might be as good as those from simulation data Banz22_num . We
here note that a similar discussion had also been given by Malijevsky et al.
Malijevsky97  in which they stated that when compared the pair distribution
functions from the MS, PY and HNC closures with the MC simulation data, the
the MS closure is the best among them.

##  IV Conclusions

In this work we have solved the SSOZ integral equation for various binary sets
of the HS mixtures using the PY and the MS approximations. We have calculated
the EOS with a virial and compressibility ways and the BMCSL technique in the
PY approximation. Obviously, the BMCSL EOS has worked better than the virial
and compressibility EOSs in this approximation. We computed the virial EOS
when the MS bridge function is used. The excess chemical potential has been
evaluated with an analytical expression while the excess Helmholtz free energy
for the system is computed with a thermodynamic expression. We have compared
our findings for thermodynamic quantities with the accurate MC and MD values.
From these comparisons, it has been shown that results from the MS
approximation are in better agreement with the MD values than those from the
PY approximation. Finally, we note that the IE theory has worked successfully
not only at moderate and high densities but also for asymmetric hard-sphere
mixtures.

##  V Acknowledgmens

This research work has been supported by the Mongolian Foundation for Science
and Technology (Project No. ShUTBIKhKhZG-2022/167).

## References

  * (1) J. P. Hansen and I. R. McDonald, Theory of Simple Liquids 4th ed. (Academic Press, New York, 2006).
  * (2) D. A. McQuarrie, Statistical Mechanics (Harper & Row, New York, 1976).
  * (3) Y. Heno and C. Regnaut, J. Chem. Phys. 95, 9204 (1991).
  * (4) I. Pagonabarraga, M. E. Cates, and G. J. Ackland, Phys. Rev. Lett. 84, 911 (2001).
  * (5) B. J. Adler, J. Chem. Phys. 40, 2724 (1964).
  * (6) A. Rotenberg, J. Chem. Phys. 43, 4377 (1965).
  * (7) G. A. Mansoori, N. F. Carnahan, K. E. Starling, and T. W. Leland, Jr., J. Chem. Phys. 54, 1523 (1971).
  * (8) J. K. Percus and G. J. Yevick, Phys. Rev. 110, 1 (1958).
  * (9) M. Barošová, A. Malikevský, S. Labík and W. R. Smith, Mol. Phys. 87, 423 (1996).
  * (10) A. Santos and R. D. Rohrmann, Phys. Rev. E 87, 052138 (2013).
  * (11) D. M. Heyes and A. Santos, J. Chem. Phys. 148, 214503 (2018).
  * (12) A. Santos, S. B. Yuste and M. L. de Haro, J. Chem. Phys. 153, 120901 (2020).
  * (13) P. Ballone, S. Pastore, G. Galli, and D. Gazzillo, Mol. Phys. 59, 275 (1986).
  * (14) A. B. Schmidt, Phys. Rev. A 45, 7636 (1992).
  * (15) A. Malikevský, M. Barošová, and W. R. Smith, Mol. Phys. 91, 65 (1997).
  * (16) A. G. Martynov and G. N. Sarkisov, Mol. Phys. 49, 1495 (1983).
  * (17) T. Boublík, J. Chem. Phys. 53, 471 (1970).
  * (18) Ts. Tsednee and T. Luchko, Phys. Rev. E, 99, 032130 (2019).
  * (19) B. Tsednee, Ts. Tsednee and Ts. Khinayat, Solid St. Phenom. (2022) accepted; arxiv: 2204.06126.
  * (20) K. Hiroike and T. Morita, J. Chem. Phys. 52, 5489 (1970).
  * (21) B. Tsednee, Ts. Tsednee and Ts. Khinayat, Sci. trans.$-$Phys., Natl. Univ. Mongolia, 33, 30 (2022).