# Feasibility Study of Function Splits in RAN Architectures with LEO
Satellites

Siva Satya Sri Ganesh Seeram1, Luca Feltrin2, Mustafa Ozger1, Shuai Zhang1,
Cicek Cavdar1  1 KTH Royal Institute of Technology, Sweden, Email: {sssgse,
ozger, shuai2<EMAIL_ADDRESS>
2 Ericsson AB, Sweden, Email<EMAIL_ADDRESS>
###### Abstract

This paper explores the evolution of Radio Access Network (RAN) architectures
and their integration into Non-Terrestrial Networks (NTN) to address
escalating mobile traffic demands. Focusing on Low Earth Orbit (LEO)
satellites as key components of NTN, we examine the feasibility of RAN
function splits (FSs) in terms of fronthaul (FH) latency, elevation angle, and
bandwidth (BW) across LEO satellites and ground stations (GS), alongside
evaluating performance of Conditional Handover (CHO) procedures under diverse
scenarios. By assessing performance metrics such as handover duration,
disconnection time, and control traffic volume, we provide insights on several
aspects such as stringent constraints for Low Layer Splits (LLSs), leading to
longer delays during mobility procedures and increased control traffic across
the feeder link in comparison with the case when gNodeB is onboard satellite.
Despite challenges, LLSs demonstrate minimal onboard satellite computational
requirements, promising reduced power consumption and payload weight. These
findings underscore the architectural possibilities and challenges within the
telecommunications industry, paving the way for future advancements in NTN RAN
design and operation.

###### Index Terms:

radio access networks, function splits, fronthaul, conditional handover, LEO
satellites.

##  I Introduction

The evolution of Radio Access Network (RAN) architectures has been a key
aspect in the advancements of wireless communication technologies. The
escalating demand for mobile traffic within the conventional Distributed-RAN
(D-RAN) paradigm catalyzes exploration and advancement toward novel RAN
technologies [1, 2]. Among these, the Centralized RAN (C-RAN) emerges as a
proposed solution to address the intensifying requirements of mobile traffic.
Nevertheless, it introduces substantial demands on the transport network,
connecting the antenna site with the central site, concerning latency and
bandwidth (BW) [1]. Thus, a critical need exists to investigate solutions that
reduce the capacity demands on this transport network. The concept of a
Function Split (FS), refers to partitioning Baseband (BB) functions among
different network elements to determine which functions remain at the antenna
site and which ones are centralized. These FSs allow for greater flexibility
and efficiency in network design and operation leading to novel architecture
solutions [1, 2, 3].

The discussions so far on RAN architectures have primarily centered on
terrestrial networks (TN), where users exhibit mobility while RAN deployment
remains stationary. However, the integration of non-terrestrial networks (NTN)
becomes imperative to achieve ubiquitous coverage. Components of NTN may
encompass satellites, high-altitude platforms, low-altitude platforms, and
other similar entities. Additionally, it is crucial to examine the feasibility
of RAN FSs across NTN entities and ground stations (GSs), considering the
constraints imposed by maximum separation distance. In this paper, our
investigation focuses specifically on Low Earth Orbit (LEO) satellites,
characterized by rapid orbital velocity and non-stationary positioning
relative to ground users.

The mobility of LEO satellites introduces challenges when deploying RAN across
NTN, leading to frequent disruptions in establishing and terminating links
between BB functions distributed across satellites and GSs. Therefore, this
study also assesses the performance of Conditional Handover (CHO) procedures
under various NTN scenarios, while assuming some feasible FS options.
Performance metrics encompass the handover duration, disconnection time, and
control traffic volume on each satellite link. This evaluation is crucial for
understanding the advantages and drawbacks of specific FS.

This paper aims to provide a comprehensive overview of FSs in RAN, elucidating
the challenges and possible solutions arising from integrating NTN to provide
connectivity to ground users. By examining the feasibility of RAN FSs across
LEO satellites and GSs, and evaluating the performance of CHO procedures, we
offer insights into the evolving landscape of wireless communication
technologies. The main contributions of this paper are as follows:

  * •

Evaluation of various FS options within the RAN architecture, particularly in
the context of a static NTN.

  * •

Detailed analysis of the constraints associated with FSs, including factors
such as distance between network nodes, latency, BW, antennae, and processing
capacity.

  * •

Performance evaluation of the CHO procedures in various mobile NTN scenarios
with different FS deployments

##  II RAN Architectures

The RAN architecture evolves with different generations of mobile
communication technologies and forms an indispensable component of the mobile
network architecture. However, the data packet or the request has to go
through a stack of protocol layers where several functions like signal
processing are performed and are known as BB functions. Three standardized
entities are defined as a subset of BB functions namely Radio unit (RU),
Distributed unit (DU), and Central unit (CU) as shown in Fig. 1(a). According
to Small Cell Forum (SCF) [7], $10$ FSs are depicted as shown in Fig. 1(a)
where the red dotted line illustrates the FS option separating the BB
functions among the entities. The functions to the right side of line are in
the DU whereas the rest are in the CU.

The RU is responsible for Radio Frequency (RF) operations and signal
processing and also hosts some lower physical layer functions depending on the
FS option, _i.e.,_ FS $6$ to $9$. In contrast, functions to the right of FS
$10$ solely represent RU without any physical layer (PHY) functions. In
general, the DU consists of the functions Radio Link Control (RLC), Medium
Access Control (MAC), and some functions from the physical layer depending on
the FS option. However, the functions can be pulled or pushed to DU/CU
depending on the FS, _i.e.,_ the DU may comprise Packet Data Convergence
Protocol (PDCP) if FS $1$ is implemented and CU might comprise RLC if FS $3$
is implemented. The CU is generally responsible for the higher layer
functions, namely PDCP and Radio Resource Control (RRC). From a downlink (DL)
perspective, the PHY function in DU is Forward Error Correction (FEC) for FS
$6$. In addition to FEC, the Quadrature Amplitude Modulation function (QAM)
and antenna mapping are present in DU for FS $7$. Similarly, for FS $8$,
resource mapping is added. For FS $9$, Inverse Fast Fourier Transform (IFFT),
and cyclic prefix are placed on the DU additionally.

(a) Baseband Function splits

(b) Reference architecture

Figure 1: Overview of RAN architecture.

The reference architecture utilized for our investigation in Section IV is
illustrated in Fig. 1(b), with entities aligned according to color-coded
references from Fig. 1(a). The prefixes “s” and “t” are used to denote the
“source” and “target” links or nodes. It can be seen that from Fig. 1(b), the
RU is connected to the User Equipment (UE) through the Uu interface or air
interface. It receives instructions from the upper layers through a Lower-
Layer Split (LLS) interface. A CU communicates with a DU using the F1
interface. Xn interface is specified for the communication between the two
RANs. Furthermore, the Core Network (CN) is composed of many functions.
However, we consider the Access and Mobility Function (AMF) and User Plane
Function (UPF), as they are responsible for mobility and some control
signaling and are connected to the RAN through the Next-generation (NG)
interface as shown in Fig. 1(b).

In a traditional base station, BB processing functions (CU+DU) are co-located
with RU at the same site, usually known as D-RAN architecture. This
architecture reduces latency. However, this increases cost, in terms of power
consumption and capital expenditure, in proportion to the cell sites [4, 5,
6]. Subsequently, first-generation C-RAN emerged, segregating RUs at cell
sites while centralizing BB processing in BB unit pools, connecting them with
fronthaul links, however, each BB processing unit is designated with a
specific RU. Next, BB processing is virtualized, called Virtualized RAN (vRAN)
offering cost, performance, and deployment advantages due to shared BB
processing, and reduced base station payload respectively [1]. It minimizes
hardware dependency by virtualizing BB functions and using general-purpose
servers. Nonetheless, centralization of all BB processing functions presents
challenges such as stringent fronthaul (FH) latency and heavy BW requirements
in the transport network. The FH interface is traditionally called CPRI.
However, in the later architectures, it is termed an LLS interface due to the
lower BB FS between the entities. Hybrid C-RAN (H-CRAN) in [3] uses the FS
model based on cell and user processing functions to distribute the BB
functions between edge (RU+DU) and central (CU) clouds. Despite these
advancements, interfaces are proprietary between entities leading to vendor
lock-in, except for the open F1 interface. Open-RAN (O-RAN) is a
standardization effort to open the interfaces and evolve towards more
intelligent vRAN [2], allowing for interoperability between entities from
different vendors, and introducing a RAN intelligent controller (RIC) for
automated and optimized RAN operations.

##  III Analysis of FSs in a Static NTN Scenario

In a static NTN scenario, which provides a momentary snapshot of an LEO
satellite serving a single ground UE, connected to the GS through a Feeder
Link (FL) at a specified elevation angle, our objective is to analyze the
feasibility of different FS configurations depicted in Fig. 1(a), linking the
GS (CU) to the satellite (DU+RU). We assess various FS options on various
dimensions such as FH latency, BW, and antennae. We analyze FS options, and
navigate challenges posed by stringent latency requirements and increasing
computational demands.

TABLE I: Function splits, Fronthaul parameters and Processing requirements. Function Split option |  One way FH latency specifications (SCF [7]) |  Max. FH distance in free space |  |  Required FH bandwidth [7]
---
[Mbps]
| Processing
---
requirements [3]
[GOPS]
Nant=2 |  Nant=64 | GS | Satellite
DL | UL | DL | UL
1 – RRC-PDCP | Non-ideal- 30 ms | 9000 km | 149.9 | 48.6 | 149.9 | 48.6 | <8 | > 36.5
2 – PDCP - RLC | Non-ideal- 30 ms | 9000 km | 150 | 48.7 | 150 | 48.7 | <8 | > 36.5
3 – RLC - MAC | Non-ideal- 30 ms | 9000 km | 150.6 | 48.9 | 150.6 | 48.9 | <8 | > 36.5
4 – hMAC- lMAC | Sub ideal- 6 ms | 1800 km | 151.3 | 49.4 | 151.3 | 49.4 | <8 | > 36.5
5 – MAC - PHY | Sub ideal- 6 ms | 1800 km | 152.3 | 49.9 | 152.3 | 49.9 | <8 | > 36.5
6 – PHY Split I | Near ideal- 2 ms | 600 km | 173.1 | 451.6 | 173.1 | 451.6 | 8 | 36.5
7 – PHY Split II |  | Near ideal- 2 ms
---
Ideal- 0.25 ms
| 600 km
---
75 km
932.6 | 903.2 | 29843 | 28901 | 15.9 | 28.6
8 – PHY Split III |  | Near ideal- 2 ms
---
Ideal- 0.25 ms
| 600 km
---
75 km
1075.2 | 921.6 | 34406 | 29491 | 18.5 | 26
9 – PHY Split IIIb |  | Near ideal- 2 ms
---
Ideal- 0.25 ms
| 600 km
---
75 km
1966.1 | 1966.1 | 62915 | 62915 | 19.8 | 24.7
10 – PHY Split IV/PHY - RF | Ideal- 0.25 ms | 75 km | 2457.6 | 2457.6 | 78643 | 78643 | 23.8 | 20.7

Table I illustrates the $10$ FS options alongside their respective FH latency
requirements [7] for different use cases. Full gNB onboard the satellite is
not tabulated since there will be only backhaul and no FH in this deployment.
Furthermore, the backhaul constraints are not that stringent when compared to
the FH. The maximum FH distance is determined as the product of the speed of
light in free space and the FH latency requirement. The separation distance is
defined as the distance between the satellite and the GS. The FH BW is
computed utilizing the equations and parameters outlined in [7]. Additionally,
the processing requirements, measured in Giga operations per second (GOPS),
are computed based on the specifications provided in [3], which is the
summation of user processing functions and cell processing functions at GS and
satellite after the split. The user processing functions include higher layer
functions namely PDCP, RLC, MAC, and PHY functions namely FEC, QAM, and
antenna mapping. The cell processing functions include remaining low PHY
functions as mentioned in Section II. Note that these processing requirements
are calculated under the assumption of a single user, thus the cell processing
requirement is not significant enough. In scenarios with multiple users, the
allocated GOPS for cell processing remains constant for all users, although
the user processing requirement increases with the users.

In Table I, as the FS shifts to lower layers, the FH latency becomes more
stringent. An FS is said to be feasible if the separation distance is less
than the maximum FH distance. Therefore, a LEO satellite orbiting at an
altitude of $600$ km, with FL elevation angles of $10^{\circ}$ and
$90^{\circ}$, will exhibit separation distances approximately measuring $1935$
km and $600$ km, respectively (as depicted in Table II). Hence, the feasible
FSs in the former case are $1$-$3$ whereas in the latter case, the feasible
FSs are $1$-$9$, assuming near ideal use case for the PHY layer splits. A
large FH distance indicates a relaxed latency requirement which may translate
to, a single hop with said distance or multiple hops as long as the total
separation distance is less than the FH distance.

From Table I, with a varying number of antenna (Nant) [7], the FH BW
requirement in both DL and uplink (UL) increases as the FS moves to lower
layers. The FH BW requirement for N${}_{ant}=2$, almost increased by $5.4$
times in DL and doubled in UL when the FS changed from $6$ to $7$. Similarly,
for N${}_{ant}=64$, the FH BW increased by $173$ times in DL and in UL by
about $64$ times when the FS changed from $6$ to $7$. Furthermore, the BW
requirements in DL and UL are about the same for the FSs $1$ to $5$ for both
cases as the BW requirement in higher layers is independent of Nant [7]. Table
I shows that FS $10$ (PHY-RF) necessitates approximately a $2.4$ Gbps and
$78.6$ Gbps FH link rate respectively, for both DL and UL within one beam of
the satellite, serving a single user. Hence, for multiple beams of a single
satellite, the required BW increases and might be a bottleneck for the
feasibility of an FS. Furthermore, Table I shows that the computational
requirements onboard the satellite decrease as the FS moves to lower layers.
Similarly, the computational requirements at the GS increase as the FS moves
to lower layers and vice versa.

The discussion above highlights how separation distances limit the feasibility
of certain FS options in NTN, especially in LLS (FS $7$-$10$). While FH
latency specifications are based on LTE with advanced TN features, they may
not directly apply to legacy New Radio (NR) NTN. However, these specifications
can be adapted to NR NTN by adjusting certain features. In legacy NR, FH
latency requirements may be relaxed using asynchronous Hybrid Automatic Repeat
Request (HARQ) specifications instead of LTE synchronous HARQ. Moreover, in
NTN Rel-17 [8], completely disabling HARQ feedback is permitted due to
significantly longer round trip delays.

Furthermore, one of the major bottlenecks in the FH latency specifications
from SCF is channel state information (CSI) reporting or reciprocity-based
channel estimation, which requires a very short delay. It is unlikely that
these kinds of features will be used in legacy NR NTN as they only boost data
rates in good terrestrial coverage or in time division duplex (TDD)
deployments. If disabled, they do not prevent basic functionalities. Hence, we
can come to a consensus that these kinds of features are not necessary in NTN.

Figure 2: Reference scenarios (a) A - 1 satellite and 1 GS, (b) B - 2
satellites and 1 GS, (c) C - 2 satellites and 2 GSs.

##  IV Analysis of FSs based on mobile NTN scenarios

In this section, we explore three distinct network scenarios (labeled A, B,
and C), each showcasing unique configurations of satellite, ground station,
and FS deployment.

Our scenario comprises two cells: a source cell (blue in Fig. 2) where a user
originates and a target cell (orange in Fig. 2) where the user moves. It
includes at least one satellite, one GS connected to the core network, and
various satellite links: Service Link (SL) between a satellite and the user,
FL between a satellite and its serving GSs, Inter-Satellite Link (ISL)
connecting two satellites, and Inter-GS Link (IGSL) connecting two GS. The
prefixes “s” and “t” are used to denote the “source” and “target” links in
which the traffic flows before or after the user performs the CHO procedure,
respectively.

In this study, three scenarios depicted in Fig. 2 are evaluated, each
representing a specific network topology, and for each scenario, three FS
deployments, which are LLS, CU-DU split and gNodeB (gNB) on-board are
analyzed. LLS option which may be referred to FS $8$ in Fig. 1(a) since there
is no clear consensus regarding LLS. As discussed in Section III, LLS is
infeasible FS for ideal use cases. However, we assume that certain features
are relaxed, rendering this FS feasible. Secondly, the CU-DU split can be
referred to FS $2$ in Fig. 1(a). Thirdly, the gNB on-board option is not an
actual FS represented in Fig. 1(a), where all functions apart from AMF/UPF are
on the satellite as in Fig. 1(b). To interpret the model for any sub-scenario
in Fig. 2, begin from the GS, indicated by a specific legend (_e.g._ , B2),
which connects to the target satellite and is linked to the source satellite
via ISL, while disregarding other legends. The depiction of the FS between the
GS and the satellite is shown in Fig. 2, where split x/y/z can represent one
of the three split options mentioned previously.

In Fig. 2(a), scenario A is characterized by a single satellite serving both
cells. Given a constellation comprising several hundreds of satellites, each
tasked with serving a substantial number of cells, this scenario is the most
probable. In scenario B in Fig. 2(b), the cells are handled by different
satellites, however, both are connected to the same GS in different ways. This
is typical at the edge of two areas handled by the two satellites. For
scenario C in Fig. 2(c), there are also two distinct GSs; however, a single
AMF is still co-located with one of the two GSs. This case is less likely than
the other ones yet it may happen at the edge of two areas handled by the same
GS, _e.g._ , at the border of two countries. In scenario C, it should be noted
that there are always two full gNBs regardless of the FS as their highest
layers are either on the two satellites or on the two GSs, but never in a
common network node.

Assuming a specific altitude and elevation angle, propagation delays for the
SL and FL can be estimated using trigonometric principles. Similarly, if a
certain number of satellites orbiting in the same plane then the distance
between the satellites can be calculated, thus determining the ISL delay.
Finally, the two GSs will lie on the edge of the satellite footprints as
defined by the FL elevation angle. On average, we can infer that they are
separated by the same angle, relative to the Earth’s center, as the two
satellites, facilitating the computation of the geodesic distance. To
accommodate factors such as non-vacuum propagation and indirect paths caused
by router usage, we can augment this distance by $20\%$ before dividing it by
the speed of light to calculate the propagation delay. Table II shows the
propagation delay of each satellite link considering various elevation angles
at $600$ km of altitude where the low elevation angles represent worst-case
scenarios. The highlighted row indicates the values utilized in the subsequent
part of the paper.

TABLE II: Different link propagation delays. Constellation |  Elevation Angles |  SL delay [ms] |  FL delay [ms] |  ISL delay [ms] |  IGSL delay [ms]
---|---|---|---|---|---
LEO@600 |  SL: 30 deg FL: 10 deg | 3.59 | 6.45 | 7.28 | 7.99
LEO@600 |  SL: 90 deg FL: 90 deg | 2.00 | 2.00 | 7.28 | 7.99

(a) Intra-DU CHO procedure

(b) Inter-DU CHO procedure

(c) Inter-gNb Intra-AMF CHO procedure

Figure 3: Complete message exchange associated with each procedure.

For the system architecture, we consider two chains of gNB functions, one
serving the source cell and one serving the target cell, the AMF/UPF as
representative of the CN as shown in Fig. 1(b). Depending on the scenario,
corresponding functions in the source and target chain may also be merged, in
which case the UE performs a specific variant of the mobility procedure within
the same gNB unit. For instance, in the case of the LLS option where two
satellites and only one GS is involved, the two RUs on the satellites (sRU and
tRU) may be connected to a common DU deployed in the GS. The UE would perform
an Intra-DU CHO procedure.

All procedures have the same behavior towards the UE, _i.e.,_ the signaling
over the tUu and sUu interfaces is the same in all cases, which consists of
the RRC CHO procedure as seen in Fig. 3. It can be divided into three phases:
setup, buffer, and execution. During the setup phase, the sgNB configures the
UE with a specific trigger condition related to a certain metric, _e.g._ ,
time, distance, or target and source cell signal strength. The UE then starts
to perform measurements regarding the configured metric. When the condition is
triggered, the buffer phase can begin. During the buffer phase, the UE
synchronizes with the tgNB by acquiring its Synchronization Signal Block (SSB)
and performs the Random Access Procedure. The UE cannot deliver data as the
RRC connection is re-established only after Msg4 is delivered. Hence, this
phase duration is a key metric to assess the quality of the FSs. The UE
resumes data delivery, often requiring additional signaling in the execution
phase to ensure correct network configuration. This phase typically involves
signaling through gNB internal or NG interface [9].

When the DU or RU is common to both source and target chains, the Intra-DU CHO
procedure is performed which consists only of the already mentioned RRC CHO
procedure over Uu [9]. When the CU is common to both source and target chains,
the Inter-DU CHO procedure is performed. During the setup phase, before
configuring the UE, the CU requests all the potential tDUs if they can accept
the UE through the F1 interface. After all replies are collected, the CU can
configure the UE through the source chain. In the buffer phase after the UE
has completed the RACH procedure, the tDU informs the CU of the successful
access so that the CU can inform the sDU that it can stop data delivery.
Finally, in the execution phase, the CU informs the sDU that it can release
the UE context [10]. Furthermore, the Inter-gNB Intra-AMF CHO is performed as
the mobility happens between two complete gNBs. This procedure may be
performed over the Xn or NG interface depending on the deployment. Here, we
assume it will be performed over Xn for the sake of simplicity. Like the
previous procedure, the sgNB asks the potential tgNBs if they can accept the
UE although now this happens through the Xn interface. During the buffer
phase, after the UE has triggered the procedure and has completed the RACH,
the tgNB informs the sgNB of the successful access. At this point, the UE can
resume data delivery, however, the network User Plane Function (UPF) is not
informed yet that the serving gNB has changed. Hence, it will keep forwarding
DL data to the sgNB which momentarily will forward it further to the tgNB
which will deliver it to the UE, causing an additional delay. The UL data is
not affected as the tgNB will still forward it directly to the UPF. To fix
this, during the execution phase, the tgNB triggers a Path Switch request
which involves several Application Programming Interface (API) calls to
various CN functions and the UPF followed by the propagation of an End Marker.
When the tgNB receives the confirmation of a successful path switch, it can
inform the sgNB that it can release the UE context. It is also assumed that
all network nodes, source, and target cells belong to the same Tracking Area
(TA) so that a TA Update procedure is not performed.

Fig. 3 shows the complete message exchange associated with each procedure.
Depending on the mapping between each architecture interface and satellite
interface, which depends on the FS and scenario considered, it is possible to
determine the propagation delay associated with each message transmission.
Furthermore, it is assumed that the reception of each message requires a $1$
ms processing delay, the SSB acquisition requires $20$ ms, and the CN API
calls performed in the Inter-gNB Intra-AMF CHO procedure require a total of
$50$ ms.

##  V Results and analysis

For Scenario A, regardless of the FS, the Intra-DU CHO procedure is always
performed as all functions in the two chains are in common. From Fig. 4(a), it
is evident that relocating RAN functions onboard the satellite results in
fewer messages directed to the GS, thereby eliminating the need for traversal
through both the SL and the FL for delivery. This substantially reduces the
overall procedure duration, particularly the time spent in a buffer state as
shown in Fig. 4(b).

In scenario B, each bar in Fig. 5 corresponds to a distinct combination of
scenario variant, FS, and CHO procedure variant. The trend is opposite as
compared to Fig. 4, as the more complexity is moved onto the satellite, the
longer the procedure takes to complete as shown in Fig. 5(b). This is mainly
due to the higher complexity of the procedure itself, the Inter-gNB Intra-AMF
CHO also involves a considerable amount of time spent in performing the Path
Switch procedure, unlike the other procedures. Each split option entails a
unique RAN function common at the GS, resulting in a distinct procedure.
However, it is notable that the time spent in the buffer state exhibits an
opposite trend, being the shortest with the gNB onboard satellite option, due
to similar reasons as observed in scenario A. Additionally, the CU-DU split
results in the highest volume of messages sent, both in FL and in overall
links as shown in Fig. 5(a). FL is typically the network bottleneck due to the
convergence of traffic from numerous cells onto the same link, potentially
posing capacity challenges for mobility procedures with the CU-DU split.

Figure 4: CHO performance for scenario A. Figure 5: CHO performance for
scenario B. Figure 6: CHO performance for scenario C.

In scenario C, from Fig. 6(a), it is observed that the LLS option exhibits the
highest volume of traffic transmitted over FL. Nevertheless, in this case, the
capacity is not significantly impacted, as the traffic from LLS is largely
independent of user/control traffic but rather depends more on the number of
cells managed by the satellite(s), as given in Table I. Furthermore, from Fig.
6(b), it is evident that as more complexity is moved onboard the satellite,
both the procedure duration and the time spent in the buffer state decrease.

The gNB on-board split option appears to be the most effective in reducing the
overhead traffic across the network and minimizing the duration of the
procedure, particularly in decreasing the time spent in the buffer state,
where the data delivery pauses. The reduction of control overhead traffic is
significant in NTN, where the mobility events predominantly occur due to
changes in serving satellites for specific cells or due to the cell movement
in the case of Earth-moving cells. These events typically occur every few
minutes, prompting all the users within the cell simultaneously to execute
mobility procedures. While strategies such as aggregating the control traffic
or distributing the handovers over time can help mitigate this issue, a
significant amount of traffic is still expected.

##  VI Conclusions

In this paper, we analyzed the feasible FSs for a static NTN scenario focusing
on factors such as FH latency, separation distance, elevation angle, FH BW and
antennae. Our findings indicate that the stringent FH latency poses
limitations on the feasibility of FSs, whereas the elevation angle of the FL
exerts a more significant impact on this feasibility. We also explored some
solutions where certain features that could be relaxed or disabled such as
HARQ and CSI reporting to adapt LTE specifications into the legacy NR NTN,
thus making LLSs feasible. Our analysis further highlights that LLS are
subject to strict separation distance constraints and experience prolonged
delays during mobility CHO procedures, notably affecting user data
transmission interruptions and control traffic volumes across the FL.
Moreover, the LLS option exhibits minimal computational demands onboard
satellites, promising a reduction in power consumption and payload.
Conversely, higher-layer splits, particularly those involving full gNB onboard
satellites, exhibit relaxed bandwidth and delay constraints, enabling coverage
of more distant areas from serving GSs at the expense of increased CPU and
payload complexity.

## Acknowledgement

This work was supported in part by the CELTIC-NEXT Project, 6G for Connected
Sky (6G-SKY), with funding received from the Vinnova, Swedish Innovation
Agency.

## References

  * [1] L. M. P. Larsen, A. Checko and H. L. Christiansen, “A Survey of the Functional Splits Proposed for 5G Mobile Crosshaul Networks,” in _IEEE Commun. Surveys & Tutorials_, vol. 21, no. 1, pp. 146-172, 2019.
  * [2] J. Pérez-Romero _et al._ , “A Tutorial on the Characterisation and Modelling of Low Layer Functional Splits for Flexible Radio Access Networks in 5G and Beyond,” in _IEEE Commun. Surveys & Tutorials_, vol. 25, no. 4, pp. 2791-2833, 2023.
  * [3] A. Alabbasi and C. Cavdar, “Delay-aware green hybrid CRAN,” in _Proc. IEEE WiOpt,_ pp. 1-7, 2017.
  * [4] X. Wang, _et al._ , “Centralize or distribute? A techno-economic study to design a low-cost cloud radio access network,” _IEEE ICC_ , pp. 1-7, 2017.
  * [5] M. Masoudi, S. S. Lisi and C. Cavdar, “Cost-Effective Migration Toward Virtualized C-RAN With Scalable Fronthaul Design,” in _IEEE Syst. J._ , vol. 14, no. 4, pp. 5100-5110, 2020.
  * [6] “C-RAN: The road towards green RAN,” China Mobile Research Institute, Hong Kong, White Paper, Oct. 2011.
  * [7] “Small Cell Virtualization Functional Splits and Use Cases,” Release 7.0, Small Cell Forum, 2016.
  * [8] 3GPP, TR 38.821 V16.2.0, Solutions for NR to support non-terrestrial networks (NTN), Mar. 2023
  * [9] 3GPP, TS 38.331 v.18.0.0, NR; Radio Resource Control (RRC); Protocol specification, Jan. 2024
  * [10] 3GPP, TS 38.401 v.18.0.0, NG-RAN; Architecture description, Jan. 2024


#  FATNN: Fast and Accurate Ternary Neural Networks††thanks: PC and BZ
contributed equally. Part of this work was done when all authors were with The
University of Adelaide. CS is the corresponding author, email<EMAIL_ADDRESS>
Peng Chen, Bohan Zhuang, Chunhua Shen
Data Science & AI, Monash University, Australia

###### Abstract

Ternary Neural Networks (TNNs) have received much attention due to being
potentially orders of magnitude faster in inference, as well as more power
efficient, than full-precision counterparts. However, 2 bits are required to
encode the ternary representation with only 3 quantization levels leveraged.
As a result, conventional TNNs have similar memory consumption and speed
compared with the standard 2-bit models, but have worse representational
capability. Moreover, there is still a significant gap in accuracy between
TNNs and full-precision networks, hampering their deployment to real
applications. To tackle these two challenges, in this work, we first show
that, under some mild constraints, computational complexity of the ternary
inner product can be reduced by $2\times$. Second, to mitigate the performance
gap, we elaborately design an implementation-dependent ternary quantization
algorithm. The proposed framework is termed Fast and Accurate Ternary Neural
Networks (FATNN). Experiments on image classification demonstrate that our
FATNN surpasses the state-of-the-arts by a significant margin in accuracy.
More importantly, speedup evaluation compared with various precision is
analyzed on several platforms, which serves as a strong benchmark for further
research.

##  1 Introduction

Equipped with high-performance computing and large-scale datasets, deep
convolution neural networks (DCNN) have become a cornerstone for most computer
vision tasks. However, a significant obstacle for deploying DCNN algorithms to
mobile/embedded edge devices with limited computing resources is the ever
growing computation complexity—in order to achieve good accuracy, the models
are becoming very heavy. To tackle this problem, much research effort has been
spent on model compression. Representative methods include model quantization
[53, 51], network pruning [24, 55] and neural architecture search for
lightweight models [56, 29]. In this paper, we focus on model quantization,
which reduces the model complexity by representing a network with low-
precision weights and activations.

Network quantization aims to map the continuous input values within a
quantization interval to the corresponding quantization level, and a low-
precision quantized value is assigned accordingly. TNNs in which both the
activations and weights are quantized to ternary, are particularly of interest
because most of the calculations can be realized with bit operations, thus
completely eliminating multiplications. However, there exists two limitations
for conventional TNNs. The first limitation is the inefficient implementation
of TNNs. Specifically, the ternary representation of $\\{-1,0,1\\}$ needs 2
bits to encode with one state wasted. As a result, with the conventional
bitwise implementation of quantized networks [51, 48], the complexity of
ternary inner product is the same with the standard 2-bit counterparts.
Another limitation is the considerable accuracy drop compared with the full-
precision counterparts due to the much more compact capacity.

To handle these drawbacks, we introduce a new framework, termed FATNN, where
we co-design the underlying ternary implementation of computation and the
quantization algorithm. In terms of implementation, we fully leverage the
property of the ternary representation to design a series of bit operations to
accomplish the ternary inner product with improved efficiency. In particular,
FATNN reduces the computational complexity of TNNs by $2\times$, which solves
the existing efficiency bottleneck. In contrast to previous works, nearly no
arithmetic operations exist in the proposed implementation. Also, FATNN works
efficiently on almost all kinds of devices (such as CPU, GPU, DSP, FPGA and
ASIC) with basic bit operation instructions available. Furthermore, we design
the compatible ternary quantization algorithm in accordance to the mild
constraints derived from the underlying implementation. Early works with
learned quantizers either propose to learn the quantized values [48] or seek
to learn the quantization intervals [20, 13]. However, most of them assume the
uniform quantizer step size, which might still be non-optimal on optimizing
network performance. To make the low-precision discrete values sufficiently
fit the statistics of the data distribution, we propose to parameterize the
step size of each quantization level and optimize them with the approximate
gradient. Besides, we suggest calibrating the distribution between the skip
connection and the low-precision branch to further improve the performance.
The overall approach is usable for quantizing both activations and weights,
and works with existing methods for back-propagation and stochastic gradient
descent.

Our main contributions are summarized as follows:

  * •

We propose a ternary quantization pipeline, in which we co-design the
underlying implementation and the quantization algorithm. To our best
knowledge, the proposed FATNN is the first solution being applied on general
platforms specific to TNNs while previous acceleration methods only target on
the dedicated hardware (FPGA or ASIC).

  * •

We devise a fast ternary inner product implementation which reduces the
complexity of TNNs by $2\times$ while keeping the bit-operation-compatible
merit. We then design a highly accurate ternary quantization algorithm in
accordance with the constraints imposed by the implementation.

  * •

We evaluate the execution speed of FATNN and make comparison with other bit
configurations on various platforms. Moreover, experiments on image
classification task demonstrate the superior performance of our FATNN over a
few competitive state-of-the-art approaches.

##  2 Related Work

Model quantization aims to quantize the weights, activations and even
backpropagation gradients into low-precision, to yield highly compact DCNN
models compared to their floating-point counterparts. As a result, most of the
multiplication operations in network inference can be replaced by more
efficient addition or bitwise operations. In particular, BNNs [38, 17, 50, 4,
5, 46, 41, 14, 30], where both weights and activations are quantized to binary
tensors, are reported to have potentially $32\times$ memory compression ratio,
and up to $58\times$ speed-up on CPU compared with the full-precision
counterparts. However, BNNs still suffer from sizable performance drop issue,
hindering them from being widely deployed. To make a trade-off between
accuracy and complexity, researchers also study ternary [23, 52] and higher-
bit quantization [51, 8, 13, 54, 28]. In general, quantization algorithms aim
at tackling two core challenges. The first challenge is to design accurate
quantizers to minimize the information loss. Early works use handcrafted
heuristic quantizers [51] while later studies propose to adjust the quantizers
to the data, basically based on matching the original data distribution [51,
6], minimizing the quantization error [48, 9] or directly optimizing the
quantizer with stochastic gradient descent [8, 20, 2]. Moreover, another
challenge is to approximate gradient of the non-differentiable quantizer. To
solve this problem, most studies focus on improving the training via loss-
aware optimization [16], regularization [12, 9, 1], knowledge distillation
[53, 32], entropy maximization [33, 35] and relaxed optimization [31, 47, 22,
44]. In addition to the quantization algorithms design, the implementation
frameworks and acceleration libraries [18, 7, 42, 19] are indispensable to
expedite the quantization technique to be deployed on energy-efficient edge
devices. For example, TBN [43] focuses on the implementation of ternary
activation and binary weight networks. daBNN [49] targets at the inference
optimization of BNNs on ARM CPU devices. GXNOR-Net [11] treats TNNs as a kind
of sparse BNNs and propose an acceleration solution on dedicated hardware
platforms. RTN [25] leverages extra linear transformation before quantization
for better performance of TNNs. Similar with GXNOR-Net, RTN demonstrates the
benefit of their implementation on dedicated devices (FPGA and ASIC). However,
there are few works targeting on improving the inference efficiency of TNNs on
general purpose computing platforms. In this paper, we propose to co-design
the underlying implementation and the quantization algorithm to achieve ideal
efficiency and accuracy simultaneously.

##  3 Inference Acceleration

###  3.1 Preliminary

As the inner product is one of the fundamental operations in convolution
neural networks, which consumes most of the execution time, we mainly focus on
the acceleration of inner product in this paper. It is worth to firstly review
how the inner product between two quantized vectors are computed in previous
literature. For BNNs, in which both the weights and activation are binarized
to $\\{-1,1\\}$, the inner product between two length-$N$ vectors
$\bm{x},\bm{y}\in\\{-1,1\\}^{N}$ can be derived using bit-wise operations:

${\bm{x}}\cdot{\bm{y}}=2\cdot{\rm{popcount}}({\rm{xnor}}({\bm{x}},{\bm{y}}))-N,$
(1)

where $\rm{popcount}$ counts the number of bits in a bit vector. Furthermore,
for quantization with more bits, the input vectors can be decomposed with a
linear combination of binary bases. For example, a $M$-bit vector $\bm{x}$ can
be encoded as $\bm{x}=\sum_{m=0}^{m=M-1}{\bm{x}}_{m}\cdot 2^{m}$ where
${\bm{x}}_{m}\in\\{-1,1\\}^{N}$ (in practical implementation, $-1$ and $1$ are
represented by $0$ and $1$ respectively). Similarly, for another $K$-bit
vector $\bm{y}$, we have $\bm{y}=\sum_{k=0}^{k=K-1}{\bm{y}}_{k}\cdot 2^{k}$,
where ${\bm{y}}_{k}\in\\{-1,1\\}^{N}$. Based on the decomposition, the binary
inner product specified in Eq. (1) can be used to compute higher bit inner
product. Generally, the inner product between two quantized vectors can be
formulated as

${\bm{x}}\cdot\bm{y}=\sum\limits_{m=0}^{M-1}{\sum\limits_{k=0}^{K-1}{{\alpha_{m}}}}{\beta_{k}}({\bm{x}}_{m}\odot{\bm{y}}_{k}),$
(2)

where $\odot$ is specially used to denote the binary inner product in
formulation of Eq. (1), $\bm{\alpha}\in{\mathbb{R}^{M}}$ and
$\bm{\beta}\in{\mathbb{R}^{K}}$ are scales to encode $\bm{x}$ and $\bm{y}$,
respectively. Specifically, $\alpha_{m}=2^{m}$ and $\beta_{k}=2^{k}$ are used
in uniform fixed-point quantization [51, 26] while $\bm{\alpha}$ and
$\bm{\beta}$ become trainable scales for non-uniform quantization [48, 2].
Considerable speedups can be achieved by Eq. (1) and Eq. (2) because all the
calculation can be realized with bit operations [17, 10, 42], thus completely
eliminating multiplications.

###  3.2 Motivations for Acceleration

If assuming the computational complexity of BNNs in Eq. (1) to be $O(N)$, the
computational complexity for higher bit quantization in Eq. (2) becomes
$O(M\cdot K\cdot N)$. We can find that higher bit quantization algorithms
acquire better task accuracy at the cost of increased computational
complexity. In particular, for the TNN case, 2 bits are required for the data
representation of $\\{-1,0,1\\}$. Thus the computational complexity for
ternary inner product is $O(4N)$, which is the same with the standard 2-bit
counterpart, however, with one of the quantization levels wasted (2 bits can
express 4 quantization levels at most). As a result, the implementation in Eq.
(2) makes TNNs less appealing to standard 2-bit models in practical.

To fully unleash the potential of TNNs, we further observe that the binary
inner product in Eq. (1) is the core for acceleration since its multiplication
and accumulation are realized by the bit operators $\rm xnor$ and $\rm
popcount$, respectively. As the input of the inner product in BNNs is
restricted to $\\{-1,1\\}$, the multiplication result is also within the set
$\\{-1,1\\}$ , which we call the “non-overflow” property. The multiplication
result can be directly obtained via $\rm xnor$ between the input vectors and
it owns the attribute that only two states exist, thus $\rm popcount$ can be
used to realize accumulation by simply counting the number of state “$1$” (or
state “$-1$”). As a result, Eq. (1) enables the same parallelism degree111 The
same parallelism degree indicates the data amount processed is the same per
instruction. More explanations are put in Section S1 in the supplementary
file. for $\rm{xnor}$ and $\rm{popcount}$, with the ALU register fully
utilized. Interestingly, we find the ternary quantized values $\\{1,0,-1\\}$
also meets the “non-overflow” property. Thus, it is potential for the TNNs to
be executed in the same parallelism degree manner for the multiplication
(i.e., $\rm xnor$) and accumulation (i.e., $\rm popcount$) operations.
Moreover, we can use this property to design a novel ternary inner product
implementation with a reduced complexity of $O(2N)$.

###  3.3 Ternary Network Acceleration

Table 1: The correspondence mapping between the quantized data space and the codec space employed in the proposed solution. Both 2’b01 and 2’b10 are taken to represent the “0” value. The design of the codec owns the attribute of “popcount(codec) $=$ data$\,+1$”. data | $-1$ | 0 | 0 | 1
---|---|---|---|---
codec | 2’b00 | 2’b01 | 2’b10 | 2’b11

We now elaborate the design of the fast ternary inner product implementation.
First, it is worth noting that the ternary values $\\{-1,0,1\\}$ will be
represented by the corresponding codec in the practical implementation. We
elaborately design the mapping between the logical level ternary values and
their implementation level codec as illustrated in Table 1. Interestingly, we
observe a property that the number of “1” in each value’s codec equals the
value plus one. Therefore, we can compute the inner product between two
ternary vectors $\bm{x},\bm{y}\in\\{-1,0,1\\}^{N}$ as follows:

$\displaystyle{\bm{x}}\cdot{\bm{y}}={\rm{popcount}}({\rm{TM}}({\bm{x}},{\bm{y}}))-N,$
(3)

where $\rm{TM(\cdot)}$ indicates the ternary multiplication, $N$ is the vector
length. In Eq. (3), we first compute the inner product in the codec space by
${\rm{popcount}}({\rm{TM}}({\bm{x}},{\bm{y}}))$, which consists of pure bit
operations. Then we simply subtract $N$ from the result to transform it into
the inner product of the logical level ternary vectors.

Second, we illustrate the way to design $\rm{TM(\cdot)}$ with pure bit
operations222We follow the C/C++ grammar in the equations. For example, $\&$
means “AND”, $\sim$ indicates “NOT”, $\wedge$ represents “XOR”.. For easy
understanding, the true value table of ternary multiplication is listed in
Table 2. Since two bits are required to encode the ternary input value and the
ternary multiplication result, there are 16 possibilities with respect to the
codec. From Table 2, we observe that most rows in the table, except the bold
ones, still follow the rule of $\rm xnor$:

${xnor}=\ \sim(x_{c}\wedge y_{c}),$ (4)

where $x_{c}$ and $y_{c}$ are the codec representation of the element in
$\bm{x}$ and $\bm{y}$, respectively. Therefore, the ternary multiplication can
be realized by $\rm xnor$ along with the exception cases fixed.

After reviewing the $\rm xnor$ correct cases and exception cases, we summarize
that the exception ones only happen when both of the operands are $0$.
Therefore, the ternary multiplication result can be fixed by locating the
“zero operand” cases and forcing the result to be $0$ (we force the result to
be $0$ if “zero operand” is detected no matter whether it is the $\rm xnor$
incorrect case or not).

Table 2: True value table of ternary multiplication. $x_{c}$ | $\cdot$ | $y_{c}$ | $=$ | $z_{c}$
---|---|---|---|---
$-1$(2’b00) |  |  $-1$(2’b00) |  | 1(2’b11)
$-1$(2’b00) |  | 0(2’b01) |  | 0(2’b10)
$-1$(2’b00) |  | 0(2’b10) |  | 0(2’b01)
$-1$(2’b00) |  | 1(2’b11) |  |  $-1$(2’b00)
0(2’b01) |  |  $-1$(2’b00) |  | 0(2’b10)
0(2’b01) |  | 0(2’b01) |  | 0(2’b01)
0(2’b01) |  | 0(2’b10) |  | 0(2’b01)
0(2’b01) |  | 1(2’b11) |  | 0(2’b01)
0(2’b10) |  |  $-1$(2’b00) |  | 0(2’b01)
0(2’b10) |  | 0(2’b01) |  | 0(2’b10)
0(2’b10) |  | 0(2’b10) |  | 0(2’b10)
0(2’b10) |  | 1(2’b11) |  | 0(2’b10)
1(2’b11) |  |  $-1$(2’b00) |  |  $-1$(2’b00)
1(2’b11) |  | 0(2’b01) |  | 0(2’b01)
1(2’b11) |  | 0(2’b10) |  | 0(2’b10)
1(2’b11) |  | 1(2’b11) |  | 1(2’b11)

Overall, our implementation of the ternary multiplication consists of three
steps: 1): Obtain the intermediate result by the $\rm xnor$ operation. 2):
Identify the 0 operand. 3): Fix the exceptions and obtain the ternary
multiplication result. The first step is easily achieved by Eq. (4). For the
second step, we here introduce an auxiliary variable $auxi$ which is a pre-
defined constant in codec $2^{\prime}b01$. In fact, the auxiliary variable
$auxi$ indicates a zero value variable (one codec of the $0$ value is
$2^{\prime}b01$), which acts as a mask to fetch specific bits in operands.
Then we propose to identify the $0$ operand using the following bit
operations:

$switch=((y_{c}>>1)\ \&\ auxi)\ |\ ((y_{c}<<1)\ \&\sim auxi),$ (5)
$mask=switch\wedge y_{c},$ (6)

With the shift and mask operations, Eq. (5) actually results in the exchange
of the two sequence bits in the operand. After that, Eq. (6) generates the
mask information by distinguishing whether the operand is $0$ (in codec
$2^{\prime}b01$ or $2^{\prime}b10$) or not. Specifically, the $mask$ variable
in Eq. (6) will be $2^{\prime}b11$ if the 0 operand is detected and
$2^{\prime}b00$ otherwise. After identifying the $0$ operand, we can easily
fix the exceptions and obtain the final ternary multiplication result as:

${\rm{TM}}(\cdot)=(mask\ \&\ auxi)\ |\ ((\sim mask)\ \&\ xnor).$ (7)

If the $mask$ is $2^{\prime}b11$ (the 0 operand is detected), then Eq. (7)
reduces to $auxi$ which equals 0. In contrast, if the $mask$ is
$2^{\prime}b00$, then Eq. (7) becomes $xnor$, where the correctness can be
examined in Table 2. In practise, we use the weight parameter ($y_{c}$) to
generate $mask$ which is determined after training.

##### Remark 1

We can derive from Eq. (3) that the computational complexity of the proposed
ternary inner product is $O(2N)$. In other words, FATNN can reduce the
computational complexity of TNNs from $O(4N)$ to $O(2N)$, which significantly
improves the efficiency (memory consumption is not changed). Therefore, even
though TNNs do not make full use of the 2-bit representational capacity, they
fortunately enjoy the faster implementation than the standard 2-bit models.
Besides, our solution can adapt to general purpose computing platforms, such
as CPU, GPU and DSP. Note that, the extra bit operations introduced in Eqs.
(5), (6) and (7) have negligible runtime overhead for deployment, as these
extra bit operations are much faster than the accumulation operation in Eq.
(3) 333Refer to the implementation details in Section S1 in the supplementary
material.. We further provide extensive benchmark results in the experiment
section to justify our analysis.

##### Constraints on the algorithm.

From the formulation discussed above, it can be learned that the designed
ternary implementation has certain requirements on the quantization algorithm.
The constraints are summarized as follows:

  * •

The ternary values for the network are limited to $\\{-1,0,1\\}$. One and only
one additional high precision coefficient is allowed to adjust the scale of
the quantized values. More than one scale coefficients will break Eq. (3). It
indicates methods such as TTN [52], in which two trainable variables
$(W_{p},W_{n})$ are learned, cannot be applied to our method.

  * •

A special case exists for the activation quantization when the $\rm{ReLU}$
non-linearity is applied, which leads to a non-negative data range. In this
situation, we advise to modify the ternary values to $\\{0,1,2\\}$ for
activations. The revision does not conflict with the first constraint as in
the inference procedure, it results in an additional constant on the output
(simply add a copy of weights on the result which are fixed after training).

##  4 Constrained Ternary Quantization

###  4.1 Quantization Function Revisited

The quantization functions can be categorized into uniform [13, 51] and non-
uniform [48, 20] ones. In particular, LSQ [13] proposes to parameterize the
uniform step size and achieves state-of-the-art performance. In this section,
we show that the principle of parameterized step size can be applied to
solving the non-uniform ternary quantization.

Let us first consider the ternary weight quantization. In particular, we
parameterize the three step sizes by introducing two learnable parameters
$\\{\alpha_{1},\alpha_{2}\\}$. In this way, the full-precision data is
partitioned into three levels with the quantization thresholds
$\\{-\alpha_{1}/2,\alpha_{2}/2\\}$, where each step size can be adjusted
accordingly during training. Then we define the weight quantizer
$Q^{w}(p;\alpha_{1},\alpha_{2})$ for data $p$ parameterized by the scale
factors {$\alpha_{1},\alpha_{2}$}. Specifically,
$Q^{w}(p;\alpha_{1},\alpha_{2})$ performs quantization by applying three
point-wise operations in order: normalization, saturate and round.

Normalization: Since we do non-uniform quantization, the tensor elements are
firstly normalized by the scale factors $\alpha_{1}$ and $\alpha_{2}$,
respectively. This operation aims to map the data from the floating-point
domain to the quantized domain. Saturate: Once normalized, the tensor elements
that are out of the range of the quantized domain are clipped accordingly:
${\rm c\mathchoice{\mathord{\lower
0.43057pt\hbox{\kern-0.57129pt\hbox{\rotatebox{10.0}{$\displaystyle\ell$}}\kern-0.57129pt}}}{\mathord{\lower
0.43057pt\hbox{\kern-0.57129pt\hbox{\rotatebox{10.0}{$\textstyle\ell$}}\kern-0.57129pt}}}{\mathord{\lower
0.43057pt\hbox{\kern-0.3999pt\hbox{\rotatebox{10.0}{$\scriptstyle\ell$}}\kern-0.3999pt}}}{\mathord{\lower
0.43057pt\hbox{\kern-0.28564pt\hbox{\rotatebox{10.0}{$\scriptscriptstyle\ell$}}\kern-0.28564pt}}}ip}(p;\beta_{1},\beta_{2})={\rm
min}({\rm max}(p,\beta_{1}),\beta_{2})$, where the scalar clipping limits
$\\{\beta_{1},\beta_{2}\\}$ are independent with the full-precision data
range. Round: We discretize the normalized and tailored tensor elements to
nearest integers using bankers rounding denoted by
$\left\lfloor\cdot\right\rceil$. Putting the above point-wise operations
together, the weight quantization function can be written as:

$Q^{w}(p)=\left\\{\begin{array}[]{ll}\left\lfloor{\rm
c\mathchoice{\mathord{\lower
0.43057pt\hbox{\kern-0.57129pt\hbox{\rotatebox{10.0}{$\displaystyle\ell$}}\kern-0.57129pt}}}{\mathord{\lower
0.43057pt\hbox{\kern-0.57129pt\hbox{\rotatebox{10.0}{$\textstyle\ell$}}\kern-0.57129pt}}}{\mathord{\lower
0.43057pt\hbox{\kern-0.3999pt\hbox{\rotatebox{10.0}{$\scriptstyle\ell$}}\kern-0.3999pt}}}{\mathord{\lower
0.43057pt\hbox{\kern-0.28564pt\hbox{\rotatebox{10.0}{$\scriptscriptstyle\ell$}}\kern-0.28564pt}}}ip}(p/\alpha_{1},-1,0)\right\rceil&{\rm{if}}\;p<0\\\
\left\lfloor{\rm c\mathchoice{\mathord{\lower
0.43057pt\hbox{\kern-0.57129pt\hbox{\rotatebox{10.0}{$\displaystyle\ell$}}\kern-0.57129pt}}}{\mathord{\lower
0.43057pt\hbox{\kern-0.57129pt\hbox{\rotatebox{10.0}{$\textstyle\ell$}}\kern-0.57129pt}}}{\mathord{\lower
0.43057pt\hbox{\kern-0.3999pt\hbox{\rotatebox{10.0}{$\scriptstyle\ell$}}\kern-0.3999pt}}}{\mathord{\lower
0.43057pt\hbox{\kern-0.28564pt\hbox{\rotatebox{10.0}{$\scriptscriptstyle\ell$}}\kern-0.28564pt}}}ip}(p/\alpha_{2},0,1)\right\rceil&\rm{otherwise}\end{array}\right.,$
(8)

For simplicity, Eq. (8) can be re-written into a unified form:

$Q^{w}(p)=\lfloor{\rm c\mathchoice{\mathord{\lower
0.43057pt\hbox{\kern-0.57129pt\hbox{\rotatebox{10.0}{$\displaystyle\ell$}}\kern-0.57129pt}}}{\mathord{\lower
0.43057pt\hbox{\kern-0.57129pt\hbox{\rotatebox{10.0}{$\textstyle\ell$}}\kern-0.57129pt}}}{\mathord{\lower
0.43057pt\hbox{\kern-0.3999pt\hbox{\rotatebox{10.0}{$\scriptstyle\ell$}}\kern-0.3999pt}}}{\mathord{\lower
0.43057pt\hbox{\kern-0.28564pt\hbox{\rotatebox{10.0}{$\scriptscriptstyle\ell$}}\kern-0.28564pt}}}ip}(p/\alpha_{1},-1,0)\rceil+\lfloor{\rm
c\mathchoice{\mathord{\lower
0.43057pt\hbox{\kern-0.57129pt\hbox{\rotatebox{10.0}{$\displaystyle\ell$}}\kern-0.57129pt}}}{\mathord{\lower
0.43057pt\hbox{\kern-0.57129pt\hbox{\rotatebox{10.0}{$\textstyle\ell$}}\kern-0.57129pt}}}{\mathord{\lower
0.43057pt\hbox{\kern-0.3999pt\hbox{\rotatebox{10.0}{$\scriptstyle\ell$}}\kern-0.3999pt}}}{\mathord{\lower
0.43057pt\hbox{\kern-0.28564pt\hbox{\rotatebox{10.0}{$\scriptscriptstyle\ell$}}\kern-0.28564pt}}}ip}(p/\alpha_{2},0,1)\rceil.$
(9)

Since the bankers rounding is non-differentiable, we use the straight through
estimator (STE) [3] to approximate the gradient through the round function
$\left\lfloor\cdot\right\rceil$ as a pass through operation, and
differentiating all other operations normally [34].

For activation quantization, there exists two cases. On the one hand, if $p$
is in the real domain (e.g., use PReLU or Tanh as non-linearity), the
activation quantizer is the same as $Q^{w}(\cdot)$. On the other hand, if $p$
is in the non-negative domain (applicable to ReLU activations), the quantized
values should be adapted accordingly. In this case, the activation quantizer
becomes:

$Q^{a}(p)=\lfloor{\rm c\mathchoice{\mathord{\lower
0.43057pt\hbox{\kern-0.57129pt\hbox{\rotatebox{10.0}{$\displaystyle\ell$}}\kern-0.57129pt}}}{\mathord{\lower
0.43057pt\hbox{\kern-0.57129pt\hbox{\rotatebox{10.0}{$\textstyle\ell$}}\kern-0.57129pt}}}{\mathord{\lower
0.43057pt\hbox{\kern-0.3999pt\hbox{\rotatebox{10.0}{$\scriptstyle\ell$}}\kern-0.3999pt}}}{\mathord{\lower
0.43057pt\hbox{\kern-0.28564pt\hbox{\rotatebox{10.0}{$\scriptscriptstyle\ell$}}\kern-0.28564pt}}}ip}(p/\alpha_{1},0,1)\rceil+\lfloor{\rm
c\mathchoice{\mathord{\lower
0.43057pt\hbox{\kern-0.57129pt\hbox{\rotatebox{10.0}{$\displaystyle\ell$}}\kern-0.57129pt}}}{\mathord{\lower
0.43057pt\hbox{\kern-0.57129pt\hbox{\rotatebox{10.0}{$\textstyle\ell$}}\kern-0.57129pt}}}{\mathord{\lower
0.43057pt\hbox{\kern-0.3999pt\hbox{\rotatebox{10.0}{$\scriptstyle\ell$}}\kern-0.3999pt}}}{\mathord{\lower
0.43057pt\hbox{\kern-0.28564pt\hbox{\rotatebox{10.0}{$\scriptscriptstyle\ell$}}\kern-0.28564pt}}}ip}(p-\alpha_{1})/\alpha_{2},0,1)\rceil.$
(10)

Note that we learn independent step sizes for weights and activations of each
quantized layer.

##### Remark 2

Above non-uniform step size quantization algorithm can be easily integrated in
the general deep learning training framework. It is worth noting that the
algorithm meets all the constraints derived in Section 3.3, which only have
requirements on the quantized values while having no requirements on the
quantization thresholds. We further visualize the non-uniform step sizes in
Section S2 in the supplementary file to provide more insights.

The proposed algorithm is featured in learning the non-uniform quantization
thresholds, which is different from two categories of methods. Compared with
methods using a uniform step size [51, 8, 13], our FATNN enjoys more flexible
step sizes to better fit the statistics of the data distribution. Moreover,
different from non-uniform LQ-Net [48] that updates the quantizer via closed-
form approximation and QIL [20] that introduces extra non-linear
transformation and hyperparameters with careful tuning during optimization,
our FATNN directly parameterizes and updates the quantization intervals by
efficient stochastic gradient descent, which shows better performance in
Section 5.

###  4.2 Calibration of Residual Block

It has been shown in the quantization literature that high-precision skip
connections are essential to reduce the accumulated quantization error and
ease the difficulty in propagating gradients through a low-precision network
due to the non-differentiable quantization function. However, few work pays
attention to the influence of quantization on the branch fusion in the
residual architecture.

Figure 1: (a). Classical residual block. (b). Calibration between different
branches by inserting an extra batch normalization (BN) layer into the
identity mapping path.

(a) Speedup on Qualcomm 821 GPU

(b) Speedup on Qualcomm 835 GPU

(c) Speedup on Nvidia 2080Ti GPU

Figure 2: Layer-wise speedup of ternary networks (‘ter’) versus 2-bit
counterparts (‘2-bit’) on different platforms.

As illustrated in Figure 1 (a), the distributions of the two inputs of the
element-wise addition, the non-negative block input $\bm{x}$ (from the
preceding residual block) and the output of the batch normalization layer, are
different. The large covariate shift between the two inputs results in
enlarged variance and value range for the output tensor. It might not cause a
problem for the full-precision network, as the representational capability
with 32-bit precision is enough to encode all the information. However, it can
lead to increased information loss for the low-precision network, because of
the deteriorated quantization error when quantizing the enlarged range of
value into a set of limited discrete values (e.g., ternary quantization).

To solve this problem, we propose to calibrate the distribution between the
skip connection and the low-precision branch. As illustrated in Figure 1(b),
the calibration can be done by inserting an extra batch normalization layer in
the identity-mapping skip connection path. The calibration helps to align the
two inputs of the element-wise addition, reducing the quantization error when
the output tensor is quantized in the succeeding block. Note that the extra
memory consumption is limited as the batch normalization layers only introduce
a few parameters (i.e., two scalar parameters for each after training). We
leverage the BN fusion when the preceding layer is a convolutional layer.
Otherwise, extra computation is introduced by the BN layer, typically in the
calibration branch. But the runtime overhead is negligible for deployment
since the computational cost of BN is orders of magnitude lower than that of
the convolutional layer. In practice, we find the execution time of all
“unfused” BN layers only occupies less than 1% of that of the whole network.

##  5 Experiments

###  5.1 Acceleration

In this paper, we propose a fast ternary inner product implementation with
$O(2N)$ complexity, which is potentially $2\times$ faster than the
implementation in previous works [51, 48]. To further justify its
effectiveness in practice, we develop the acceleration code (in C++ and
OpenCL) for binary, ternary and 2-bit convolutions444We include the
implementation details in Section S1 of the supplementary material..
Measurements of the actual execution time on different devices are conducted.
We cover both the embedded-side devices (Qualcomm 821 and Qualcomm 835555We
conduct experiments on Google Pixel and Xiaomi 6 phones which equip with the
Qualcomm 821/835 chips respectively. Any platform with the same chip is
expected to produce the similar result.) and server-side devices (Nvidia
2080Ti) to demonstrate the flexibility of our solution. We keep the
experimental setting the same across all devices.

Table 3: Execution time ($\rm ms$) and speedup ratios for overall quantized layers. “bin” means binary and “ter” is ternary. ‘Q821’ and ‘Q835’ indicate the platform of Qualcomm 821 and Qualcomm 835, respectively. We run 5 times and report the results with mean and standard deviation. Device | Network | bin | ter | 2-bit | bin vs. ter | bin vs. 2-bit | ter vs. 2-bit
---|---|---|---|---|---|---|---
Q835 | ResNet-18 |  12.1$\pm$0.2 |  20.1$\pm$0.2 |  43.0$\pm$0.7 | 1.7 | 3.6 | 2.1
ResNet-34 |  25.3$\pm$0.3 |  42.1$\pm$0.5 |  87.0$\pm$1.5 | 1.7 | 3.4 | 2.1
Q821 | ResNet-18 |  15.7$\pm$0.3 |  25.2$\pm$0.3 |  52.3$\pm$1.0 | 1.6 | 3.3 | 2.1
ResNet-34 |  32.3$\pm$0.6 |  51.4$\pm$0.6 |  105.2$\pm$2.1 | 1.6 | 3.3 | 2.0

####  5.1.1 Layer-wise Speedup

First, we report the layer-wise speedup in Figure 2, where convolution layers
(kernel size = $3\times 3$, padding = 1, stride = 1, the same number of input
and output channels and batch size = 1) with six different shape
configurations are tested. For the first four cases on each platform in Figure
2, we fix the channel number to be 64 and increase the resolution from 28 to
224. For the last two cases, we fix the resolution to be 56 and double the
channel number from 64 to 256. We report the relative acceleration ratios
between ternary and 2-bit models on Qualcomm 821, 835 and Nvidia 2080Ti
platforms in Figure 2. From Figure 2, we observe that, compared with the 2-bit
quantization, the proposed ternary quantization is much faster. Specifically,
the ternary convolutional layer is $1.6\times$ to $2.2\times$ faster on the
Qualcomm 821 platform, $1.4\times$ to $2.5\times$ faster on the Qualcomm 835
platform and $2.4\times$ to $3.5\times$ faster on the Nvidia 2080Ti platform.
The variance of the speedup on different platforms is caused by the macro-
architectures of different devices.

Table 4: Influence of the extra bit operations in the FATNN implementation. ‘base’ indicates the total layer-wise execution time ($\mu s$) of the ternary convolutional layer. ‘extra’ represents the layer-wise execution time of the extra bit operations described in Eqs. (5)-(7). We report both absolute time and relative ratios. We run 5 times and report the mean results. ‘Q821’, ‘Q835’ and ‘2080Ti’ indicate the platforms of Qualcomm 821, Qualcomm 835 and Nvidia 2080Ti, respectively.  Device |  | case1 | case2 | case3 | case4 | case5 | case6
---|---|---|---|---|---|---|---
Q821 | base | 594 | 1190 | 3425 | 12373 | 3631 | 13202
extra | 61 (10.3%) | 56 (4.7%) | 36 (1.1%) | 21 (0.2%) | 95 (2.6%) | 676 (5.1%)
Q835 | base | 485 | 1007 | 2847 | 11023 | 2606 | 12122
extra | 155 (32.0%) | 150 (14.9%) | 471 (16.5%) | 177 (1.6%) | 5 (0.2%) | 268 (2.2%)
2080Ti | base | 12 | 15.5 | 29 | 92.5 | 35.5 | 88.5
extra | 1 (8.3%) | 1 (6.5%) | 3 (10.3%) | 2.5 (2.7%) | 1.5 (4.2%) | 1.5 (1.7%)

####  5.1.2 Overall Speedup

Besides the layer-wise analysis, it is also interesting to get a knowledge of
the overall speed for some classical networks. We present the whole execution
time of all quantized layers (first and last layers are excluded while non-
linear and skip connection layers are included) for ResNet-18 and ResNet-34 on
ImageNet in Table 3. We set the batch size to 1. From Table 3, we can learn
that both the 2-bit and ternary models run faster than the theoretical speedup
versus the binary counterpart (2 and 4 respectively). Moreover, the speedup of
the ternary models compared with the 2-bit ones (last column) again
demonstrates that our proposed FATNN can run about $2\times$ faster than the
2-bit models or conventional TNNs.

####  5.1.3 Ablation Study

In Remark 1, we claim the influence of the extra bit operations is negligible.
To verify the impact, we test the exact execution time of the six cases
introduced in Section 5.1.1 on different devices in Table 4. The total
execution time of the implementation described in Section 3.3 is taken as the
baseline. We then measure the cost with respect to the extra bit operations
and list the absolute time and relative ratios in Table 4. From Table 4, we
observe that the influence is highly relative to the devices and layer shapes.
On Qualcomm 821, Qualcomm 835 and Nvidia 2080Ti, the relative ratios of the
extra bit operations range from $0.2\%$ to $10.3\%$, $0.2\%$ to $32.0\%$ and
$1.7\%$ to $10.3\%$ on the evaluated cases, respectively. Besides, the
occurrence of the $32.0\%$ ratio happens only on the small input shape ($\rm
c=64,w=h=28$) while most of the larger input sizes have relative small ratios.
Together with the overall speedup in Section 5.1.2, we can conclude that the
extra bit operations have a small impact on the total execution time.

###  5.2 Quantization Accuracy

We perform experiments on the ImageNet [39] dataset. The ImageNet contains
about 1.2 million training and 50K validation images of 1,000 object
categories. To verify the effectiveness of the proposed auxiliary learning
strategy, we compare with various representative quantization approaches,
including uniform approaches LSQ [13] and DoReFa-Net [51] and non-uniform
methods LQ-Net [48], HWGQ [6] and QN [47]. Ternary-oriented quantization
methods, such as RTN [25], TBN [43] and LSB [36] are also included.
Comparisons on other datasets, such as CIFAR-10, can be found in Section S4 in
the supplementary file.

####  5.2.1 Evaluation on ImageNet

##### Experimental setup.

For ImageNet classification, all the images are re-scaled with the shorter
edge to be 256. Training images are then randomly cropped into resolution of
$224\times 224$. After that, the images are normalized using the mean and
standard deviation. No additional augmentations are performed except the
random horizontal flip. Validation images follow a similar procedure except
the random crop is replaced with the center crop and no flip is applied. We
conduct experiments on the ResNet models [15]. As in previous works [51, 48],
we do not quantize the first and last layers. Following LSQ [13] and IR-net
[37], we leverage weight normalization along with the standard batch
normalization during training to make the optimization more stable. If not
specially mentioned, the initial learning rate is set to 1e-2 and the cosine
annealing decay is employed. Other default hyper-parameters include: SGD
optimizer with a momentum of $0.9$, a weight decay of 2e-5, and a maximum
training epoch of 90. The quantization related parameters $\alpha_{1}$ and
$\alpha_{2}$ are initialized to 1.0 for weights and activations in all
quantized layers. We initialize the quantized network with the pretrained
full-precision weights at the beginning of the quantization.

Table 5: Accuracy (%) comparisons between our FATNN and other algorithms on ImageNet. “A/W” in the second column indicates the bit configuration for activations and weights respectively. “ter” denotes ternary. Results for LSQ are based on our own implementation. Results for algorithms, including TBN, RTN, LSB, LQ-Net, HWGQ, DoReFa-Net and QN, are directly cited from the original papers. Method | A/W | ResNet-18 | ResNet-34 | ResNet-50
---|---|---|---|---
Top-1 | Top-5 | Top-1 | Top-5 | Top-1 | Top-5
32/32 | 69.8 | 89.1 | 73.3 | 91.4 | 76.1 | 92.9
FATNN (Ours) | ter/ter | 66.0 | 86.4 | 69.8 | 89.1 | 72.4 | 90.6
LSQ [13] | ter/ter | 64.7 | 85.6 | 69.0 | 88.8 | 71.2 | 90.1
RTN [25] | ter/ter | 64.5 | - | - | - | - | -
TBN [43] | ter/1 | 55.6 | 79.0 | 58.2 | 81.0 | - | -
LSB [36] | ter/1 | 62.0 | 83.6 | - | - | - | -
LSQ [13] | 2/1 | 64.9 | 85.8 | 69.1 | 88.8 | 71.0 | 90.0
LQ-Net [48] | 2/1 | 62.6 | 84.3 | 66.6 | 86.9 | 68.7 | 88.4
HWGQ [6] | 2/1 | 59.6 | 82.2 | 64.3 | 85.7 | 64.6 | 85.9
DoReFa-Net [51] | 2/1 | 53.4 | - | - | - | - | -
QN [47] | 2/1 | 63.4 | 84.9 | - | - | - | -

##### Performance analysis.

Firstly, we compare with the current best performed quantization algorithm
LSQ, which learns a uniform step size and thus is directly comparable to our
FATNN. To make a fair comparison, we implement LSQ under exactly the same
training process with FATNN. We report the quantization results in Table 5.
From Table 5, we observe steady Top-1 accuracy improvement of our FATNN over
LSQ on all comparing architectures in the ternary case. This result strongly
justifies the effectiveness of the proposed distribution-aware quantization
strategy. To be emphasized, our FATNN boosts the implementation speed of LSQ
by a factor of two according to Table 3 while still achieving the state-of-
the-art accuracy.

Secondly, to make a comprehensive comparison, we also include the results of
the “2-bit activation and binary weight” networks which have a similar
computational complexity $O(2N)$ with our FATNN. We can learn from Table 5
that our FATNN is able to achieve steady accuracy gain over the non-uniform
quantization algorithms, such as LQ-Net, HWGQ and QN. For example, FATNN
outperforms LQ-Net by 3.4% on the Top-1 accuracy on ResNet-18. This justifies
the superiority of our gradient-based approach to learn the step sizes.
Moreover, compared to the uniform quantization algorithms, for example DoReFa-
Net and LSQ, we also achieve the best performance on various architectures.
This further shows that the employed algorithm can better fit the statistics
of the data distribution to reduce the quantization error.

####  5.2.2 Ablation Study

In this section, we further investigate the effect of the distribution-aware
quantization strategies proposed in Section 4, including the non-uniform
quantization functions and the calibration of residual block, to the final
performance respectively. We conduct the experiments with ResNet-18 on
ImageNet and report the results in Table 6. We observe that the branch
calibration strategy deteriorates the performance in the full-precision
setting, leading to $0.2\%$ Top-1 accuracy drop. In contrast, the calibration
strategy brings $0.6\%$ performance improvement to our ternary FATNN in terms
of the Top-1 accuracy. It empirically justifies that calibrating the residual
block is particularly designed for the low-precision network. Moreover, the
non-uniform step size quantization of FATNN further brings $0.7\%$ Top-1
accuracy increase, which justifies that the parameterized quantization
thresholds can fit the statistics of the data distribution effectively.

Table 6: Impact of the distribution-aware ternary quantization algorithms with ResNet-18 on ImageNet. ‘N’ indicates the non-uniform step size quantization algorithm. ‘C’ implies the distribution calibration between the skip connection and the low-precision branch. Method | A/W | N | C | Top-1 | Top-5
---|---|---|---|---|---
| 32/32 |  |  | 69.8 | 89.1
| 32/32 |  | ✓ | 69.6 | 88.9
LSQ [13] (baseline) | ter/ter |  |  | 64.7 | 85.6
FATNN (Ours) | ter/ter | ✓ |  | 65.4 | 86.2
ter/ter | ✓ | ✓ | 66.0 | 86.4

##  6 Conclusion

In this paper, we have proposed a fast ternary neural network, named FATNN.
Specifically, we emphasize that the underlying implementation and the
quantization algorithm are highly correlated and should be co-designed. From
the implementation perspective, we exploit the “non-overflow” property to
design a novel ternary inner product with fully bit operations. As a result,
our FATNN can achieve $2\times$ less complexity than the conventional TNNs.
Moreover, we have designed an efficient quantization algorithm in accordance
with the constraints of the implementation. Extensive experiments demonstrate
that FATNN improves the previous TNNs in both execution time and quantization
accuracy. Thus, we advocate to rethink the value of TNNs and hope that FATNN
can serve as a strong benchmark for further research.

## References

  * [1] Yu Bai, Yu-Xiang Wang, and Edo Liberty.  Proxquant: Quantized neural networks via proximal operators.  In Proc. Int. Conf. Learn. Repren., 2019.
  * [2] Chaim Baskin, Eli Schwartz, Evgenii Zheltonozhskii, Natan Liss, Raja Giryes, Alex M Bronstein, and Avi Mendelson.  Uniq: Uniform noise injection for non-uniform quantization of neural networks.  arXiv preprint arXiv:1804.10969, 2018.
  * [3] Yoshua Bengio, Nicholas Léonard, and Aaron Courville.  Estimating or propagating gradients through stochastic neurons for conditional computation.  arXiv preprint arXiv:1308.3432, 2013.
  * [4] Joseph Bethge, Marvin Bornstein, Adrian Loy, Haojin Yang, and Christoph Meinel.  Training competitive binary neural networks from scratch.  arXiv preprint arXiv:1812.01965, 2018.
  * [5] Joseph Bethge, Haojin Yang, Christian Bartz, and Christoph Meinel.  Learning to train a binary neural network.  arXiv preprint arXiv:1809.10463, 2018.
  * [6] Zhaowei Cai, Xiaodong He, Jian Sun, and Nuno Vasconcelos.  Deep learning with low precision by half-wave gaussian quantization.  In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 5918–5926, 2017\.
  * [7] Tianqi Chen, Thierry Moreau, Ziheng Jiang, Lianmin Zheng, Eddie Yan, Haichen Shen, Meghan Cowan, Leyuan Wang, Yuwei Hu, Luis Ceze, et al.  TVM: An automated end-to-end optimizing compiler for deep learning.  In USENIX Symp. Operating Systems Design & Implementation, pages 578–594, 2018.
  * [8] Jungwook Choi, Zhuo Wang, Swagath Venkataramani, Pierce I-Jen Chuang, Vijayalakshmi Srinivasan, and Kailash Gopalakrishnan.  Pact: Parameterized clipping activation for quantized neural networks.  arXiv preprint arXiv:1805.06085, 2018.
  * [9] Yoojin Choi, Mostafa El-Khamy, and Jungwon Lee.  Learning low precision deep neural networks through regularization.  ArXiv, abs/1809.00095, 2018.
  * [10] Matthieu Courbariaux, Yoshua Bengio, and Jean-Pierre David.  Binaryconnect: Training deep neural networks with binary weights during propagations.  In Proc. Adv. Neural Inf. Process. Syst., pages 3123–3131, 2015\.
  * [11] Lei Deng, Peng Jiao, Jing Pei, Zhenzhi Wu, and Guoqi Li.  Gxnor-net: Training deep neural networks with ternary weights and activations without full-precision memory under a unified discretization framework.  Neural Networks, 100:49–58, 2018.
  * [12] Ruizhou Ding, Ting-Wu Chin, Zeye Liu, and Diana Marculescu.  Regularizing activation distribution for training binarized deep networks.  In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 11408–11417, 2019.
  * [13] Steven K Esser, Jeffrey L McKinstry, Deepika Bablani, Rathinakumar Appuswamy, and Dharmendra S Modha.  Learned step size quantization.  In Proc. Int. Conf. Learn. Repren., 2020.
  * [14] Yiwen Guo, Anbang Yao, Hao Zhao, and Yurong Chen.  Network sketching: Exploiting binary structure in deep cnns.  In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 5955–5963, 2017\.
  * [15] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 770–778, 2016\.
  * [16] Lu Hou and James T Kwok.  Loss-aware weight quantization of deep networks.  In Proc. Int. Conf. Learn. Repren., 2018.
  * [17] Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio.  Binarized neural networks.  In Proc. Adv. Neural Inf. Process. Syst., pages 4107–4115, 2016\.
  * [18] Andrey Ignatov, Radu Timofte, William Chou, Ke Wang, Max Wu, Tim Hartley, and Luc Van Gool.  Ai benchmark: Running deep neural networks on android smartphones.  In Proc. Eur. Conf. Comp. Vis., 2018.
  * [19] Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko.  Quantization and training of neural networks for efficient integer-arithmetic-only inference.  In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., 2018.
  * [20] Sangil Jung, Changyong Son, Seohyung Lee, Jinwoo Son, Jae-Joon Han, Youngjun Kwak, Sung Ju Hwang, and Changkyu Choi.  Learning to quantize deep networks by optimizing quantization intervals with task loss.  In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 4350–4359, 2019\.
  * [21] Alex Krizhevsky and Geoffrey Hinton.  Learning multiple layers of features from tiny images.  2009\.
  * [22] Cong Leng, Hao Li, Shenghuo Zhu, and Rong Jin.  Extremely low bit neural network: Squeeze the last bit out with ADMM.  CoRR, abs/1707.09870, 2017.
  * [23] Fengfu Li, Bo Zhang, and Bin Liu.  Ternary weight networks.  arXiv preprint arXiv:1605.04711, 2016.
  * [24] Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf.  Pruning filters for efficient convnets.  arXiv preprint arXiv:1608.08710, 2017.
  * [25] Yuhang Li, Xin Dong, Sai Qian Zhang, Haoli Bai, Yuanpeng Chen, and Wei Wang.  Rtn: Reparameterized ternary network.  In AAAI, 2020.
  * [26] Darryl Lin, Sachin Talathi, and Sreekanth Annapureddy.  Fixed point quantization of deep convolutional networks.  In Proc. Int. Conf. Mach. Learn., pages 2849–2858, 2016.
  * [27] Min Lin, Qiang Chen, and Shuicheng Yan.  Network in network.  arXiv preprint arXiv:1312.4400, 2013.
  * [28] Xiaofan Lin, Cong Zhao, and Wei Pan.  Towards accurate binary convolutional neural network.  In Proc. Adv. Neural Inf. Process. Syst., pages 344–352, 2017.
  * [29] Hanxiao Liu, Karen Simonyan, and Yiming Yang.  Darts: Differentiable architecture search.  In Proc. Int. Conf. Learn. Repren., 2019.
  * [30] Zechun Liu, Wenhan Luo, Baoyuan Wu, Xin Yang, Wei Liu, and Kwang-Ting Cheng.  Bi-real net: Binarizing deep network towards real-network performance.  arXiv preprint arXiv:1811.01335, 2018.
  * [31] Christos Louizos, Matthias Reisser, Tijmen Blankevoort, Efstratios Gavves, and Max Welling.  Relaxed quantization for discretized neural networks.  In Proc. Int. Conf. Learn. Repren., 2019.
  * [32] Asit Mishra and Debbie Marr.  Apprentice: Using knowledge distillation techniques to improve low-precision network accuracy.  In Proc. Int. Conf. Learn. Repren., 2018.
  * [33] Eunhyeok Park, Junwhan Ahn, and Sungjoo Yoo.  Weighted-entropy-based quantization for deep neural networks.  In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 5456–5464, 2017\.
  * [34] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer.  Automatic differentiation in pytorch.  In Proc. Adv. Neural Inf. Process. Syst. Workshops, 2017.
  * [35] Antonio Polino, Razvan Pascanu, and Dan Alistarh.  Model compression via distillation and quantization.  In Proc. Int. Conf. Learn. Repren., 2018.
  * [36] Hadi Pouransari, Zhucheng Tu, and Oncel Tuzel.  Least squares binary quantization of neural networks.  In Proc. IEEE Conf. Comp. Vis. Patt. Recogn. Workshops, 2020.
  * [37] Haotong Qin, Ruihao Gong, Xianglong Liu, Mingzhu Shen, Ziran Wei, Fengwei Yu, and Jingkuan Song.  Forward and backward information retention for accurate binary neural networks.  In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., 2020.
  * [38] Mohammad Rastegari, Vicente Ordonez, Joseph Redmon, and Ali Farhadi.  Xnor-net: Imagenet classification using binary convolutional neural networks.  In Proc. Eur. Conf. Comp. Vis., pages 525–542, 2016.
  * [39] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al.  Imagenet large scale visual recognition challenge.  Int. J. Comp. Vis., 115(3):211–252, 2015.
  * [40] Karen Simonyan and Andrew Zisserman.  Very deep convolutional networks for large-scale image recognition.  In Proc. Int. Conf. Learn. Repren., 2015.
  * [41] Wei Tang, Gang Hua, and Liang Wang.  How to train a compact binary neural network with high accuracy?  In Proc. AAAI Conf. on Arti. Intel., pages 2625–2631, 2017.
  * [42] Yaman Umuroglu, Nicholas Fraser, Giulio Gambardella, Michaela Blott, Philip Leong, Magnus Jahre, and Kees Vissers.  Finn: A framework for fast, scalable binarized neural network inference.  In Proc. ACM/SIGDA Int. Symp. Field-Programmable Gate Arrays, pages 65–74. ACM, 2017.
  * [43] Diwen Wan, Fumin Shen, Li Liu, Fan Zhu, Jie Qin, Ling Shao, and Heng Tao Shen.  Tbn: Convolutional neural network with ternary inputs and binary weights.  In Proc. Eur. Conf. Comp. Vis., September 2018.
  * [44] Peisong Wang, Qinghao Hu, Yifan Zhang, Chunjie Zhang, Yang Liu, and Jian Cheng.  Two-step quantization for low-bit neural networks.  In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 4376–4384, 2018\.
  * [45] Athanasios Xygkis, Lazaros Papadopoulos, David Moloney, Dimitrios Soudris, and Sofiane Yous.  Efficient winograd-based convolution kernel implementation on edge devices.  In Proc. 55th Annual Design Automation Conf., DAC ’18, pages 136:1–136:6, New York, NY, USA, 2018.
  * [46] Haojin Yang, Martin Fritzsche, Christian Bartz, and Christoph Meinel.  Bmxnet: An open-source binary neural network implementation based on mxnet.  In Proc. of the ACM Int. Conf. on Multimedia., pages 1209–1212. ACM, 2017.
  * [47] Jiwei Yang, Xu Shen, Jun Xing, Xinmei Tian, Houqiang Li, Bing Deng, Jianqiang Huang, and Xian-sheng Hua.  Quantization networks.  In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., June 2019.
  * [48] Dongqing Zhang, Jiaolong Yang, Dongqiangzi Ye, and Gang Hua.  Lq-nets: Learned quantization for highly accurate and compact deep neural networks.  In Proc. Eur. Conf. Comp. Vis., 2018.
  * [49] Jianhao Zhang, Yingwei Pan, Ting Yao, He Zhao, and Tao Mei.  dabnn: A super fast inference framework for binary neural networks on arm devices.  In Proc. ACM Int. Conf. Multimedia, pages 2272–2275, 2019.
  * [50] Tianli Zhao, Xiangyu He, Jian Cheng, and Jing Hu.  Bitstream: Efficient computing architecture for real-time low-power inference of binary neural networks on CPUs.  In Proc. ACM Int. Conf. Multimedia, pages 1545–1552, 2018.
  * [51] Shuchang Zhou, Yuxin Wu, Zekun Ni, Xinyu Zhou, He Wen, and Yuheng Zou.  Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients.  arXiv preprint arXiv:1606.06160, 2016.
  * [52] Chenzhuo Zhu, Song Han, Huizi Mao, and William J Dally.  Trained ternary quantization.  In Proc. Int. Conf. Learn. Repren., 2017.
  * [53] Bohan Zhuang, Chunhua Shen, Mingkui Tan, Lingqiao Liu, and Ian Reid.  Towards effective low-bitwidth convolutional neural networks.  In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., 2018.
  * [54] Bohan Zhuang, Chunhua Shen, Mingkui Tan, Lingqiao Liu, and Ian Reid.  Structured binary neural network for accurate image classification and semantic segmentation.  In Proc. IEEE Conf. Comp. Vis. Patt. Recogn., 2019.
  * [55] Zhuangwei Zhuang, Mingkui Tan, Bohan Zhuang, Jing Liu, Yong Guo, Qingyao Wu, Junzhou Huang, and Jinhui Zhu.  Discrimination-aware channel pruning for deep neural networks.  In Proc. Adv. Neural Inf. Process. Syst., pages 875–886, 2018.
  * [56] Barret Zoph and Quoc V Le.  Neural architecture search with reinforcement learning.  In Proc. Int. Conf. Learn. Repren., 2017.

## Supplementary Material

##  S1 Acceleration Details

As a fundamental operation in convolutional neural networks, the vector inner
product is one of the core components in acceleration. In this section, we
first elaborate more about the “non-overflow” property and the same
parallelism degree as well as their importance to the fast implementation.
Then we present the detailed implementation of the proposed fast ternary inner
product and introduce its application in the convolutional and fully-connected
layers.

Non-overflow property: If the range of input quantized vectors and the
multiplication result in the inner product keeps the same, we term it “non-
overflow” property. The multiplication result here, indicates the intermediate
result of multiplication between the two input quantized vectors. Binary
quantized values $\\{1,-1\\}$ and ternary quantized values $\\{1,0,-1\\}$ are
examples. We attribute the fast implementation of our FATNN to the “non-
overflow” property because it enables the same parallelism degree for the
multiplication (i.e., $\rm xnor$) and accumulation (i.e., $\rm popcount$)
operations. Specifically, for the BNNs case, 8 full-precision values can be
packed into one byte. When $\rm xnor$ is employed for the multiplication, the
parallelism degree is 8. Moreover, $\rm popcount$ also accumulates 8 data at
the same time (same parallelism degree with the $\rm xnor$ operation). For the
TNNs case, only 4 full-precision values can be packed into one byte. Thus, the
parallelism degree of our $\rm{TM(\cdot)}$ in Eq. (3) is 4. Interestingly, the
$\rm popcount$ operation can also accumulate the 4 data simultaneously.
However, if standard 2-bit quantization values $\\{0,1,2,3\\}$ are leveraged,
the multiplication by combination of bit-wise operators (such as $\rm xnor$)
can own the parallelism degree of 4 if packed in byte. However, 4 bits are
required to encode the multiplication result. Thus, the parallelism degree for
the accumulation procedure is 2 at most (less than 4). Consequently, the
computational efficiency will be halved. Based on the analysis, it can be
learnt that the “non-overflow” property is an important attribute to enable
the fast implementation of ternary and binary networks.

Input: (1): Full-precision weight vector $\bm{w}\in\mathbb{R}^{N}$ and
activation vector $\bm{a}\in\mathbb{R}^{N}$. (2): Pre-allocated temporary
buffer $\hat{\bm{w}}$ and $\hat{\bm{a}}$ with unsigned char type in the length
of $N/4$. (3): Quantization parameters $\alpha_{1}^{w},\alpha_{2}^{w}$ and
$\alpha_{1}^{a},\alpha_{2}^{a}$ which are used to parameterize the step sizes
for weights and activations, respectively.

Output: The ternary inner product result $z$.

1 Step (1): data packing;

2 for $i\leftarrow 0$ to $\frac{N-1}{4}$ do

3 Pack 4 values of $\bm{a}[4i:4i+3]$ into one unsigned char and store it in
$\hat{\bm{a}}[i]$;

4 Pack 4 values of $\bm{w}[4i:4i+3]$ into one unsigned char and store it in
$\hat{\bm{w}}[i]$;

5

6 end for

7Step (2): ternary inner product;

8 $acc$ = 0;

9 for $i\leftarrow 0$ to $\frac{N-1}{4}$ do

10 $acc$ += popcount(TM($\hat{\bm{w}}[i]$, $\hat{\bm{a}}[i]$));

11

12 end for

$z=acc-N$

Algorithm 1 Fast Ternary Inner Product

As explained in the Section 3, we design a fast implementation for TNNs by
exploiting the “non-overflow” property. Algorithm 1 summarizes the inference
flow of the proposed fast ternary inner product. In Algorithm 1, lines $2\sim
5$ quantize the full-precision input vector into the codec of the quantized
values. As 2 bits are required to encode one ternary value, 4 full-precision
values can be packed into one byte. It is worth noting that it is also
possible to pack the full-precision data into other data structures. For
example, 8 full-precision data can be packed into $\rm short$ type or 16 full-
precision data can be packed into 32-bit $\rm int$ type. More specifically,
during the packing, each full-precision data is compared with the
corresponding quantization thresholds characterized by {$\alpha_{1}$,
$\alpha_{2}$} and assigned to the corresponding codec value. After that, based
on Eqs. (5), (6), (7), the accumulation is performed as lines $8\sim 10$ in
Algorithm 1. Finally the logic level inner product result is obtained
according to Eq. (3) in line 11.

Following the implementation of the fast ternary inner product, the
convolutional layer can be realized by first expanding the input activation
into a matrix ($\rm im2co\mathchoice{\mathord{\lower
0.43057pt\hbox{\kern-0.57129pt\hbox{\rotatebox{10.0}{$\displaystyle\ell$}}\kern-0.57129pt}}}{\mathord{\lower
0.43057pt\hbox{\kern-0.57129pt\hbox{\rotatebox{10.0}{$\textstyle\ell$}}\kern-0.57129pt}}}{\mathord{\lower
0.43057pt\hbox{\kern-0.3999pt\hbox{\rotatebox{10.0}{$\scriptstyle\ell$}}\kern-0.3999pt}}}{\mathord{\lower
0.43057pt\hbox{\kern-0.28564pt\hbox{\rotatebox{10.0}{$\scriptscriptstyle\ell$}}\kern-0.28564pt}}}$)
and then conducting the matrix multiplication ($\rm gemm$). To enhance the
efficiency, the data packing in Algorithm 1 is integrated into the $\rm
im2co\mathchoice{\mathord{\lower
0.43057pt\hbox{\kern-0.57129pt\hbox{\rotatebox{10.0}{$\displaystyle\ell$}}\kern-0.57129pt}}}{\mathord{\lower
0.43057pt\hbox{\kern-0.57129pt\hbox{\rotatebox{10.0}{$\textstyle\ell$}}\kern-0.57129pt}}}{\mathord{\lower
0.43057pt\hbox{\kern-0.3999pt\hbox{\rotatebox{10.0}{$\scriptstyle\ell$}}\kern-0.3999pt}}}{\mathord{\lower
0.43057pt\hbox{\kern-0.28564pt\hbox{\rotatebox{10.0}{$\scriptscriptstyle\ell$}}\kern-0.28564pt}}}$
operation. After that, the matrix multiplication is realized based on the
inner product step in Algorithm 1. Tricks, such as winograd [45], are commonly
employed in the $\rm gemm$ operation, however we do not integrate these tricks
for simplicity. The fully-connected layer is similar to the implementation of
the convolutional layer, which can be regarded as a special case of the latter
with kernel size being equal to the feature map resolution. Other operations,
such as the ReLU non-linearity and skip connection layers, can be fused in the
$\rm im2co\mathchoice{\mathord{\lower
0.43057pt\hbox{\kern-0.57129pt\hbox{\rotatebox{10.0}{$\displaystyle\ell$}}\kern-0.57129pt}}}{\mathord{\lower
0.43057pt\hbox{\kern-0.57129pt\hbox{\rotatebox{10.0}{$\textstyle\ell$}}\kern-0.57129pt}}}{\mathord{\lower
0.43057pt\hbox{\kern-0.3999pt\hbox{\rotatebox{10.0}{$\scriptstyle\ell$}}\kern-0.3999pt}}}{\mathord{\lower
0.43057pt\hbox{\kern-0.28564pt\hbox{\rotatebox{10.0}{$\scriptscriptstyle\ell$}}\kern-0.28564pt}}}$
procedure. Besides, we fuse the batch normalization layers into the
corresponding convolutional or fully-connected layers.

##  S2 Visualization of Non-uniform Step Sizes

Figure S3: The proposed non-uniform ternary quantization for (a) a tensor in
the real domain; (b) a tensor which only contains non-negative values. The
vertical axis represents the quantized domain and the horizontal axis denotes
the real domain. The green curve indicates the distribution of the full-
precision tensor and the blue line shows the quantized values by discretizing
the full precision data according to the learned thresholds. We aim to learn
the optimal step size of each quantization level.

(a) layer1_1_conv2

(b) layer2_1_conv2

(c) layer3_1_conv2

(d) layer4_1_conv2

Figure S4: Weight quantization for ResNet-18.

(a) layer1_1_conv2

(b) layer2_1_conv2

(c) layer3_1_conv2

(d) layer4_1_conv2

Figure S5: Activation quantization for ResNet-18.

In this paper, we discretize the full-precision tensor into the ternary
quantized values with trainable non-uniform step sizes. For each quantized
layer, we learn two parameters $\alpha=\\{\alpha_{1},\alpha_{2}\\}$ for
weights and activations separately. As illustrated in Figure S3, the
quantization thresholds are directly related to the learnt parameters
$\alpha$. When the two scale factors are identical ($\alpha_{1}=\alpha_{2}$),
the proposed quantization algorithm reduces to the uniform step size
quantization [13, 51, 20]. Thus, it is interesting to investigate the
properties of the learnt $\alpha$. We plot the distribution of the full-
precision weights and activations as well as the corresponding quantization
thresholds on ResNet-18 in Figure S4 and S5, respectively. We list the
statistics of four layers in ResNet-18. Besides, the max value, min value and
the two quantization thresholds of the tensors are marked in each sub-figure.
On the one hand, from Figure S4, we observe that the distribution of the
weight in the model varies a lot in different layers. In order to reduce the
information loss during quantization, we propose to learn the quantization
thresholds automatically to better fit the data distribution. On the other
hand, Figure S5 demonstrates that, the full-precision activations consist of
dense relative small values and sparse relative large values. The number of
elements of each interval is unbalanced. It can be seen from Figure S5 that
the quantization step sizes learnt based on the stochastic gradient descent
are non-uniform ones and differ at different layers.

##  S3 More Execution Time Benchmarks

We present more acceleration benchmark results in this section.

First, for convenience of comparison, we list the the exact execution time for
the layer-wise benchmark described in experiments section in Table S7 and
Table S8, respectively. We use Q821/Q835/2080Ti to indicate the Qualcomm 821,
835 and Nvidia 2080Ti separately. Besides, “bin” represents binary and “ter”
means ternary in Table S7 and Table S8. We measure the layer-wise execution
time for convolution layers (kernel size = $3\times 3$, padding = 1, stride =
1, the same number of input and output channels and batch size = 1) with six
different shape configurations. For the first four cases, we fix the channel
number to be 64 and increase the resolution from 28 to 224. For the last two
cases, we fix the resolution to be 56 and double the channel number from 64 to
256.

Table S7: Exact execution time ($\rm ms$) on embedded-side platforms. We run 5 times and report the mean results. Device | A/W | case1 | case2 | case3 | case4 | case5 | case6
---|---|---|---|---|---|---|---
Q821 | bin/bin | 0.6 | 1.2 | 2.6 | 8.3 | 2.3 | 6
ter/ter | 0.9 | 1.7 | 4.4 | 14.8 | 4.2 | 13.5
2/2 | 1.9 | 3.5 | 8 | 24.9 | 7.2 | 21.2
Q835 | bin/bin | 0.7 | 0.9 | 1.9 | 6.1 | 1.8 | 4.8
ter/ter | 0.9 | 1.5 | 3.7 | 13.2 | 3.2 | 12.7
2/2 | 2.1 | 3 | 6.5 | 20.5 | 5.6 | 15.8
Table S8: Exact execution time ($\mu s$) on server-side platforms. We run 5 times and report the mean results. Device | A/W | case1 | case2 | case3 | case4 | case5 | case6
---|---|---|---|---|---|---|---
2080Ti | bin/bin | 11 | 12.5 | 19.5 | 56.5 | 22 | 55.5
ter/ter | 11 | 14.5 | 26 | 90 | 34 | 87
2/2 | 38 | 44 | 73.5 | 274 | 91 | 217.5

##  S4 Evaluation on CIFAR-10

###  S4.1 Training hyper-parameters on CIFAR-10

We follow the hyper-parameter setting in previous works [48, 44] to train the
networks on the classification task. Specifically, for ResNet-20 on CIFAR-10,
we train up to 200 epochs. The initial learning rate starts from 0.1 and is
divided by 10 at epoch of 82 and 123, respectively. We use a weight decay of
1e-4 and a batch size of 128. For VGG-Small on CIFAR-10, the learning rate
begins with 0.02 and is divided by 10 at epoch of 80 and 160, separately. The
weight decay is set to be 5e-4, batch size to be 128 and total epochs to be
200. For NIN on CIFAR-10, we train 90 epochs with the initial learning rate to
be 1e-2. The learning rate is divided by 10 at epoch of 30 and 60. Weight
decay is set to be 1e-5.

###  S4.2 Comparison on CIFAR-10

We further demonstrate the effectiveness of the proposed method on CIFAR-10
[21] dataset. For CIFAR-10, the input images are firstly padded with 4 pixels
and then cropped into 32$\times$32 samplings. Random horizontal flip is
employed for data augmentation. NIN [27], VGG-Small [40] and ResNet-20 are
employed for the evaluation. Comparisons between our FATNN and other
quantization algorithms are listed in Table S9.

Table S9: Top-1 accuracy (%) comparisons between our FATNN and other
algorithms, including RTN [25], HWGQ [6], LQ-Net [48] and TSQ [44] on CIFAR-10
dataset.

Method | A/W | ResNet-20 | NIN | VGG-Small
---|---|---|---|---
| 32/32 | 92.1 | 89.8 | 93.8
FATNN | ter/ter | 90.2 | 89.9 | 93.7
RTN | ter/ter | - | 89.6 | -
HWGQ | 2/1 | - | - | 92.5
LQ-Net | 2/1 | 88.4 | - | 93.4
TSQ | 2/ter | - | - | 93.5
LQ-Net | 2/2 | 90.2 | - | 93.5

From Table S9, we observe that for ResNet-20 and VGG-Small, the proposed
method is able to achieve comparable accuracy for ternary networks compared
with higher bit counterparts (“2/2” or “2/ter”) quantized by LQ-Net and TSQ.
Moreover, it surpasses RTN, HWGQ and LQ-Net on all the corresponding “2/1” or
“ter/ter” networks. For NIN, our FATNN even beats the full-precision
counterpart. It implies that network quantization has certain kind of
regularization effect to suppress the over-fitting problem. Overall, the
proposed FATNN demonstrates superior performance on the CIFAR-10 dataset.