# Non-Brownian diffusion and chaotic rheology of autophoretic disks

R. Kailasham  Department of Chemical Engineering, Carnegie Mellon University,
Pittsburgh, PA 15213, USA  Aditya S. Khair<EMAIL_ADDRESS>Department
of Chemical Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, USA

###### Abstract

The dynamics of a two dimensional autophoretic disk is quantified as a minimal
model for the chaotic trajectories undertaken by active droplets. Via direct
numerical simulations, we show that the mean-square displacement of the disk
in a quiescent fluid is linear at long times. Surprisingly, however, this
apparently diffusive behavior is non-Brownian, owing to strong cross-
correlations in the displacement tensor. The effect of a shear flow field on
the chaotic motion of a 2D autophoretic disk is examined. Here, the stresslet
on the disk is chaotic for weak shear flows; a dilute suspension of such disks
would exhibit a chaotic shear rheology. This chaotic rheology is quenched
first into a periodic state and ultimately a steady state as the flow strength
is increased.

Droplets that undergo isotropic solubilization at their interface can exhibit
spontaneous self-propulsion [1, 2, 3, 4], proceeding through a cascade of
instabilities as the Peclet number, i.e., the dimensionless ratio of the
strength of advective to diffusive transport of the products of
solubilization, is increased. Such active droplets display varying patterns of
motion, including straight, curvilinear, or meandering, before entering a
chaotic regime at high values of the Péclet number [5, 6]. The occurrence of
chaotic dynamics is intriguing since the fluid flow around the drop is at low
Reynolds number, where nonlinear inertial forces are absent.

In this Letter, we consider a two-dimensional autophoretic disk as a minimal
model for an active drop and discover two important features about its
dynamics. First, although such particles undergo normal diffusion at long time
scales [7], we show that they are not Brownian due to strong cross-
correlations in their displacement components. Second, we calculate the motion
of an autophoretic disk in a shear flow field. Here, the velocity and
stresslet of the disk are chaotic at low values of the shear rate, implying
that a dilute suspension of disks would exhibit a chaotic shear rheology. On
increasing the strength of the shear flow (at fixed Péclet number) the chaos
is quenched, such that the rheology becomes periodic and eventually steady.
Thus, we illustrate that external flows have a dramatic influence on the
dynamics of autophoretic disks, and, more generally, active droplet emulsions.

Our model consists of a circular disk of radius $a^{*}$ that is immersed in an
incompressible Newtonian solvent of viscosity $\eta^{*}$ at temperature
$T^{*}$. A steady simple shear flow with shear rate $\dot{\gamma}^{*}$ is
imposed on the fluid, whose flow obeys the Stokes equations. The disk
isotropically emits solute particles from its surface at a rate
$\mathcal{A}^{*}$, and the solute diffusivity is $D^{*}$. The uniform solute
concentration far away from the disk is $C^{*}_{\infty}$, and the excess
solute concentration with respect to this far-field value is denoted by
$c^{*}=C^{*}-C^{*}_{\infty}$. The disk interacts with the solute particles
through a short-ranged potential of characteristic length $b^{*}$, which is
much smaller than the radius of the disk [8]. These interactions set up a
tangential phoretic slip velocity, whose magnitude is determined by a mobility
parameter $\mathcal{M}^{*}=\pm k_{B}T^{*}b^{*2}/\eta^{*}$, where $k_{B}$ is
Boltzmann’s constant, and the concentration gradient on the surface of the
disk. The sign of the mobility parameter is positive (negative) for repulsive
(attractive) interactions [9]. Following [10, 11], the scales for length,
time, fluid velocity, pressure, and concentration are defined as $a^{*}$,
${a^{*}D^{*}}/{|\mathcal{A}^{*}\mathcal{M}^{*}|}$,
$U^{*}={|\mathcal{A}^{*}\mathcal{M}^{*}|}/{D^{*}}$, ${\eta^{*}U^{*}}/{a^{*}}$,
and ${a^{*}|\mathcal{A}^{*}|}/{D^{*}}$, respectively. It is convenient to
define the scaled emission rate and mobility parameter as
$A=\mathcal{A}^{*}/|\mathcal{A}^{*}|$ and
$M=\mathcal{M}^{*}/|\mathcal{M}^{*}|$, respectively. The onset of phoretic
self-propulsion requires $|AM|=1$ [10, 12], and we set $A=M=1$ henceforth. The
intrinsic Péclet number quantifies the chemical activity of the disk and is
defined as $Pe={a^{*}|\mathcal{A}^{*}\mathcal{M}^{*}|}/{D^{*2}}$. The
dimensionless shear rate is given by
$\epsilon=\dot{\gamma}^{*}{a^{*}D^{*}}/{|\mathcal{A}^{*}\mathcal{M}^{*}|}$,
which can be viewed as a ratio of phoretic to flow time scales.

The flow and solute concentration fields around the autophoretic disk placed
in an ambient shear flow are calculated by simultaneously solving the
transient advection-diffusion equation for the solute concentration, and the
Stokes equations

$Pe\left(\dfrac{\partial c}{\partial
t}+\bm{v}\cdot\boldsymbol{\nabla}c\right)=\nabla^{2}c;\boldsymbol{\nabla}p=\nabla^{2}\bm{v},\,\boldsymbol{\nabla}\cdot\bm{v}=0,$
(1)

with $t$, $\bm{v}$ and $p$ denoting the dimensionless time, velocity, and
pressure respectively, and subject to $\left(\partial c/\partial
r\right)(1,\theta,t)=-A$, and the attenuation condition $c(r\to\infty,t)\to
0$. In numerical computations we employ a finite size of the computational
domain, $R_{o}$, and impose the far-field Dirichlet condition $c(r=R_{o})=0$.
Considering a frame-of-reference attached to the centroid of the disk, the
slip velocity at the surface of the disk is given by
$\bm{v}\left(r=1,\theta\right)=\bm{v}_{\text{s}}=M\nabla_{\text{s}}c$, with
the surface-gradient operator defined as
$\nabla_{\text{s}}=\left(1/r\right)\,\bm{e}_{\theta}\partial/\partial\theta$,
where $\bm{e}_{\theta}$ is the unit vector in the tangential direction,
$\theta$. The far-field velocity is given by
$\bm{v}\left(R_{o},\theta\right)=-\bm{U}+\epsilon y\bm{e}_{x}$, where
$\bm{e}_{x}$ the Cartesian unit vector in the $x$-direction, $y$ is the co-
ordinate in the direction of the shear gradient, and the phoretic velocity
$\bm{U}(t)$ may be written in terms of the slip velocity using the reciprocal
theorem [13] as

$\bm{U}(t)=-\dfrac{1}{2\pi}\int_{0}^{2\pi}\bm{v}_{\text{s}}d\theta.$ (2)

The time-dependent base state, $c_{0}(r,\theta,t)$, corresponding to the state
of zero phoretic motion ($\bm{U}(t)=0$) is computed by solving the transient
advection-diffusion equation subject to an initial condition of
$c_{0}(r,\theta,t=0)=0$ over a duration $t_{\text{b}}$, at specified $Pe$ and
$\epsilon$. The value of the base state at $t_{\text{b}}$ is perturbed,
$c(r=1,\theta,t=0)=c_{0}(r=1,\theta,t_{\text{b}})-\delta_{\text{per}}\cos\theta$,
with $|\delta_{\text{per}}|<1$, and used as the initial condition while
solving eq. (1). A spectral element solver, described in detail in [14, 11],
is used for the numerical solution of (1).

Figure 1: (Colour online) Dynamics of a two-dimensional autophoretic disk in a
quiescent fluid ($\epsilon=0$) at $Pe=20$. (a) Trajectory in the $x-y$ plane;
solid and hollow circles denote the beginning and end of the trajectory,
respectively, (b) elements of the mean square displacement tensor
$\boldsymbol{\beta}(\tau)$, with the $xx$ component denoted by  , $yy$
component by  , $xy$ component by  and $\text{tr}(\boldsymbol{\beta})$ by  ,
(c) velocity autocorrelation, and (d) stresslet component autocorrelation. The
black dotted line in (c) has the functional form
$f_{1}\exp[-\lambda_{1}\tau]+f_{2}\exp[-\lambda_{2}\tau]$, with the parameter
values given by $f_{1}=0.42$, $\lambda_{1}=0.006$, $f_{2}=0.58$, and
$\lambda_{2}=0.0006$. Absolute values of $\beta_{xy}$ have been plotted in
(b). Figure 2: (Colour online) (a) Trajectory in the $x-y$ plane; (b) elements
of the mean square displacement tensor $\boldsymbol{\beta}(\tau)$, with the
same legend scheme as in fig. 1 (b), (c) velocity autocorrelation, and (d)
stresslet component autocorrelation for an autophoretic particle at $Pe=20$,
placed in a shear flow field of strength $\epsilon=0.01$.

The solution methodology for an active disk suspended in a quiescent fluid
($\epsilon=0$) is slightly different, as explained in detail in the
Supplemental Material [15]. A value of $R_{o}=200$ has been used in all the
simulations, after establishing convergence [15]. Using a procedure analogous
to that in [16], the stresslet induced by the two-dimensional disk is derived
as

$\bm{S}=-2\int_{0}^{2\pi}\left[\bm{n}\bm{v}_{\text{s}}+\bm{v}_{\text{s}}\bm{n}\right]d\theta,$
(3)

where $\bm{n}=\bm{e}_{r}$ is the unit outward normal on the surface of the
disk. In a dilute suspension of non-interacting autophoretic disks, the $xy$
component of the stresslet would contribute to the effective shear viscosity
of the suspension.

Results for an autophoretic disk suspended in a quiescent fluid ($\epsilon=0$)
are presented first (fig. 1), to provide context for our findings on the
dynamics and rheology of disks placed in shear flow. The base state for a two-
dimensional disk has no fluid flow and time-dependent solute diffusion, which
we evaluate by numerically solving the transient diffusion equation. Our
approach is in contrast to [7, 17] who assume a steady base state, in which
the purely diffusive concentration field is set to zero at a finite distance
$R_{o}$. In [15] we show that both approaches yield qualitatively similar
dynamics in the chaotic regime; however, there are important differences
regarding the onset of self-propulsion.

An autophoretic disk in a quiescent fluid undergoes qualitative changes in its
trajectory as the Péclet number is increased, evolving from a stationary state
to one of steady linear motion, followed by a meandering regime that
transitions via an intermittency scenario to chaotic motion at sufficiently
large $Pe$ [15, 7]. The chaotic dynamics is characterized by a transition to
diffusive scaling of the mean square displacement of the particle. We choose a
representative value of $Pe=20$ that is deep in the chaotic regime. The mean
square displacement tensor of the disk at a lag time $\tau$ is
$\boldsymbol{\beta}(\tau)=\left<\left[\bm{r}(t+\tau)-\bm{r}(t)\right]\left[\bm{r}(t+\tau)-\bm{r}(t)\right]\right>$,
where $\bm{r}(t)$ denotes the instantaneous position of the particle at time
$t$, and the angular brackets represent the average evaluated over the
trajectory of the particle. The off-diagonal components of this $2\times 2$
tensor are identical, and the scalar mean square displacement is given by the
trace of $\beta$, i.e, $\text{MSD}\equiv\text{tr}(\bm{\beta})$. Even though
$\text{MSD}\sim\tau$ at long times, seemingly indicative of Brownian motion,
the cross correlation $\beta_{xy}$ does not vanish, which is markedly
different from a passive Brownian particle (fig. 1 (b)). To represent the data
on logarithmic axes, we plot the absolute values of the off-diagonal term,
$\beta_{xy}$. The growth of $\beta_{xy}$ with respect to the lag time proceeds
with abrupt dips, which have been observed by Suga  _et al._ [5] in their
experiments on active liquid crystal droplets swimming in a surfactant
solution in a two-dimensional geometry. We notice that the dips in fig. 1 (b)
correspond to time-instances where the raw values of $\beta_{xy}$ undergo a
change of sign. The experimental trajectory of large droplets in [5] are
predominantly loop-like structures involving sharp changes in the droplet
orientation, and consequently the overall MSD shows an oscillatory behavior.
Such loop-like shapes represent only a small portion of the overall trajectory
of the chaotic autophoretic disks examined in our work (fig. 1 (a)); and,
consequently, the overall MSD does not oscillate in time, even though the off-
diagonal term does. The velocity and stress autocorrelations are evaluated as
$C_{v}(\tau)=\left<\bm{U}(t)\cdot\bm{U}(t+\tau)/|\bm{U}(t)||\bm{U}(t+\tau)|\right>$,
and $C_{s}(\tau)=\left<s(t)s(t+\tau)\right>/\left<s^{2}(t)\right>$,
respectively, where $s$ represents either $S_{xy}$ or $S_{yy}$. We observe
that $C_{v}$ in fig. 1 (c) is well-approximated by a sum of two exponentials.
The autocorrelation of both the $xy$ and $yy$ components of the stresslet
decay nearly identically, going to zero at $\tau\approx 300$ (fig. 1 (d)).
This is to be expected because there is no ambient flow that would lead to a
distinction between the components of the stresslet.

Figure 3: Time series of velocity magnitude at shear rate values of (a)
$\epsilon=0.01$, (b) $\epsilon=0.03$, (c) $\epsilon=0.04$, and (d)
$\epsilon=0.6$, at a fixed value of the Péclet number, $Pe=20$.

A widely used reduced-order framework for self-propelled micro-scale objects
is the Active Brownian Particle (ABP) [18, 19, 20], wherein a particle is
assumed to move with a constant speed, while its instantaneous orientation is
selected from a Gaussian white noise distribution. Peruani and Morelli [21]
(hereafter PM) prescribe a model for a self-propelling particle in two
dimensions which allows for fluctuations in both the magnitude and direction
of velocity. By assuming that the instantaneous velocity magnitude may be
selected stochastically from a Poisson distribution, and the orientational
distribution function obeys a diffusion equation, they analytically derive an
expression for MSD and $C_{v}$ of such a particle. Both the ABP and PM models
predict an MSD that transitions to long-time diffusion following an early-time
ballistic behavior. However, while the ABP predicts that a single-exponential
may be used to describe the variation in $C_{v}$, the PM model posits that the
velocity autocorrelation is a sum of two exponentials. In fig. 1 (c), we show
agreement with this bi-exponential form and the computed velocity
autocorrelation of an autophoretic disk. Therefore, we conclude that the
velocity time-series of an autophoretic disk may be well-approximated by a
reduced-order model that allows for the instantaneous velocity and orientation
to be chosen independently.

Figure 4: (Colour online) Time series of $xy$ component of the stresslet at
shear rate values of (a) $\epsilon=0.01$, (b) $\epsilon=0.03$, (c)
$\epsilon=0.04$, and (d) $\epsilon=0.6$, at a fixed value of the Péclet
number, $Pe=20$.

For a passive Brownian sphere in shear flow, the mean square displacement in
the flow direction ($\beta_{xx}$) scales as $\tau^{3}$ at long times, an
example of Taylor dispersion, whereas that in the direction of the gradient
($\beta_{yy}$) scales linearly with the lag time [22]. The cross-correlation,
$\beta_{xy}$, scales as $\tau^{2}$. The effect of a shear-flow field on the
dynamics of an autophoretic disk is illustrated in fig. 2. The position of the
autophoretic disk is evaluated as,

$\begin{split}r_{y}(t+\Delta t)&=r_{y}(t)+U_{y}(t)\Delta t;\\\ r_{x}(t+\Delta
t)&=r_{x}(t)+\left[U_{x}(t)+\epsilon y(t)\right]\Delta t,\end{split}$ (4)

while the expression for the mean square displacement tensor remains
unchanged. In fig. 2 (b), the components of $\boldsymbol{\beta}(\tau)$ for an
autophoretic disk placed in a shear-flow field of $\epsilon=0.01$ are plotted
as a function of the lag time, and it is observed that they all scale as
$\tau^{2}$, in marked contrast to the scaling observed for a passive disk. The
autocorrelation of the phoretic velocity is not significantly altered in
comparison to the quiescent case (fig. 2 (c)). The autocorrelation of the
stresslet components (fig. 2 (d)) do not vanish in the long-time limit due to
the presence of an ambient shear flow. Furthermore, the $xy$ and $yy$
components of the stress tensor may be clearly differentiated from their
autocorrelation signals, unlike that in the quiescent case, since the imposed
flow has a fixed direction, along the $x$-axis.

Figure 5: (Colour online) Power spectrum of the $xy$ component of the
stresslet at various values of the shear rate ($\epsilon$), and a fixed Péclet
number of $Pe=20$. The ordinates of the data-series have been multiplied by a
scale factor to render them well-spaced on the $y$-axis, for clarity. From
bottom to top, the values of $\epsilon$ (and the associated scale factors) are
0.036 ($10^{-22}$), 0.0366($10^{-15}$), 0.037($10^{-8}$), 0.0372(1),
0.0373($10^{10}$), and 0.0374($10^{18}$).

Magnetic fields [23], buoyant forces [24, 25], and mechanical impulses [26]
may be used to relaminarize a high-Reynolds number flow [27]. The use of an
external force field in quenching the chaotic variation of the velocity field
around an autophoretic particle has very recently been demonstrated through
numerical simulations [11]. The influence of gravity on counteracting the
chemical activity-induced chaotic motion of oil droplets settling in an
aqueous surfactant solution has also been studied recently through experiments
[28]. In fig. 3, the effect of increasing the shear strength on the phoretic
velocity of the particle at a fixed value of the Péclet number is illustrated,
over a representative time window. At low values of $\epsilon$, the time-
series is chaotic, and then gradually settles into a periodic pattern upon
increasing $\epsilon$, before vanishing completely as the shear rate is
increased further. The triangular wave pattern of the phoretic velocity in
fig. 3 (c) is qualitatively similar to that predicted for self-propelled two-
dimensional droplets [17]. The external flow field, therefore, modulates the
self-propulsive motion of the autophoretic disk. A similar trend is observed
in fig. 4 for the stresslet component $S_{xy}$, which notably attains a time-
independent value at sufficiently large shear rates. A negative value for the
steady-state stresslet has also been observed for the case of axisymmetric
swimmers with a “pusher” type flow pattern [10, 16]. We therefore anticipate
that a dilute suspension of active droplets would exhibit a chaotic rheology
at small values of the shear rate, before first transitioning to a time
periodic and ultimately steady rheology as the shear rate is increased.

A more detailed picture of the chaotic-to-steady quenching transition is
apparent from the power spectrum of the stresslet, as plotted in fig. 5 over a
narrow window in the shear rate window. At $\epsilon=0.036$, the broadband
spectrum signifies chaotic dynamics. An increase in the shear rate is
accompanied by a coalescence of various frequencies from the broadband
spectrum into discrete peaks of the periodic motion in fig. 4 (c).

The minimal model of an autophoretic disk can qualitatively capture several
features of active droplets observed in experiments [5, 3, 6]. We have shown
using such a model that the dynamics of an autophoretic disk, in a quiescent
fluid and in the presence of a shear flow field, is markedly different from a
passive Brownian particle. The dynamics is also richer than that of an ABP
with respect to the bi-exponential decay of the velocity autocorrelation
function. Furthermore, we showed that the rheology of a dilute suspension of
such disks is chaotic at small values of the shear rate; effectively,
increasing the shear provides a route to removing chaotic dynamics. We expect
these findings to hold, at least qualitatively, in experimentally realizable
active droplet systems. Finally, our work may be useful for the synthesis and
manipulation of active droplet emulsions. Here, one may have to account for
the coupled chaotic dynamics of many active droplets, which is certainly an
interesting direction for future work.

###### Acknowledgements.

We gratefully acknowledge the support of the Charles E. Kaufmann Foundation of
the Pittsburgh Foundation.

## References

  * Izri  _et al._ [2014] Z. Izri, M. N. Van Der Linden, S. Michelin, and O. Dauchot, Self-propulsion of pure water droplets by spontaneous marangoni-stress-driven motion, Phys. Rev. Lett. 113, 248302 (2014).
  * Maass  _et al._ [2016] C. C. Maass, C. Krüger, S. Herminghaus, and C. Bahr, Swimming Droplets, Ann. Rev. Cond. Matt. Phys. 7, 171 (2016).
  * Suda  _et al._ [2021] S. Suda, T. Suda, T. Ohmura, and M. Ichikawa, Straight-to-Curvilinear Motion Transition of a Swimming Droplet Caused by the Susceptibility to Fluctuations, Phys. Rev. Lett. 127, 088005 (2021).
  * Michelin [2023] S. Michelin, Self-propulsion of chemically-active droplets, Ann. Rev. Fluid Mech. (in press) (2023).
  * Suga  _et al._ [2018] M. Suga, S. Suda, M. Ichikawa, and Y. Kimura, Self-propelled motion switching in nematic liquid crystal droplets in aqueous surfactant solutions, Phys. Rev. E 97, 062703 (2018).
  * Hokmabad  _et al._ [2021] B. V. Hokmabad, R. Dey, M. Jalaal, D. Mohanty, M. Almukambetova, K. A. Baldwin, D. Lohse, and C. C. Maass, Emergence of Bimodal Motility in Active Droplets, Phys. Rev. X 11, 011043 (2021).
  * Hu  _et al._ [2019] W. F. Hu, T. S. Lin, S. Rafai, and C. Misbah, Chaotic Swimming of Phoretic Particles, Phys. Rev. Lett. 123, 238004 (2019).
  * Anderson [1989] J. Anderson, Colloid Transport By Interfacial Forces, Ann. Rev. Fluid Mech. 21, 61 (1989).
  * Michelin and Lauga [2014] S. Michelin and E. Lauga, Phoretic self-propulsion at finite Péclet numbers, J. Fluid Mech. 747, 572 (2014).
  * Michelin  _et al._ [2013] S. Michelin, E. Lauga, and D. Bartolo, Spontaneous autophoretic motion of isotropic particles, Phys. Fluids 25, 061701 (2013).
  * Kailasham and Khair [2022] R. Kailasham and A. S. Khair, Dynamics of forced and unforced autophoretic particles, J. Fluid Mech. 948, A41 (2022).
  * Saha  _et al._ [2021] S. Saha, E. Yariv, and O. Schnitzer, Isotropically active colloids under uniform force fields: From forced to spontaneous motion, J. Fluid Mech. 916, A47 (2021).
  * Squires and Bazant [2006] T. M. Squires and M. Z. Bazant, Breaking symmetries in induced-charge electro-osmosis and electrophoresis, J. Fluid Mech. 560, 65 (2006).
  * Chisholm  _et al._ [2016] N. G. Chisholm, D. Legendre, E. Lauga, and A. S. Khair, A squirmer across Reynolds numbers, J. Fluid Mech. 796, 233 (2016).
  * [15] See Supplemental Material for a discussion of the solution methodology adopted for autophoretic disks in a quiescent fluid ($\epsilon=0$), and convergence studies with respect to the size of the computational domain and the timestep width used for numerical integration.
  * Lauga and Michelin [2016] E. Lauga and S. Michelin, Stresslets Induced by Active Swimmers, Phys. Rev. Lett. 117, 148001 (2016).
  * Li [2022] G. Li, Swimming dynamics of a self-propelled droplet, J. Fluid. Mech. 934, A20 (2022).
  * Bechinger  _et al._ [2016] C. Bechinger, R. D. Leonardo, C. Reichhardt, and G. Volpe, Active particles in complex and crowded environments, Rev. Mod. Phys. 88, 045006 (2016).
  * Marchetti  _et al._ [2016] M. C. Marchetti, Y. Fily, S. Henkes, A. Patch, and D. Yllanes, Minimal model of active colloids highlights the role of mechanical interactions in controlling the emergent behavior of active matter, Curr. Opin. Colloid Interface Sci. 21, 34 (2016).
  * Zeitz  _et al._ [2017] M. Zeitz, K. Wolff, and H. Stark, Active Brownian particles moving in a random Lorentz gas, Eur. Phys. J. E 40, 23 (2017).
  * Peruani and Morelli [2007] F. Peruani and L. G. Morelli, Self-propelled particles with fluctuating speed and direction of motion in two dimensions, Phys. Rev. Lett. 99, 010602 (2007).
  * Sandoval  _et al._ [2014] M. Sandoval, N. K. Marath, G. Subramanian, and E. Lauga, Stochastic dynamics of active swimmers in linear flows, J. Fluid Mech. 742, 50 (2014).
  * Moffatt [1967] H. K. Moffatt, On the suppression of turbulence by a uniform magnetic field, J. Fluid Mech. 28, 571 (1967).
  * He  _et al._ [2016] S. He, K. He, and M. Seddighi, Laminarisation of flow at low Reynolds number due to streamwise body force, J. Fluid Mech. 809, 31 (2016).
  * Marensi  _et al._ [2021] E. Marensi, S. He, and A. P. Willis, Suppression of turbulence and travelling waves in a vertical heated pipe, J. Fluid Mech. 919, A17 (2021).
  * Kühnen  _et al._ [2018] J. Kühnen, B. Song, D. Scarselli, N. B. Budanur, M. Riedl, A. P. Willis, M. Avila, and B. Hof, Destabilizing turbulence in pipe flow, Nat. Phys. 14, 386 (2018).
  * Hanratty and Mito [2009] T. J. Hanratty and Y. Mito, A unifying explanation for the damping of turbulence by additives or external forces, Flow Turbulence Combust 83, 293 (2009).
  * Castonguay  _et al._ [2022] A. C. Castonguay, R. Kailasham, C. M. Wentworth, C. H. Meredith, A. S. Khair, and L. D. Zarzar, Gravitational settling of active droplets, 10.26434/chemrxiv-2022-43g7d (2022).


# F8Net: Fixed-Point 8-bit Only Multiplication for Network Quantization

Qing Jin1,2 Jian Ren1 Richard Zhuang1 Sumant Hanumante1 Zhengang Li2
Zhiyu Chen3 Yanzhi Wang2 Kaiyuan Yang3 Sergey Tulyakov1
1Snap Inc. 2Northeastern University, USA 3 Rice University, USA Work done
during an internship at Snap Inc. Code is available at
https://github.com/snap-research/F8Net.  Qing Jin1,2 Jian Ren1 Richard Zhuang1
Sumant Hanumante1 Zhengang Li2
Zhiyu Chen3 Yanzhi Wang2 Kaiyuan Yang3 Sergey Tulyakov1
1Snap Inc. 2Northeastern University, USA 3 Rice University, USA Work done
during an internship at Snap Inc. Code is available at
https://github.com/snap-research/F8Net.

###### Abstract

Neural network quantization is a promising compression technique to reduce
memory footprint and save energy consumption, potentially leading to real-time
inference. However, there is a performance gap between quantized and full-
precision models. To reduce it, existing quantization approaches require high-
precision INT32 or full-precision multiplication during inference for scaling
or dequantization. This introduces a noticeable cost in terms of memory,
speed, and required energy. To tackle these issues, we present F8Net, a novel
quantization framework consisting of only fixed-point 8-bit multiplication. To
derive our method, we first discuss the advantages of fixed-point
multiplication with different formats of fixed-point numbers and study the
statistical behavior of the associated fixed-point numbers. Second, based on
the statistical and algorithmic analysis, we apply different fixed-point
formats for weights and activations of different layers. We introduce a novel
algorithm to automatically determine the right format for each layer during
training. Third, we analyze a previous quantization algorithm—parameterized
clipping activation (PACT)—and reformulate it using fixed-point arithmetic.
Finally, we unify the recently proposed method for quantization fine-tuning
and our fixed-point approach to show the potential of our method. We verify
F8Net on ImageNet for MobileNet V1/V2 and ResNet18/50. Our approach achieves
comparable and better performance, when compared not only to existing
quantization techniques with INT32 multiplication or floating-point
arithmetic, but also to the full-precision counterparts, achieving state-of-
the-art performance.

##  1 Introduction

Real-time inference on resource-constrained and efficiency-demanding platforms
has long been desired and extensively studied in the last decades, resulting
in significant improvement on the trade-off between efficiency and accuracy
(Han et al., 2015; Liu et al., 2018; Mei et al., 2019; Tanaka et al., 2020; Ma
et al., 2020; Mishra et al., 2020; Liang et al., 2021; Jin et al., 2021; Liu
et al., 2021). As a model compression technique, quantization is promising
compared to other methods, such as network pruning (Tanaka et al., 2020; Li et
al., 2021; Ma et al., 2020; 2021a; Yuan et al., 2021) and slimming (Liu et
al., 2017; 2018), as it achieves a large compression ratio (Krishnamoorthi,
2018; Nagel et al., 2021) and is computationally beneficial for integer-only
hardware. The latter one is especially important because many hardwares (e.g.,
most brands of DSPs (Ho, 2015; QCOM, 2019)) only support integer or fixed-
point arithmetic for accelerated implementation and cannot deploy models with
floating-point operations. However, the drop in performance, such as
classification accuracy, caused by quantization errors, restricts wide
applications of such methods (Zhu et al., 2016).

To address this challenge, many approaches have been proposed, which can be
categorized into simulated quantization, integer-only quantization, and fixed-
point quantization (Gholami et al., 2021). Fig. 1 shows a comparison between
these implementations. For simulated quantization, previous works propose to
use trainable clipping-levels (Choi et al., 2018), together with scaling
techniques on activations (Jin et al., 2020b) and/or gradients (Esser et al.,
2019), to facilitate training for the quantized models. However, some
operations in these works, such as batch normalization (BN), are conducted
with full-precision to stabilize training (Jin et al., 2020b; Esser et al.,
2019), limiting the practical application of integer-only hardware. Meanwhile,
integer-only quantization, where the model inference can be implemented with
integer multiplication, addition, and bit shifting, has shown significant
progress in recent studies (Jacob et al., 2018; Yao et al., 2021; Kim et al.,
2021). Albeit floating-point operations are removed to enable models running
on devices with limited support of operation types, INT32 multiplication is
still required for these methods. On the other hand, fixed-point quantization,
which also applies low-precision logic for arithmetic, does not require INT32
multiplication or integer division. For example, to replace multiplication by
bit shifting, Jain et al. (2019) utilize trainable power-of-2 scale factors to
quantize the model.

(a) Full-precision.

(b) Simulated quant.

(c) Integer-only quant.

(d) Fixed-point quant.

Figure 1:  Inspired by Gholami et al. (2021), we show the comparison of full-
precision model (presented in (a)) and different quantizations settings: (b)
simulated quantization; (c) integer-only quantization; and (d) fixed-point
quantization. Note the combination of last two operations in integer-only
quantization is termed as dyadic scaling in literature (Yao et al., 2021).

In this work, we adopt fixed-point quantization. Our work differs from
previous efforts (Jain et al., 2019) in three major aspects. First, to
determine the minimum error quantization threshold, we conduct statistical
analysis on fixed-point numbers. Second, we unify parameterized clipping
activation (PACT) and fixed-point arithmetic to achieve high performance and
high efficiency. Third, we discuss and propose quantization fine-tuning
methods for different models. We dub our method as F8Net, as it consists in
only Fixed-point 8-bit multiplication employed for Network quantization. We
thoroughly study the problem with fixed-point numbers, where only INT8
multiplication is involved, without any INT32 multiplication, neither
floating-point nor fixed-point types. Throughout this paper we focus on 8-bit
quantization, the most widely supported case for different devices and is
typically sufficient for efficiency and performance requirements. Our
contribution can be elaborated as follows.

  * •

We show 8-bit fixed-point number is able to represent a wide range of values
with negligible relative error, once the format is properly chosen (see Fig. 3
and Fig. 4). This critical characteristic enables fixed-point numbers a much
stronger representative capability than integer values.

  * •

We propose a method to determine the fixed-point format, also known as
fractional length, for weights and activations using their variance. This is
achieved by analyzing the statistical behaviors of fixed-point values of
different formats, especially those quantized from random variables with
normal distribution of different variances. The analysis reveals the
relationship between relative quantization error and variance, which further
helps us build an approximated formula to determine the fractional length from
the variance.

  * •

We develop a novel training algorithm for fixed-point models by unifying
fixed-point quantization and PACT (Choi et al., 2018). Besides, we show the
impact of fractional length sharing for residual blocks, which is also
important to obtain good performance for quantized models.

  * •

We validate our approach for various models, including MobileNet V1/V2 and
ResNet18/50 on ImageNet for image classification, and demonstrate better
performance than existing methods that resort to 32-bit multiplication. We
also integrate the recent proposed fine-tuning method to train quantized
models from pre-trained full-precision models with ours for further
verification.

##  2 Related Work

Quantization is one of the most widely-used techniques for neural network
compression (Courbariaux et al., 2015; Han et al., 2015; Zhu et al., 2016;
Zhou et al., 2016; 2017; Mishra et al., 2017; Park et al., 2017; Banner et
al., 2018), with two types of training strategies: Post-Training Quantization
directly quantizes a pre-trained full-precision model (He & Cheng, 2018; Nagel
et al., 2019; Fang et al., 2020a; b; Garg et al., 2021); Quantization-Aware
Training uses training data to optimize quantized models for better
performance (Gysel et al., 2018; Esser et al., 2019; Hubara et al., 2020;
Tailor et al., 2020). In this work, we focus on the latter one, which is
explored in several directions. One area uses uniform-precision quantization
where the model shares the same precision (Zhou et al., 2018; Wang et al.,
2018; Choukroun et al., 2019; Gong et al., 2019; Langroudi et al., 2019; Jin
et al., 2020a; Bhalgat et al., 2020; Chen et al., 2020; Yang et al., 2020;
Darvish Rouhani et al., 2020; Oh et al., 2021). Another direction studies
mixed-precision that determines bit-width for each layer through search
algorithms, aiming at better accuracy-efficiency trade-off (Dong et al., 2019;
Wang et al., 2019; Habi et al., 2020; Fu et al., 2020; 2021; Yang & Jin, 2020;
Zhao et al., 2021a; b; Ma et al., 2021b). There is also binarization network,
which only applies 1-bit (Rastegari et al., 2016; Hubara et al., 2016; Cai et
al., 2017; Bulat et al., 2020; Guo et al., 2021). Despite the fact that
quantization helps reduce energy consumption and inference latency, it is
usually accompanied by performance degradation. To alleviate this problem,
several methods are proposed.

One type of effort focuses on simulated quantization. The strategy is to leave
some operations, e.g., BN, in full-precision for the stabilized training of
quantized models (Choi et al., 2018; Esser et al., 2019; Jin et al., 2020b).
Nevertheless, these methods limit the application of the quantized models on
resource-demanding hardware, such as DSP, where full-precision arithmetic is
not supported for accelerated computing (QCOM, 2019; Ho, 2015). To completely
eliminate floating-point operations from the quantized model, integer-only
quantization techniques emulate the full-precision multiplication by 32-bit
integer multiplication followed by bit shifting (Jacob et al., 2018; Zhu et
al., 2020; Wu et al., 2020; Yao et al., 2021; Kim et al., 2021). However, the
calculation of INT32 multiplication in these works requires one more
operation, which results in extra energy and higher latency (Gholami et al.,
2021). In parallel, recent work (Jain et al., 2019) proposes to restrict all
scaling factors as power-of-2 values for all weights and activations, which
belongs to fixed-point quantization methods (Lin et al., 2016; Jain et al.,
2019; Kim & Kim, 2021; Mitschke et al., 2019; Enderich et al., 2019b; Chen et
al., 2017; Enderich et al., 2019a; Zhang et al., 2020; Goyal et al., 2021).
This enables the model to only incorporate INT8 or even INT4 multiplications,
followed by INT32 bit shifting. However, there still a lack of a thorough
study of the benefits of using fixed-point arithmetic. Also, the power-of-2
scaling factors are directly determined from the training data without
theoretical analysis and guidance. In this work, we give an extensive
analysis, especially on the potential and theoretical principle of using
fixed-point values for quantized models, and demonstrate that with proper
analysis and design, a model quantized with only INT8 multiplication involved
is able to achieve comparable and even better performance to the integer-only
methods implemented with INT32 multiplication.

##  3 Analysis of Fixed-Point Representation

In this section, we first introduce the fixed-point multiplication (Smith et
al., 1997; Tan & Jiang, 2018) and analyze the distribution of weight from
different layers in a well-trained full-precision model (Sec. 3.1). We then
investigate the statistical property of fixed-point numbers, and demonstrate
the potential of approximating full-precision values by 8-bit fixed-point
numbers with different formats (Sec. 3.2). After that, we study the
relationship between standard deviation of random variables and the optimal
fixed-point format with the smallest quantization error. Finally, we derive an
approximated formula relating the standard deviation and fixed-point format,
which is verified empirically and employed in our final algorithms (Sec. 3.3).

###  3.1 Advantages of Fixed-Point Arithmetic

(a) Weight range.

(b) Weight and activation fractional length.

Figure 2:  (a) Value range of effective weight (see Sec. 4.2) for a pre-
trained full-precision (FP) model, and (b) fractional lengths of each layer
for a well-trained fixed-point model for MobileNet V2.

Fixed-point number is characterized by its format, which includes both the
word length indicating the whole bit-width of the number and the fractional
length (FL) characterizing the range and resolution of the represented values
(Smith et al., 1997). Fixed-point arithmetic—especially fixed-point
multiplication—is widely utilized for applications in, e.g., digital signal
processing (Smith et al., 1997; Tan & Jiang, 2018). Compared with integer or
floating-point multiplication, fixed-point multiplication has two major
characteristics: First, multiplying two fixed-point numbers is more efficient
than multiplying two floating-point numbers, especially on resource-
constrained devices such as DSP. Second, it is more powerful than its integer
counterpart due to its versatility and the representative ability of fixed-
point numbers (there can be tens of different implementations for fixed-point
multiplication but only one for integer and floating-point ones (Smith et al.,
1997)). This efficiency and versatility make fixed-point quantization a more
appealing solution than integer-only quantization. Specifically, as shown in
Fig. 2(a), the scales of weights from different layers in a pre-trained full-
precision model can vary in orders, ranging from less than $0.1$ to nearly
$4$. Direct quantization with only integers inevitably introduces considerable
quantization error, unless more precision and more operations are involved,
such as using INT32 multiplication together with bit shifting for scaling as
shown in Fig. 1(c). On the other hand, employing fixed-point numbers has the
potential to reduce quantization error without relying on high-precision
multiplication, as weights and activations from different layers have the
extra degree of using different formats during quantization. Indeed, as shown
in Fig. 2(b) for a well-trained MobileNet V2 with 8-bit fixed-point numbers,
the fractional lengths for weights and activations vary from layer to layer.
This raises the question of how to determine the formats for each layer. In
the following, we study this for 8-bit fixed-point models.

(a) Signed quant. for Gaussian R.V.

(b) Unsigned quant. for Rectified Gaussian R.V.

Figure 3:  Representing potential for 8-bit signed (a) and unsigned (b) fixed-
point numbers with different formats. The figures plot the relationship
between relative quantization error and the standard deviation for different
fixed-point formats. Both are experimented on zero-mean Gaussian random
variables (R.V.), with ReLU applied on (b).

###  3.2 Statistical Analysis for Fixed-Point Format

For a predefined bit-width, integer, which is a special case of fixed-point
numbers with zero fractional length, has a predefined set of values that it
can take, which severely constrains the potential of integer-only
quantization. On the other hand, fixed-point numbers, with an extra degree of
freedom, i.e., the fractional length, are able to represent a much wider range
of full-precision values by selecting the proper format, and thus they are
more suitable for quantization. As an example, Fig. 3 shows the relative
quantization error with 8-bit fixed-point values using different formats for a
set of random variables, which are sampled from normal distributions (both
signed and unsigned, with the latter processed by ReLU before quantization)
with zero-mean and different standard deviations $\sigma$ (more experimental
details in Appx. 7.2). From the experiments, we make the following two
observations.

Observation 1: Fixed-point numbers with different formats have different
optimal representing regions, and the minimum relative error and optimal
standard deviation (annotated as a star) varies for different fractional
lengths (Fig. 3). This is because the format controls the value magnitude and
the representation resolution (the least significant bit).

Observation 2: Larger fractional lengths are more robust to represent smaller
numbers, while smaller fractional lengths are more suitable for larger ones.
For a given standard deviation, using small fractional length has the risk of
underflow, while large fractional length might cause overflow issue.
Specifically, integers (black curves in Fig. 4) are much more prone to
underflow issues and have large relative errors for small enough values to
quantize.

(a) Optimal fractional length and minimum relative error for signed quant.

(b) Relationship between threshold standard deviation and fractional length
for signed quant.

(c) Optimal fractional length and minimum relative error for unsigned quant.

(d) Relationship between threshold standard deviation and fractional length
for unsigned quant.

Figure 4:  Determining optimal fractional length from standard deviation. (a)
and (c) illustrate optimal fractional length and minimum relative quantization
error against standard deviation for signed and unsigned 8-bit fixed-point
quantization for Gaussian and rectified Gaussian random variables. (b) and (d)
show the relationship between threshold standard deviation and fractional
length.

###  3.3 Choosing Optimal Fixed-Point Format

With the above observations, we are interested in answering two questions:

(1) Can we achieve a small fixed-point quantization error for a wide range of
full-precision values by always using the optimal fractional length
corresponding to the smallest relative error?

To answer this, we first plot the smallest possible relative error amongst all
the candidate fixed-point formats against the standard deviation. As shown in
red lines from Fig. 4(a) and Fig. 4(c), for zero-mean normal distribution, by
always choosing the optimal fixed-point format, we are able to achieve a
relative quantization error smaller than 1% for standard deviation with a
range of order of at least around $3$. For example, for signed quantization,
the standard deviation can range from $0.1$ to around $40$ to achieve less
than 1% error, and for unsigned quantization, the standard deviation can range
from $0.1$ to $100$. The experiments verify our presumption that using fixed-
point values with the optimal formats is able to achieve negligible
quantization error.

(2) Can we have a simple way to determine the optimal fractional length?

To answer this, we plot the optimal fractional length from the statistics of
the full-precision values against the standard deviation, as shown in the blue
lines in Fig. 4(a) and Fig. 4(c). We find that the threshold $\sigma$ value
corresponding to the jumping point is almost equidistant on the log scale of
the standard deviation. This is expected as the representing region of
different formats are differed by a factor of 2’s exponents. Plotting the
threshold standard deviation (on a log-scale) against the corresponding
optimal fractional length (Fig. 4(b) and Fig. 4(d)), we find their
relationship is almost linear, leading to the following semi-empirical
approximating formulas to determine the optimal fractional length
$\mathrm{FL}^{*}$ from the standard deviation (more discussion in Appendix
7.7):

$\displaystyle\mathrm{Signed:}\qquad\mathrm{FL}^{*}$
$\displaystyle\approx\lfloor\log_{2}\frac{40}{\sigma}\rfloor,$
$\displaystyle\mathrm{Unsigned:}\qquad\mathrm{FL}^{*}$
$\displaystyle\approx\lfloor\log_{2}\frac{70}{\sigma}\rfloor.$ (1)

In the following, unless specifically stated, we use (1) to determine the
fractional length for both weight and activation quantization. Note that we
only calculate the standard deviation during training.

##  4 Methods

In this section, we discuss our proposed training technique for neural network
quantization with fixed-point numbers, where the formats of weights and
activations in each layer are determined based on (1) during training. We
first analyze how to unify PACT and fixed-point quantization (Sec. 4.1). Then
we show how to quantize weights and activations, especially updating for BN
running statistics and fractional lengths (Sec. 4.2). Finally, we discuss the
necessity of relating scaling factors from two adjacent layers to calculate
the effective weights for quantization, especially for residual blocks where
some layers have several layers following them (Sec. 4.3).

###  4.1 Unifying PACT and Fixed-Point Quantization

To quantize a positive value $x$ with unsigned fixed-point number of format
$(\mathrm{WL},\mathrm{FL})$, where $\mathrm{WL}$ and $\mathrm{FL}$ denotes
word length and fractional length for the fixed-point number, respectively, we
have the quantization function $\mathrm{fix\\_quant}$ as:

$\mathrm{fix\\_quant}(x)=\frac{1}{2^{\mathrm{FL}}}\mathrm{round}\left(\mathrm{clip}\left(x\cdot
2^{\mathrm{FL}},0,2^{\mathrm{WL}}-1\right)\right),$ (2)

where $\mathrm{clip}$ is the clipping function, and
$0\leq\mathrm{FL}\leq\mathrm{WL}$ for unsigned fixed-point numbers. Note that
fixed-point quantization has two limitations: overflow, which is caused by
clipping into its representing region, and underflow, which is introduced by
the rounding function. Both of these introduce approximation errors. To
minimize the error, we determine the optimal fractional length for each layer
based on the analysis in Sec. 3.3.

To achieve a better way to quantize a model using fixed-point numbers, we take
a look at one of the most successful quantization techniques, PACT (Choi et
al., 2018), which clips on the full-precision value with a learned clipping-
level $\alpha$ before quantization:

$\mathrm{PACT}(x)=\frac{\alpha}{M}\mathrm{round}\left(\frac{M}{\alpha}\mathrm{clip}\left(x,0,\alpha\right)\right),$
(3)

where $M$ is a pre-defined scale factor mapping the value from $[0,1]$ to
$[0,M]$. The formal similarity between (2) and (3) inspires us to relate them
with each other as (more details in the Appx. 7.3):

$\mathrm{PACT}(x)=\frac{2^{\mathrm{FL}}\alpha}{2^{\mathrm{WL}}-1}\mathrm{fix\\_quant}(\frac{2^{\mathrm{WL}}-1}{2^{\mathrm{FL}}\alpha}x),$
(4)

where we have set $M=2^{\mathrm{WL}}-1$, which is the typical setting. With
this relationship, we can implement PACT and train the clipping-level $\alpha$
implicitly with fixed-point quantization.

###  4.2 Updating BN and Fractional Length

Double Forward for BN Fusion. To quantize the whole model with only 8-bit
fixed-point multiplication involved, we need to tackle the scaling factor from
BN layer, including both the weight and running variance. Specifically, we
need to quantize the effective weight that fuses the weight of convolution
layers with the weight and running variance from BN (Jacob et al., 2018; Yao
et al., 2021). This raises the question of how to determine the running
statistics during training. To solve this problem, we apply forward
computation twice. For the first forward, we apply the convolution using
quantized input yet full-precision weight of the convolution layer, and use
the output to update the running statistics of BN. In this way, the effective
weight to quantize is available. Note there is no backpropagation for this
step. For the second forward, we quantize the combined effective weight to get
the final output of the two layers of convolution and BN and do the
backpropagation.

Updating Fractional Length. Different from existing work that directly trains
the fractional length (Jain et al., 2019), we define the fractional length for
weight on-the-fly during training by inferring from current value of weight,
using (1). For the fractional length of activation, we use a buffer to store
and update the value with a momentum of $0.1$, similar to how to update BN
running statistics. Once the fractional lengths are determined after training,
we keep them fixed for inference.

###  4.3 Relating Scaling Factors between Adjacent Layers

As shown in (4), there are still two extra factors during the quantization
operation, which we denote as a fix scaling factor $\eta_{\mathrm{fix}}$:

$\eta_{\mathrm{fix}}=\frac{2^{\mathrm{FL}}\alpha}{2^{\mathrm{WL}}-1}.$ (5)

Now $\alpha$ is a trainable parameter with full-precision, which means the fix
scaling factor is also in full-precision. To eliminate undesired extra
computation, we absorb it into the above-mentioned effective weights for
quantization (Sec. 4.2). However, the fix scaling factor occurs twice, one for
rescaling after quantization ($\eta_{\mathrm{fix}}$) and the other for scaling
before quantization ($1/\eta_{\mathrm{fix}}$). To completely absorb it, we
need to relate two adjacent layers. In fact, for a mapping that includes
convolution, BN, and ReLU (more details are shown in Appx. 7.5), we apply PACT
quantization to relate the activation between two adjacent layers as:

$q^{(l+1)}_{i}=\mathrm{fix\\_quant}\left(\sum_{j=1}^{n^{(l)}}{\color[rgb]{0,0,0}\underbrace{\vrule
width=0.0pt,height=0.0pt,depth=17.22217pt\frac{\gamma^{(l)}_{i}}{\sigma^{(l)}_{i}}\frac{\eta_{\mathrm{fix}}^{(l)}}{\eta_{\mathrm{fix}}^{(l+1)}}W^{(l)}_{ij}}_{\text{Effective
Weight}}}{\color[rgb]{0,0,0}q^{(l)}_{j}}+{\color[rgb]{0,0,0}\underbrace{\vrule
width=0.0pt,height=0.0pt,depth=17.22217pt\frac{1}{\eta_{\mathrm{fix}}^{(l+1)}}\left(\beta^{(l)}_{i}-\frac{\gamma^{(l)}_{i}}{\sigma^{(l)}_{i}}\mu^{(l)}_{i}\right)}_{\text{Effective
Bias}}}\right),$ (6)

where $q$ is the fixed-point activation, $W$ the full-precision weight of the
convolution layer, $i$ and $j$ the spatial indices, $n$ the total number of
multiplication, and the superscript $(l)$ indicates the $l$-th block
consisting of convolution and BN. $\gamma$, $\beta$, $\sigma$, $\mu$ are the
learned weight, bias, running standard deviation, and running mean for the BN
layer, respectively. Also, we set $\mathrm{WL}=8$ for all layers. As can be
seen from (6), to obtain the final effective weight for fixed-point
quantization, for the $l$-th Conv-BN block, we need to access the fix scaling
factor, or equivalently, the clipping-level $\alpha$ and the activation
fractional length $\mathrm{FL}$, from its following $(l+1)$-th block(s). To
achieve this, we apply two techniques.

Pre-estimating Fractional Length. As mentioned above, we determine the
activation fractional length from its standard deviation. Also, (5) indicates
that the fix scaling factor relies on such fractional length for each layer.
However, in (6), we need the fix scaling factor from the next layer to
determine the effective weight under quantization, which we have not yet
updated. Thus, when calculating the effective weights during training, we use
the activation fractional length stored in the buffer, instead of the one for
quantizing the input of the next layer.

Clipping-Level Sharing. As shown in Fig. 5, for residual blocks, some layers
have two following layers (which we also name as child layer). Since we need
the fix scaling factor from the child layer to calculate the effective weight
for the parent (see (6)), inconsistent fix scaling factors between all
children layers will be a problem. To this end, we define one layer as master
and force all its siblings to share its clipping-level. In fact, the best way
is to share both the clipping-level and the fractional length among siblings,
but we find sharing fractional length leads to considerable performance drop,
especially for deep models such as MobileNet V2 and ResNet50. This is because
the fractional lengths play two roles here: one is for the fix scaling factor,
and the other is for the representing region (or equivalently the clipping-
level). Using different fractional lengths effectively enables different
clipping-levels (although only differ by a factor of power-of-2, see Appx.
7.6), which can be beneficial because the activation scales might vary from
layer to layer. Moreover, breaking the constraint of sharing activation
fractional length does not introduce much computational cost, as the value
only differs in storing format, and typically the values are stored in 32-bit,
i.e., the accumulation results are only quantized into 8-bit for
multiplication. Note that when computing the effective weight of the parent
layer, we only use the master child’s activation fractional length. For
effective weight of each child layer and fixed-point quantization on its
input, we use its own fractional length.

(a) ResBlock with direct connection.

(b) ResBlock with downsampling.

Figure 5:  The illustration of residual connections. For a layer with several
layers (named children layers) directly following it, we choose one to be
master, and all its sibling layers use the master layer’s clipping level. On
the other hand, since using different fractional length only cause bit
shifting or different fixed-point quantization formats, and the values are
stored in 32-bit before quantized into 8-bit, we do not share the fractional
formats to allow more degrees of freedom. The two figures show the case of
direct residual connection (a) and that with downsampling convolution layer
(b).

##  5 Experiments

Table 1: 8-bit quantization with conventional training for ResNet18 and
MobileNet V1/V2b. Following Yao et al. (2021), we abbreviate Integer-Only
Quantization as “Int”, INT8-Multiplication-Only Quantization as “8-bit”, the
Baseline Accuracy as “BL”, and Top-1 Accuracy as “Top-1”. All models are for
8-bit weight and activation quantization. For MobileNet V2, we are using
MobileNet V2b version as it is the most typical one.

(a) ResNet18

Method | Int | 8-bit | BL | Top-1
---|---|---|---|---
Baseline (FP) | ✗ | ✗ | 70.3 | 70.3
RVQuant (Park et al., 2018) | ✗ | ✗ | 69.9 | 70.0
PACT (Choi et al., 2018) | ✗ | ✗ | 70.2 | 69.8
LSQ (Esser et al., 2019) | ✗ | ✗ | 70.5 | 71.1
CPT (Fu et al., 2021) | ✗ | ✗ | - | 69.6
F8Net (ours) | ✓ | ✓ | 70.3 | 71.1

(b) MobileNet V1

Method | Int | 8-bit | BL | Top-1
---|---|---|---|---
Baseline (FP) | ✗ | ✗ | 72.4 | 72.4
PACT (Choi et al., 2018) | ✗ | ✗ | 72.1 | 71.3
TQT (Jain et al., 2019) | ✓ | ✓ | 71.1 | 71.1
SAT (Jin et al., 2020b) | ✗ | ✗ | 71.7 | 72.6
F8Net (ours) | ✓ | ✓ | 72.4 | 72.8

(c) MobileNet V2b

Method | Int | 8-bit | BL | Top-1
---|---|---|---|---
Baseline (FP) | ✗ | ✗ | 72.7 | 72.7
PACT (Choi et al., 2018) | ✗ | ✗ | 72.1 | 71.7
TQT (Jain et al., 2019) | ✓ | ✓ | 71.7 | 71.8
SAT (Jin et al., 2020b) | ✗ | ✗ | 71.8 | 72.5
F8Net (ours) | ✓ | ✓ | 72.7 | 72.6

In this section, we present our results for various models on ImageNet (Deng
et al., 2009) for classification task and compare the results with previous
works that focus on quantization-aware training to verify the effectiveness of
our method. We show the results for two sets of training. First, we discuss
the conventional training method following Jin et al. (2020b). Second, we
unify our method with one recent fine-tuning method that quantizes full-
precision models with high accuracy (Yao et al., 2021). More detailed
experimental settings are described in Appx. 7.1.

Conventional training. We first apply our method using conventional training
(Choi et al., 2018; Esser et al., 2019; Jin et al., 2020b; Fu et al., 2021),
where the quantized model is trained with the simplest setting as those for
full-precision model (more details in Appx. 7.1). To verify the effectiveness
of our method, we perform experiments on several models including ResNet18 and
MobileNet V1/V2. As shown in Table 1, our method achieves the state-of-the-art
results for all models. Additionally, we obtain comparable or even better
performance than the full-precision counterparts.

Compared with previous works on simulated quantization (Choi et al., 2018;
Park et al., 2018; Esser et al., 2019; Jin et al., 2020b; Fu et al., 2021)
that requires full-precision rescaling after INT8 convolution, our approach is
not only more efficient but also achieves better performance. On the other
hand, compared with previous fixed-point quantization (Jain et al., 2019), our
approach gives better results. This might partially due to that our method is
based on a more systematic analysis, as explained above in Section 3.3.

To further understand the significance of our method, we plot the fractional
lengths for weight and activation for each layer. Illustrated in Fig. 2(b) for
MobileNet V2, we find that the fractional lengths for both weight and
activation vary from layer to layer. Specifically, for weight quantization,
since some layers have relatively large value range of effective weight,
especially some depthwise layers, small fractional length is necessary to
avoid overflow issue. On the other hand, for layers with small weight scale,
large fractional length has more advantages to overcome the underflow problem.
The same conclusion also applies for the fractional length for activation.
Indeed, for some early layers in front of depthwise convolution layer, the
activation fractional length needs to be small, yet for the later-stages,
larger fractional length is desired. This further verifies our finding that
using different fractional lengths for layers with the same parent is critical
for good performance, because layers at different depths might be siblings and
requires different fractional lengths (see Fig. 5).

Table 2: 8-bit quantization with tiny fine-tuning on well-trained full-
precision model. Following Yao et al. (2021), we abbreviate Integer-Only
Quantization as “Int”, INT8-Multiplication-Only Quantization as “8-bit”,
Layer-Wise Quantization as “Layer”, the baseline accuracy as “BL”, Top-1
Accuracy as “Top-1”, and Top-1 Accuracy Drop with respect to the baseline as
“Drop”. We use two baselines for ResNet50, one from PytorchCV (Sémery, 2021)
(Baseline #1) and another from Nvidia (Nvidia, 2021) (Baseline #2), and we use
ResNet50b version. Note that the OMPQ (Ma et al., 2021b) is mixed-precision
quantization.

(a) ResNet18

Method | Int | 8-bit | Layer | BL | Top-1 | Drop
---|---|---|---|---|---|---
Baseline (FP) | ✗ | ✗ | - | 71.5 | 71.5 | -
HAWQ-V3 (Yao et al., 2021) | ✓ | ✗ | ✗ | 71.5 | 71.6 | 0.1
HAWQ-V3 (Yao et al., 2021) | ✓ | ✗ | ✓ | 71.5 | 70.9 | -0.6
OMPQ (Ma et al., 2021b) | ✓ | ✗ | ✗ | 73.1 | 72.3 | -0.8
F8Net (ours) | ✓ | ✓ | ✓ | 73.1 | 72.4 | -0.7

(b) ResNet50b

Method | Int | 8-bit | Layer | BL | Top-1 | Drop
---|---|---|---|---|---|---
Baseline #1 (FP) | ✗ | ✗ | - | 77.6 | 77.6 | -
HAWQ-V3 (Yao et al., 2021) | ✓ | ✗ | ✗ | 77.6 | 77.5 | -0.1
HAWQ-V3 (Yao et al., 2021) | ✓ | ✗ | ✓ | 77.6 | 77.1 | -0.5
F8Net (ours) | ✓ | ✓ | ✓ | 77.6 | 77.6 | 0.0
Baseline #2 (FP) | ✗ | ✗ | - | 78.5 | 78.5 | -
HAWQ-V3 (Yao et al., 2021) | ✓ | ✗ | ✗ | 78.5 | 78.1 | -0.4
HAWQ-V3 (Yao et al., 2021) | ✓ | ✗ | ✓ | 78.5 | 76.7 | -1.8
F8Net (ours) | ✓ | ✓ | ✓ | 78.5 | 78.1 | -0.4

Tiny fine-tuning on full-precision model. Recent work (Yao et al., 2021) focus
on investigating the potential of neural network quantization. To this end,
they suggest to tiny fine-tune on a well-pretrained full-precision model with
high accuracy. In this way, it might help to avoid misleading conclusion
coming from improper comparison between weak full-precision models with strong
quantized model. To further investigate the power of our method and compare it
with these advanced techniques, we also apply our method and fine-tune on
several full-precision models with high accuracy. Also, given the number of
total fine-tuing steps is very small, we apply grid search to determine the
optimal fractional lengths for this experiment. The results are listed in
Table 2, and we can find that our method is able to achieve better performance
than previous method (Yao et al., 2021), without time- and energy-consuming
high-precision multiplication (namely dyadic scaling shown in Fig. 1(c)).

Our method reveals that the high-precision rescaling, no matter implemented in
full-precision, or approximated or quantized with INT32 multiplication
followed by bit-shifting (a.k.a. dyadic multiplication), is indeed unnecessary
and is not the key part for quantized model to have good performance. This is
not well-understood in previous literature. Specifically, we demonstrate that
by properly choosing the formats for weight and activation in each layer, we
are able to achieve comparable and even better performance with 8-bit fixed-
point numbers, which can be implemented more efficiently on specific hardwares
such as DSP that only supports integer operation.

##  6 Conclusion

Previous works on neural network quantization typically rely on 32-bit
multiplication, either in full-precision or with INT32 multiplication followed
by bit-shifting (termed dyadic multiplication). This raises the question of
whether high-precision multiplication is critical to guarantee high-
performance for quantized models, or whether it is possible to eliminate it to
save cost. In this work, we study the opportunities and challenges of
quantizing neual networks with 8-bit only fixed-point multiplication, via
thorough statistical analysis and novel algorithm design. We validate our
method on ResNet18/50 and MobileNet V1/V2 on ImageNet classification. With our
method, we achieve the state-of-the-art performance without 32-bit
multiplication, and the quantized model is able to achieve comparable or even
better performance than their full-precision counterparts. Our method
demonstrates that high-precision multiplication, implemented with either
floating-point or dyadic scaling, is not necessary for model quantization to
achieve good performance. One future direction is to perform an in-depth
statistical analysis of fixed-point numbers with smaller word-lengths for
neural network quantization.

## References

  * Banner et al. (2018) Ron Banner, Yury Nahshan, Elad Hoffer, and Daniel Soudry.  Post-training 4-bit quantization of convolution networks for rapid-deployment.  _arXiv preprint arXiv:1810.05723_ , 2018.
  * Bhalgat et al. (2020) Yash Bhalgat, Jinwon Lee, Markus Nagel, Tijmen Blankevoort, and Nojun Kwak.  Lsq+: Improving low-bit quantization through learnable offsets and better initialization.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops_ , pp. 696–697, 2020.
  * Bulat et al. (2020) Adrian Bulat, Brais Martinez, and Georgios Tzimiropoulos.  High-capacity expert binary networks.  _arXiv preprint arXiv:2010.03558_ , 2020.
  * Cai et al. (2017) Zhaowei Cai, Xiaodong He, Jian Sun, and Nuno Vasconcelos.  Deep learning with low precision by half-wave gaussian quantization.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pp. 5918–5926, 2017.
  * Chen et al. (2020) Jianfei Chen, Yu Gai, Zhewei Yao, Michael W Mahoney, and Joseph E Gonzalez.  A statistical framework for low-bitwidth training of deep neural networks.  _arXiv preprint arXiv:2010.14298_ , 2020.
  * Chen et al. (2017) Xi Chen, Xiaolin Hu, Hucheng Zhou, and Ningyi Xu.  Fxpnet: Training a deep convolutional neural network in fixed-point representation.  In _2017 International Joint Conference on Neural Networks (IJCNN)_ , pp. 2494–2501. IEEE, 2017.
  * Choi et al. (2018) Jungwook Choi, Zhuo Wang, Swagath Venkataramani, Pierce I-Jen Chuang, Vijayalakshmi Srinivasan, and Kailash Gopalakrishnan.  Pact: Parameterized clipping activation for quantized neural networks.  _arXiv preprint arXiv:1805.06085_ , 2018.
  * Choukroun et al. (2019) Yoni Choukroun, Eli Kravchik, Fan Yang, and Pavel Kisilev.  Low-bit quantization of neural networks for efficient inference.  In _ICCV Workshops_ , pp. 3009–3018, 2019.
  * Courbariaux et al. (2015) Matthieu Courbariaux, Yoshua Bengio, and Jean-Pierre David.  Binaryconnect: Training deep neural networks with binary weights during propagations.  In _Advances in neural information processing systems_ , pp. 3123–3131, 2015.
  * Darvish Rouhani et al. (2020) Bita Darvish Rouhani, Daniel Lo, Ritchie Zhao, Ming Liu, Jeremy Fowers, Kalin Ovtcharov, Anna Vinogradsky, Sarah Massengill, Lita Yang, Ray Bittner, et al.  Pushing the limits of narrow precision inferencing at cloud scale with microsoft floating point.  _Advances in Neural Information Processing Systems_ , 33, 2020.
  * Deng et al. (2009) Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In _2009 IEEE conference on computer vision and pattern recognition_ , pp. 248–255. Ieee, 2009.
  * Dong et al. (2019) Zhen Dong, Zhewei Yao, Amir Gholami, Michael W Mahoney, and Kurt Keutzer.  Hawq: Hessian aware quantization of neural networks with mixed-precision.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pp. 293–302, 2019.
  * Enderich et al. (2019a) Lukas Enderich, Fabian Timm, Lars Rosenbaum, and Wolfram Burgard.  Fix-net: pure fixed-point representation of deep neural networks.  2019a.
  * Enderich et al. (2019b) Lukas Enderich, Fabian Timm, Lars Rosenbaum, and Wolfram Burgard.  Learning multimodal fixed-point weights using gradient descent.  _arXiv preprint arXiv:1907.07220_ , 2019b.
  * Esser et al. (2019) Steven K Esser, Jeffrey L McKinstry, Deepika Bablani, Rathinakumar Appuswamy, and Dharmendra S Modha.  Learned step size quantization.  _arXiv preprint arXiv:1902.08153_ , 2019.
  * Fang et al. (2020a) Jun Fang, Ali Shafiee, Hamzah Abdel-Aziz, David Thorsley, Georgios Georgiadis, and Joseph Hassoun.  Near-lossless post-training quantization of deep neural networks via a piecewise linear approximation.  _arXiv preprint arXiv:2002.00104_ , pp. 4, 2020a.
  * Fang et al. (2020b) Jun Fang, Ali Shafiee, Hamzah Abdel-Aziz, David Thorsley, Georgios Georgiadis, and Joseph H Hassoun.  Post-training piecewise linear quantization for deep neural networks.  In _European Conference on Computer Vision_ , pp. 69–86. Springer, 2020b.
  * Fu et al. (2020) Yonggan Fu, Haoran You, Yang Zhao, Yue Wang, Chaojian Li, Kailash Gopalakrishnan, Zhangyang Wang, and Yingyan Lin.  Fractrain: Fractionally squeezing bit savings both temporally and spatially for efficient dnn training.  _arXiv preprint arXiv:2012.13113_ , 2020.
  * Fu et al. (2021) Yonggan Fu, Han Guo, Meng Li, Xin Yang, Yining Ding, Vikas Chandra, and Yingyan Lin.  Cpt: Efficient deep neural network training via cyclic precision.  _arXiv preprint arXiv:2101.09868_ , 2021.
  * Garg et al. (2021) Sahaj Garg, Joe Lou, Anirudh Jain, and Mitchell Nahmias.  Dynamic precision analog computing for neural networks.  _arXiv preprint arXiv:2102.06365_ , 2021.
  * Gholami et al. (2021) Amir Gholami, Sehoon Kim, Zhen Dong, Zhewei Yao, Michael W Mahoney, and Kurt Keutzer.  A survey of quantization methods for efficient neural network inference.  _arXiv preprint arXiv:2103.13630_ , 2021.
  * Gong et al. (2019) Ruihao Gong, Xianglong Liu, Shenghu Jiang, Tianxiang Li, Peng Hu, Jiazhen Lin, Fengwei Yu, and Junjie Yan.  Differentiable soft quantization: Bridging full-precision and low-bit neural networks.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pp. 4852–4861, 2019.
  * Goyal et al. (2017) Priya Goyal, Piotr Dollár, Ross Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, Andrew Tulloch, Yangqing Jia, and Kaiming He.  Accurate, large minibatch sgd: Training imagenet in 1 hour.  _arXiv preprint arXiv:1706.02677_ , 2017.
  * Goyal et al. (2021) Rishabh Goyal, Joaquin Vanschoren, Victor Van Acht, and Stephan Nijssen.  Fixed-point quantization of convolutional neural networks for quantized inference on embedded platforms.  _arXiv preprint arXiv:2102.02147_ , 2021.
  * Guo et al. (2021) Nianhui Guo, Joseph Bethge, Haojin Yang, Kai Zhong, Xuefei Ning, Christoph Meinel, and Yu Wang.  Boolnet: Minimizing the energy consumption of binary neural networks.  _arXiv preprint arXiv:2106.06991_ , 2021.
  * Gysel et al. (2018) Philipp Gysel, Jon Pimentel, Mohammad Motamedi, and Soheil Ghiasi.  Ristretto: A framework for empirical study of resource-efficient inference in convolutional neural networks.  _IEEE transactions on neural networks and learning systems_ , 29(11):5784–5789, 2018.
  * Habi et al. (2020) Hai Victor Habi, Roy H Jennings, and Arnon Netzer.  Hmq: Hardware friendly mixed precision quantization block for cnns.  In _Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXVI 16_ , pp. 448–463. Springer, 2020.
  * Han et al. (2015) Song Han, Huizi Mao, and William J Dally.  Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding.  _arXiv preprint arXiv:1510.00149_ , 2015.
  * He & Cheng (2018) Xiangyu He and Jian Cheng.  Learning compression from limited unlabeled data.  In _Proceedings of the European Conference on Computer Vision (ECCV)_ , pp. 752–769, 2018.
  * Ho (2015) Joshua Ho.  Qualcomm details hexagon 680 dsp in snapdragon 820 accelerated imaging, 2015.  URL https://www.anandtech.com/show/9552/qualcomm-details-hexagon-680-dsp-in-snapdragon-820-accelerated-imaging.
  * Howard et al. (2017) Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam.  Mobilenets: Efficient convolutional neural networks for mobile vision applications.  _arXiv preprint arXiv:1704.04861_ , 2017.
  * Hubara et al. (2016) Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio.  Binarized neural networks.  _Advances in neural information processing systems_ , 29, 2016.
  * Hubara et al. (2020) Itay Hubara, Yury Nahshan, Yair Hanani, Ron Banner, and Daniel Soudry.  Improving post training neural quantization: Layer-wise calibration and integer programming.  _arXiv preprint arXiv:2006.10518_ , 2020.
  * Jacob et al. (2018) Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko.  Quantization and training of neural networks for efficient integer-arithmetic-only inference.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pp. 2704–2713, 2018.
  * Jain et al. (2019) Sambhav R Jain, Albert Gural, Michael Wu, and Chris H Dick.  Trained quantization thresholds for accurate and efficient fixed-point inference of deep neural networks.  _arXiv preprint arXiv:1903.08066_ , 2019.
  * Jin et al. (2020a) Qing Jin, Linjie Yang, and Zhenyu Liao.  Adabits: Neural network quantization with adaptive bit-widths.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 2146–2156, 2020a.
  * Jin et al. (2020b) Qing Jin, Linjie Yang, Zhenyu Liao, and Xiaoning Qian.  Neural network quantization with scale-adjusted training.  In _BMVC_ , 2020b.
  * Jin et al. (2021) Qing Jin, Jian Ren, Oliver J Woodford, Jiazhuo Wang, Geng Yuan, Yanzhi Wang, and Sergey Tulyakov.  Teachers do more than teach: Compressing image-to-image models.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 13600–13611, 2021.
  * Kim et al. (2021) Sehoon Kim, Amir Gholami, Zhewei Yao, Michael W Mahoney, and Kurt Keutzer.  I-bert: Integer-only bert quantization.  _arXiv preprint arXiv:2101.01321_ , 2021.
  * Kim & Kim (2021) Sungrae Kim and Hyun Kim.  Zero-centered fixed-point quantization with iterative retraining for deep convolutional neural network-based object detectors.  _IEEE Access_ , 9:20828–20839, 2021.
  * Krishnamoorthi (2018) Raghuraman Krishnamoorthi.  Quantizing deep convolutional networks for efficient inference: A whitepaper.  _arXiv preprint arXiv:1806.08342_ , 2018.
  * Langroudi et al. (2019) Hamed F Langroudi, Zachariah Carmichael, David Pastuch, and Dhireesha Kudithipudi.  Cheetah: Mixed low-precision hardware & software co-design framework for dnns on the edge.  _arXiv preprint arXiv:1908.02386_ , 2019.
  * Li et al. (2021) Zhengang Li, Geng Yuan, Wei Niu, Pu Zhao, Yanyu Li, Yuxuan Cai, Xuan Shen, Zheng Zhan, Zhenglun Kong, Qing Jin, et al.  Npas: A compiler-aware framework of unified network pruning and architecture search for beyond real-time mobile acceleration.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 14255–14266, 2021.
  * Liang et al. (2021) Tailin Liang, John Glossner, Lei Wang, Shaobo Shi, and Xiaotong Zhang.  Pruning and quantization for deep neural network acceleration: A survey.  _Neurocomputing_ , 461:370–403, 2021.
  * Lin et al. (2016) Darryl Lin, Sachin Talathi, and Sreekanth Annapureddy.  Fixed point quantization of deep convolutional networks.  In _International conference on machine learning_ , pp. 2849–2858. PMLR, 2016.
  * Liu et al. (2021) Ning Liu, Geng Yuan, Zhengping Che, Xuan Shen, Xiaolong Ma, Qing Jin, Jian Ren, Jian Tang, Sijia Liu, and Yanzhi Wang.  Lottery ticket preserves weight correlation: Is it desirable or not?  In _International Conference on Machine Learning_ , pp. 7011–7020. PMLR, 2021.
  * Liu et al. (2017) Zhuang Liu, Jianguo Li, Zhiqiang Shen, Gao Huang, Shoumeng Yan, and Changshui Zhang.  Learning efficient convolutional networks through network slimming.  In _Proceedings of the IEEE international conference on computer vision_ , pp. 2736–2744, 2017.
  * Liu et al. (2018) Zhuang Liu, Mingjie Sun, Tinghui Zhou, Gao Huang, and Trevor Darrell.  Rethinking the value of network pruning.  _arXiv preprint arXiv:1810.05270_ , 2018.
  * Luo et al. (2018) Ping Luo, Xinjiang Wang, Wenqi Shao, and Zhanglin Peng.  Towards understanding regularization in batch normalization.  _arXiv preprint arXiv:1809.00846_ , 2018.
  * Ma et al. (2020) Xiaolong Ma, Wei Niu, Tianyun Zhang, Sijia Liu, Sheng Lin, Hongjia Li, Wujie Wen, Xiang Chen, Jian Tang, Kaisheng Ma, et al.  An image enhancing pattern-based sparsity for real-time inference on mobile devices.  In _European Conference on Computer Vision_ , pp. 629–645. Springer, 2020.
  * Ma et al. (2021a) Xiaolong Ma, Geng Yuan, Xuan Shen, Tianlong Chen, Xuxi Chen, Xiaohan Chen, Ning Liu, Minghai Qin, Sijia Liu, Zhangyang Wang, et al.  Sanity checks for lottery tickets: Does your winning ticket really win the jackpot?  _Advances in Neural Information Processing Systems_ , 34, 2021a.
  * Ma et al. (2021b) Yuexiao Ma, Taisong Jin, Xiawu Zheng, Yan Wang, Huixia Li, Guannan Jiang, Wei Zhang, and Rongrong Ji.  Ompq: Orthogonal mixed precision quantization.  _arXiv preprint arXiv:2109.07865_ , 2021b.
  * Mei et al. (2019) Jieru Mei, Yingwei Li, Xiaochen Lian, Xiaojie Jin, Linjie Yang, Alan Yuille, and Jianchao Yang.  Atomnas: Fine-grained end-to-end neural architecture search.  _arXiv preprint arXiv:1912.09640_ , 2019.
  * Mishra et al. (2017) Asit Mishra, Eriko Nurvitadhi, Jeffrey J Cook, and Debbie Marr.  Wrpn: Wide reduced-precision networks.  _arXiv preprint arXiv:1709.01134_ , 2017.
  * Mishra et al. (2020) Rahul Mishra, Hari Prabhat Gupta, and Tanima Dutta.  A survey on deep neural network compression: Challenges, overview, and solutions.  _arXiv preprint arXiv:2010.03954_ , 2020.
  * Mitschke et al. (2019) Norbert Mitschke, Michael Heizmann, Klaus-Henning Noffz, and Ralf Wittmann.  A fixed-point quantization technique for convolutional neural networks based on weight scaling.  In _2019 IEEE International Conference on Image Processing (ICIP)_ , pp. 3836–3840. IEEE, 2019.
  * Nagel et al. (2019) Markus Nagel, Mart van Baalen, Tijmen Blankevoort, and Max Welling.  Data-free quantization through weight equalization and bias correction.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pp. 1325–1334, 2019.
  * Nagel et al. (2021) Markus Nagel, Marios Fournarakis, Rana Ali Amjad, Yelysei Bondarenko, Mart van Baalen, and Tijmen Blankevoort.  A white paper on neural network quantization.  _arXiv preprint arXiv:2106.08295_ , 2021.
  * Nvidia (2021) Nvidia.  Nvidia models, 2021.  URL https://ngc.nvidia.com/catalog/models/nvidia:resnet50_pyt_amp.
  * Oh et al. (2021) Sangyun Oh, Hyeonuk Sim, Sugil Lee, and Jongeun Lee.  Automated log-scale quantization for low-cost deep neural networks.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 742–751, 2021.
  * Park et al. (2017) Eunhyeok Park, Junwhan Ahn, and Sungjoo Yoo.  Weighted-entropy-based quantization for deep neural networks.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pp. 5456–5464, 2017.
  * Park et al. (2018) Eunhyeok Park, Sungjoo Yoo, and Peter Vajda.  Value-aware quantization for training and inference of neural networks.  In _Proceedings of the European Conference on Computer Vision (ECCV)_ , pp. 580–595, 2018.
  * QCOM (2019) QCOM.  Qualcomm® hexagontm dsp, 2019\.  URL https://developer.qualcomm.com/software/hexagon-dsp-sdk.
  * Rastegari et al. (2016) Mohammad Rastegari, Vicente Ordonez, Joseph Redmon, and Ali Farhadi.  Xnor-net: Imagenet classification using binary convolutional neural networks.  In _European conference on computer vision_ , pp. 525–542. Springer, 2016.
  * Sémery (2021) Oleg Sémery.  Pytorchcv library, 2021.  URL https://pypi.org/project/pytorchcv/.
  * Smith et al. (1997) Steven W Smith et al.  The scientist and engineer’s guide to digital signal processing.  1997\.
  * Tailor et al. (2020) Shyam A Tailor, Javier Fernandez-Marques, and Nicholas D Lane.  Degree-quant: Quantization-aware training for graph neural networks.  _arXiv preprint arXiv:2008.05000_ , 2020.
  * Tan & Jiang (2018) Lizhe Tan and Jean Jiang.  _Digital signal processing: fundamentals and applications_.  Academic Press, 2018.
  * Tanaka et al. (2020) Hidenori Tanaka, Daniel Kunin, Daniel LK Yamins, and Surya Ganguli.  Pruning neural networks without any data by iteratively conserving synaptic flow.  _arXiv preprint arXiv:2006.05467_ , 2020.
  * Wang et al. (2019) Kuan Wang, Zhijian Liu, Yujun Lin, Ji Lin, and Song Han.  Haq: Hardware-aware automated quantization with mixed precision.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 8612–8620, 2019.
  * Wang et al. (2018) Peisong Wang, Qinghao Hu, Yifan Zhang, Chunjie Zhang, Yang Liu, and Jian Cheng.  Two-step quantization for low-bit neural networks.  In _Proceedings of the IEEE Conference on computer vision and pattern recognition_ , pp. 4376–4384, 2018.
  * Wu et al. (2020) Hao Wu, Patrick Judd, Xiaojie Zhang, Mikhail Isaev, and Paulius Micikevicius.  Integer quantization for deep learning inference: Principles and empirical evaluation.  _arXiv preprint arXiv:2004.09602_ , 2020.
  * Yang & Jin (2020) Linjie Yang and Qing Jin.  Fracbits: Mixed precision quantization via fractional bit-widths.  _arXiv preprint arXiv:2007.02017_ , 1, 2020.
  * Yang et al. (2020) Zhaohui Yang, Yunhe Wang, Kai Han, Chunjing Xu, Chao Xu, Dacheng Tao, and Chang Xu.  Searching for low-bit weights in quantized neural networks.  _arXiv preprint arXiv:2009.08695_ , 2020.
  * Yao et al. (2021) Zhewei Yao, Zhen Dong, Zhangcheng Zheng, Amir Gholami, Jiali Yu, Eric Tan, Leyuan Wang, Qijing Huang, Yida Wang, Michael Mahoney, et al.  Hawq-v3: Dyadic neural network quantization.  In _International Conference on Machine Learning_ , 2021.
  * Yuan et al. (2021) Geng Yuan, Xiaolong Ma, Wei Niu, Zhengang Li, Zhenglun Kong, Ning Liu, Yifan Gong, Zheng Zhan, Chaoyang He, Qing Jin, et al.  Mest: Accurate and fast memory-economic sparse training framework on the edge.  _Advances in Neural Information Processing Systems_ , 34, 2021.
  * Zhang et al. (2020) Xishan Zhang, Shaoli Liu, Rui Zhang, Chang Liu, Di Huang, Shiyi Zhou, Jiaming Guo, Qi Guo, Zidong Du, Tian Zhi, et al.  Fixed-point back-propagation training.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 2330–2338, 2020.
  * Zhao et al. (2021a) Kang Zhao, Sida Huang, Pan Pan, Yinghan Li, Yingya Zhang, Zhenyu Gu, and Yinghui Xu.  Distribution adaptive int8 quantization for training cnns.  In _Proceedings of the Thirty-Fifth AAAI Conference on Artificial Intelligence_ , 2021a.
  * Zhao et al. (2021b) Sijie Zhao, Tao Yue, and Xuemei Hu.  Distribution-aware adaptive multi-bit quantization.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 9281–9290, 2021b.
  * Zhou et al. (2017) Aojun Zhou, Anbang Yao, Yiwen Guo, Lin Xu, and Yurong Chen.  Incremental network quantization: Towards lossless cnns with low-precision weights.  _arXiv preprint arXiv:1702.03044_ , 2017.
  * Zhou et al. (2018) Aojun Zhou, Anbang Yao, Kuan Wang, and Yurong Chen.  Explicit loss-error-aware quantization for low-bit deep neural networks.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pp. 9426–9435, 2018.
  * Zhou et al. (2016) Shuchang Zhou, Yuxin Wu, Zekun Ni, Xinyu Zhou, He Wen, and Yuheng Zou.  Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients.  _arXiv preprint arXiv:1606.06160_ , 2016.
  * Zhu et al. (2016) Chenzhuo Zhu, Song Han, Huizi Mao, and William J Dally.  Trained ternary quantization.  _arXiv preprint arXiv:1612.01064_ , 2016.
  * Zhu et al. (2020) Feng Zhu, Ruihao Gong, Fengwei Yu, Xianglong Liu, Yanfei Wang, Zhelong Li, Xiuqi Yang, and Junjie Yan.  Towards unified int8 training for convolutional neural network.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 1969–1979, 2020.

##  7 Appendix

###  7.1 More Experimental Details

More Details for Conventional Training. For conventional training method, we
train the quantized model initialized with a pre-trained full-precision one.
The training of full-precision and quantized models shares the same
hyperparameters, including learning rate and its scheduler, weight decay,
number of epochs, optimizer, and batch size. For ResNet18 and MobileNet V1, we
use an initial learning rate of $0.05$, and for MobileNet V2, it is $0.1$. We
find the value of learning rate, i.e., $0.1$ and $0.05$, does not have much
impact on the final performance. Totally, $150$ epochs of training are
conducted, with cosine learning rate scheduler without restart. The warmup
strategy is adopted with linear increasing ($\mathrm{batchsize}/256\times
0.05$) (Goyal et al., 2017) during the first five epochs before cosine
learning rate scheduler. The input image is randomly cropped to $224\times
224$ and randomly flipped horizontally, and is kept as 8-bit unsigned fixed-
point numbers with $\mathrm{FL}=8$ and without standardization. For ResNet18
and MobileNet V1/V2, we use batch size of $2048$ and run the experiments on
$8$ A100 GPUs. The parameters are updated with SGD optimizer and Nesterov
momentum with a momentum weight of $0.9$ without damping. The original
structure of MobileNet V2 uses ReLU6 as its activation. Since our unified PACT
and the fixed-point quantization already has clipping operation, and can be
equivalently formulated with ReLU6 by rescaling weight or activation, we
eliminate ReLU6 in our implementation.

Discussion for Weight Decay. We set weight decay to $4\times 10^{-5}$, and
find the weight decay scheme is critical for good performance, especially for
the quantized model. We analyze weight decay for different models as follows:

  * •

For ResNet18, we apply weight decay on all layers, including convolution,
fully-connected, and BN layers.

  * •

For MobileNet V1, previous methods only apply weight decay on conventional
convolution and fully-connected layers, but not on depthwise convolution and
BN (Howard et al., 2017). We find this leads to the overfitting problem,
making some early convolution layers have large weights, which is not friendly
for quantization. We further observe that some channels of some depthwise
convolution layers have all zero inputs, due to some channels of previous
layer become all negative and ReLU is applied afterwards, making the running
statistics of the corresponding channels in the following BN layer almost
zero. This breaks the regularization effect of BN (Luo et al., 2018). Since
each output channel only depends on one input channel for depthwise
convolution layers, the weights connecting them become uncontrolled, and the
effective weights become large, leading to an overfitting problem. Applying
weight decay on the depthwise convolution and BN layers helps to alleviate
this problem, and the resulting effective weights become small.

  * •

For MobileNet V2, we find overfitting plays the role of reducing the
validation error (although the training error is lower), and applying weight
decay on depthwise convolution or BN weights impairs the training procedure.
The underlying reason might be related to the residual connecting structure of
this model (note MobileNet V1 does not use residual connection).

In summary, we apply weight decay on all layers, including depthwise
convolution and BN layers for ResNet18 and MobileNet V1, and do not apply
weight decay on depthwise convolution and BN layers for MobileNet V2.

More Details for Tiny Fine-tuning. For tiny fine-tuning on full-precision
models, we follow the same strategy proposed in Yao et al. (2021).
Specifically, we use a constant learning rate of $10^{-4}$, with $500$
iterations of fine-tuning (or equivalently data ratio of around $0.05$ with
batch size of $128$). Different from (Yao et al., 2021), we find fixed BN is
not helpful, and we allow it to update during the whole fine-tuning step. As
mentioned in Sec. 5, we apply grid search to determine the fractional lengths
for both weight and input, as the training cost is very small and applying
grid search does not introduce too much effort or training time. Also, since
the original full-precision model uses the normalized input, we also apply
normalization on the images and quantize images with signed fixed-point
numbers (and format determined with grid search) before being fed into the
first convolution layer of the model.

###  7.2 More Details for Statistical Analysis

For the toy example in Fig. 3, we sample $10,000$ zero-mean Gaussian random
variables with different standard deviations, and apply ReLU activation for
the rectified Gaussian variables with unsigned quantization. The variables are
then quantized with fixed-point quantization given in (2) and (9),
respectively. We calculate the relative quantization error and plot against
the standard deviation for each fixed-point format. Note that zero-mean is a
reasonable simplifying assumption if we assume to neglect the impact of bias
in BN for analysis purposes.

###  7.3 derivation for fixed-point and PACT relation

Here we derive the relationship between PACT and fixed-point quantization
shown in (4). Specifically, the PACT quantization in (3) can be formulated as
follows for positive $\alpha$:

$\displaystyle\mathrm{PACT}(x)$
$\displaystyle=\frac{\alpha}{M}\mathrm{round}\left(\frac{M}{\alpha}\mathrm{clip}\left(x,0,\alpha\right)\right)$
(7a)
$\displaystyle=\frac{\alpha}{M}\mathrm{round}\left(M\mathrm{clip}\left(\frac{x}{\alpha},0,1\right)\right)$
(7b)
$\displaystyle=\frac{\alpha}{M}\mathrm{round}\left(\frac{M}{2^{\mathrm{WL}}-1}\mathrm{clip}\left(\frac{{2^{\mathrm{WL}}-1}}{\alpha}x,0,2^{\mathrm{WL}}-1\right)\right)$
(7c)
$\displaystyle=\frac{2^{\mathrm{\mathrm{WL}}}-1}{M}\frac{2^{\mathrm{\mathrm{FL}}}\alpha}{2^{\mathrm{\mathrm{WL}}}-1}\frac{1}{2^{\mathrm{\mathrm{FL}}}}\mathrm{round}\left(\frac{M}{2^{\mathrm{\mathrm{WL}}}-1}\mathrm{clip}\left(\frac{2^{\mathrm{\mathrm{WL}}}-1}{2^{\mathrm{\mathrm{FL}}}\alpha}x*2^{\mathrm{\mathrm{FL}}},0,2^{\mathrm{\mathrm{WL}}}-1\right)\right).$
(7d)

For $M=2^{\mathrm{\mathrm{WL}}}-1$, which is the typical setting for
quantization, we have:

$\mathrm{PACT}(x)=\frac{2^{\mathrm{\mathrm{FL}}}\alpha}{2^{\mathrm{\mathrm{WL}}}-1}\frac{1}{2^{\mathrm{\mathrm{FL}}}}\mathrm{round}\left(\mathrm{clip}\left(\frac{2^{\mathrm{\mathrm{WL}}}-1}{2^{\mathrm{\mathrm{FL}}}\alpha}x*2^{\mathrm{\mathrm{FL}}},0,2^{\mathrm{\mathrm{WL}}}-1\right)\right).$
(8)

Comparing with the expression for fixed-point quantization (2), we can
immediately get (4).

###  7.4 Double Side Quantization for Weight and MobileNet V2

In (2), we only give the formula for fixed-point quantization of unsigned
case. For weight and activation from some layer without following ReLU
nonlinearity (such as some layers in MobileNet V2), signed quantization is
necessary, and the expression is similarly given as:

$\mathrm{fix\\_quant}(x)=\frac{1}{2^{\mathrm{FL}}}\mathrm{round}\left(\mathrm{clip}\left(x\cdot
2^{\mathrm{FL}},-2^{\mathrm{WL}-1}+1,2^{\mathrm{WL}-1}-1\right)\right),$ (9)

where $\mathrm{clip}$ is the clipping function, and
$0\leq\mathrm{FL}\leq\mathrm{WL}-1$.

###  7.5 Derivation of Effective Weight

Here we derive the equation of effective weights relating two adjacent layers
in Sec. 4.3. Specifically, for a Conv-BN-ReLU block with conventional PACT
quantization using input clipping, quantization and dequantization, the
general procedure can be described as

Nonlinear: $\displaystyle\widetilde{x}^{(l)}_{i}$
$\displaystyle=\mathrm{clip}{\left(x^{(l-1)}_{i},0,\alpha^{(l)}\right)},$
(10a) Input Quant (uint8): $\displaystyle\widehat{q}^{(l)}_{i}$
$\displaystyle=\mathrm{round}\left(\frac{M}{\alpha^{(l)}}\widetilde{x}^{(l)}_{i}\right),$
(10b) Input Dequant: $\displaystyle\widetilde{q}^{(l)}_{i}$
$\displaystyle=\frac{\alpha^{(l)}}{M}\widehat{q}^{(l)}_{i},$ (10c) Conv:
$\displaystyle y^{(l)}_{i}$
$\displaystyle=\sum_{j=1}^{n^{(l)}}W^{(l)}_{ij}\widetilde{q}^{(l)}_{j},$ (10d)
BN: $\displaystyle x^{(l)}_{i}$
$\displaystyle=\gamma^{(l)}_{i}\frac{y^{(l)}_{i}-\mu^{(l)}_{i}}{\sigma^{(l)}_{i}}+\beta^{(l)}_{i}$
(10e)
$\displaystyle=\frac{\gamma^{(l)}_{i}}{\sigma^{(l)}_{i}}y^{(l)}_{i}+\left(\beta^{(l)}_{i}-\frac{\gamma^{(l)}_{i}}{\sigma^{(l)}_{i}}\mu^{(l)}_{i}\right),$
(10f)

where $x$ is the input before clipping, $\widehat{q}$ is the integer input
after quantization, $\widetilde{q}$ is the full-precision input after
dequantization, $\mathrm{clip}$ is the clipping function, $\alpha$ is the
clipping-level, $M=2^{\mathrm{WL}}-1$ is the scaling for quantization,
$W_{ij}$ is weight from convolution layer, and $\gamma$, $\beta$, $\sigma$,
$\mu$ are weight, bias, running standard deviation, and running mean from BN
layer, respectively, and $i$ and $j$ are spatial indices. We first note that
(10a), (10b) and (10c) can be combined as:

$\displaystyle\widetilde{q}^{(l)}_{i}$
$\displaystyle=\mathrm{PACT}(x^{(l-1)}_{i})$ (11a)
$\displaystyle=\eta_{\mathrm{fix}}^{(l)}\mathrm{fix\\_quant}\left(\frac{1}{\eta_{\mathrm{fix}}^{(l)}}x^{(l-1)}_{i}\right)$
(11b) $\displaystyle=\eta_{\mathrm{fix}}^{(l)}q^{(l)}_{i},$ (11c)

where $q$ is the fixed-point activation and we have used the relationship
given by (4) and the definition in (5). From this we can derive that:

$\displaystyle q^{(l+1)}_{i}$
$\displaystyle=\mathrm{fix\\_quant}\left(\frac{1}{\eta_{\mathrm{fix}}^{(l+1)}}x^{(l)}_{i}\right)$
(12a)
$\displaystyle=\mathrm{fix\\_quant}\Bigg{(}\frac{1}{\eta_{\mathrm{fix}}^{(l+1)}}\bigg{(}\frac{\gamma^{(l)}_{i}}{\sigma^{(l)}_{i}}y^{(l)}_{i}+\Big{(}\beta^{(l)}_{i}-\frac{\gamma^{(l)}_{i}}{\sigma^{(l)}_{i}}\mu^{(l)}_{i}\Big{)}\bigg{)}\Bigg{)}$
(12b)
$\displaystyle=\mathrm{fix\\_quant}\Bigg{(}\frac{1}{\eta_{\mathrm{fix}}^{(l+1)}}\bigg{(}\frac{\gamma^{(l)}_{i}}{\sigma^{(l)}_{i}}\sum_{j=1}^{n^{(l)}}W^{(l)}_{ij}\widetilde{q}^{(l)}_{j}+\Big{(}\beta^{(l)}_{i}-\frac{\gamma^{(l)}_{i}}{\sigma^{(l)}_{i}}\mu^{(l)}_{i}\Big{)}\bigg{)}\Bigg{)}$
(12c)
$\displaystyle=\mathrm{fix\\_quant}\Bigg{(}\frac{1}{\eta_{\mathrm{fix}}^{(l+1)}}\bigg{(}\frac{\gamma^{(l)}_{i}}{\sigma^{(l)}_{i}}\sum_{j=1}^{n^{(l)}}W^{(l)}_{ij}\eta_{\mathrm{fix}}^{(l)}q^{(l)}_{j}+\Big{(}\beta^{(l)}_{i}-\frac{\gamma^{(l)}_{i}}{\sigma^{(l)}_{i}}\mu^{(l)}_{i}\Big{)}\bigg{)}\Bigg{)}$
(12d)
$\displaystyle=\mathrm{fix\\_quant}\left(\sum_{j=1}^{n^{(l)}}\frac{\gamma^{(l)}_{i}}{\sigma^{(l)}_{i}}\frac{\eta_{\mathrm{fix}}^{(l)}}{\eta_{\mathrm{fix}}^{(l+1)}}W^{(l)}_{ij}q^{(l)}_{j}+\frac{1}{\eta_{\mathrm{fix}}^{(l+1)}}\left(\beta^{(l)}_{i}-\frac{\gamma^{(l)}_{i}}{\sigma^{(l)}_{i}}\mu^{(l)}_{i}\right)\right),$
(12e)

which is just (6).

###  7.6 Private Fractional Lengths Enabling Different Clipping-Levels

Here we analyze the effect of using private fractional lengths between sibling
layers to indicate that this effectively enables private clipping-levels for
them. In fact, the original PACT quantization step is given as

$\displaystyle\widetilde{q}$ $\displaystyle=\mathrm{PACT}(x)$ (13a)
$\displaystyle=\frac{2^{\mathrm{\mathrm{FL}}}\alpha}{2^{\mathrm{\mathrm{WL}}}-1}\frac{1}{2^{\mathrm{\mathrm{FL}}}}\mathrm{round}\left(\mathrm{clip}\left(\frac{2^{\mathrm{\mathrm{WL}}}-1}{2^{\mathrm{\mathrm{FL}}}\alpha}x*2^{\mathrm{\mathrm{FL}}},0,2^{\mathrm{\mathrm{WL}}}-1\right)\right),$
(13b)

where we have omitted layer and spatial indices for simplification. Now if we
use private fractional lengths for sibling layers while require them to share
the same clipping level, and use the master child’s fractional length for
calculating the effective weight in (6), denoting the fractional length of the
master layer as $\mathrm{FL}^{\mathrm{m}}$, the above function becomes

$\displaystyle\widetilde{q}$
$\displaystyle=\frac{2^{\mathrm{\mathrm{FL}}}\alpha}{2^{\mathrm{\mathrm{WL}}}-1}\frac{1}{2^{\mathrm{\mathrm{FL}}}}\mathrm{round}\left(\mathrm{clip}\left(\frac{2^{\mathrm{\mathrm{WL}}}-1}{2^{\mathrm{\mathrm{FL}}^{\mathrm{m}}}\alpha}x*2^{\mathrm{\mathrm{FL}}},0,2^{\mathrm{\mathrm{WL}}}-1\right)\right)$
(14a)
$\displaystyle=2^{\mathrm{FL}-\mathrm{FL}^{\mathrm{m}}}\frac{2^{\mathrm{FL}}\alpha^{\prime}}{2^{\mathrm{\mathrm{WL}}}-1}\frac{1}{2^{\mathrm{\mathrm{FL}}}}\mathrm{round}\left(\mathrm{clip}\left(\frac{2^{\mathrm{\mathrm{WL}}}-1}{2^{\mathrm{\mathrm{FL}}}\alpha^{\prime}}x*2^{\mathrm{\mathrm{FL}}},0,2^{\mathrm{\mathrm{WL}}}-1\right)\right),$
(14b)

where
$\alpha^{\prime}=2^{\mathrm{\mathrm{FL}}^{\mathrm{m}}-\mathrm{FL}}\alpha$.
From this we see that using private fractional lengths effectively enables
different clipping-levels between sibling layers, and the cost is only some
bit shifting.

###  7.7 More Discussion of the Optimal Fractional Length

Here we give some further discussion of using standard deviation to determine
the optimal fractional length. The main reason is that standard deviation is a
more robust statistics than others, such as dynamic range, and is an easily-
estimated parameter for Gaussian distributed weights and pre-activations.
Considering depth-wise convolution layers that contain much fewer weights and
inputs, using robust statistics becomes essential as these layers might
include weights or inputs with strange behavior, _e.g._ , the pre-activation
values of some channels become all negative with large magnitude. Therefore,
the standard deviation is more suitable and robust than the dynamic range.

###  7.8 Fractional Length for ResNet50

Here we provide more results of fractional lengths distribution in Fig. 6 for
the well-trained ResNet50 with 8-bit fixed-point numbers finetuned from the
Baseline #2 in Table 2(b). As we can see, the optimal fractional lengths are
layer-dependent and their distribution is highly different from those in
MobileNet V2 (as shown in Fig. 2(b)). Specifically, for MobileNet V2, some
layers have vanishing weight fractional lengths and less than $4\%$ of all
layers have an activation fractional length less than 4, while for ResNet50,
more than $88\%$ of all layers have an activation fractional length that is
less or equal to 4.

Figure 6:  Fractional lengths of each layer for a well-trained fixed-point model for ResNet50. Table 3: Analysis of the impact of the searching space for fractional length (ResNet50 on ImageNet). Method | Frac. Len. Range | BL | Top-1
---|---|---|---
Baseline (FP) | - | 77.6 | 77.6
F8Net (ours) | ${6,7,8}$ | 77.6 | 72.4
F8Net (ours) | $0-8$ | 77.6 | 77.6

###  7.9 Analyzing Searching Space of Fractional Lengths

In the main paper, we adopt the largest possible searching space for the
fractional lengths of 8-bit fixed-point. As shown in Fig. 2(b), many layers
have a fractional length less than $4$, either for input or weight. Here we
study whether it is possible to use only fractional lengths between $6$ and
$8$. To this end, we finetune on ResNet50b using the Baseline #1. The results
are listed in Table 3, from which we find that restricting the fractional
lengths between $6$ to $8$ significantly impacts the performance of the final
quantized model, as the top-1 accuracy drops from $77.6\%$ to $72.4\%$.