# A Novel Solution for the General Diffusion

Luisiana Cundin

###### Abstract

The Fisher–KPP equation is a reaction–diffusion equation originally proposed
by Fisher to represent allele propagation in genetic hosts or population. It
was also proposed by Kolmogorov for more general applications. A novel method
for solving nonlinear partial differential equations is applied to produce a
unique, approximate solution for the Fisher–KPP equation. Analysis proves the
solution is counter–intuitive. Although still satisfying the maximum
principle, time dependence collapses for all time greater than zero,
therefore, the solution is highly irregular and not smooth, invalidating the
traveling wave approximation so often employed.

Keywords— KPP equation or Fisher–KPP equation

##  1 On the general diffusion…

Diffusion is incontrovertibly one of the most fundamental physical processes
describing the transport of energy, mass and other phenomena through space and
time. The linear transfer of energy is certainly in the purview of Nature’s
design, as evidenced all around us, everyday; but, nonlinear transfer has long
been sought as an extension to rational systems where the medium reacts to the
gradient of concentration. A nonlinear diffusive process would describe some
process that, in addition to gradient dependence, would be further dependent
upon the medium response to the gradient of concentration, reacting in some
nonlinear, chaotic fashion [1, 2]. The concept of some unusual medium response
is predicated upon the idea that the medium carries some special reactive
properties, like memory or retention or something of the sort that would
account for some additional behavior atop ordinary flux allowed by the medium.

It may be that any such nonlinear diffusive behavior could simply be captured
in a modified diffusion constant, rather, experimentalists may find unique
diffusion constants for various media, and, never think it due to some
reactive medium response [3, 4, 5]. In fact, any exotic, complex diffusivity
should be focused upon the diffusion constant, _per se_ , and not extricated
to describe the medium response itself. In general, the media response is
typically random in nature, as _per_ Brownian motion; moreover, although
randomness is often perceived nonlinear, it is rational nonetheless [6].
Whether it is physically possible for media to exhibit nonlinear reactive
responses is certainly questionable, in fact, it may all together be not
possible for such systems to exists; because, linear diffusion already
describes the flux of energy and mass subject to the local gradient or
concentration, therefore, already accounts for the actual medium response in
its description of mass/energy transfer [7]. To ascribe additional ”nonlinear”
attributes to media would imbue into physical materialé some ghost process or
special physical phenomenon yet discovered.

In many cases, for example, reaction–diffusion processes, as are certain
chemical reactions, nonlinear diffusion equations are used in an attempt to
capture the media’s actual response to changes in the gradient as the chemical
reaction progresses in time and space, therefore, there is some rationalé
behind attempting to formulate some type of equation that mimics a very
complex, intricate play in reversible concentrations, as the chemical reaction
progresses to homeostasis [8]. Nevertheless, to formulate or reduce such a
complex process to one, all encompassing equation may or may not be suitable,
adequate or advisable, for the process is truly an interplay of many reaction
chemical kinetic equations, interacting over time and space, with an intricate
dance in concentrations diminishing and evolving in time. Complex systems,
generally speaking, require a milieu of equations, each formulated
specifically, to properly model each element/species involved in the overall
process [9]. In this manner, a researcher adequately accounts for each kinetic
species in the reaction and does not attempt to ’cut corners’ by reducing the
entire system to some modified ”nonlinear” system, which is often forced, in
some manner, into a one–dimensional system; this done in hopes it will mimic
the system in question and admit ease in solving.

There are many reasons the simple addition of some multiplicative medium
response is not advisable nor should be considered physically correct. Of the
many reasons, Ostrogradsky’s theorem concerning mechanics stands tall [10].
The theorem shows why no system with higher order derivatives than second
order are physically acceptable or seen in mechanics, furthermore, it proves
third order and higher systems are unstable in time and space, therefore,
describe unrealistic physical processes, or, in the least, doomed processes.
All systems are bounded from above by energy and by material properties
(strengths); but, to say a system’s Hamiltonian is unbounded from below
envisions a system which allows a particle at rest to suddenly jump from rest
to some unbounded state of energy, moreover, a negative state of energy, which
is improper, to say the least. All energy is positive, although, in a relative
system or coordinate system, negative energies are possible, even though, such
formulations are sloppy, at best, but improper, at worst.

Nonlinear equations admit special solutions, usually, involving hyperbolic
trigonometric equations, which are typically periodic functions and suffer
from branch points, singularities and other non-analytic properties. The fact
poles exist within the domain of the solution should indicate to the wary that
the domain is not entire, therefore, the solution is hemmed in or restricted
in some way or another. A typical path for solving nonlinear equations
involves the traveling wave approximation, which assumes any deflection in the
solution is small enough and smooth enough to allow for equivalence under
D’Alembert’s principle, thereby, enabling reduction of an equation of some
dimension to a one–dimensional canonical equation, whose behavior is assumed
equivalent [11, 12]. Nonlinear equations, in general, are not smooth and this
fact brings any solution obtained via the traveling wave approximation under
serious scrutiny, as will be seen for the Fisher–KPP equation, shortly. The
additional multiplicative medium response places extraordinary restraints upon
the system and any solution.

##  2 Fisher-KPP equation

Consider the general Fisher–KPP equation with real coefficients (D, b, r),
where D describes the diffusivity of the material under question, b the linear
response of the medium (if applicable), and r the nonlinear coefficient,
_viz.:_

$\frac{\partial}{\partial t}u(x,t)=D\frac{\partial^{2}}{\partial
x^{2}}u(x,t)-b\;u(x,t)+r\;u(x,t)^{2}/;\;\\{(D,b,r)|(D,b,r)\in\Re\\}$ (1)

Assume a solution in the form of $u(x,t)=G(x,t)\ast f(x,t)$, where $G(x,t)$
represents Green’s function which solves the linear equation, inept, $r=0$;
_viz.:_

$G(x,t)=\frac{e^{-x^{2}/4Dt}}{\sqrt{4\pi Dt}}\subset e^{-(2\pi
s)^{2}Dt}e^{-bt}=g(s,t),$ (2)

where symbol $\subset$ is borrowed from Bracewell, indicating the Fourier
transform of the given function [13].

Since Green’s function is a solution to the linear partial differential
equation with linear medium response already included, then the task at hand
is to reduce the main equation to some residual equation involving only the
nonlinearity. Employing Green’s solution realizes several cancellations
immediately, cancelling several terms after applying the time derivative,
where partial derivatives distribute across convolution integrals, see Theorem
3. Moreover, one retains the choice onto which function a derivative is acted
upon, if the convolution integral involves the same variable, to wit, the
choice has been made to apply both spatial derivatives to Green’s function,
_viz.:_

$\cancel{(G^{\prime}\ast f)(x)}+(G\ast
f^{\prime})(x)=\cancel{D\,\left(G_{xx}\ast f\right)(x)}-\cancel{b\;\left(G\ast
f\right)(x)}+r\;\left(G\ast f\right)^{2}(x),$ (3)

where prime indicates derivative with respect to time.

Since all terms are removed except the nonlinear term and the corresponding
time derivative needed to cancel this term, the residual equation, _viz.:_

$G\ast f^{\prime}=r\;\left(G\ast f\right)^{2}(x)$ (4)

No immediate solution comes to mind for the above equation, so, a sequence of
approximations for the convolution integral will be employed, namely,

######  Theorem 1 (Convolution integral theorem).

The convolution integral is greater than or equal to the product of both
functions involved in the convolution integral

$f\ast g\geq fg$ (5)

Theorem 1 is true in general because the convolution integral sums an area
either larger than or equal to the area of both functions multiplied together.
This approximation enables bypassing the convolution integral and solving the
resulting ordinary differential equation.

##  3 Zeroth approximation

With the aid of the approximation for a convolution integral, theorem 1, the
residual nonlinear equation can be transformed into a Bernoulli equation,
which has an exact solution. Applying the approximation once will yield
interesting results and they will be investigated.

Firstly, equation (4) will be brought to bear in the frequency domain to yield
a zeroth approximation. In the original domain, the approximation must be
applied to two convolution integrals, whereby, in the codomain, the
approximation is only for one convolution integral, therefore, the overall
approximation should be closer. In the transform domain, the residual
nonlinear equation takes on the following form, plus, the approximation, after
applying theorem 1, is to the right of the inequality, _viz.:_

$gF_{t}=r\left(gF\ast gF\right)(s)\geq gF_{t}=rg^{2}F^{2}$ (6)

The approximation initiates a set of applications to yield a solution, _viz.:_

$\displaystyle gF_{t}=$ $\displaystyle rg^{2}F^{2},$
$\displaystyle\frac{F_{t}}{F^{2}}=$ $\displaystyle rg,$
$\displaystyle-\frac{1}{F}=$ $\displaystyle
C(s)+r\int{g\,\mathrm{d}t},\mathrm{finally,}$ $\displaystyle F=$
$\displaystyle\frac{1}{C(s)-r\int{g\,\mathrm{d}t}},$

where $C(s)$ is a constant of integration.

The solution states the following, in the frequency domain, _viz.:_

$u(s,t)=gF$ (7)

The solution, as it stands now, is quite acceptable, for it does reduce to the
solution to the linear equation for nonlinear coefficient r equal to zero;
assuming the constant of integration, $C(s)$, equals unity.

A general inverse Fourier transform to the original domain is prohibitive,
given the nature of the solution, but an expansion around small nonlinear
coefficient r yields an approximate solution that satisfies what many
researchers yearn for from the Fisher-KPP equation; since most applications of
the Fisher-KPP equation involve small nonlinear coefficient r. Since the
denominator of the function F involves the integral of the Green’s function in
the transform domain, this function’s maximum is unity and declines as either
the frequency increases or time increases, therefore, for small nonlinear
coefficient r, the requirements for a binomial expansion are satisfied.

Before expanding, it is advisable to define what the constant of integration
is, and, for the purposes of this monograph, the property that the solution
approach the Dirac Delta function in space as time approaches zero ensures the
solution is normalized on the domain. To that end, setting time equal to zero
in the finction F yields a constant of integration equal to $C(s)=1-r/((2\pi
s)^{2}D+b)$, which will effectively cancel the contribution from the algebraic
term in the limit of time to zero, yet enable a bounded formulæ.

Figure 1: Zeroth approximation after binomial expansion (gray area is clipped
by software [14]) for $D=1,b=1,r=0.1$ and domain
$\mathcal{D}\subset\\{x|x=(-3,3)\\},\\{t|t=(0,2)\\}$.

Lumping together all functions under the symbol zeta, $\zeta$, the following
relationship holds for the binomial approximation:

$F\approx\frac{1}{1-r\;\zeta}=\sum_{n=0}^{\infty}{(r\,\zeta)^{n}}$ (8)

The inverse Fourier transform of the first term of the expansion is shown for
small nonlinear coefficient (r):

$u(s,t)=gF\approx e^{-(2\pi s)^{2}Dt-bt}-\frac{r\,e^{-(2\pi s)^{2}Dt-
bt}}{(2\pi s)^{2}D+b}+\frac{r\,e^{-2(2\pi s)^{2}Dt-2bt}}{(2\pi
s)^{2}D+b}+\text{higher order terms}$ (9)

All relevant inverse Fourier transforms are shown below:

$\displaystyle e^{-(2\pi s)^{2}Dt-bt}$
$\displaystyle\supset\frac{e^{-\frac{x^{2}}{4Dt}}}{\sqrt{4\pi Dt}}e^{-bt}$
(10) $\displaystyle\frac{1}{(2\pi s)^{2}D+b}$
$\displaystyle\supset\frac{e^{x\sqrt{\frac{b}{D}}}}{2\sqrt{bD}}\theta(-x)+\frac{e^{-x\sqrt{\frac{b}{D}}}}{2\sqrt{bD}}\theta(x)$
(11) $\displaystyle\frac{e^{-(2\pi s)^{2}Dt}e^{-bt}}{(2\pi s)^{2}D+b}$
$\displaystyle\supset\frac{e^{\frac{bx}{\sqrt{bD}}}e^{bt}\text{erfc}\left(\frac{2t\sqrt{bD}+x}{2\sqrt{Dt}}\right)\theta(-x)}{4\sqrt{bD}}+\frac{e^{bt}e^{-\frac{bx}{\sqrt{bD}}}\text{erfc}\left(\frac{2t\sqrt{bD}-x}{2\sqrt{Dt}}\right)\theta(x)}{4\sqrt{bD}}$
(12) $\displaystyle\frac{e^{-2(2\pi s)^{2}Dt}e^{-2bt}}{(2\pi s)^{2}D+b}$
$\displaystyle\supset\frac{e^{\frac{bx}{\sqrt{2bD}}}e^{-bt}\text{erfc}\left(\frac{2t\sqrt{2bD}+x}{2\sqrt{2Dt}}\right)\theta(-x)}{4\sqrt{2bD}}+\frac{e^{-bt}e^{-\frac{bx}{\sqrt{2bD}}}\text{erfc}\left(\frac{2t\sqrt{2bD}-x}{2\sqrt{2Dt}}\right)\theta(x)}{4\sqrt{2bD}},$
(13)

where $\theta(x)$ is Heaviside’s step function.

The zeroth approximation, under the binomial expansion, mimics the linear
solution, hence, the modification is slight, indeed. The surface shown in
Figure 1 is slightly depressed from the linear surface, hence, the effect of
the nonlinear coeiffient is to compress and spread the energy faster than
under normal circumstances. The solution still approaches a Dirac Delta in
space in the limit of time to zero, ensuring a solution satisfying all
requirements for a Green’s function, therefore, the solution shown is entire
and analytic throughout the domain $\mathcal{D}$. In addition, the solution
satisfies all releavant boundary conditions, specifically, the solution is
zero along the outer boundary of the domain $\partial\mathcal{D}$.

##  4 Successive approximations

A single application of the approximation for convolution integrals, theorem
5, provided a meaningful solution to the nonlinear reaction–diffusion
equation, equation (1); nonetheless, it is only one application of the
approximation and successive applications will reveal a more accurate
solution.

To that end, assume the following solution for equation (1):

$u(x,t)=G\ast f_{1}\ast f_{2}$ (14)

Applying the time derivative and maintaining all spacial derivatives upon
Green’s function yields the following residual function in the frequency
domain:

$gf_{1}^{\prime}f_{2}+gf_{1}f_{2}^{\prime}=r\left(gf_{1}f_{2}\ast
gf_{1}f_{2}\right)(s)\geq
gf_{1}^{\prime}f_{2}+gf_{1}f_{2}^{\prime}=rg^{2}f_{1}^{2}f_{2}^{2},$ (15)

where, similarly, the approximation has been applied to the right of the
inequality, also, prime represents derivation with respect to time.

The prime of $f_{1}$ is known, it is simply the zeroth order approximation,
after applying the derivative, $f_{1}^{\prime}=-rgf_{1}^{2}$, which will aid
in resolving the subsequent Bernoulli ordinary differential equation, where
function $f_{2}$ is the unknown functional [15], _viz.:_

Another application of the approximation

$gf_{1}^{\prime}f_{2}+gf_{1}f_{2}^{\prime}=rg^{2}f_{1}^{2}f_{2}^{2}$ (16)

Evaluating the prime against function $f_{1}$ yields:

$-rgf_{1}^{2}f_{2}+f_{1}f_{2}^{\prime}=rgf_{1}^{2}f_{2}^{2},$ (17)

Then substituting $f_{2}=h^{m}$ and $f_{2}^{\prime}=mh^{m-1}h^{\prime}$,
yields:

$-rgf_{1}h^{m}+mh^{m-1}h^{\prime}=rgf_{1}f_{2}^{2m}$ (18)

With m, Bernoulli’s free parameter, equal to minus unity, the equation resuces
to the following:

$rgf_{1}h+h^{\prime}=-rgf_{1}$ (19)

Solving for the integrating factor:

$he^{\int{rgf_{1}\mathrm{d}\,t}}$ (20)

which yields the following as a solution for the unknown function:

$f_{2}=\frac{e^{\int{rgf_{1}\mathrm{d}\,t}}}{C_{2}(s)-\int{rgf_{1}e^{\int{rgf_{1}\mathrm{d}\,t}}\mathrm{d}\,t}}$
(21)

Repeated iterations, as above, reveal the following sequence of functions:

$\displaystyle\begin{split}f_{1}&=\frac{1}{C_{1}(s)-\int{r\,g\,\mathrm{d}\,t}}\\\
f_{2}&=\frac{e^{\int{r\,gf_{1}\,\mathrm{d}\,t}}}{C_{2}(s)-\int{r\,gf_{1}\,e^{\int{r\,gf_{1}\,\mathrm{d}\,t}}\mathrm{d}\,t}}\\\
f_{3}&=\frac{e^{\int{r\,gf_{1}f_{2}\,\mathrm{d}\,t}}}{C_{3}(s)-\int{r\,gf_{1}f_{2}\,e^{\int{r\,gf_{1}f_{2}\,\mathrm{d}\,t}}\mathrm{d}\,t}}\\\
&\hskip 50.0pt\vdots\\\ f_{n+1}&=\frac{e^{\int{r\,gf_{1}f_{2}\ldots
f_{n}\mathrm{d}\,t}}}{C_{n}(s)-\int{r\,gf_{1}f_{2}\ldots
f_{n}\,e^{\int{r\,gf_{1}f_{2}\ldots
f_{n}\,\mathrm{d}\,t}}\mathrm{d}\,t}}\end{split}$ (22)

Inductive reasoning reveals the $f_{n+1}$ functional, whose limit to infinity
approaches a functional $f_{s}$, _viz.:_

$\lim\limits_{n\rightarrow\infty}f_{n+1}\longrightarrow
f_{s}/;\\{f_{s}\subset\mathcal{D}\\}$ (23)

The functional $f_{s}$ is a product functional and has the property of
equaling zero for all values of time and frequency variable greater than zero,
otherwise, the functional equals unity in the limit of time to zero, hence,
the functional behaves as a Dirac Delta functional in time, _viz.:_

$f_{s}=\begin{cases}\delta(t)&t>0\\\
\delta^{\prime}(t)=r\delta(t)&t=0\end{cases}$ (24)

##  5 Concluding remarks

Much has been written on the subject of Fisher’s equation in the literature,
primarily, traveling wave solutions; albeit, such solutions, whether analytic
or numeric solutions, if based upon the traveling wave approximation, are
completely invalid. This conclusion was suspected by this author, given the
Ostrogradsky theorem concerning mechanics.

The analysis shows the solution is not smooth and irregular, therefore, the
traveling wave approximation should not be applied. Besides the serious
implications for this particular equation studied, the Fisher-KPP equation,
there are a host of nonlinear equation where the traveling wave approximation
has been historically employed, placing solutions on shaky ground, to say the
least. In fact, after analysis, a whole class of apparent published solutions
have been placed under disbelief. As mentioned in the introduction, the very
viability of a nonlinear diffusion is under scrutiny, for mechanical and
rational reasons; but, analysis proves these functions are truly meaningless.

> It illustrates an empirical truth called "the BERS principle": GOD watches
> over applied mathematicians. Let us hope he continues to do so [16].
>
> ##  6 Theorems
>
> ######  Theorem 2 (DERIVATIVE OF A CONVOLUTION INTEGRAL).
>
> The derivative of a convolution integral is equivalent to the derivative of
> either function involved in the convolution; said differently, one may
> selectively apply the derivative to either function of interest.
>
> $\frac{\partial}{\partial x}\left(g\ast f\right)(x)=f^{\prime}\ast g=f\ast
> g^{\prime},$
>
> where prime indicates the derivative with respect to the variable of
> integration.
>
> See R.N. Bracewell [13].
>
> ######  Theorem 3 (DERIVATIVE OF A CONVOLUTION INTEGRAL).
>
> The derivative of a convolution integral by any variable not involved in the
> integral distributes across each function involve in the convolution.
>
> $\frac{\partial}{\partial t}\left(f\ast g\right)(x)=f^{\prime}\ast g+f\ast
> g^{\prime}$
>
> See R.N. Bracewell [13].
>
> ######  Theorem 4 (CONVOLUTION THEOREM).
>
> The convolution of two functions results in the multiplication of each
> Fourier transform of the functions, _viz.:_
>
> $f\ast g=FG$
>
> See R.N. Bracewell [13].
>
> ## References
>
>   * [1] R. A. FISHER.  The wave of advance of advantageous genes.  Annals of
> Eugenics, 7(4):355–369, 1937.
>   * [2] A. Kolmogorov, I. Petrovskii, and N. Piscunov.  A study of the
> equation of diffusion with increase in the quantity of matter, and its
> application to a biological problem.  Byul. Moskovskogo Gos. Univ.,
> 1(6):1–25, 1937.
>   * [3] L. R. Ahuja and D. Swartzendruber.  An improved form of soil-water
> diffusivity function.  Soil Science Society of America Journal, 36(1):9–14,
> 1972.
>   * [4] Ying Hou and Ruth E. Baltus.  Experimental measurement of the
> solubility and diffusivity of co2 in room-temperature ionic liquids using a
> transient thin-liquid-film method.  Industrial & Engineering Chemistry
> Research, 46(24):8166–8175, 2007.
>   * [5] Roman Cherniha and Vasyl’ Davydovych.  Nonlinear reaction–diffusion
> systems with a non-constant diffusivity: Conditional symmetries in no-go
> case.  Applied Mathematics and Computation, 268:23–34, Oct 2015.
>   * [6] Tomoshige Miyaguchi, Takashi Uneyama, and Takuma Akimoto.  Brownian
> motion with alternately fluctuating diffusivity: Stretched-exponential and
> power-law relaxation.  Phys. Rev. E, 100:012116, Jul 2019.
>   * [7] E. K. Lenzi, M. K. Lenzi, H. V. Ribeiro, and L. R. Evangelista.
> Extensions and solutions for nonlinear diffusion equations and random walks.
> Proceedings of the Royal Society A: Mathematical, Physical and Engineering
> Sciences, 475(2231):20190432, 2019.
>   * [8] Nicholas F Britton et al.  Reaction-diffusion equations and their
> applications to biology. Academic Press, 1986.
>   * [9] Audrius B. Stundzia and Charles J. Lumsden.  Stochastic simulation
> of coupled reaction–diffusion processes.  Journal of Computational Physics,
> 127(1):196–207, 1996.
>   * [10] M. Ostrogradsky.  Démonstration d’un théorème du calcul intégral.
> Mem. Ac. St. Petersbourg, IV:385, 1850.
>   * [11] C. H. Glocker.  Discussion of d’alembert’s principle for non-smooth
> unilateral constraints.  ZAMM - Journal of Applied Mathematics and Mechanics
> / Zeitschrift für Angewandte Mathematik und Mechanik, 79(S1):91–94, 1999.
>   * [12] Jennifer Coopersmith.  D’alembert’s principle.  2017\.
>   * [13] Ronald N Bracewell.  The Fourier transform and its applications,
> volume 31999.  McGraw-Hill New York, 1986.
>   * [14] Wolfram Research, Inc.  Mathematica, Version 12.2.  Champaign, IL,
> 2020.
>   * [15] Boyce, W.E. Boyce, and R.C. DiPrima.  Elementary Differential
> Equations.  Wiley, 1997.
>   * [16] C. Truesdell.  Springer-Verlag New York, 1 edition, 1984.
>


# GKT: A Novel Guidance-Based Knowledge Transfer Framework For Efficient
Cloud-edge Collaboration LLM Deployment

Yao Yao1,2,3, Zuchao Li4,∗ and Hai Zhao1,2,3,
1Department of Computer Science and Engineering, Shanghai Jiao Tong University
2Shanghai Key Laboratory of Trusted Data Circulation and Governance in Web3
3Key Laboratory of Shanghai Education Commission for Intelligent Interaction
and Cognitive Engineering, Shanghai Jiao Tong University
4National Engineering Research Center for Multimedia Software,
School of Computer Science, Wuhan University, Wuhan, 430072, P. R. China
<EMAIL_ADDRESS><EMAIL_ADDRESS>
<EMAIL_ADDRESS>
Corresponding author. This research was supported by the Joint Research
Project of Yangtze River Delta Science and Technology Innovation Community
(No. 2022CSJGG1400), the National Natural Science Foundation of China (No.
62306216), the Natural Science Foundation of Hubei Province of China (No.
2023AFB816), the Fundamental Research Funds for the Central Universities (No.
2042023kf0133).

###### Abstract

The burgeoning size of Large Language Models (LLMs) has led to enhanced
capabilities in generating responses, albeit at the expense of increased
inference times and elevated resource demands. Existing methods of
acceleration, predominantly hinged on knowledge distillation, generally
necessitate fine-tuning of considerably large models, posing a challenge for
average users. Furthermore, present techniques for expediting inference and
reducing costs operate independently. To address these issues, we introduce a
novel and intuitive Guidance-based Knowledge Transfer (GKT) framework. This
approach leverages a larger LLM as a “teacher” to create guidance prompts,
paired with a smaller “student” model to finalize responses. Remarkably, GKT
requires no fine-tuning and doesn’t necessitate the teacher and student models
to have the same vocabulary, allowing for extensive batch generation to
accelerate the process while ensuring user customization. GKT can be
seamlessly integrated into cloud-edge collaboration architectures, and is
versatile enough for plug-and-play application across various models. When
utilizing ChatGPT as teacher model and Llama2-70B as the student model, we can
achieve 95.00% of ChatGPT’s performance at 52% of the cost. The results
highlight substantial enhancements in accuracy and processing speed on the
GSM8K and CSQA datasets, surpassing the performance of using either the
student or teacher models in isolation. Our code is publicly available at
https://github.com/Zoeyyao27/GKT

GKT: A Novel Guidance-Based Knowledge Transfer Framework For Efficient Cloud-
edge Collaboration LLM Deployment

Yao Yao1,2,3, Zuchao Li4,∗ and Hai Zhao1,2,3,††thanks:  Corresponding author.
This research was supported by the Joint Research Project of Yangtze River
Delta Science and Technology Innovation Community (No. 2022CSJGG1400), the
National Natural Science Foundation of China (No. 62306216), the Natural
Science Foundation of Hubei Province of China (No. 2023AFB816), the
Fundamental Research Funds for the Central Universities (No. 2042023kf0133).
1Department of Computer Science and Engineering, Shanghai Jiao Tong University
2Shanghai Key Laboratory of Trusted Data Circulation and Governance in Web3
3Key Laboratory of Shanghai Education Commission for Intelligent Interaction
and Cognitive Engineering, Shanghai Jiao Tong University 4National Engineering
Research Center for Multimedia Software, School of Computer Science, Wuhan
University, Wuhan, 430072, P. R. China<EMAIL_ADDRESS>zcli-
<EMAIL_ADDRESS><EMAIL_ADDRESS>

##  1 Introduction

Figure 1: Cloud-edge collaboration: The GKT framework facilitates cloud-edge
collaboration by deploying the larger teacher model on remote cloud servers
and the smaller student model on lightweight mobile devices. GKT allows for
brief guidance prompts to be easily transmitted to mobile devices,
significantly reducing data transmission costs. In cloud-edge collaboration,
users can also perform simple personalized generation settings on their mobile
devices.

The swift advancement of large language models (LLMs) has dramatically pushed
the frontiers of AI technology. LLMs, with their vast number of parameters,
are exceptionally adept at comprehending human intentions, offering high-
quality reasoning, and responses Wei et al. (2022b); Zhang et al. (2023); Yao
et al. (2023); Kojima et al. (2022). However, the immense size of these models
is a double-edged sword. While it improves model performance, it also leads to
slower inference times and higher computational costs. As the demand for LLM
usage increases, relying solely on LLMs for auto-regressive inference actually
demands an overwhelming amount of computational resources and time, posing a
substantial deployment challenges for cloud services and resource-constrained
devices.

Consequently, many recent studies Leviathan et al. (2023a); Ning et al.
(2023); Jiang et al. (2023) have taken steps to improve the inference
efficiency of LLMs. One prevalent and widely adopted approach is knowledge
distillation Hinton et al. (2015); Yim et al. (2017); Tunstall et al. (2023);
Jiang et al. (2023); Chiang et al. (2023); Li et al. (2023). The majority of
knowledge distillation frameworks utilize large language models as “teacher”
models to generate training samples. These samples are then used to train more
compact “student” language models, effectively teaching them to mimic the
performance and capabilities of teacher model. However, this process still
demands a carefully crafted data generation mechanism and the subsequent
training of the student model. Despite being smaller in size, to achieve
satisfactory results in general tasks, many student models still maintain a
considerable parameter size, often around 7B Tunstall et al. (2023) and 13B
Jiang et al. (2023); Chiang et al. (2023). This is in line with the
observations by Wei et al. (2022a), who noted that the emergent abilities of
LLMs for language understanding typically become evident when the model size
exceeds 10 billion parameters.

Feature | KD | SD | GKT
---|---|---|---
Accelerates Inference | ✓ | ✓ | ✓
Preserves Model Architecture | ✓ | ✓ | ✓
Eliminates Additional Fine-tuning | ✗ | ✓ | ✓
Allows Custom Settings | ✗ | ✗ | ✓
Enables Cloud-Edge Collaboration | ✗ | ✗ | ✓
Allows Different vocabulary | ✗ | ✗ | ✓

Table 1: Comparative analysis of Knowledge Distillation (KD), Speculative
Decoding (SD), and Guidance-based Knowledge Transfer (GKT).

To solve this limitation, another research line focuses on speculative
decoding Leviathan et al. (2023a); He et al. (2023); Leviathan et al. (2023b),
which involves using a more efficient, smaller model to generate token
predictions, which the larger target model then evaluates. If a token
prediction is accepted, it’s used; if not, it’s discarded, and the target
model generates a new token. The method is shown to significantly speed up
inference without needing changes to the model’s architecture or training
procedures. However, users often have diverse generation requirements, such as
the desire to adjust generation parameters like temperature and top_p. While
speculative decoding can ensure consistency with the output of the final large
model, employing batch generation to save time during high concurrent access
can impede the ability to meet user-specific generation settings.

Figure 2: System overview. Our framework consists of two steps: guidance
generation and response completion. In guidance generation, teacher model
generates guidance prompts using batch generation to process concurrent user
inputs. In response completion, student model receives guidance prompt and
complete the response. Student model generates output with a batch size of 1
which allows customize generation settings by the user

Considering the limitations of these methods, we were inspired by the common
human experience of “Getting started is the hardest part.” and sociological
studies Goldberg et al. (2014) showing that effective prompts provided by
teachers in classrooms can significantly improve student performance in exams.
Thus, we propose a novel knowledge transfer framework: Guidance-based
Knowledge Transfer (GKT). Our framework involves two steps: firstly, using an
LLM as a teacher model to generate guidance prompts from concurrent user
inputs through batch generation. Secondly, a smaller LM acts as the student
model, which simply completes the answers based on the guidance prompts,
allowing for user-customized generation settings.

GKT reduces the burden of LLM inference, thereby speeding up response
generation. Unlike knowledge distillation, our framework does not require
generating dataset from teacher model and fine-tuning the student model. It
also doesn’t necessitate the teacher and student models to have the same
vocabulary, allowing for extensive batch generation to accelerate the process
while ensuring user customization. GKT can also be seamlessly integrated into
cloud-edge collaboration architectures as shown in Figure 1. GKT deploys the
larger teacher model on remote cloud servers and the smaller student model on
lightweight mobile devices, such as smartphones. This setup allows for brief
guidance prompts to be easily transmitted to mobile devices, significantly
reducing data transmission costs. In cloud-edge collaboration, users can also
perform simple personalized generation settings on their mobile devices. The
table 1 succinctly delineates the pros and cons between the Guidance-based
Knowledge Transfer (GKT) framework, Knowledge Distillation (KD) and
Speculative Decoding (SD), providing a clear comparative perspective.

Finally, our study follows the philosophy that “No such thing as bad student.
Only bad teacher.” We conducted various experiments to explore the optimal
guidance strategies for the teacher model, seeking answers to questions such
as how much and what type of help a teacher should provide to maximize the
student model’s accuracy while minimizing the teacher model’s inference time.
The results demonstrate that GKT not only achieves a significant accuracy
improvement of 14.18% on the GSM8K dataset, but also enhances speed by 10.72
$\times$. Furthermore, on the CSQA dataset, it records a noteworthy accuracy
increase of 14.00% along with a 7.73 $\times$ acceleration in inference speed.

Dataset | Model | Output Length | ACC${}_{\text{teacher}}(\%)$ | ACC(%) | $\Delta(ACC)$ | Time(s) | Speed Up
---|---|---|---|---|---|---|---
GSM8K | Single Model |
Llama2-7B | 200 | - | 13.87 | - | 6945.70 | 1.31$\times$
| 300 | - | 14.40 | - | 10304.18 | 1.38$\times$/13.98$\times$
Llama2-13B | 200 | - | 21.23 | 7.36 | 9066.17 | -
| 300 | - | 23.65 | 9.25 | 14215.12 | 10.13$\times$
Llama2-70B | 300 | - | 56.63 | 42.23 / 32.98 | 144018.55 | -
Bloom-3B | 300 | - | 2.35 | - | 4376.05 | 1.00$\times$
Bloom-7B | 300 | - | 4.40 | 2.05 | 4392.74 | -
GKT Framework |
Llama2-13B$\xrightarrow{}$Llama2-7B | 30$\xrightarrow{}$200 | 3.33 | 17.66 | 3.79 | 7762.62 | 1.17$\times$
| 30$\xrightarrow{}$300 | 3.33 | 17.82 | 3.42 | 10793.34 | 1.32$\times$
| 40$\xrightarrow{}$300 | 4.62 | 19.18 | 4.78 | 10871.01 | 1.31$\times$
| 40(concise)$\xrightarrow{}$300 | 4.62 | 19.26 | 4.86 | 10707.43 | 1.33$\times$
Llama2-70B$\xrightarrow{}$Llama2-7B | 40$\xrightarrow{}$300 | 7.81 | 28.58 | 14.18 | 13440.71 | 10.72 $\times$
Llama2-70B$\xrightarrow{}$Llama2-13B | 40$\xrightarrow{}$300 | 7.81 | 35.41 | 11.76 | 16250.34 | 8.86$\times$
Bloom-7B$\xrightarrow{}$Bloom-3B | 40$\xrightarrow{}$300 | 2.05 | 2.58 | 0.23 | 5198.60 | 0.84$\times$
| Llama2-7B$\xrightarrow{}$Bloom-3B | 40$\xrightarrow{}$300 | 3.71 | 6.82 | 4.47 | 4154.40 | 1.06$\times$
| Llama2-13B$\xrightarrow{}$Bloom-3B | 40$\xrightarrow{}$300 | 4.62 | 7.73 | 5.38 | 4186.22 | 3.40$\times$
| Llama2-13B$\xrightarrow{}$Bloom-7B | 40$\xrightarrow{}$300 | 4.62 | 10.31 | 5.91 | 3275.90 | 3.82$\times$
CSQA | Single Model |  |  |  |  |  |
Llama2-7B | 100 | - | 60.69 | - | 3239.04 | 1.31 $\times$ / 13.36$\times$
| 300 | - | 60.61 | - | 9321.51 | 1.30$\times$
Llama2-13B | 100 | - | 71.17 | 10.48 | 4235.46 | 10.22$\times$
| 300 | - | 71.09 | 10.48 | 12128.50 | -
Llama2-70B | 100 | - | 76.58 | 15.89 / 5.41 | 43285.73 | -
Bloom-3B | 100 | - | 20.88 | - | 2157.97 | 1.05$\times$
Bloom-7B | 100 | - | 21.79 | - | 2269.35 | -
GKT Framework |  |  |  |  |  |
Llama2-13B$\xrightarrow{}$Llama2-7B | 10$\xrightarrow{}$100 | 0.00 | 61.02 | 0.33 | 3472.33 | 1.22$\times$
| 20$\xrightarrow{}$100 | 0.00 | 64.70 | 4.01 | 3656.64 | 1.16$\times$
| 30$\xrightarrow{}$100 | 18.76 | 69.86 | 9.17 | 3579.21 | 1.18$\times$
| 30 $\xrightarrow{}$ 300 | 18.76 | 69.86 | 9.25 | 10171.32 | 1.19$\times$
| 40 $\xrightarrow{}$300 | 62.74 | 71.01 | 10.40 | 10068.97 | 1.20$\times$
| 50 $\xrightarrow{}$300 | 70.43 | 71.09 | 10.48 | 10361.78 | 1.17$\times$
Llama2-70B$\xrightarrow{}$Llama2-7B | 30$\xrightarrow{}$100 | 18.84 | 74.69 | 14.00 | 5600.38 | 7.73$\times$
Llama2-70B$\xrightarrow{}$Llama2-13B | 30$\xrightarrow{}$100 | 18.84 | 76.16 | 4.99 | 6598.43 | 6.56$\times$
Bloom-7B$\xrightarrow{}$Bloom-3B | 20$\xrightarrow{}$100 | 0.00 | 20.96 | 0.08 | 2395.36 | 0.95$\times$
| 30$\xrightarrow{}$100 | 11.06 | 20.63 | -0.25 | 2250.04 | 1.01$\times$
| Llama2-7B$\xrightarrow{}$Bloom-3B | 30$\xrightarrow{}$100 | 21.86 | 40.05 | 19.17 | 2134.58 | 1.52$\times$
| Llama2-13B$\xrightarrow{}$Bloom-3B | 30$\xrightarrow{}$100 | 18.76 | 41.20 | 20.32 | 2210.29 | 1.92 $\times$
| Llama2-13B$\xrightarrow{}$Bloom-7B | 30$\xrightarrow{}$100 | 18.76 | 39.80 | 18.01 | 2405.20 | 1.76$\times$

Table 2: Results for GSM8K and CSQA. “$\xrightarrow{}$” signifies the
transition from the teacher model to the student model, with settings on the
left of the arrow (Model, Output Length) pertaining to the teacher model, and
those on the right corresponding to the student model.
“ACC${}_{\text{teacher}}(\%)$” and “ACC(%)” denotes the mean accuracy(%)
achieved by the teacher model alone and the overall framework, respectively.
“$\Delta$(ACC)” denotes the improvement in accuracy (%) achieved by the GKT
framework compared to using only the student model ($\Delta$(ACC) for
Llama2-70B shows two numbers separated by “/”, The number on the left (right)
of the “/” is the change in accuracy when using Llama2-7B (Llama2-13B) as the
student model). “Speed Up” indicates the acceleration factor of the GKT
framework relative to using only the teacher model. (“Speed Up” for Llama2-7B
shows two numbers separated by “/”, The number on the left (right) of the “/”
is the change in accuracy when using Llama2-13B (Llama2-70B) as the student
model). “concise” denotes that we use concise guidance generation method and
the detailed analysis can be found in section 4.2

##  2 Method

The schematic representation of our Guidance-Based Knowledge Transfer (GKT)
framework is depicted in Figure 2. GKT encompasses a two-step process:
guidance generation and response completion. During the guidance generation
phase, a large language model serves as the “teacher” model. This model
processes concurrent user inputs and employs batch generation to craft
guidance prompts. Subsequently, in the response completion stage, a smaller
language model functions as the “student” model. This model offers flexibility
in generation settings, allowing for user customization. The guidance prompt
created in the first stage is then fed into this smaller model, facilitating
the completion of the response with enhanced efficiency. We will now elaborate
on each stage:

##### Guidance Generation

Given concurrent user inputs $Q=\\{q_{1},q_{2},...,q_{n}\\}$, where $q_{i}$
represents the input question from user $i$, the teacher model $M_{t}$ batch
generates the guidance prompts:

$G=\\{g_{1},g_{2},...,g_{n}\\}=\mathbf{F}(M_{t}(Q))$

Here, $G$ represents the batch-generated guidance, $g_{i}$ is the guidance for
user $i$, and $\mathbf{F}(\cdot)$ denotes the projection operation that
generates the guidance prompt from the generated text. In this paper, we
explore different projection operations including: (1) Cut-off guidance
generation (2) Concise guidance generation (3) Hint guidance generation. We
will elaborate on these methods in Section 4.2

##### Response generation

In response generation, we use a smaller language model $M_{s}$ as student
model. For every user $i$, $M_{s}$ generates the final response $r_{i}$ by:

$r_{i}=M_{s}^{i}(g_{i})$

Where $M_{s}^{i}$ stands for $M_{s}$ under the user $i$’s custom generation
setting. Figure 6 in Appendix D demonstrates a concrete example for GKT
framework.

##  3 Experimental Settings

##### Datasets

In this paper we use two challenge but widely used dataset: GSM8K Cobbe et al.
(2021) for arithmetic reasoning and CSQA Talmor et al. (2019) for commonsense
reasoning. GSM8K focus on arithmetic reasoning which is a collection of grade
school math word problems, each requiring 2 to 8 steps to solve. The solutions
mainly involve a series of basic arithmetic calculations to arrive at the
final answer. CSQA focuses on commonsense question answering which includes
multiple-choice questions that require commonsense knowledge for answering.
The detailed dataset statistics can be found in Appendix A. In all our
experiments, we use the same prompt settings in Manual-CoT Wei et al. (2022b)
otherwise stated. The full prompt can be found in Appendix E

##### Hyperparameter

All experiments were conducted on an NVIDIA A800 GPU. Detailed hyperparameter
settings for the experiments are provided in Appendix B. In our experiments,
we tested various teacher models, including Flan-t5-xl Scao et al. (2022),
Bloom-7BScao et al. (2022), Llama2-70BTouvron et al. (2023) and Llama2-13B.
Correspondingly, the student models used were Flan-t5-large, Bloom-3B,
Llama2-13B and Llama2-7B.

##  4 Results and Exploration

In this section, we delve into the empirical results of our comprehensive
analysis using the GKT framework. The essence of this exploration lies in
quantifying the effectiveness of GKT in enhancing the accuracy of student
models while ensuring computational efficiency. We first report the overall
results for the GKT on the GSM8K and CSQA datasets. Then, we explore various
dimensions of knowledge transfer including the optimal guidance generation
methods, the intriguing dynamics between different types of teacher and
student models, the influence that a teacher can exert on student and the
optimal guidance length.

###  4.1 Overall Results

The overall results for the Guidance-based Knowledge Transfer (GKT) framework
on the GSM8K and CSQA datasets are presented in Table 2. The results indicate
that on the GSM8K dataset, we achieved a maximum accuracy improvement of 14.18
% compared to simply using the student model, along with a 10.72$\times$ speed
up relative to the teacher model. On the CSQA dataset, we observed a accuracy
improvement of 14.00 % compared to the student model, and a maximum
7.73$\times$ speed-up in comparison to the teacher model.

###  4.2 How to facilitate student learning effectively?

Model | Prompt | Output Length | ACC(%) | $\Delta(ACC)$ | Time(s) | Speed Up
---|---|---|---|---|---|---
Single Model
Llama2-7B | - | 300 | 14.40 | - | 10304.18 | 1.38$\times$
Llama2-13B | - | 300 | 23.65 | 9.25 | 14215.12 | -
GKT Framework
Llama2-13B$\xrightarrow{}$Llama2-7B | - | 40$\xrightarrow{}$300 | 19.18 | 4.78 | 10871.01 | 1.31$\times$
"Provide the answer in a brief manner: " | 40(concise)$\xrightarrow{}$300 | 19.26 | 4.86 | 10707.43 | 1.33$\times$
"Provide a brief hint for the question: " | 40(hint)$\xrightarrow{}$300 | 19.11 | 4.71 | 11190.12 | 1.27$\times$

Table 3: Results of different guidance generation methods on GSM8K. “concise”
and “hint” denotes Concise Guidance Generation and Hint Guidance Generation
respectively

To investigate how to generate better guidance to assist student models in
answering questions, we experimented with three different guidance generation
methods as described in Section 2: (1) Cut-off Guidance Generation, (2)
Concise Guidance Generation, and (3) Hint Guidance Generation.

Cut-off Guidance Generation: We employed the simplest method of cutting off,
where the teacher model generates only a fixed number of the first $m$ tokens
as guidance.

Concise Guidance Generation: In this approach, we added the prompt: “Provide
the answer in a brief manner:” to guide the model to generate more concise
guidance responses.

Hint Guidance Generation: Here, we introduced the prompt “Provide a brief hint
for the question:” to encourage the teacher model not to give direct answers
but to offer hints in a guiding manner, aiding the student model in generating
responses.

The detailed results of these experiments on GSM8K can be seen in Table 3. To
more intuitively understand the acceleration effect of each component in the
GKT framework, we have created a trace diagram of GKT’s performance on the
GSM8K dataset, as illustrated in Figure 3. Intriguingly, we found that
providing hints leads to poorer outcomes compared to directly giving the
answer. We speculate that this may be due to the limited inferential and
reasoning capabilities of the smaller models. Instead of giving hints for them
to infer, it might be more effective to provide direct answers. We also
observed that Concise Guidance Generation was relatively effective, as the
brevity of the guidance reduces the inferential workload for the student
model. By prompting the teacher model to produce shorter answers, this method
not only improved the accuracy of the model’s responses but also accelerated
the inference speed.

###  4.3 GKT for Cloud-edge Collaboration LLM Deployment

Figure 3 demonstrates that utilizing only the Llama2-13B model for the GSM8K
dataset, comprising a total of 1319 examples, results in an average response
time of 10.78 seconds per example ($14215.12\div 1319=10.78s$). This implies a
single-user service capability within this timeframe. Conversely, the
deployment of the GKT framework in a Cloud-Edge collaboration environment for
LLMs markedly reduces the large model’s response time to 0.38 seconds on
average ($506.73\div 1319=0.38s$), and the small model processes each example
in 7.86 seconds ($10364.28\div 1319=7.86s$), culminating in a total response
time of 8.24 seconds. Therefore, theoretically, by employing batch processing,
the GKT framework can facilitate simultaneous service to 24 users within the
8.24-second window.  This efficiency stems from the large model’s capacity for
batch processing in the cloud, which can concurrently serve multiple users
(with a batch size of 24). At the same time, the small models are deployed on
distinct edge devices, facilitating parallel operations and enabling
personalized user experiences. In stark contrast, reliance solely on the
Llama2-13B model limits service to a single user within the 10.78-second
timeframe. Hence, the GKT framework substantially augments the parallelism in
user service provision in Cloud-Edge collaborative LLM deployments.

Additionally, compared to speculative decoding, GKT can control transmission
losses in low-bandwidth environments by adjusting the length of the guidance
prompt. We provide a comprehensive analysis of GKT and speculative decoding in
Appendix C.

Figure 3: The trace diagram for GKT framework on GSM8K. The vertical axis
represents the guidance generation method used (where “Llama2-13B” indicates
the exclusive use of the Llama2-13B model for response generation, and
“Llama2-7B” denotes the sole use of the Llama2-7B model). The horizontal axis
represents the runtime (s) of the GKT across the entire dataset.

###  4.4 How To Identify the Right Teacher for the Right Student ?

To find the right student for the right teacher, we experimented with the
decoder-only model Llama and the encoder-decoder model Flan-t5. Our
experiments show that both types of models can achieve certain improvements
through GKT. However, an intriguing phenomenon we observed was that replacing
the teacher model of Flan-t5 with a larger Llama model led to an accuracy
decrease, rather than an increase. We speculate that one possible reason could
be that for encoder-decoder models, the encoder and decoder use different
parameters. Using a large model’s answers as input for the smaller model’s
encoder may disrupt the coherence of thought, leading to reduced inferential
ability. In contrast, for decoder-only models, directly concatenating inputs
and targets seems to aid in better inference. The significant structural
differences between T5 and Llama models could result in inconsistent thinking
patterns between models, implying that the teacher and student are not
perfectly in sync, thereby diminishing the overall effectiveness.

Model | Output Length | ACC(%) | $\Delta(ACC)$ | Time(s) | Speed Up
---|---|---|---|---|---
Single Model
Flan-t5-large (0.8B) | 300 | 6.90 | - | 2343.06 | 1.09$\times$
Flan-t5-xl (3B) | 300 | 11.30 | 4.40 | 2557.76 | -
Llama2-7B | 300 | 14.40 | - | 10304.18 | 1.38$\times$
Llama2-13B | 300 | 23.65 | 12.35 | 14215.12 | -
GKT Framework
Flan-t5-xl$\xrightarrow{}$Flan-t5-large | 40$\xrightarrow{}$300 | 7.88 | 0.98 | 2190.05 | 1.17$\times$
Llama2-13B$\xrightarrow{}$Llama2-7B | 40$\xrightarrow{}$300 | 19.18 | 4.78 | 10871.01 | 1.31$\times$
Llama2-13B$\xrightarrow{}$Flan-t5-large | 40$\xrightarrow{}$300 | 6.52 | -0.38 | 2330.48 | 6.10$\times$
Llama2-7B$\xrightarrow{}$Flan-t5-large | 40$\xrightarrow{}$300 | 6.21 | -0.69 | 2284.67 | 4.51$\times$

Table 4: Decoder-only Model VS Encoder-Decoder Model. We experimented with the
decoder-only model Llama and the encoder-decoder model Flan-t5.

###  4.5 How Can a Teacher Influence His Student?

Based on the above findings, we conducted further experiments to explore the
impact of a teacher’s guidance on student models, specifically focusing on
decoder-only models. In these experiments, we used Llama2 and Bloom, both
decoder-only models. The overall results can be seen in Table 5. Table 5
presents a detailed overview of the impact of teacher model choice on the
performance and efficiency of student models, as tested on the GSM8K and CSQA
datasets. The results highlight the differential effects on accuracy and
processing speed, depending on the combination of teacher and student models
used. The results also revealed that the Llama model possesses stronger
inferential capabilities and stores more common-sense knowledge, performing
better on both datasets. When we provided the same Bloom student model with
the more experienced teacher model Llama, under similar-sized teacher model
conditions, we observed that the Llama model, as a teacher, could enhance the
student model’s accuracy by 20 %. In contrast, using Bloom-7B as the teacher
resulted in a decrease in accuracy. This outcome underscores the critical
influence of the teacher on the student.

Additionally, by comparing results on the GSM8K and CSQA datasets, we found
that this approach of using larger models to guide smaller ones can better
transfer the common-sense knowledge stored in the teacher model. However, it
had a less pronounced effect on mathematical reasoning abilities in the GSM8K
dataset. On the CSQA dataset, when employing a Llama2-7B teacher model,
Bloom-3B model’s accuracy is improved by nearly 20 %, with a 1.52 times speed
increase. These results convincingly demonstrate that our method can
effectively transfer the knowledge and reasoning abilities stored in larger
models to smaller ones.

Model | Output Length | ACC(%) | $\Delta(ACC)$ | Time(s) | Speed Up
---|---|---|---|---|---
GSM8K
Llama2-13B$\xrightarrow{}$Llama2-7B | 40$\xrightarrow{}$300 | 19.18 | 4.78 | 10871.01 | 1.31$\times$
Bloom-7B$\xrightarrow{}$Bloom-3B | 40$\xrightarrow{}$300 | 2.58 | 0.23 | 5198.60 | 0.84$\times$
Llama2-7B$\xrightarrow{}$Bloom-3B | 40$\xrightarrow{}$300 | 6.82 | 4.47 | 4154.40 | 1.06$\times$
Llama2-13B$\xrightarrow{}$Bloom-3B | 40$\xrightarrow{}$300 | 7.73 | 5.38 | 4186.22 | 3.40$\times$
Llama2-13B$\xrightarrow{}$Bloom-7B | 40$\xrightarrow{}$300 | 10.31 | 5.91 | 3725.90 | 3.82$\times$
CSQA
Llama2-13B$\xrightarrow{}$Llama2-7B | 30$\xrightarrow{}$100 | 69.86 | 9.17 | 3579.21 | 1.18$\times$
Bloom-7B$\xrightarrow{}$Bloom-3B | 30$\xrightarrow{}$100 | 20.63 | -0.25 | 2250.04 | 1.01$\times$
Llama2-7B$\xrightarrow{}$Bloom-3B | 30$\xrightarrow{}$100 | 40.05 | 19.17 | 2134.58 | 1.52$\times$
Llama2-13B$\xrightarrow{}$Bloom-3B | 30$\xrightarrow{}$100 | 41.2 | 20.32 | 2210.29 | 1.92$\times$
Llama2-13B$\xrightarrow{}$Bloom-7B | 30$\xrightarrow{}$100 | 39.8 | 18.01 | 2405.2 | 1.76$\times$

Table 5: Comparative analysis of teacher influence on student models using
decoder-only models, Llama and Bloom.

| Average Output Length | ACC | $\Delta(ACC)$
---|---|---|---
| ChatGPT | Ours
Full ChatGPT | 76.68 | - | 68.16 | -
Llama2-7B | - | 10 $\xrightarrow{}$ 300 | 19.41 | 0.23
- | 20 $\xrightarrow{}$ 300 | 27.52 | 8.34
- | 30 $\xrightarrow{}$ 300 | 32.83 | 13.65
- | 40 $\xrightarrow{}$ 300 | 40.79 | 21.61
Llama2-13B | - | 40 $\xrightarrow{}$ 300 | 48.14 | 24.49
Llama2-70B | - | 40 $\xrightarrow{}$ 300 | 64.75 | 8.12

Table 6: Result on GSM8K when using ChatGPT as teacher model and Llama2-7B,
13B, 70B as student model. “$\xrightarrow{}$” signifies the output length
transition from ChatGPT to the Llama2 model. “$\Delta$(ACC)” denotes the
improvement in accuracy (%) achieved by GKT framework compared to using only
the corresponding Llama2 model. “Full ChatGPT” denotes the ChatGPT performance
on GSM8K without GKT

To delve deeper into the extent of influence a teacher model can have on a
student model, we utilized the superior-performing ChatGPT OpenAI (2023) as
the teacher model, with Llama2-7B, 13B, and 70B serving as the student model.
The Table 6 clearly demonstrates that using a more powerful teacher model
significantly enhances performance. When the number of guidance tokens is kept
constant, employing ChatGPT as the teacher model results in a substantial
improvement over using Llama2-13B as the teacher model. Specifically, on the
GSM8K dataset, there’s an increase of 21.53 % compared to Llama2-13B teacher
(from 19.26% to 40.79% ). We can also see from the table that stronger student
model can preserve more teacher model abilities. From another perspective,
this approach allows for more cost-effective outcomes. When utilizing
ChatGPT’s API interface and employing the best performance Llama2-70B as the
student model, we can achieve 95.00% of ChatGPT’s performance at 52% of the
cost, effectively showcasing the GKT framework’s “cheap and cheerful” charm.

Model | Output Length | ACC${}_{\text{teacher}}$(%) | ACC(%) | $\Delta(ACC)$ | Time(s) | Speed up
---|---|---|---|---|---|---
Llama2-7B | 100 | - | 27.17 | - | 707.02 | 13.05$\times$
Llama2-70B | 100 | - | 40.16 | 12.99 | 9224.76 | -
Llama2-70B$\xrightarrow{}$ Llama2-7B | 20$\xrightarrow{}$100 | 5.51 | 30.31 | 3.14 | 1070.02 | 8.62$\times$

Table 7: Results on AQuA-RAT dataset

###  4.6 Further Exploration

####  4.6.1 Results on additional datasets

To better demonstrate the effectiveness of GKT across different datasets, we
conducted supplementary experiments on the AQuA-RAT Ling et al. (2017)
dataset. The results for the AQuA-RAT dataset are presented in the Table 7
which shows that GKT also exhibits superior capabilities on the AQuA-RAT
dataset, powerfully affirming the effectiveness of the GKT framework and its
robustness across various datasets.

Model | Rouge |  $\Delta$(Rouge) | Time(s)
---|---|---|---
Llama2-7B | 0.1475 | - | 10304.18
Llama2-13B | 0.1631 | 0.0156 | 14215.12
Llama2-13B$\xrightarrow{}$Llama2-7B | 0.1578 | 0.0103 | 10871.01

Table 8: Rouge scores on GSM8K rationale dataset. We set the output length for
the teacher model to 40 tokens and for the student model to 300 tokens.

To validate the performance of the GKT framework in more open-ended generative
tasks, we also calculated and compared the Rouge scores of the rationales
generated by the GKT model on the GSM8K dataset. The GSM8K dataset contains
two types of labels. The first is the rationale for each question, which is a
text string providing a solution to the question. The second label is the
numeric format answer to the question. Examples of the GSM8K dataset can be
found in Figure 7, where the blue part is the rationale, and the pink part is
the answer. In our evaluation, we expanded beyond just assessing the final
numeric format answers (Accuracy) by also evaluating the rationale provided
for answering the question (Rouge). This approach helps in assessing how well
the GKT framework adapts to tasks that require extensive content generation.
The results in terms of Rouge scores are presented in the Table 8.

####  4.6.2 How much guidance should a teacher offer to students ?

To investigate the optimal amount of assistance a teacher should provide to a
student, or in other words, to determine the most suitable guidance length for
maximizing student benefits, we conducted experiments. Intuitively, one might
assume that the more a teacher model outputs, the higher the accuracy of the
model’s response. However, longer guidance tends to weaken the model’s
acceleration effect. Therefore, we plotted a line graph showing the changes in
accuracy as the guidance length varied from 10 to 40, in intervals of 10, as
depicted in Figure 4. Based on our experiments, we established that for the
GSM8K dataset, the teacher model should output the first 40 tokens of the
answer. For the CSQA dataset, which generally requires shorter responses, we
set the teacher model to output the first 30 tokens.

$10$$20$$30$$40$$20$$40$$60$Guidance LengthAccuracy(%)GSM8KCSQA Figure 4:
Performance of different length ranges. We use Llama2-13B as teacher model and
Llama2-7B as student model.

####  4.6.3 How few-shot exemplars affect students?

$2$$4$$6$$8$$20$$40$$60$Number of Few-shot ExemplarsAccuracy(%)GSM8KCSQA
Figure 5: Performance of different few-shot exemplar number

In this investigation, we focused on the role of few-shot exemplars in
influencing the performance of student models. We used a range of few-shot
exemplars, provided through Manual-CoT Wei et al. (2022b), acros GSM8K and
CSQA. The variation in accuracy with the change in the number of few-shot
exemplars is illustrated in Figure 5. As depicted in Figure 5, it can be
observed that the performance of the student models improves with the increase
in the number of exemplars. Thus, in this experiment, we select 8-shot for
GSM8K and 7-shot for CSQA in line with the Manual-CoT.

##  5 Related Work

The aspect most relevant to our work is knowledge distillationTunstall et al.
(2023); Jiang et al. (2023); Chiang et al. (2023); Li et al. (2023); Chen et
al. (2023). Knowledge distillation involves condensing the expertise from
larger models into more compact versions, thereby ensuring these smaller
models retain high efficiency while still achieving impressive performance. Li
et al. (2023) introduces Symbolic Chain-of-Thought Distillation (SCoTD), which
trains a smaller "student" language model using the outputs (reasoning chains)
of a larger "teacher" model. The teacher model first generates multiple
reasoning chains for a given task, showcasing step-by-step problem-solving.
The student model then are fine-tuned on these examples, essentially mimicking
the teacher’s reasoning process. This training enables the student model to
perform complex reasoning tasks more effectively, despite its smaller size.
Tunstall et al. (2023) presents a distillation framework named Distilled
Direct Preference Optimization (dDPO) result in a 7B model named ZEPHYR. The
method comprises three steps: (1) distilled supervised fine-Tuning (2) AI
feedback through preferences and (3) distilled direct preference optimization.
Different from knowledge distillation, our work focuses on leveraging the
knowledge of larger models to enhance the overall efficiency and performance.
The proposed GKT framework circumvents the usual requirements of producing a
distillation dataset from a teacher model or fine-tuning the student model,
streamlining the knowledge transfer process.

Another line of study that is related to our work is speculative decoding
Leviathan et al. (2023a). Speculative decoding uses two models: the original
target model and a much smaller approximate model. The smaller model handles
autoregressive sampling, while the larger assesses the output. Simple tokens
are generated by the smaller model, with complex tokens handled by the larger.
Based on speculative decoding, He et al. (2023) proposed Retrieval-Based
Speculative Decoding (REST) which combines speculative decoding with retrieval
techniques. Instead of using a smaller LM for draft generation, REST bypasses
the need for an additional small LM by retrieving draft tokens from a pre-
built datastore containing context-continuation pairs. These drafts are then
verified by a large LM. Recently, Ning et al. (2023) proposed skeleton-of-
thought (SoT). SoT first guides LLMs to first create a concise “skeleton” of
an answer and then fill in each point of the skeleton in parallel, speeding up
the response process. Xu et al. (2023) introduces Super In-Context Learning
(SuperICL), a method enhancing the performance of LLMs by integrating them
with smaller, locally fine-tuned models. These smaller models, acting as plug-
ins, provide task-specific knowledge and predictions. The process involves
fine-tuning a small model on task-specific data, using it to generate
predictions and confidence scores for in-context examples, and combining these
with the LLM’s general language understanding. While SuperICL also integrates
LLMs and LMs, it primarily focuses on enhancing model performance without
considering the efficiency of the framework. This focus on performance
enhancement can even negatively impact overall efficiency. Whereas, GKT
focuses on finding the optimal balance between efficiency and effectiveness

##  6 Conclusion

We introduce the innovative Guidance-based Knowledge Transfer (GKT) framework,
designed to leverage the knowledge of larger models to enhance the efficiency
and performance of smaller models, while maintaining the flexibility of
personalized generation settings, allowing users to freely adjust the
generation parameters. The unique collaborative framework of GKT seamlessly
integrates into cloud-edge architectures, deploying smaller models on edge
devices to minimize data transmission delays and expedite response generation.
Our results demonstrate remarkable improvements: a maximum accuracy increase
of 14.18% and a 10.72$\times$ speed-up on the GSM8K dataset, and a 14.00%
accuracy enhancement with a 7.73$\times$ speed increase on the CSQA dataset.
Moreover, when utilizing ChatGPT as teacher model and Llama2-70B as the
student model, we can achieve 95.00% of ChatGPT’s performance at 52% of the
cost. GKT signify major strides in performance metrics, combining accuracy
with computational speed-ups - all wrapped up in a “cheap and cheerful”
package.

## Limitation

The Guidance Knowledge Transfer (GKT) framework exhibits certain limitations
compared to traditional knowledge distillation methods. Knowledge distillation
primarily relies on a pre-trained smaller model, often leading to faster
inference speeds. However, the GKT framework still depends on a large model to
generate guidance prompts during inference. This approach aims to reduce the
limitations brought about by fine-tuning through performance loss during
inference while supporting customized configurations of the framework.

Moreover, the length of guidance prompts generated by the GKT framework for
different datasets is specific. Although the generation method is universally
applicable, finding a universally appropriate length for guidance prompts that
suits various scenarios remains a challenge. This means that additional
adjustments and optimizations may be necessary for different datasets and
application contexts to ensure optimal performance. Therefore, one of the
future research directions is to explore how to more effectively determine the
appropriate lengths of guidance prompts to enhance the universality and
flexibility of the GKT framework.

## References

  * Chen et al. (2023) Zeming Chen, Qiyue Gao, Antoine Bosselut, Ashish Sabharwal, and Kyle Richardson. 2023.  DISCO: distilling counterfactuals with large language models.  In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, Toronto, Canada, July 9-14, 2023_ , pages 5514–5528. Association for Computational Linguistics.
  * Chiang et al. (2023) Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023.  Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality.
  * Cobbe et al. (2021) Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021.  Training verifiers to solve math word problems.  _CoRR_ , abs/2110.14168.
  * Goldberg et al. (2014) Saryn R Goldberg, Jennifer Andrea Rich, and Amy Masnick. 2014.  The use of metacognitive writing-to-learn prompts in an engineering statics class to improve student understanding and performance.  In _2014 ASEE Annual Conference & Exposition_, pages 24–1251.
  * He et al. (2023) Zhenyu He, Zexuan Zhong, Tianle Cai, Jason D. Lee, and Di He. 2023.  REST: retrieval-based speculative decoding.  _CoRR_ , abs/2311.08252.
  * Hinton et al. (2015) Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. 2015.  Distilling the knowledge in a neural network.  _CoRR_ , abs/1503.02531.
  * Jiang et al. (2023) Yuxin Jiang, Chunkit Chan, Mingyang Chen, and Wei Wang. 2023.  Lion: Adversarial distillation of closed-source large language model.  _arXiv preprint arXiv:2305.12870_.
  * Kojima et al. (2022) Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022.  Large language models are zero-shot reasoners.  In _Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022_.
  * Leviathan et al. (2023a) Yaniv Leviathan, Matan Kalman, and Yossi Matias. 2023a.  Fast inference from transformers via speculative decoding.  In _International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA_ , volume 202 of _Proceedings of Machine Learning Research_ , pages 19274–19286. PMLR.
  * Leviathan et al. (2023b) Yaniv Leviathan, Matan Kalman, and Yossi Matias. 2023b.  Fast inference from transformers via speculative decoding.  In _Proceedings of the 40th International Conference on Machine Learning_ , volume 202 of _Proceedings of Machine Learning Research_ , pages 19274–19286. PMLR.
  * Li et al. (2023) Liunian Harold Li, Jack Hessel, Youngjae Yu, Xiang Ren, Kai-Wei Chang, and Yejin Choi. 2023.  Symbolic chain-of-thought distillation: Small models can also “think” step-by-step.  In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 2665–2679, Toronto, Canada. Association for Computational Linguistics.
  * Ling et al. (2017) Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. 2017.  Program induction by rationale generation: Learning to solve and explain algebraic word problems.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers_ , pages 158–167. Association for Computational Linguistics.
  * Ning et al. (2023) Xuefei Ning, Zinan Lin, Zixuan Zhou, Huazhong Yang, and Yu Wang. 2023.  Skeleton-of-thought: Large language models can do parallel decoding.  _CoRR_ , abs/2307.15337.
  * OpenAI (2023) OpenAI. 2023.  GPT-4 technical report.  _CoRR_ , abs/2303.08774.
  * Scao et al. (2022) Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilic, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, Jonathan Tow, Alexander M. Rush, Stella Biderman, Albert Webson, Pawan Sasanka Ammanamanchi, Thomas Wang, Benoît Sagot, Niklas Muennighoff, Albert Villanova del Moral, Olatunji Ruwase, Rachel Bawden, Stas Bekman, Angelina McMillan-Major, Iz Beltagy, Huu Nguyen, Lucile Saulnier, Samson Tan, Pedro Ortiz Suarez, Victor Sanh, Hugo Laurençon, Yacine Jernite, Julien Launay, Margaret Mitchell, Colin Raffel, Aaron Gokaslan, Adi Simhi, Aitor Soroa, Alham Fikri Aji, Amit Alfassy, Anna Rogers, Ariel Kreisberg Nitzav, Canwen Xu, Chenghao Mou, Chris Emezue, Christopher Klamm, Colin Leong, Daniel van Strien, David Ifeoluwa Adelani, and et al. 2022.  BLOOM: A 176b-parameter open-access multilingual language model.  _CoRR_ , abs/2211.05100.
  * Talmor et al. (2019) Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019.  CommonsenseQA: A question answering challenge targeting commonsense knowledge.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4149–4158, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Touvron et al. (2023) Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton-Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurélien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. 2023.  Llama 2: Open foundation and fine-tuned chat models.  _CoRR_ , abs/2307.09288.
  * Tunstall et al. (2023) Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro von Werra, Clémentine Fourrier, Nathan Habib, Nathan Sarrazin, Omar Sanseviero, Alexander M. Rush, and Thomas Wolf. 2023.  Zephyr: Direct distillation of LM alignment.  _CoRR_ , abs/2310.16944.
  * Wei et al. (2022a) Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. 2022a.  Emergent abilities of large language models.  _Trans. Mach. Learn. Res._ , 2022.
  * Wei et al. (2022b) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022b.  Chain-of-thought prompting elicits reasoning in large language models.  In _NeurIPS_.
  * Xu et al. (2023) Canwen Xu, Yichong Xu, Shuohang Wang, Yang Liu, Chenguang Zhu, and Julian J. McAuley. 2023.  Small models are valuable plug-ins for large language models.  _CoRR_ , abs/2305.08848.
  * Yao et al. (2023) Yao Yao, Zuchao Li, and Hai Zhao. 2023.  Beyond chain-of-thought, effective graph-of-thought reasoning in large language models.  _CoRR_ , abs/2305.16582.
  * Yim et al. (2017) Junho Yim, Donggyu Joo, Ji-Hoon Bae, and Junmo Kim. 2017.  A gift from knowledge distillation: Fast optimization, network minimization and transfer learning.  In _2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017_ , pages 7130–7138. IEEE Computer Society.
  * Zhang et al. (2023) Zhuosheng Zhang, Yao Yao, Aston Zhang, Xiangru Tang, Xinbei Ma, Zhiwei He, Yiming Wang, Mark Gerstein, Rui Wang, Gongshen Liu, and Hai Zhao. 2023.  Igniting language intelligence: The hitchhiker’s guide from chain-of-thought reasoning to language agents.  _CoRR_ , abs/2311.11797.

## Appendix

##  Appendix A Dataset Statistics

The detailed dataset statistics can be found in Table 9

Dataset | GSM8K | CSQA
---|---|---
#Instance | 1319 | 1221
Average input length | 47 | 28
(Words)
Answer Format | int | str
Question Format | grade school | single-choice
math problems | question
Few-shot | 8 | 7

Table 9: Dataset statistics

##  Appendix B Hyperparameters

The detailed hyperparameter settings can be found in Table 10

Parameters | Value
---|---
batch size (Bloom 7B) | 32
batch size (Llama2 13B) | 24
batch size (Llama2 70B) | 10
top_p | 0.9
temperature | 0.8
max_seq_len | 1024

Table 10: Hyperparameters

##  Appendix C Speculative Decoding VS GKT

We provide a more comprehensive analysis of the deployment of speculative
decoding and GKT within a cloud-edge framework in this section. For the Llama
model, with a vocabulary size of 32K, the number of bits ($b$) required to
store each token can be calculated using the formula:

$b=\lceil\log_{2}(N)\rceil$

where $\lceil x\rceil$ denotes the ceiling function. Therefore, the number of
bits needed to store each token is 15 bits.

In the scenario of GSM8K (40->300), for the GKT framework, to respond to a
user request, it only needs to transmit 40 tokens of guidance prompts from the
cloud LLM. However, for speculative decoding, the model needs to transmit 300
* 2 = 600 tokens (the small model transmits the draft token to the cloud LLM,
and the LLM sends back the validated tokens to the small model).

In a network environment with limited bandwidth, say 5K per second,
transmitting a 15-bit token takes about 0.003 seconds. Therefore, for GKT,
transmitting 40 tokens would only require 0.12 seconds. However, for
speculative decoding, it would take 1.8 seconds. A significant advantage of
the GKT framework over speculative decoding lies in its ability to control
transmission losses by adjusting the length of the prompt. In contrast, the
speculative decoding framework needs to transmit all tokens output by the LLM.
Although speculative decoding can preserve the performance of LLM, it cannot
be efficiently deployed in cloud-edge scenarios. On the other hand, GKT offers
flexible applicability in these settings.

##  Appendix D Example for GKT framework

An concrete example for GKT framework can be found in Figure 6, where we show
the input and output of the cloud teacher LLM and the edge student LLM.

Figure 6: Example for GKT framework

##  Appendix E Prompt used for GSM8K and CSQA

The prompt used for GSM8K and CSQA can be found in Figure 7 and Figure 8
respectively.

Figure 7: The prompt used for GSM8K Figure 8: The prompt used for CSQA


Large Tate–Shafarevich orders from good $abc$ triples
David Broadhurst, Open University, UK

<EMAIL_ADDRESS>
14 November 2021

Record values are determined for the order $|\Sha|$ of the Tate–Shafarevich
group of an elliptic curve $E$, computed analytically by the Birch–Swinnerton-
Dyer conjecture, and for the Goldfeld–Szpiro ratio $G=|\Sha|/\sqrt{N}$, where
$N$ is the conductor of $E$. The curves have rank zero and are isogenous to
quadratic twists of Frey curves constructed from coprime positive integers
$(a,b,c)$ with $a+b=c$ and $c>r^{1.4}$, where the radical $r$ is the product
of the primes dividing $abc$. Curves with $|\Sha|>250000^{2}$ and $G>12$ are
found in 20 isogeny classes. Three curves have $G>150$. The largest value of
$|\Sha|$ is $1937832^{2}>3.755\times 10^{12}$. This is more than 3.5 times the
previous record, which had been computed at a cost about 600 times greater
than that for the new record. The primes 25913, 27457, 36929 and 49253 are
identified as divisors of $|\Sha|$ values.

##  1 Definitions and results

Let $a$ and $b$ be coprime integers with $b>a>0$. Let $c=a+b$ and
$\lambda=\log(c)/\log(r)$, where the radical $r$ is the product of the primes
$p|abc$. There are 241 known $abc$ triples [8] with $\lambda>1.4$. Given such
a triple and a square-free integer $q$, one may construct an isogeny class of
4 elliptic curves over the rationals [9], as follows:

$\displaystyle E_{1}$ $\displaystyle:$ $\displaystyle
y^{2}=x\left(x^{2}-2q(b+c)x+q^{2}a^{2}\right)$ (1) $\displaystyle E_{2}$
$\displaystyle:$ $\displaystyle y^{2}=x\left(x^{2}+2q(a+c)x+q^{2}b^{2}\right)$
(2) $\displaystyle E_{3}$ $\displaystyle:$ $\displaystyle
y^{2}=x\left(x^{2}-2q(a-b)x+q^{2}c^{2}\right)$ (3) $\displaystyle E_{4}$
$\displaystyle:$ $\displaystyle y^{2}=x(x+qb)(x+qc)$ (4)

where $E_{4}$ is a quadratic twist of a Frey curve. If these curves have rank
zero, then the conjecture of Birch and Swinnerton-Dyer asserts that the
central value of the $L$-function associated with this isogeny class is given
by

$L=L(1)=\frac{\pi C_{k}|\Sha(E_{k})|}{\sqrt{|q|c}\,{\rm
AGM}(1,\alpha)},\quad\alpha=\left\\{\begin{array}[]{l}\sqrt{b/c}~{}{\rm
for}~{}q>0\\\ \sqrt{a/c}~{}{\rm for}~{}q<0\end{array}\right.$ (5)

where $C_{k}$ is a positive integer, determined by Tamagawa factors at primes
dividing the conductor $N$, and $|\Sha(E_{k})|$ is the order of the Tate-
Shafarevich group  of $E_{k}(c,a,q)$ and is assumed to be the square of an
integer. The integers $C_{k}$ differ only by powers of 4. Tables 1 to 4 give
the smallest value of $k$ for which $C_{k}$ is minimal and the corresponding
Goldfeld-Szpiro ratio [1, 2, 4] $G=|\Sha|/\sqrt{N}$ is hence maximal.

Table 1: Curves with $|\Sha|>250000^{2}$ and $G>12$.

$\displaystyle\begin{array}[]{|c|c|c|c|c|c|c|}\hline\cr|\Sha|&c&a&q&k&L&G\\\
\hline\cr
1937832^{2}&3^{19}11^{4}463^{5}&5^{4}19^{13}103&285&3&21.1540&153.084\\\
804572^{2}&3^{22}7^{14}43\cdot 83&2^{5}67^{8}107\cdot
22381&7&2&5.64497&163.119\\\
793656^{2}&2^{9}5^{16}11^{9}79&29^{4}2213^{2}&23&1&17.3059&162.256\\\
589080^{2}&3^{19}11^{4}463^{5}&5^{4}19^{13}103&-95&1&6.98206&24.5023\\\
574656^{2}&3^{13}5^{8}11^{3}53\cdot
73^{2}89^{2}103&7^{5}61&39&1&4.61571&24.4672\\\
514672^{2}&11^{8}109^{2}3677^{3}&2\cdot 5^{10}13^{4}&429&3&7.22644&24.4535\\\
487408^{2}&2\cdot 3^{15}7^{2}31^{10}&5\cdot
67^{3}127^{2}19219&62&3&2.82526&20.0273\\\
480512^{2}&2^{37}89^{3}167^{2}1823&3^{22}9787^{2}&29&1&4.85512&28.8220\\\
479144^{2}&5^{9}139^{6}&2^{2}11&104945&1&37.9640&96.2939\\\
439312^{2}&5^{9}139^{6}&2^{2}11&114998&1&30.4875&54.6806\\\
421216^{2}&5^{11}7^{10}79\cdot 389^{2}&11\cdot
103^{8}&778&1&0.78741&12.7045\\\ 394024^{2}&2\cdot 3^{15}7^{2}31^{10}&5\cdot
67^{3}127^{2}19219&26&3&2.85120&20.2112\\\
393216^{2}&3^{38}13^{4}5233&71^{8}233^{3}&5&1&1.46770&34.8223\\\
338122^{2}&3^{38}397&13^{5}19^{3}&5161&1&20.7254&16.9191\\\
324440^{2}&5^{15}179^{4}2141&2^{12}13^{3}223^{3}&30&3&11.1895&39.8368\\\
321584^{2}&5^{9}139^{6}&2^{2}11&5282&3&38.1136&31.3650\\\ 298500^{2}&2\cdot
11^{6}193^{4}20551&3\cdot 5^{6}7^{8}53&5010&3&25.1811&34.9472\\\
295432^{2}&3^{13}5^{8}11^{3}53\cdot
73^{2}89^{2}103&7^{5}61&1&1&11.4278&14.2781\\\
288182^{2}&2^{4}3^{19}17^{8}29&5^{2}23^{10}106531&108273&2&13.3365&15.3808\\\
277548^{2}&3^{30}13^{4}277&5^{11}31\cdot 191&4966&3&5.44574&14.7707\\\
\hline\cr\end{array}$

Table 2: More curves with $|\Sha|>10^{10}$ and $G>7$.

$\displaystyle\begin{array}[]{|c|c|c|c|c|c|c|}\hline\cr|\Sha|&c&a&q&k&L&G\\\
\hline\cr 245520^{2}&2\cdot 11^{6}193^{4}20551&3\cdot
5^{6}7^{8}53&10615&2&5.85180&16.2426\\\
243952^{2}&11^{8}109^{2}3677^{3}&2\cdot 5^{10}13^{4}&186&1&4.93145&8.34373\\\
240848^{2}&5^{9}139^{6}&2^{2}11&807&3&27.3470&23.9241\\\
217564^{2}&2^{15}17^{2}331\cdot
1061^{4}&3^{3}241^{3}&10065&1&12.0412&20.1124\\\
211784^{2}&5^{9}139^{6}&2^{2}11&272857&1&9.19960&11.6672\\\
205720^{2}&5^{15}17\cdot
53093^{2}&2^{11}3^{4}101^{4}29221&5151&2&6.20109&18.2946\\\
202760^{2}&2^{26}5^{12}1873&13^{10}37^{2}&49913&1&6.67897&22.8167\\\
197256^{2}&2\cdot 3^{3}5^{23}953&7^{2}41^{2}311^{3}&63755&1&5.00374&7.17519\\\
185028^{2}&5^{9}11^{8}2489197589&3^{4}7^{2}41&33&1&6.16228&25.7619\\\
183200^{2}&5^{9}139^{6}&2^{2}11&92883&3&14.7483&7.99828\\\
160368^{2}&3^{30}13^{4}277&5^{11}31\cdot 191&4011&1&4.04598&7.75986\\\
127542^{2}&11^{9}29^{4}101^{3}&2^{2}3^{4}163^{3}1006151&163&1&6.18013&12.2005\\\
116224^{2}&11^{12}389^{2}6841&2^{6}5^{2}7^{13}13^{2}463&-1945&2&5.75633&7.67883\\\
114704^{2}&2\cdot 3^{3}5^{23}953&7^{2}41^{2}311^{3}&-1555&2&9.52652&15.5354\\\
112704^{2}&2\cdot 3^{3}5^{23}953&7^{2}41^{2}311^{3}&4043&1&0.81083&9.30159\\\
110268^{2}&2\cdot 3^{3}5^{23}953&7^{2}41^{2}311^{3}&3157&1&5.27005&10.0761\\\
109976^{2}&2\cdot 11^{6}193^{4}20551&3\cdot
5^{6}7^{8}53&1837&2&8.46717&7.83401\\\ 106064^{2}&2^{30}5^{2}127\cdot
353^{2}&1&10590&1&8.43066&14.0020\\\ 103312^{2}&5^{15}17\cdot
53093^{2}&2^{11}3^{4}101^{4}29221&-39&2&7.68388&18.7474\\\
103296^{2}&2^{26}5^{12}1873&13^{10}37^{2}&9365&1&10.0047&13.6712\\\
\hline\cr\end{array}$

Table 3: More curves with $|\Sha|>10^{8}$ and $G>12$.

$\displaystyle\begin{array}[]{|c|c|c|c|c|c|c|}\hline\cr|\Sha|&c&a&q&k&L&G\\\
\hline\cr 87920^{2}&3^{9}5^{6}13^{5}23\cdot
191&17^{4}&4485&1&16.3832&13.7675\\\
83168^{2}&5^{9}139^{6}&2^{2}11&1390&1&4.96930&17.8253\\\
70400^{2}&5^{9}139^{6}&2^{2}11&302&1&7.63893&27.4015\\\
63036^{2}&3^{38}397&13^{5}19^{3}&1&1&12.9372&42.2448\\\
56932^{2}&2^{28}3^{12}11^{3}67&7^{11}19&70&1&10.9192&20.9225\\\
40800^{2}&19^{3}139^{6}&3\cdot 13^{2}1049&278&3&45.1256&30.4809\\\
40800^{2}&2^{11}73^{7}83^{2}197&3^{5}5^{15}13^{5}&1&1&3.34266&17.4567\\\
34168^{2}&19^{11}59\cdot 7207&2^{46}23&6&3&3.40399&13.1792\\\
33600^{2}&3^{16}103^{3}127&73&2823&1&6.90932&38.6702\\\
32272^{2}&19^{3}139^{6}&3\cdot 13^{2}1049&1807&3&5.53691&29.9201\\\
26608^{2}&5^{15}17\cdot
53093^{2}&2^{11}3^{4}101^{4}29221&-1&2&6.36599&21.9655\\\
24304^{2}&3^{16}103^{3}127&73&7519&1&4.43014&12.3974\\\
24064^{2}&2^{26}5^{12}1873&13^{10}37^{2}&2&1&14.8618&35.9005\\\
22816^{2}&2^{30}5^{2}127\cdot 353^{2}&1&6&1&8.19496&27.2210\\\
22076^{2}&13^{2}251^{6}&2^{19}367^{3}&13&3&8.26382&58.6943\\\
17264^{2}&3^{27}13^{4}&7^{3}29^{5}151^{2}&755&3&1.16852&14.8598\\\
15888^{2}&5^{9}139^{6}&2^{2}11&5&1&15.1187&15.3391\\\ 15384^{2}&3^{22}13\cdot
47^{2}263&2^{7}23^{8}&437&2&14.0272&18.8405\\\ 12664^{2}&19^{3}139^{6}&3\cdot
13^{2}1049&13&3&5.02615&19.2050\\\ 12372^{2}&7\cdot
131^{7}1373&3^{4}23^{6}1013^{2}&5&1&10.9177&16.7405\\\ \hline\cr\end{array}$

Table 4: Curves with $a=44$ and $G>4$.

$\displaystyle\begin{array}[]{|c|c|c|c|c|c|c|}\hline\cr|\Sha|&c&a&q&k&L&G\\\
\hline\cr 479144^{2}&5^{9}139^{6}&44&104945&1&37.9640&96.2939\\\
439312^{2}&5^{9}139^{6}&44&114998&1&30.4875&54.6806\\\
321584^{2}&5^{9}139^{6}&44&5282&3&38.1136&31.3650\\\
240848^{2}&5^{9}139^{6}&44&807&3&27.3470&23.9241\\\
211784^{2}&5^{9}139^{6}&44&272857&1&9.19960&11.6672\\\
183200^{2}&5^{9}139^{6}&44&92883&3&14.7483&7.99828\\\
176832^{2}&5^{9}139^{6}&44&198770&1&15.0289&6.73874\\\
144752^{2}&5^{9}139^{6}&44&243265&1&6.82733&5.77241\\\
117552^{2}&5^{9}139^{6}&44&129965&1&4.10674&5.20828\\\
117168^{2}&5^{9}139^{6}&44&5838&3&4.81256&6.52483\\\
83536^{2}&5^{9}139^{6}&44&4865&1&2.67976&5.13812\\\
83168^{2}&5^{9}139^{6}&44&1390&1&4.96930&17.8253\\\
78048^{2}&5^{9}139^{6}&44&2310&3&6.78950&4.60258\\\
70400^{2}&5^{9}139^{6}&44&302&1&7.63893&27.4015\\\
55336^{2}&5^{9}139^{6}&44&3322&1&14.2300&5.10444\\\
51872^{2}&5^{9}139^{6}&44&3315&3&2.50348&8.98022\\\
49388^{2}&5^{9}139^{6}&44&4983&3&1.85105&6.63988\\\
38920^{2}&5^{9}139^{6}&44&170&3&6.22360&11.1623\\\
24428^{2}&5^{9}139^{6}&44&663&3&1.24148&4.45330\\\
15888^{2}&5^{9}139^{6}&44&5&1&15.1187&15.3391\\\ \hline\cr\end{array}$

##  2 Examples and strategy

Example 1: Consider the curve with $|\Sha|=479144^{2}$ in the first entry of
Table 4. The large value $G>96$ results from three circumstances. First, the
factorization of

$b=c-a=5^{9}\times 139^{6}-44=3^{2}\times 13^{10}\times 17\times 151\times
4423$ (6)

is reasonably smooth, giving the radical of $abc$ as

$r=2\times 3\times 5\times 11\times 13\times 17\times 139\times 151\times
4423=6770408926710,$ (7)

which is significantly smaller than $c$, with
$\lambda=\log(c)/\log(r)=1.49243$. Secondly, $q=5\times 139\times 151=104945$
is a divisor of $r$. In general, the conductor is given by

$N=\frac{2^{s-1}q^{2}r}{{\rm gcd}(q,r)}$ (8)

with an integer $s\in[0,5]$. In this example, $N=8qr$ and $q$ neatly
disappears from

$\frac{G}{L}\approx\frac{r^{(\lambda-1)/2}}{2^{15/2}\pi}=2.53645,$ (9)

with an arithmetic-geometric mean in (5) differing from unity by less than
$10^{-18}$ and the Tamagawa factors giving $C_{1}=64$. Finally, the large
value $L=37.9640$ gives $G=96.2939$, exceeding the previous record in [5],
with $|\Sha|=1832^{2}$ and $G=42.2653$.

Example 2: Similar remarks apply to the second entry of Table 4, with
$|\Sha|=439312^{2}$ and $G>54$. Here $C_{1}=64$ and $q=2\times 13\times
4423=114998$ divides $r$ in (7). The conductor in (8) is $N=16qr$, giving
$G/L\approx r^{(\lambda-1)/2}/(2^{8}\pi)=1.79354.$ Then $L=30.4875$ gives
$G=54.6806$.

The computation of central $L$-values proceeds by truncation of the infinite
sum in [3]

$L=L(1)=2\sum_{n>0}\frac{a(n)}{n}\exp(-2\pi n/\sqrt{N})$ (10)

where $a(n)=\prod_{i}a(p_{i}^{k_{i}})$ is determined by the factorization of
$n=\prod_{i}p_{i}^{k_{i}}$ in powers of primes and $a(1)=1$. For powers $k>1$
of primes that do not divide $N$, we have
$a(p^{k})=a(p^{k-1})a(p)-a(p^{k-2})p$. If $p|N$, then $a(p^{k})=a(p)^{k}$. If
$p^{2}|N$, then $a(p)=0$.

Suppose that we truncate (10) at $n=m$. Then $L$ is in error by a term

$\delta_{m}=2\sum_{n>m}\frac{a(n)}{n}\exp(-2\pi n/\sqrt{N}).$ (11)

For $2\pi m\gg\sqrt{N}\gg 1$, we expect that $\exp(2\pi
m/\sqrt{N})\delta_{m}=O(1)$, since $a(n)=O(n^{1/2})$ varies in sign for
$n>m\gg 1$. We can tolerate such an error because $|\Sha|$ is assumed to be
the square of an integer. Moreover, the maximal value of $|\Sha|$ in an
isogeny class is divisible by $4^{t}$ with an integer $t\in[0,5]$ determined
by the ratio of the largest and smallest values of the coefficients $C_{k}$ in
(5). If we assume that $\exp(2\pi m/\sqrt{N})|\delta_{m}|<K$, then we may
truncate at

$m\approx\frac{|\Sha|}{2\pi G}\log\left(\frac{K\sqrt{|\Sha|}}{2^{t}L}\right)$
(12)

where $|\Sha|=G\sqrt{N}$ is the maximal order of the Tate-Shafarevich group in
the isogeny class. In [3] it is stated (without proof) that we may set $K=4$,
for sufficiently large $N$. This value was adopted by Abderrahmane Nitaj in
[5] and by Andrzej Dabrowski, in collaboration with Mariusz Wodzicki in [2]
and with Lucjan Szymaszkiewicz in [1].

The main burden in computing $|\Sha|$ comes from obtaining the coefficients
$a(p)$ in (10) for primes $p<m$, with $m$ estimated by setting $K=4$ in (12).
For this, I used the ellap command of Pari-GP [6], which implements the baby-
step/giant-step method of Shanks and Mestre and runs in time roughly
proportional to $p^{1/4}$, implying that the total burden is roughly
proportional to $m^{5/4}$. For each curve in the tables, I used a single
thread of Pari-GP, running at 2.6 GHz on a modest laptop, with 3.2 GB of
memory allocated to this process. This allocation is sufficient to allow
storage of (and rapid access to) values of $a(n)$ for $n<10^{8}$. If $n$ is
divisible by a prime power $p^{k}>10^{8}$, more work is required. For example,
with $m=3\times 10^{10}$ and prime $p>10^{8}$ one might need to recompute
$a(p)$ almost 300 times. However, there is a saving that comes from the set
${\cal S}$ of primes $\ell$ such that $\ell^{2}|N$. If $n$ is divisible by any
prime $\ell\in{\cal S}$, then $a(n)=0$. Thus, for $|\Sha|>10^{8}$, I quantify
the burden by the integer nearest to

$B=\left(\frac{|\Sha|}{(2\pi
G)10^{8}}\log\left(\frac{4\sqrt{|\Sha|}}{2^{t}L}\right)\right)^{5/4}\prod_{\ell^{2}|N}\left(1-\frac{1}{\ell}\right),$
(13)

where (12) has been divided by $10^{8}$, to make $B$ a convenient size.

For each of the 20 curves in Table 1, with $|\Sha|>250000^{2}$ and $G>12$,
truncations of (10) were monitored as $n$ increased in steps of $10^{7}$ and
the process was stopped when the approximate values of $\sqrt{|\Sha|}/2^{t}$
for 20 successive steps differed from the same integer by less than 1/20. The
wall-clock time, in hours, was multiplied by 2.6 to give the actual cost $T$,
in GHz-hours, rounded to the nearest integer. I found that (13) is a fair
estimate of the upper limit for $T$, with $T/B=0.8\pm 0.2$. Examples 1 and 2,
with $B=31$ and $B=61$, took $T=28$ and $T=47$ GHz-hours, respectively.

Example 3: The first entry of Table 1 gives the curve with the record value

$|\Sha|=2^{6}\times 3^{2}\times 13^{2}\times 6211^{2}=1937832^{2}>3.755\times
10^{12}$ (14)

achieved with $G=153.084$. The factorization of

$b=c-a=3^{19}\times 11^{4}\times 463^{5}-5^{4}\times 19^{13}\times
103=2^{13}\times 13^{9}\times 29\times 2441\times 7673^{2}$ (15)

gives the radical of $abc$ as

$r=2\times 3\times 5\times 11\times 13\times 19\times 29\times 103\times
463\times 2441\times 7673=2111349279492568830$ (16)

for a triple with the good value $\lambda=\log(c)/\log(r)=1.44935$, found by
Frank Rubin in 2010 [8]. Its merit, as defined in [8], is
$(\lambda-1)^{2}\log(r)\log(\log(r))=31.8833$, which is exceeded by only 14
known triples. A refinement of the $abc$ conjecture in [7] suggests an upper
limit of 48 for the merit of triples. The current record merit is less than
39. The quadratic twist $q=3\times 5\times 19=285$ divides $r$, giving a
conductor $N=qr$. The Tamagawa factors give $C_{3}=576$ as the minimal integer
in (5), where ${\rm AGM}(1,\sqrt{b/c})\approx 0.999998$ differs little from
unity. Thus we have $G/L\approx r^{(\lambda-1)/2}/(576\pi)>7.2366$ and hence
$G>153$, thanks to a reasonably large central value $L=21.154$. It is this
large value of $G$ that makes the burden (13) tolerable. With $B=958$ and
$T/B\approx 0.8$, I estimated that it would take about 12 days at 2.6 GHz to
determine $|\Sha|$ running a single Pari-GP process, unthreaded. In the event,
the elapsed time was 11.4 days, using the stopping criterion described above.

Example 4: The second entry of Table 1 gives a curve with $G>163$, which is
the current record, achieved using a triple with $r=2249854771815161490$,
$\lambda=1.41022$ and merit 26.6224, found by Frank Rubin in 2019 [8]. With
$q=7$, $N=qr$ and $C_{2}=64$, this gives
$G/L=0.999898\,r^{(\lambda-1)/2}/(64\pi)=28.8963$ and hence $G=163.119$, with
a rather modest central value $L=5.64497$. The burden $B=189$ suggested a run
time of about 2 days. In the event, it took 51 hours to determine that
$|\Sha|=804572^{2}$.

Example 5: The third entry of Table 1 gives a curve with $G>162$, from a
triple with $r=655240793892471930$, $\lambda=1.41235$ and merit 25.9073, found
by Tim Dokchitser in 2003 [8]. With $q=23$, $N=qr$ and $C_{1}=160$, this gives
$G/L\approx r^{(\lambda-1)/2}/(160\pi)=9.37574$ and hence $G=162.256$, with
$L=17.3059$. The burden is $B=167$. In the event, it took 39 hours to
determine that $|\Sha|=793656^{2}$.

My strategy for selecting the 20 curves of Table 1 was based on experience
gained from the curves in Tables 2 and 3, where the burdens are smaller. For a
given triple $(a,b,c)$ and quadratic twist $q$, it takes little time to
compute the arithmetic-geometric mean and Tamagawa factors that determine
$G/L$. The burden $B$ of determining an analytic value of $|\Sha|$ decreases
rather rapidly with increasing $G$, thanks to the factor of $1/G^{5/4}$ in
(13). The benefit of a large central value is much weaker, since $L$ appears
only in the denominator of the logarithm of (13). Hence I scanned the 241 good
triples in [8], with square-free twists of magnitudes $|q|<10^{6}$, selecting
promising values of $G/L$ that might give burdens $B<10^{3}$ for $L>1$. Only
in such cases did I make a very rough estimate of $L$, by truncating the sum
in (10) at $n=10^{7}$, which is far too small to give a reliable value of $L$.
If the very tentative estimate of $G$ seemed promising, I went up to
$n=10^{8}$, which takes about 6 minutes.

Thereafter it was usually clear whether the curve was promising. However,
there were some expensive disappointments, where the estimate of $L$ decreased
radically when going up to $n=10^{9}$, or beyond, as well as happier cases,
where it increased significantly.

In the 5 examples given so far, $q$ was a positive divisor of the radical $r$.
If $q$ is negative, the AGM in (5) dilutes $G/L$, since $a<b$. If $q$ is
divisible by an odd prime $p$ coprime to $r$, then $G/L$ is diluted by a
factor $p^{-1/2}$. The following examples show that neither of these dilutions
is an insuperable obstacle to achieving $G>20$.

Example 6: The fourth curve in Table 1, with $|\Sha|=589080^{2}$, is based on
the triple of the record-breaking Example 3, but with a negative twist,
$q=-95$, which dilutes $G/L=0.215528\,r^{(\lambda-1)/2}/(256\pi)=3.50932$.
However, it achieves $G=24.5023$, thanks to $L=6.98206$.

Example 7: The fourth curve in Table 4, with $|\Sha|=240848^{2}$, is based on
the triple of Examples 1 and 2, but with a twist $q=3\times 269=807$, where
269 is coprime to $r$ and hence dilutes $G/L\approx
r^{(\lambda-1)/2}/(32\sqrt{269}\pi)=0.874835$. However, it achieves
$G=23.9241$, thanks to $L=27.3470$.

##  3 Comparison with previous work

In [9], de Weger set the records $|\Sha|=224^{2}$ and $G=6.98260$, using an
untwisted triple from the relation $7^{3}+5^{13}\times 181=2^{4}\times 3\times
11\times 13^{2}\times 19^{5}$.

In [5], Nitaj improved those records to $|\Sha|=1832^{2}$ and $G=42.2653$,
twisting the relation $5^{14}\times 19+2^{5}\times 3\times 7^{13}=11^{7}\times
37^{2}\times 353$ by $q=11$.

Thereafter much work was done to find good $abc$ triples [8], yet no
improvement for $G$ was reported. Rather, Dabrowski and colleagues [1, 2]
determined large values of $|\Sha|$ using an Ansatz that produces rather low
values of $G$ and hence entails burdens that are much higher than in the
present work. In [2], Dabrowski and Wodzicki found 85 isogeny classes with
$|\Sha|>10^{6}$, only two of which have $G>1$, while none has $G>2$. The
largest value $|\Sha|=63408^{2}$ has $G=0.911597$.

More recently, Dabrowski and Szymaszkiewicz [1] listed 40 isogeny classes with
$|\Sha|>50000^{2}$, only three of which have $G>1$, while none has $G>2$. The
following example illustrates how burdensome was the Ansatz that they adopted.

Example 8: Let $a=29$, $b=3^{46}$ and $c=a+b$. The factorization $c=2\times
17216879\times 257390962660901$ shows that the radical of $abc$ is greater
than $c$, with $r/c=3\times 29=87$. With twist $q=3$, we obtain a conductor
$N=48r=4176c$ for the isogeny class (1,2,3,4). The curves $E_{1}$ and $E_{2}$
give the minimal factors $C_{1}=C_{2}=2$ and hence the maximal value of
$|\Sha|$. The AGM in (5) differs from unity by less than $10^{-21}$ and we
obtain $G/L\approx 1/(8\sqrt{87}\pi)$, at very high precision. This is about
2200 times smaller than the value of $G/L$ for the new record in Example 3.

In [1] it was reported that Example 8 has $|\Sha|=1029212^{2}$, with
$L=40.8169$ giving the poor value $G=0.174117$. With $t=2$ in (13), from the
maximal factor $C_{3}=32=4^{2}C_{1}$, I obtain $B=578970$ as the notional
burden. This is more than 600 times the burden $B=958$ for the new record in
Example 3.

It took less than 12 days, on a single thread running at 2.6 GHz with 3.2 GB
of memory, to increase the record for $|\Sha|$ by a factor of more than 3.5.
It might have taken 20 years, with this modest resource, to check the previous
record in [1], which was obtained by distributing the task to a large number
of processors.

##  4 Statistics and prospects

The strategy of Section 2 yielded 231 isogeny classes with $|\Sha|>10^{8}$ and
$G>1$, from twists of 78 good triples. The scan of curves with $N<10^{18}$,
for twists with $|q|<10^{6}$, was reasonably thorough, yielding 173 classes.
Yet it is inevitable that some cases were missed, since the light initial
screening by truncating (10) at $n=10^{7}$ can be misleading. For
$N\in[10^{18},10^{19}]$, I was more selective, cherry-picking twists that give
good values of $G/L$, before spending the 6 minutes that were needed to
truncate at $n=10^{8}$, for a better estimate of $L$. This resulted in 31 new
classes. The remaining 27 classes, with $N>10^{19}$, were found by picking the
very ripest cherries. In particular, the 3 conspicuous cases with
$|\Sha|>6\times 10^{11}$ and $G>150$ emerged from candidates that were
identified at an early stage of this work and then left to run their courses,
while investigating less promising cases on a second thread of the dual
processor.

In conclusion:

  1. 1.

Table 1 gives 20 classes with $\sqrt{|\Sha|}>250000$, $G>12$ and burden
$B<1000$.

  2. 2.

Table 2 gives 20 of the 29 classes with $\sqrt{|\Sha|}\in[100000,250000]$.

  3. 3.

Table 3 gives 20 of the 182 classes with $\sqrt{|\Sha|}\in[10000,100000]$.

  4. 4.

Table 4 gives 20 of the 38 classes with $c=5^{9}\times 139^{6}$ and $a=44$.

  5. 5.

The triple with $c=2^{26}\times 5^{12}\times 1873$, $a=13^{10}\times 37^{2}$
and merit 26.1686, found by Tim Dokchitser in 2003 [8], is also rather
fruitful. Its twists yielded 16 classes.

  6. 6.

The previous record prime $p$ dividing $|\Sha|$, found in [1], was $p=19861$.
The list is hereby extended to include the primes 25913, 27457, 36929 and
49253.

  7. 7.

There are prospects of improving the record $|\Sha|=1937832^{2}\approx
3.755\times 10^{12}$, by exploiting triples of high merit. By way of example,
I was able to identify a class with $|\Sha|=(4.99\pm 0.01)\times 10^{12}$ and
$G>50$.

Acknowledgements: I thank Bart de Smit, for collating $abc$ triples of high
merit, notably from Tim Dokchitser and Frank Rubin. Without the notable ABC
triple of Bill Allombert, Karim Belabas and Henri Cohen, who provide the
excellent free-ware [6] of Pari-GP, none of this work would have been feasible
for me. I am grateful to Kevin Acres and Michael St Clair Oakes for advice and
encouragement.

## References

  * [1] Andrzej Dabrowski and Lucjan Szymaszkiewicz, Elliptic curves with exceptionally large analytic order of the Tate-Shafarevich groups, https://arxiv.org/abs/2103.11001 .
  * [2] Andrzej Dabrowski and Mariusz Wodzicki, Elliptic curves with large analytic order of $\Sha(E)$, in: Algebra, Arithmetic and Geometry (ed: Tschinkel and Zarhin), Progress in Mathematics, vol 269 (2009), Birkhäuser, Boston, https://www.math.nyu.edu/~tschinke/.manin/final/dabrowski/dabrowski.pdf .
  * [3] Josef Gebel and Horst G. Zimmer, Computing the Mordell–Weil group of an elliptic curve over Q, in: Elliptic Curves and Related Topics, CRM Proc. Lecture Notes 4, Amer. Math. Soc. (1994) 61–83, https://www.ams.org/books/crmp/004 .
  * [4] Dorian Goldfeld and Lucian Szpiro, Bounds for the order of the Tate-Shafarevich group, Compositio Math. 97 (1995) 71-87, http://www.numdam.org/item/CM_1995__97_1-2_71_0.pdf .
  * [5] Abderrahmane Nitaj, Invariants des courbes de Frey-Hellegouarch et grands groupes de Tate-Shafarevich, Acta Arithmetica 93 (2000) 303-327, https://eudml.org/doc/207416 .
  * [6] Pari-GP version 2.13.2, Univ. Bordeaux, June 2021, http://pari.math.u-bordeaux.fr .
  * [7] Olivier Robert, Cameron L. Stewart and Gérald Tenenbaum, A refinement of the abc conjecture, Bull. London Math. Soc. 46 (2014) 1156-1166, https://tenenb.perso.math.cnrs.fr/PPP/abc.pdf .
  * [8] Bart de Smit, ABC triples by quality, https://pub.math.leidenuniv.nl/~smitbde/abc/?set=2 .
  * [9] Benjamin M.M. de Weger, $A+B=C$ and big ’s, Quart. J. Math. 49 (1998) 105-128, http://deweger.xs4all.nl/papers/[25]dW-ABCSha-QuJMathOxf[1998].pdf .