# Cauchy matrix approach to novel extended semi-discrete KP-type systems

Hong-juan Tian1,2,3, Abdselam Silem4111Corresponding author. Email:
<EMAIL_ADDRESS>
1 College of Computer and Information Engineering, Henan Normal University,
Xinxiang 453007, China
2School of Physics, Henan Normal University, Xinxiang 453007, PR China
3 Engineering Lab of Intelligence Business and Internet of Things, Henan
Province
4 Department of Mathematics, Zhejiang University of Technology, Hangzhou
310014, China

###### Abstract

Two novel extended semi-discrete KP-type systems, namely partial differential-
difference systems with one continuous and two discrete variables, are
investigated. Introducing an arbitrary function into the Cauchy matrix
function or the plane wave factor allows the implementation of extended
integrable systems within the Cauchy matrix approach. We introduce the
bilinear $D\Delta^{2}$KP system, the extended $D\Delta^{2}$pKP,
$D\Delta^{2}$pmKP, and $D\Delta^{2}$SKP systems, all of which are based on the
Cauchy matrix approach. This results in a diversity of solutions for these
extended systems as contrasted to the usual multiple soliton solutions.

Key Words: Squared eigenfunction, Semi-discrete KP-type system, Generalized
Cauchy matrix approach, Exact solutions.

##  1 Introduction

Over the past decades, the extensive investigations of the theory of discrete
integrable systems [1, 2, 3, 4, 5] have resulted in numerous significant
accomplishments within this domain. For instance, integrable semi-discrete
models [9, 6, 7, 10, 8, 11] are a fundamental subject in the integrable
systems field. It is worth mentioning that these equations can serve as the
superposition formula for B$\ddot{a}$cklund transformations of their
corresponding continuous equations. Moreover, they also possess a significant
mathematical value due to their underlying rich algebraic structure[12, 13].
There are numerous methods for searching for integrable discretization of
differential equations[14, 15], with one particularly significant approach
being the construction of transformations between solutions [14]. The core
concept of this discretization method is the establishment of discrete linear
dispersion relations [16]. The Cauchy matrix technique, introduced by Nijhoff
et al., was initially proposed to study the soliton solutions of the lattice
equations [17]. Motivated by these works, a generalized Cauchy matrix
approach[18, 19] was developed, and has been used to derive several kinds of
solutions for continuous and discrete integrable systems. The approach employs
the Sylvester equation, which is associated with certain evolutionary
relationships between the vectors $\mathbf{r}$ and $\mathbf{s}$. These
relationships form the determining equation set (DES), where continuous and
discrete KP equations [20] are governed entirely by the product of the plane
wave factors.

In this paper, we propose two novel semi-discrete KP-type integrable systems,
they are related to the following differential-difference system (involving
one continuous and two discrete variables)

$\partial_{x}(\widehat{\mathrm{u}}-\widetilde{\mathrm{u}})=(\widehat{\mathrm{u}}-\widetilde{\mathrm{u}})(\mathrm{u}+\widehat{\widetilde{\mathrm{u}}}-\widehat{\mathrm{u}}-\widetilde{\mathrm{u}}).$
(1.1)

Here, we consider the potential function $\mathrm{u}=\mathrm{u}(x,n:p,m:q)$
and adopt notations of forward and backward shifts as follows:

$\displaystyle\widetilde{\mathrm{u}}\doteq\mathrm{u}(x,n+1,m),\widehat{\mathrm{u}}\doteq\mathrm{u}(x,n,m+1),\widehat{\widetilde{\mathrm{u}}}\doteq\mathrm{u}(x,n+1,m+1)$
(1.2a)
$\displaystyle\underset{\widetilde{}}{\mathrm{u}}\doteq\mathrm{u}(x,n-1,m),\underset{\widehat{}}{\mathrm{u}}\doteq\mathrm{u}(x,n,m-1),\underset{\widehat{\widetilde{}}}{\mathrm{u}}\doteq\mathrm{u}(x,n-1,m-1)$
(1.2b)

Equation (1.1) was implicitly proposed in [21] and appeared earlier in the
above form as a master equation for generating the discrete Calogero–Moser
model (see Ref.[22]). Extended integrable systems can be generated by making
modifications to the plane wave factor or, alternatively, by adding an
arbitrary function to the Cauchy function matrix function [23, 24, 25].

The paper is organized as follows. In Section.2, we briefly introduce the
infinite matrix formalism of the Cauchy matrix approach. Next, in Section.3 we
demonstrate the efficiency of the Cauchy matrix approach in deriving the novel
semi-discrete KP-type equations and the two types of extended integrable
systems. Section.4 provides the explicit formula for all possible vectors
$\mathbf{r},\mathbf{s}$ and the matrix $\mathbf{M}$. These formulas yield a
variety of solutions compared to the typical $N$-soliton solutions (NSS) for
these extended KP-type systems.

##  2 Cauchy Matrix Approach

In addition to deriving soliton solutions, The Cauchy matrix [19] approach
also proved powerful in constructing new equations and their features. For the
three dimensional systems, the Cauchy-type matrix $M=(m_{i,j})_{N\times
N},m_{i,j}=\frac{r_{i}s_{j}}{k_{i}+l_{j}}$ is introduced and satisfies the
Sylvester equation

$\mathbf{KM}+\mathbf{ML}=\mathbf{rs}^{T},$ (2.1)

where $\mathbf{s}=(s_{1},\cdots,s_{N})^{T}$ and $(r_{1},\cdots,r_{N})^{T}$ are
constant vectors. Assume that
$\mathcal{E}(\mathbf{K})\bigcap\mathcal{E}(-\mathbf{L})=\varnothing$, where
$\mathcal{E}(\mathbf{L})$ and $\mathcal{E}(-\mathbf{K})$ are the eigenvalue
set of $\mathbf{K}$ and $-\mathbf{L}$, respectively. This will guarantee that
the Sylvester equation (2.1) for a given
$(\mathbf{L},\mathbf{K},\mathbf{r},\mathbf{s})$ is solvable and has a unique
solution $\mathbf{M}$. Scalar functions
$\mathbf{S}^{(i,j)}=\mathbf{s}^{T}\mathbf{L}^{j}(\mathbf{I}+\mathbf{M})^{-1}\mathbf{K}^{i}\mathbf{r}$
meet certain recurrence relations, and some relations have closed forms that
lead to integrable systems. Note that the generalized Cauchy matrix approach
is characterized by the Sylvester equation (2.1) that includes an unknown
matrix $\mathbf{M}$ and constant matrices $\mathbf{K,L}$. Motivated by the
approach’s concept, dispersion relations will be imposed on $\mathbf{r}$ and
$\mathbf{s}$ or on $\mathbf{M}$. Moreover, we will further concentrate on the
evolution of the scalar function $\mathbf{S}^{(i,j)}$. It has the potential to
not only integrate the Cauchy matrix technique in discrete and continuous
integrable systems, but also to discover further connections linking the
discrete systems with their continuous counterparts.

The key point of discretizing the Cauchy matrix approach is to introduce the
discrete linear dispersions. For instance, evolutions of discrete, semi-
discrete, and continuous KP hierarchies are fully characterized by the product
of the plane wave factors

$\rho(k)=\exp{\sum^{\infty}_{j=0}k^{j}x_{j}}\prod^{\infty}_{i=1}(p_{i}+k)^{n_{i}},\sigma(l)=\exp{(-\sum^{\infty}_{j=0}l^{j}x_{j}})\prod^{\infty}_{i=1}(p_{i}-l)^{-n_{i}},$
(2.2)

whith $k$ and $l$ being two distinct spectral parameters. Note that the
factors related to $n_{j}$ can be seen as discretisations of the exponents of
$x_{j}$[2]. To clarify, when we substitute the continuous independent
variables $x_{j}$ with discrete variables $n_{j}$ according to equation (2.2),
a series of equations of different types (continuous, semi-discrete, full-
discrete) emerge, successively. The matrix
$\mathbf{S}=(S^{(i,j)})_{\infty\times\infty}$ reads

$S^{(i,j)}=\mathbf{s}^{\mathrm{T}}\mathbf{L}^{j}(\mathbf{I}+\mathbf{M})^{-1}\mathbf{K}^{i}\mathbf{r}=\mathbf{s}^{T}\mathbf{L}^{j}\mathbf{u}^{(i)}=\mathbf{w}^{(j)}\mathbf{K}^{i}\mathbf{r},$
(2.3)

where the auxiliary $N$-th order vectors $\mathbf{u}^{(i)}$ and
$\mathbf{w}^{(j)}$ are defined as

$\mathbf{u}^{(i)}=(\mathbf{I}+\mathbf{M})^{-1}\mathbf{K}^{i}\mathbf{r},~{}~{}\mathbf{w}^{(j)}=\mathbf{s}^{T}\mathbf{L}^{j}(\mathbf{I+M})^{-1}.$
(2.4)

###  2.1  Semi-discrete KP-type equations

Based on the generalized Cauchy matrix method, the D$\Delta^{2}$pKP equation
is firstly constructed. Then modifying the plane wave factor and the Cauchy
function matrix via adding an arbitrary function, the two type extended
D$\Delta^{2}$pKP systems were obtained. Taking $\mathbf{r,s,M}$ as functions
of $(n,m,x)$, different dispersion relations can be obtained for the same
Sylvester equation (2.1).

######  Proposition 2.1.

For given Sylvester equation (2.1) and dispersion relations

$\displaystyle\widetilde{\mathbf{r}}=(p\mathbf{I}+\mathbf{K})\mathbf{r},~{}\widehat{\mathbf{r}}=(q\mathbf{I}+\mathbf{K})\mathbf{r},~{}\mathbf{r}_{x}=\mathbf{K}\mathbf{r},$
(2.5a)
$\displaystyle\mathbf{s}=(p\mathbf{I}-\mathbf{L})\widetilde{\mathbf{s}},~{}\mathbf{s}=(q\mathbf{I}-\mathbf{L})\widehat{\mathbf{s}},~{}\mathbf{s}_{x}=\mathbf{L}\mathbf{s},$
(2.5b)

we can obtain the relations :

$\displaystyle S^{(i,j)}_{x}$
$\displaystyle=S^{(i,j+1)}+S^{(i+1,j)}-S^{(0,j)}S^{(i,0)},$ (2.6a)
$\displaystyle p\widetilde{S}^{(i,j)}$
$\displaystyle=\widetilde{S}^{(i,j+1)}+pS^{(i,j)}+S^{(i+1,j)}-\widetilde{S}^{(i,0)}S^{(0,j)},$
(2.6b) $\displaystyle q\widehat{S}^{(i,j)}$
$\displaystyle=\widehat{S}^{(i,j+1)}+qS^{(i,j)}+S^{(i+1,j)}-\widehat{S}^{(i,0)}S^{(0,j)}.$
(2.6c)

###### Proof.

Based on the uniqueness of the solution and dispersion relation (2.5);
together with the Sylvester equation (2.1) w.r.t discrete variables
$n-,m-$direction; and the derivative w.r.t continuous variable $x$, one can
obtain the relations

$\widetilde{\mathbf{M}}=\mathbf{M}+\mathbf{r}\widetilde{\mathbf{s}}^{T},~{}\widehat{\mathbf{M}}=\mathbf{M}+\mathbf{r}\widehat{\mathbf{s}}^{T},~{}\mathbf{M}_{x}=\mathbf{rs}^{T}.$
(2.7)

Then, according to the definition $\mathbf{u}^{(i)},\mathbf{w}^{(i)}\eqref{ui-
wj}$ and the (2.7), we have

$\displaystyle\mathbf{u}_{x}^{(i)}=\mathbf{u}^{(i+1)}-\mathbf{u}^{(0)}S^{(i,0)},~{}\mathbf{w}_{x}^{(j)}=\mathbf{w}^{(j+1)}-\mathbf{w}^{(0)}S^{(0,j)},$
(2.8a)
$\displaystyle\widetilde{\mathbf{u}}^{(i)}=p\mathbf{u}^{(i)}+\mathbf{u}^{(i+1)}-\mathbf{u}^{(0)}\widetilde{S}^{(i,0)},~{}\mathbf{w}^{(j)}=p\widetilde{\mathbf{w}}^{(j)}-\widetilde{\mathbf{w}}^{(j+1)}+\widetilde{\mathbf{w}}^{(0)}S^{(0,j)},$
(2.8b)
$\displaystyle\widehat{\mathbf{u}}^{(i)}=q\mathbf{u}^{(i)}+\mathbf{u}^{(i+1)}-\mathbf{u}^{(0)}\widehat{S}^{(i,0)},~{}\mathbf{w}^{(j)}=q\widehat{\mathbf{w}}^{(j)}-\widehat{\mathbf{w}}^{(j+1)}+\widehat{\mathbf{w}}^{(0)}S^{(0,j)}.$
(2.8c)

Using $S^{(i,j)}=\mathbf{s}^{T}\mathbf{L}^{j}\mathbf{u}^{(i)}$, when taking
the shift in the $n,m$ directions, and the derivative w.r.t $x$, by means of
dispersion relation (2.5) and relations, we obtain (2.6). ∎

From the evolution relation of Proposition 2.1, for specific values of $i$ and
$j$, various semi-discrete KP-type equations are constructed accordingly.

  * •

D$\Delta^{2}$pKP equation

For (2.6), let $i=j=0$, $u=S^{(0,0)}$and eliminate $S^{(0,1)},S^{(1,0)}$, it
is the D$\Delta^{2}$pKP equation

$\partial_{x}(\widehat{u}-\widetilde{u})=(p-q+\widehat{u}-\widetilde{u})(u+\widehat{\widetilde{u}}-\widehat{u}-\widetilde{u}).$
(2.9)

Under transform $\mathrm{u}=u-pn-mq$, equations (2.9) and (1.1) are
equivalent.

  * •

D$\Delta^{2}$KP equation

The equation (2.9) can be deformed as
$[\ln(\widehat{\mathrm{u}}-\widetilde{\mathrm{u}})]_{x}=\mathrm{u}+\widehat{\widetilde{\mathrm{u}}}-\widetilde{\mathrm{u}}-\widehat{\mathrm{u}},$
then, let $Q=\widehat{\mathrm{u}}-\widetilde{\mathrm{u}}$, we have

$(\ln\widetilde{Q}-\ln\widehat{Q})_{x}=\widehat{\widetilde{Q}}-\widetilde{Q}-\widehat{Q}+Q,$
(2.10)

where equation (1.1) is the potential form of (2.10).

  * •

D$\Delta^{2}$pmKP equation

For the relations (2.6), taking $v=1-S^{(-1,0)},~{}w=1-S^{(0,-1)}$, the
D$\Delta^{2}$pmKP equation can be obtained as

$\frac{p\widehat{v}-q\widetilde{v}}{\widehat{\widetilde{v}}}-\frac{\widetilde{v}_{x}+pv}{\widetilde{v}}+\frac{\widehat{v}_{x}+qv}{\widehat{v}}=0,~{}or~{}\frac{\widehat{w}_{x}}{\widehat{w}}-\frac{p\widetilde{w}_{x}-q\widehat{w}_{x}}{p\widetilde{w}-q\widehat{w}}+p(\frac{\widetilde{w}}{w}-\frac{\widehat{\widetilde{w}}}{\widehat{w}})=0.$
(2.11)

  * •

D$\Delta^{2}$SKP equation

For the relations (2.6), take
$z=S^{(-1,-1)}-\frac{n}{p}-\frac{m}{q}-\frac{h}{r}$ and use the identity
relation
$\frac{(vw)^{\widehat{}}}{(vw)^{\widetilde{}}}=\frac{(\widehat{v}w)}{(\widetilde{v}w)}\frac{(\widetilde{v}w)^{\widehat{}}}{(\widehat{v}w){\widetilde{}}},$
it is the D$\Delta^{2}$SKP equation

$\frac{1-\widehat{z}_{x}}{1-\widetilde{z}_{x}}=\frac{(z-\widehat{z})(\widehat{z}-\widehat{\widetilde{z}})}{(z-\widetilde{z})(\widetilde{z}-\widetilde{\widehat{z}})}.$
(2.12)

  * •

Bilinear D$\Delta^{2}$KP system

Definitions the function $\tau=\tau_{n,m,x}=\det(\mathbf{I}+\mathbf{M})$ and
$Y(a)=1-\mathbf{s}^{T}(a+\mathbf{L})^{-1}\mathbf{u}^{(0)}$,
$V(a)=1-\mathbf{w}^{(0)}(a+\mathbf{K})^{-1}\mathbf{r}$, it can be obtained
$\frac{\widetilde{\tau}}{\tau}=Y(-p)=\frac{1}{\widetilde{V}(p)},~{}\frac{\widehat{\tau}}{\tau}=Y(-q)=\frac{1}{\widehat{V}(q)},~{}\frac{\tau_{x}}{\tau}=u.$
Using (2.6), the following closed relationship can be obtained

$(p-q)\widehat{\widetilde{\tau}}\tau+\widehat{\tau}\widetilde{\tau}_{x}-\widetilde{\tau}\widehat{\tau}_{x}=0.$
(2.13)

Under transformation $\tau\rightarrow(p-q)^{mn+m+n}\tau$, equation (2.13) is
normalized to

$\widehat{\widetilde{\tau}}\tau+\widehat{\tau}\widetilde{\tau}_{x}-\widetilde{\tau}\widehat{\tau}_{x}=0.$
(2.14)

###  2.2  First-type extended systems

######  Proposition 2.2.

For Proposition 2.1, replacing dispersion relation (2.5) with

$\mathbf{r}_{x}=\mathbf{K}\mathbf{r}+C(x)\mathbf{r},~{}\mathbf{s}_{x}=\mathbf{L}^{T}\mathbf{s}+C^{T}(x)\mathbf{s}$
(2.15)

and meet conditions
$C(x)\mathbf{L}=\mathbf{L}C(x),~{}C(x)\mathbf{K}=\mathbf{K}C(x)$, where
$C(x)=\mathbf{Diag}(c_{1}(x),\cdots,~{}c_{n}(x))$, we obtain the new relations
of $x$:

$S^{(i,j)}_{x}=S^{(i,j+1)}+S^{(i+1,j)}-S^{(0,j)}S^{(i,0)}+\mathbf{w}^{(j)}[2C(x)]\mathbf{u}^{(i)}.$
(2.16)

Simultaneously, the evolutionary relation of the direction of $n,~{}m$ is
consistent with the propositions 2.1.

###### Proof.

Similar calculation with Proposition 2.1, we can obtain the relation (2.16). ∎

  * •

First-type extended D$\Delta^{2}$pKP system

Making use of (2.6b), (2.6c), (2.16), and (2.8a)and taking $u=S^{(0,0)}$ and
$\Phi=(\phi_{1},\phi_{2},\cdots,\phi_{N})$,
$\Psi=(\psi_{1},\psi_{2},\cdots,\psi_{N})^{T}$, with
$\psi_{j}=\sqrt{2c_{j}(x)}[\mathbf{u}^{(0)}]_{j}$,
$\phi_{j}=\sqrt{2c_{j}(x)}[\mathbf{w}^{(0)}]_{j}$, we have

$\displaystyle\partial_{x}(\widehat{u}-\widetilde{u})=(p-q+\widehat{u}-\widetilde{u})(u+\widehat{\widetilde{u}}-\widehat{u}-\widetilde{u})+(E_{m}-E_{n})\Phi\Psi,$
(2.17a)
$\displaystyle(E_{n}-E_{m})\Psi=(p-q-\widetilde{u}+\widehat{u})\Psi,~{}(E_{m}-E_{n})\Phi=(p-q-\widetilde{u}+\widehat{u})\widehat{\widetilde{\Phi}},$
(2.17b)

where $E_{n},~{}E_{m}$ are the shift operators in $n,m$ direction. Under the
transformation $\mathrm{u}=u-pn-mq$, the system (2.17), just the first-type
extended D$\Delta^{2}$pKP system (ED$\Delta^{2}$pKP-1).

$\displaystyle\partial_{x}(\widehat{\mathrm{u}}-\widetilde{\mathrm{u}})=(\widehat{\mathrm{u}}-\widetilde{\mathrm{u}})(\mathrm{u}+\widehat{\widetilde{\mathrm{u}}}-\widehat{\mathrm{u}}-\widetilde{\mathrm{u}})+(E_{m}-E_{n})\Phi\Psi,$
(2.18a)
$\displaystyle(E_{n}-E_{m})\Psi=(\widehat{\mathrm{u}}-\widetilde{\mathrm{u}})\Psi,\
(E_{m}-E_{n})\Phi=(\widehat{\mathrm{u}}-\widetilde{\mathrm{u}})\widehat{\widetilde{\Phi}}.$
(2.18b)

  * •

First-type extended D$\Delta^{2}$pmKP system

From (2.6b), (2.6c), (2.16) and (2.8a), taking
$v=1-S^{(-1,0)},~{}w=1-S^{(0,-1)}$ and
$\psi_{j}=\sqrt{2c_{j}(x)}[\mathbf{u}^{(-1)}]_{j},$
$\phi_{j}=\sqrt{2c_{j}(x)}[\mathbf{w}^{(0)}]_{j}$, we have the
D$\Delta^{2}$pmKP with source

$\displaystyle\frac{p\widehat{v}-q\widetilde{v}}{\widehat{\widetilde{v}}}-\frac{\widetilde{v}_{x}+pv}{\widetilde{v}}+\frac{\widehat{v}_{x}+qv}{\widehat{v}}=\frac{\widetilde{\Phi}\widetilde{\Psi}}{\widetilde{v}}-\frac{\widehat{\Phi}\widehat{\Psi}}{\widehat{v}},$
(2.19a)
$\displaystyle(\widehat{\Phi}-\widetilde{\Phi})\widehat{\widetilde{v}}=\widetilde{\widehat{\Phi}}(p\widehat{v}-q\widetilde{v}),\
\widetilde{\Psi}\widehat{v}-\widehat{\Psi}\widetilde{v}=\Psi(p\widehat{v}-q\widetilde{v}).$
(2.19b)

Equations (2.19) can be named as the first-type extended D$\Delta^{2}$pmKP
system (ED$\Delta^{2}$pmKP-1).

  * •

First-type extended D$\Delta^{2}$SKP system

From (2.6b), (2.6c), (2.16) and (2.8a), taking
$z=S^{(-1,-1)}-\frac{n}{p}-\frac{m}{q}-\frac{h}{r}$ and
$\psi_{j}=\sqrt{2c_{j}(x)}[\mathbf{u}^{(-1)}]_{j}$,
$\phi_{j}=\sqrt{2c_{j}(x)}[\mathbf{w}^{(-1)}]_{j}$, the first-type extended
D$\Delta^{2}$SKP system (ED$\Delta^{2}$SKP-1) can be written as

$\displaystyle\frac{1-\widehat{z}_{x}+\widehat{\Phi}\widehat{\Psi}}{1-\widetilde{z}_{x}+\widetilde{\Phi}\widetilde{\Psi}}$
$\displaystyle=\frac{(z-\widehat{z})(\widehat{z}-\widehat{\widetilde{z}})}{(z-\widetilde{z})(\widetilde{z}-\widetilde{\widehat{z}})},$
(2.20a)
$\displaystyle(\widehat{\Phi}-p\widehat{\widetilde{\Phi}})p(\widehat{z}-\widehat{\widetilde{z}})$
$\displaystyle=q(\widetilde{z}-\widehat{\widetilde{z}})(\widetilde{\Phi}-q\widetilde{\widehat{\Phi}}),$
(2.20b) $\displaystyle(\widehat{\Psi}-q\Psi)p(z-\widetilde{z})$
$\displaystyle=q(z-\widehat{z})(\widetilde{\Psi}-p\Psi).$ (2.20c)

  * •

Extended bilinear D$\Delta^{2}$KP system

First, for the definition of the $\tau$ function in the case of adding any
function $C(x)$, using the definitions
$Y(a)=1-\mathbf{s}^{T}(a+\mathbf{L})^{-1}\mathbf{u}^{(0)}$,
$V(a)=1-\mathbf{w}^{(0)}(a+\mathbf{K})^{-1}\mathbf{r}$, it can be obtained

$\frac{\widetilde{\tau}}{\tau}=Y(-p)=\frac{1}{\widetilde{V}(p)},~{}\frac{\widehat{\tau}}{\tau}=Y(-q)=\frac{1}{\widehat{V}(q)},~{}\frac{\tau_{x}}{\tau}=u.$
(2.21)

Using (2.6b), (2.6c), (2.16), the closed relationship can be obtained as

$(p-q)\widehat{\widetilde{\tau}}\tau+\widehat{\tau}\widetilde{\tau}_{x}-\widetilde{\tau}\widehat{\tau}_{x}=(\widehat{\tau}\widetilde{\mathbf{s}}^{T}-\widetilde{\tau}\widehat{\mathbf{s}}^{T})(\mathbf{I}+\mathbf{M})^{-1}[C(x)-\mathbf{M}C(x)]\mathbf{u}^{(0)}\tau.$
(2.22)

Next, let $\sigma:=(\sigma_{1},\cdots,\sigma_{N}),~{}\sigma_{j}=\tau
A\psi_{j}$, i.e.$\sigma=\tau\Psi$. At the same time
$\rho^{*}:=(\rho_{1},\cdots,\rho_{N}),~{}\rho_{j}=\tau\phi_{j},~{}A=\frac{1}{2}(\mathbf{I}-\mathbf{M})$,
i.e. $\rho^{*}=\tau\Phi$, where the definition of $\phi_{j},~{}\psi_{j}$ is
the same as in (2.17b). Using (2.8a) and
$p-q-\widetilde{u}+\widehat{u}=(p-q)\frac{\widetilde{\widehat{\tau}}\tau}{\widetilde{\tau}\widehat{\tau}}$,
it can be verified separately

$\displaystyle\widehat{\tau}\widetilde{\sigma}-\widetilde{\tau}\widehat{\sigma}$
$\displaystyle=(p-q)\widetilde{\widehat{\tau}}\sigma,$ (2.23a)
$\displaystyle\widetilde{\tau}\widehat{\rho}^{*}-\widehat{\tau}\widetilde{\rho}^{*}$
$\displaystyle=(p-q)\tau\widetilde{\widehat{\rho}}^{*}.$ (2.23b)

On the basis of (2.21), auxiliary functions
$\mathbf{w}(b)=\mathbf{s}^{T}(b+\mathbf{L})^{-1}(\mathbf{I}+\mathbf{M})^{-1}$,
$V(a)=1-\mathbf{w}_{0}(a+\mathbf{K})^{-1}\mathbf{r}$ are used to obtain
$\mathbf{w}(-p)=-\frac{\widetilde{\mathbf{w}}_{0}}{\widetilde{V}(p)}$. Making
similar processing of $\mathbf{w}(-r)$, it can be simplified to
$\mathbf{w}(-r)=-\frac{\widehat{\mathbf{w}}_{0}}{\widehat{V}(r)}.$ Therefore,
the source form of (2.22) can be expressed as

$(\widehat{\tau}\widetilde{\mathbf{s}}^{T}-\widetilde{\tau}\widehat{\mathbf{s}}^{T})(\mathbf{I}+\mathbf{M})^{-1}[C(x)-\mathbf{M}C(x)]\mathbf{u}_{0}\tau_{x}=(p-q)\widehat{\widetilde{\rho}}^{*}\sigma,$

with $\rho^{*},~{}\sigma$ satisfy (2.23a) and (2.23b). Based on the above
discussion and under transformation $\tau\rightarrow(p-q)^{mn+m+n}\tau$, the
first-type extended bilinear D$\Delta^{2}$KP system can be normalized as

$\displaystyle\widehat{\widetilde{\tau}}\tau+\widehat{\tau}\widetilde{\tau}_{x}-\widetilde{\tau}\widehat{\tau}_{x}=\widehat{\widetilde{\rho}}^{*}\sigma,$
(2.24a)
$\displaystyle\widehat{\tau}\widetilde{\sigma}-\widetilde{\tau}\widehat{\sigma}=\widetilde{\widehat{\tau}}\sigma,~{}\widetilde{\tau}\widehat{\rho}^{*}-\widehat{\tau}\widetilde{\rho}^{*}=\tau\widetilde{\widehat{\rho}}^{*}.$
(2.24b)

###  2.3  Second-type extended systems

######  Proposition 2.3.

For the given Sylvester equation and dispersion relation (2.5), using the
dressed Cauchy kernel $\mathbf{M}^{+}=\mathbf{M}+C(m),$ the infinite matrix
and exchange conditions can be redefined as
$U_{\alpha,\beta}=\mathbf{s}^{T}\mathbf{L}^{\beta}(\mathbf{I}+\mathbf{M}^{+})^{-1}\mathbf{K}^{\alpha}\mathbf{r}=\mathbf{s}^{T}\mathbf{L}^{\beta}\mathbf{u}_{\alpha}=\mathbf{w}_{\beta}\mathbf{K}^{\alpha}\mathbf{r},~{}C(m)\mathbf{L}=\mathbf{L}C(m),~{}C(m)\mathbf{K}=\mathbf{K}C(m).$
Then, the evolutionary relations can be obtained as

$\displaystyle\partial_{x}U_{\alpha,\beta}=U_{\alpha,\beta+1}+U_{\alpha+1,\beta}-U_{0,\beta}U_{\alpha,0},$
(2.25a) $\displaystyle
p\widetilde{U}_{\alpha,\beta}=\widetilde{U}_{\alpha,\beta+1}+pU_{\alpha,\beta}+U_{\alpha+1,\beta}-\widetilde{U}_{\alpha,0}U_{0,\beta},$
(2.25b) $\displaystyle
q\widehat{U}_{\alpha,\beta}=\widehat{U}_{\alpha,\beta+1}+qU_{\alpha,\beta}+U_{\alpha+1,\beta}-\widehat{U}_{\alpha,0}U_{0,\beta}-\mathbf{w}_{\beta}[C(m+1)-C(m)]\widehat{\mathbf{u}}_{\alpha}.$
(2.25c)

###### Proof.

Similarly, for the new dressed Cauchy kernel
$\mathbf{M}^{+},\mathbf{u}_{\alpha},\mathbf{w}_{\beta}$, we have the relations

$\displaystyle\widetilde{\mathbf{M}}^{+}=\mathbf{M}^{+}+\mathbf{r}\widetilde{\mathbf{s}}^{T},~{}\mathbf{M}^{+}_{x}=\mathbf{rs}^{T},~{}\widehat{\mathbf{M}}^{+}=\mathbf{M}^{+}+\mathbf{r}\widehat{\mathbf{s}}^{T}+C(m+1)-C(m),$
(2.26a)
$\displaystyle\partial_{x}\mathbf{u}_{\alpha}=\mathbf{u}_{\alpha+1}-\mathbf{u}_{0}U_{\alpha,0},~{}\partial_{x}\mathbf{w}_{\beta}=\mathbf{w}_{\beta+1}-\mathbf{w}_{0}U_{0,\beta},$
(2.26b)
$\displaystyle\widetilde{\mathbf{u}}_{\alpha}=p\mathbf{u}_{\alpha}+\mathbf{u}_{\alpha+1}-\widetilde{U}_{\alpha,0}\mathbf{u}_{0},~{}\mathbf{w}_{\beta}=p\widetilde{\mathbf{w}}_{\beta}-\widetilde{\mathbf{w}}_{\beta+1}+U_{0,\beta}\widetilde{\mathbf{w}}_{0}$
(2.26c)
$\displaystyle\widehat{\mathbf{u}}_{\alpha}=q\mathbf{u}_{\alpha}+\mathbf{u}_{\alpha+1}-\widehat{U}_{\alpha,0}\mathbf{u}_{0}-(\mathbf{I}+\mathbf{M}^{+})^{-1}[C(m+1)-C(m)]\widehat{\mathbf{u}}_{\alpha}.$
(2.26d)

Using the relations (2.26) and definitions $U_{\alpha,\beta}$, it can be
calculated as (2.25). ∎

  * •

Second-type extended D$\Delta^{2}$pKP system

Taking $u=U_{0,0}$ and
$\Phi=(\phi_{1},\phi_{2},\cdots,\phi_{N}),~{}\Psi=(\psi_{1},\psi_{2},\cdots,\psi_{N})^{T}$
with $\psi_{j}=[\mathbf{u}_{0}]_{j},$
$\phi_{j}=[c_{j}(m+1)-c_{j}(m)][\mathbf{w}_{0}]_{j}$ , from (2.25), we can get

$\displaystyle\partial_{x}(\widehat{u}-\widetilde{u})=(p-q+\widehat{u}-\widetilde{u})(u+\widehat{\widetilde{u}}-\widehat{u}-\widetilde{u})+(1-E_{n})\Phi\widehat{\Psi},$
(2.27a)
$\displaystyle\widetilde{\Psi}-\Psi_{x}=(p+u-\widetilde{u})\Psi,~{}\Phi+\widetilde{\Phi}_{x}=(p+u-\widetilde{u})\widetilde{\Phi}.$
(2.27b)

Under the transformation $\mathrm{u}=u-pn-mq$, the equtaions (2.27) can be
rewrite to obtain second-type extended D$\Delta^{2}$pKP system
(ED$\Delta^{2}$pKP-2),

$\displaystyle\partial_{x}(\widehat{\mathrm{u}}-\widetilde{\mathrm{u}})=(\widehat{\mathrm{u}}-\widetilde{\mathrm{u}})(\mathrm{u}+\widehat{\widetilde{\mathrm{u}}}-\widehat{\mathrm{u}}-\widetilde{\mathrm{u}})+(1-E_{n})\Phi\widehat{\Psi},$
(2.28a)
$\displaystyle\widetilde{\Psi}-\Psi_{x}=(\mathrm{u}-\widetilde{\mathrm{u}})\Psi,~{}\Phi+\widetilde{\Phi}_{x}=(\mathrm{u}-\widetilde{\mathrm{u}})\widetilde{\Phi}.$
(2.28b)

  * •

Second-type extended D$\Delta^{2}$pmKP system

From (2.25), taking $v=1-U_{-1,0},w=1-U_{0,-1}$ and
$\psi_{j}=[\mathbf{u}_{-1}]_{j},~{}\phi_{j}=[c_{j}(m+1)-c_{j}(m)][\mathbf{w}_{0}]_{j}$,
we have

$\displaystyle
p\Bigl{(}\frac{\widehat{v}}{\widetilde{\widehat{v}}}-\frac{v}{\widetilde{v}}\Bigr{)}+q\Bigl{(}\frac{v}{\widehat{v}}-\frac{\widetilde{v}}{\widehat{\widetilde{v}}}\Bigr{)}+\Bigl{(}\frac{\widehat{v}_{x}}{\widehat{v}}-\frac{\widetilde{v}_{x}}{\widetilde{v}}\Bigr{)}=\bigl{(}\frac{1}{\widehat{\widetilde{v}}}-\frac{1}{\widehat{v}}\bigr{)}\Phi\widehat{\Psi},$
(2.29a)
$\displaystyle(\Phi-\widetilde{\Phi}_{x})\widetilde{v}=\widetilde{\Phi}(\widetilde{v}_{x}+pv),~{}\partial_{x}\Psi\widetilde{v}=(\widetilde{\Psi}-p\Psi)v.$
(2.29b)

Equations (2.29) can be named as second-type extended D$\Delta^{2}$pmKP system
(ED$\Delta^{2}$pmKP-2)

  * •

Second-type extended D$\Delta^{2}$SKP system

Taking $z=U_{(-1,-1)}-\frac{n}{p}-\frac{m}{q}-\frac{h}{r}$ and
$\psi_{j}=[\mathbf{u}_{-1}]_{j},~{}\phi_{j}=[c_{j}(m+1)-c_{j}(m)][\mathbf{w}_{-1}]_{j}$,
we have

$\displaystyle\frac{1-\widehat{z}_{x}}{1-\widetilde{z}_{x}}=\frac{(\widehat{z}-\widehat{\widetilde{z}})\\{q(z-\widehat{z})+\Phi\widehat{\Psi}\\}}{(z-\widetilde{z})\\{q(\widetilde{z}-\widetilde{\widehat{z}})+\widetilde{\Phi}\widetilde{\widehat{\Psi}}\\}},$
(2.30a)
$\displaystyle\partial_{x}\Phi=(p\widetilde{\Phi}-\Phi)\frac{1-\widetilde{z}_{\xi}}{p(z-\widetilde{z})},\partial_{x}\Psi=(\widetilde{\Psi}-p\Psi)\frac{1-z_{\xi}}{p(z-\widetilde{z})}.$
(2.30b)

Equations (2.30) are referred to as second-type extended D$\Delta^{2}$SKP
system (ED$\Delta^{2}$SKP-2).

##  3 Solutions

To get the solution of semi-discrete KP-type equations and second-type
extended semi-discrete KP-type systems, for the given matrices
$(\mathbf{L},~{}\mathbf{K})$, we need to define the wave factors for semi-
discrete cases

$\displaystyle\rho_{i}=(p+k_{i})^{n}(q+k_{i})^{m}\mathrm{e}^{\eta_{i}},~{}\eta_{i}=k_{i}x+\eta^{(0)}_{i},k_{i},~{}\eta^{(0)}_{i}\in\mathbb{C},$
(3.1a)
$\displaystyle\sigma_{i}=(p-l_{i})^{-n}(q-l_{i})^{-m}\mathrm{e}^{\zeta_{i}},~{}\zeta_{i}=l_{i}x+\zeta^{(0)}_{i},l_{i},~{}\zeta^{(0)}_{i}\in\mathbb{C}.$
(3.1b)

To get the solution of first-type extended semi-discrete KP-type systems, for
the given matrices $(\mathbf{L},~{}\mathbf{K},~{}C(x))$, we need to redefine
the wave factors for semi-discrete cases

$\displaystyle\rho_{i}=(p+k_{i})^{n}(q+k_{i})^{m}\mathrm{e}^{\eta_{i}},~{}\eta_{i}=k_{i}x+\int_{0}^{x}C(z)dz+\eta^{(0)}_{i},k_{i},~{}\eta^{(0)}_{i}\in\mathbb{C},$
(3.2a)
$\displaystyle\sigma_{i}=(p-l_{i})^{-n}(q-l_{i})^{-m}\mathrm{e}^{\zeta_{i}},~{}\zeta_{i}=l_{i}x+\int_{0}^{x}C(z)dz+\zeta^{(0)}_{i},l_{i},~{}\zeta^{(0)}_{i}\in\mathbb{C}.$
(3.2b)

Next, the solutions of Sylvester equation and dispersion relation can be,
similarly, listed as in Ref.[24].

Case 1. When $\mathbf{K}=\mathbf{D}^{[N]}(\\{k_{i}\\}^{N}_{1})$,
$\mathbf{L}=\mathbf{D}^{[N]}(\\{l_{i}\\}^{N}_{1})$,
$C(x)=\mathbf{D}^{[N]}(\\{c_{i}(x)\\}^{N}_{1}),$ or
$C(m)=\mathbf{D}^{[N]}(\\{c_{i}(m)\\}^{N}_{1}),$ the matrices
$\mathbf{r}=\mathbf{r}_{D}^{[N]}(\\{k_{i}\\}_{1}^{N})$ and
$\mathbf{s}=\mathbf{s}_{D}^{[N]}(\\{l_{i}\\}_{1}^{N})$ are composed of
$r_{i}=\rho_{i},s_{j}=\sigma_{j}$ respectively, where

$\mathbf{M}=\Bigl{(}\frac{r_{i}s_{j}}{k_{i}+l_{j}}\Bigr{)}_{N\times N}\,.$
(3.3)

Case 2. When
$\mathbf{K}=\mathbf{J}^{[N]}(k_{1}),~{}\mathbf{L}=\mathbf{J}^{[N]}(l_{1}),$
$C(x)=\mathbf{J}^{[N]}(c_{1}(x))$ or $C(m)=\mathbf{J}^{[N]}(c_{1}(m))$, we
have $\mathbf{r}=\mathbf{r}_{J}^{[N]}(k_{1})$ and
$\mathbf{s}=\mathbf{s}_{J}^{[N]}(l_{1})$ composed of
$r_{i}=\frac{\partial^{i-1}_{k_{1}}\rho_{1}}{(i-1)!},\
s_{j}=\frac{\partial^{N-j}_{l_{1}}\sigma_{1}}{(N-j)!}$ respectively, and
$\mathbf{M}=\mathbf{FGH}$ where

$\mathbf{F}=\mathbf{T}^{[N]}(\\{r_{i}\\}^{N}_{1}),~{}~{}\mathbf{G}=\mathbf{G}^{[N;N]}_{JJ}(k_{1};l_{1}),~{}~{}\mathbf{H}=\mathbf{H}^{[N]}(\\{s_{j}\\}^{N}_{1}).$
(3.4)

Case 3. When

$\displaystyle\mathbf{K}=\mathrm{Diag}\\{\mathbf{D}^{[N_{1}]}(\\{k_{i}\\}^{N_{1}}_{1}),\mathbf{J}^{[N_{2}]}(k_{N_{1}+1}),\mathbf{J}^{[N_{3}]}(k_{N_{1}+2}),\cdots,\mathbf{J}^{[N_{s}]}(k_{N_{1}+(s-1)})\\},$
(3.5a)
$\displaystyle\mathbf{L}=\mathrm{Diag}\\{\mathbf{D}^{[N_{1}]}(\\{l_{i}\\}^{N_{1}}_{1}),\mathbf{J}^{[N_{2}]}(l_{N_{1}+1}),\mathbf{J}^{[N_{3}]}(l_{N_{1}+2}),\cdots,\mathbf{J}^{[N_{s}]}(l_{N_{1}+(s-1)})\\},$
(3.5b)

with $\sum_{i=1}^{s}N_{i}=N$,
$C(x)=\mathrm{Diag}\\{\mathbf{D}^{[N_{1}]}(\\{c_{i}(x)\\}^{N_{1}}_{1}),\mathbf{J}^{[N_{2}]}(c_{N_{1}+1}(x)),\cdots,\mathbf{J}^{[N_{2}]}(c_{N_{1}+(s-1)}(x))\\}$
or
$C(m)=\mathrm{Diag}\\{\mathbf{D}^{[N_{1}]}(\\{c_{i}(m)\\}^{N_{1}}_{1}),\mathbf{J}^{[N_{2}]}(c_{N_{1}+1}(m)),\cdots,\mathbf{J}^{[N_{2}]}(c_{N_{1}+(s-1)}(m))\\}$,
we have

$\mathbf{r}=\left(\begin{array}[]{l}\mathbf{r}_{D}^{[N_{1}]}(\\{k_{i}\\}_{1}^{N_{1}})\\\
\mathbf{r}_{J}^{[N_{2}]}(k_{N_{1}+1})\\\ ~{}~{}\vdots\\\
\mathbf{r}_{J}^{[N_{s}]}(k_{N_{1}+(s-1)})\end{array}\right),~{}~{}~{}\mathbf{s}=\left(\begin{array}[]{l}\mathbf{s}_{D}^{[N_{1}]}(\\{l_{j}\\}_{1}^{N_{1}})\\\
\mathbf{s}_{J}^{N_{2}}(l_{N_{1}+1})\\\ ~{}~{}\vdots\\\
\mathbf{s}_{J}^{[N_{s}]}(l_{N_{1}+(s-1)})\end{array}\right),$ (3.6)

where

$\displaystyle\mathbf{F}=\mathrm{Diag}\\{\mathbf{D}^{[N_{1}]}(\\{r_{i}\\}^{N_{1}}_{1}),\mathbf{T}^{[N_{2}]}(k_{N_{1}+1}),\mathbf{T}^{[N_{3}]}(k_{N_{1}+2}),\cdots,\mathbf{T}^{[N_{s}]}(k_{N_{1}+(s-1)})\\},$
(3.7)
$\displaystyle\mathbf{H}=\mathrm{Diag}\\{\mathbf{D}^{[N_{1}]}_{D}(\\{s_{j}\\}^{N_{1}}_{1}),\mathbf{H}^{[N_{2}]}(l_{N_{1}+1}),\mathbf{H}^{[N_{3}]}(l_{N_{1}+2}),\cdots,\mathbf{H}^{[N_{s}]}(l_{N_{1}+(s-1)})\\},$
(3.8)

with $\mathbf{M}=\mathbf{FGH}$, and $\mathbf{G}$ is a block matrix
$\mathbf{G}=(\mathbf{G}_{i,j})_{s\times s}$, where

$\begin{array}[]{l}\mathbf{G}_{1,1}=\Bigl{(}\frac{\rho_{i}\sigma_{j}}{k_{i}+l_{j}}\Bigr{)}_{N_{1}\times
N_{1}};~{}~{}\mathbf{G}_{1,j}=\mathbf{G}^{[N_{1};N_{j}]}_{DJ}(\\{k_{i}\\}^{N_{1}}_{1};l_{N_{1}+j-1}),~{}(1<j\leq
s);\\\
\mathbf{G}_{i,1}=\mathbf{G}^{[N_{i};N_{1}]}_{JD}(k_{N_{1}+i-1};\\{l_{j}\\}^{N_{1}}_{1}),~{}(1<i\leq
s);~{}~{}\mathbf{G}_{i,j}=\mathbf{G}^{[N_{i};N_{j}]}_{JJ}(k_{N_{1}+i-1};l_{N_{1}+j-1}),~{}(1<i,j\leq
s).\end{array}$

Note that the commutative conditions are satisfied for the above three cases.

##  4 Conclusion

Semi-discrete versions of soliton systems, i.e. systems given by integrable
partial differential-difference equations, has significant roles in
mathematics and physics. Based on KP-type equations involving one continuous
variable and two discrete variables, we derive the D$\Delta^{2}$pKP,
D$\Delta^{2}$pmKP and D$\Delta^{2}$SKP equations, as well as two kinds of
corresponding extended systems. The two types of extensions are given in terms
of squared eigenfunctions. This also demonstrates, to some extent, the
capability of the Cauchy matrix method in constructing new integrable systems.

For KP-type equation with one continuous variable and two discrete variables,
we can continue to study its conservation law and related properties. At the
same time, the extended Volterra chain system can be further investigated in
the future. Furthermore, for the extended D${}^{2}\Delta$KP equation, the
squared eigenfunction was given as a symmetry of the D${}^{2}\Delta$KP
equation in [8]. Whether the two types of squared eigenfunction extensions can
be utilized as symmetries is worth exploring. The research on the
relationships between integrable systems and symmetries [26, 27] not only
sheds new light on the current theory of integrable systems, but also enhances
many areas of pure mathematics.

### Acknowledgments

This project is supported by Natural Science Foundation of Henan
(No.242300421687) and Henan Provincial Science and Technology Research Project
(No.222102210258).

### Conflict of Interest

The authors of this work declare that they have no conflicts of interest.

## References

  * [1] E. Date, M. Jimbo, T. Miwa, Method for generating discrete soliton equations. II, J. Phys. Soc. Japan, 51 (1982) 4125–4131.
  * [2] T. Miwa, On Hirota’s difference equations. Proc. Japan Acad., 58 (1981) 9-12.
  * [3] F. W. Nijhoff, H. W. Capel, G. L. Wiersma, G. R. W. Quispel, Bäcklund transformations and three-dimensional lattice equations. Phys. Lett. A, 105 (1984) 267–72
  * [4] H. Eisenberg, Y. Silberberg, Discrete Solitons, Springer Berlin Heidelberg, 2003.
  * [5] J. Hietarinta, N. Joshi, F. W. Nijhoff, Discrete Systems and Integrability, Cambridge University Press, 2016.
  * [6] D. N. Christodoulides, R. I. Joseph, Discrete self-focusing in nonlinear arrays of coupled wave-guides, Opt. Lett., 13 (1988) 794-796.
  * [7] H. S. Eisenberg, Y. Silberberg , R. Morandotti, A. R. Boyd, J. S. Aitchison, Discrete Spatial Optical Solitons in Waveguide Arrays, Phys. Rev. Lett., 81 (1998) 3383-3386.
  * [8] K. Chen, X. Deng, D.J. Zhang, Symmetry constraint of the differential-difference KP hierarchy and a second discretization of the ZS-AKNS system. J. Nonlinear Math. Phys., 24 (2017) 18-35.
  * [9] W. Fu, L. Huang, K. M. Tamizhmani, D. J. Zhang, Integrability properties of the differential-difference Kadomtsev-Petviashvili hierarchy and continuum limits, Nonlinearity, 26 (2013) 3197-3229.
  * [10] S. K. Vel, K. M. Tamizhmani, Lax Pairs, symmetries and conservation Laws of a differential-difference equation-Sato’s approach, Chaos Solit. Fractals, 8 (1997) 917–931.
  * [11] G. L. Wiersma, H. W. Capel, Lattice equations, hierarchies and Hamiltonian structures: III. The 2D Toda and KP hierarchies, Physica A, 149 (1988) 75-106.
  * [12] E. V.Ferapontov, V. S. Novikov, I. Roustemoglou, On the classification of discrete Hirota-type equations in 3D, Int. Math. Res. Notices, 13(2015), 4933-4974.
  * [13] I. T. Habibullin, A. R. Khakimova, On the classification of nonlinear integrable three-dimensional chains via characteristic Lie algebras, Theor. Math. Phys., 217(2023) 1541-1573.
  * [14] G. L. Wiersma, H. W. Capel, Lattice equations, hierarchies and Hamiltonian structures: II. KP-type of hierarchies on 2D lattices, Physica A, 149 (1988), 49–74.
  * [15] V. E. Adler, S. Ya. Startsev, Discrete analogues of the Liouville equation, Theor. Math. Phys., 121(1999) 1484-1495.
  * [16] Y. Yin, W. Fu, Integrable semi-discretisation of the Drinfel’d–Sokolov hierarchies, Nonlinearity, 35 (2021) 3324-3357
  * [17] F. W. Nijhoff, J. Atkinson, J. Hietarinta, Soliton solutions for ABS lattice equations: I. Cauchy matrix approach, J. Phys. A: Math. Theor., 42 (2009) 404005 (34pp).
  * [18] D. J. Zhang, S. L. Zhao, Solutions to ABS lattice equations via generalized Cauchy matrix approach, Stud. Appl. Math., 131 (2013) 72-103.
  * [19] D.D. Xu, D.J. Zhang, S.L. Zhao, The Sylvester equation and integrable equations: I. The Korteweg-de Vries system and sine-Gordon equation, J. Nonl. Math. Phys., 21 (2014) 382-406.
  * [20] W. Feng, S. L. Zhao, Generalized Cauchy matrix approach for lattice KP-type equations, Commun. Nonlinear Sci. Numer. Simulat, 18 (2013) 1652–1664.
  * [21] F. W. Nijhoff, H. W. Capel, G. L. Wiersma, Integrable lattice systems in two and three dimensions, Lecture Notes in Physics 239 (1985) 263–302.
  * [22] F. W. Nijhoff, G. D. Pang, A time-discretized version of the Calogero–Moser model Phys. Lett. A, 191 (1994) 101–107.
  * [23] H. J. Tian, D. J. Zhang, Cauchy matrix structure of the Mel’nikov model of long-short wave interaction, Commun. Theor. Phys., 72 (2020) 125006 (11pp).
  * [24] H. J. Tian, D. J. Zhang, Cauchy matrix approach to integrable equations with self-consistent sources and the Yajima–Oikawa system. Appl. Math. Lett., 103 (2020) 106165 (7pp).
  * [25] H. J. Tian, X. J. Feng, W. M. Liu, Extended KdV equations generated by squared eigenfunction symmetry. Appl. Math. Lett., 128(2022) 107852 (7pp).
  * [26] C. Z. Li, J.P. Cheng, K. L. Tian, M. H. Li, J. S. He, Ghost symmetry of the discrete KP hierarchy, Monat. für. Math., 180(2015) 815-832.
  * [27] Y.Q. Yao , X.J. Liu, Y.B. Zeng, A new extended discrete KP hierarchy and a generalized dressing method, J. Phys. A: Math. Theor., 42(2009) 454026 (10pp).
  * [28]


# Partial Rewriting for Multi-Stage ASR

Antoine Bruguier, David Qiu, Yanzhang He
Google LLC, USA

<EMAIL_ADDRESS>
###### Abstract

For many streaming automatic speech recognition tasks, it is important to
provide timely intermediate streaming results, while refining a high quality
final result. This can be done using a multi-stage architecture, where a small
left-context only model creates streaming results and a larger left- and
right-context model produces a final result at the end. While this
significantly improves the quality of the final results without compromising
the streaming emission latency of the system, streaming results do not benefit
from the quality improvements. Here, we propose using a text manipulation
algorithm that merges the streaming outputs of both models. We improve the
quality of streaming results by around 10%, without altering the final
results. Our approach introduces no additional latency and reduces flickering.
It is also lightweight, does not require retraining the model, and it can be
applied to a wide variety of multi-stage architectures.

##  1 Introduction

Streaming automatic speech recognition (ASR) processes the audio as it is
being spoken, without waiting for the entire utterance to be finished before
generating a transcript. The intermediate results are called partial results,
while the single result produced at the end of the utterance is called a final
result. This is in contrast to batch ASR which waits until the entire
utterance is spoken before generating only a final result. Streaming ASR is
useful in applications where it is important to provide feedback to the user
as soon as possible, such as voice-activated assistants, dictation, or live
video captioning.

The first streaming ASR systems were developed several decades ago Yu and Deng
(2015); Hinton et al. (2012); Brown et al. (1982); Selfridge et al. (2011);
Fink et al. (1998); Saraclar et al. (2002), but more recently new approaches
relying on end-to-end models Li et al. (2019); Yeh et al. (2019); Sainath et
al. (2020); He et al. (2019) have been developed. However, batch ASR usually
outperforms streaming ASR because it can better understand the context of the
speech and therefore make more accurate predictions.

Cascaded recognizers Narayanan et al. (2021); Sainath et al. (2021) attempt to
have the best of both worlds by combining the strengths of both streaming and
batch ASR systems. They do so by using two models. A causal model only uses
the left context outputs and therefore has low latency. A cascaded model with
both left and right contexts looks ahead in the audio to output results and
therefore has better accuracy. The cascaded model is also more accurate
because it has more weights than the causal one. While both models can
generate partial and final results, typically only the causal model is used to
generate the partial results so that the latency is low, and only the cascaded
model is used to generate the final result so that it is more accurate Sainath
et al. (2022). This approach enables low-latency streaming ASR while achieving
a high quality for the final results. However, this leaves a gap in quality
because the partial results still have relatively lower quality.

Further, running two models independently can cause issues because even though
the models decode the same audio and even share some of their weights, there
is no coupling between the two outputs and the partial results may
occasionally differ substantially from the final result. Recent improvements
have been able to tie in several models together Mahadeokar et al. (2022); Li
et al. (2023). By running the two models together, they are able to improve
the partial results, but this comes at the cost of not having a right context
for high-quality final results. In essence, both models are causal, but one is
still larger than the other. Further, the two models now have to be run
synchronously, and the overall architecture is fixed at training time.

In parallel to these modelling efforts, there have been some improvements in
how partial results can be evaluated along three dimensions: flickering,
partial quality, and partial latency. Flickering occurs when words that have
already been decoded in a previous partial results are changed in subsequent
partial results. This negatively impacts the experience for the users Noble et
al. (2022); Muller et al. (2016) as well as potentially increasing the
computation cost of other systems (e.g. translation Iranzo-Sánchez et al.
(2020) or assistants Chang et al. (2020)). The amount of flickering can be
measured using the metric defined in Shangguan et al. (2020). Work by Bruguier
et al. (2022) defines metrics for partial quality and latency. This allows us
to measure improvements to partial results along these three dimensions. We
will describe the three metrics of interest in more detail in section 2.3.

##  2 Goals and constraints

###  2.1 Overview

In the present work, we want to improve the quality of the partial results by
using a large model that has a right context, but we want to do so without
scarifying latency. Contrary to Narayanan et al. (2021); Sainath et al. (2021)
we no longer see partial results as a by-product of the decoding.

Instead, we want to keep two models running completely independently as in
Narayanan et al. (2021) and still allow the larger model to have a right
context. We also do not require a fixed architecture at training time. Our
work will leave the final results unchanged as they already benefit from the
larger model and the use of a right context, but it will improve the quality
of partial results significantly.

We want the approach to be cheap computationally. It should not increase the
memory budget in any measurable way, nor increase power consumption.

###  2.2 Combining results from two models

The only assumption we make is that we have two models that output a stream of
partial results. The results of the first model have low latency, but
relatively lower quality. The results from the second model have much higher
latency, but are of better quality. Like Narayanan et al. (2021); Sainath et
al. (2021, 2022) we want the best of both world, but this time for partial
results without tying in models architectures as done in Mahadeokar et al.
(2022); Li et al. (2023) nor require retraining.

In order to validate our approach experimentally, we modify how the cascaded
architecture creates partials. While we still only use the cascaded model to
create final results, we now allow for both causal and cascaded model to
create partial results. Then, our proposed approach combines these two streams
of partial results into a single stream of new partial results that still have
low latency, but better quality. We do not rely on the specificity of the
cascaded architecture and only assume that we have two models running
concurrently and decoding the same audio. We only modify the partial results,
and leave the final results unchanged.

###  2.3 Metrics of interest

In order to measure the validity of our approach, we must be able to measure
the quality of our newly created partial results. For this we heavily rely on
a partial word error rate (PWER) metric defined in section 2.2 of Bruguier et
al. (2022). Roughly speaking, PWER is the WER averaged over all the partials,
ignoring the missing words at the end of each partial.

The causal model outputs partial results with a very low latency, and we do
not wish to introduce any new latency. We thus rely on a metric that estimates
the latency of all partials, as defined in section 2.3 of Bruguier et al.
(2022). Roughly speaking, the partial latency (PL) metric is the average
appearance time of every correct word since the beginning of the utterance,
where only the change between experiments is the meaningful number.

Finally, an algorithm could increase the amount of flickering of the partial
results. In order to control for this potential negative change, we measure
the unstable partial word ratio (UPWR) as defined in Shangguan et al. (2020).
Roughly speaking, UPWR is the fraction of already decoded words in all the
partial results that are changed by a subsequent partial or final result.
However, in order to measure the effect of flickering more accurately, we
expand the definition of the metric by measuring UPWR on different set of
results. If the decoding of an utterance produces $N$ partial results and 1
final result, we measure UPWR on three different sets: 1) Only the $N$ partial
results, 2) Only the single $N^{\text{th}}$ partial result and the final
result, and 3) The $N$ partial results followed by the final result. This
allows us to measure how much flickering occurs during the streaming decoding,
during the transition from streaming decoding to final result, and overall,
respectively.

##  3 Proposed algorithm

###  3.1 Intuition and core algorithm

The algorithm we propose relies on text manipulation only. We take as input
the two streams of partial results, one from the causal model and one from the
cascaded model, and then create a new stream of composite partial results.

_rozaee_how_are_you_rosalie_how$0$$1$$2$$3$$4$$5$$6$$1$$0$$1$$2$$3$$4$$5$$2$$1$$1$$2$$3$$4$$5$$3$$2$$2$$2$$3$$4$$5$$4$$3$$3$$3$$3$$4$$5$$5$$4$$4$$4$$3$$4$$5$CISSCDD
Figure 1: Example of Levenshtein alignment of two decoded sequences. The
vertical is the partial result from the cascaded model $x[1,\ldots,m]$ with
$m=5$, while the horizontal is the partial result from the causal model
$y[1,\ldots,n]$ with $n=6$. The numbers correspond to the alignment cost
$C(i,j)$ and the arrows the best path with C=correct, I=insertion,
S=substitution, and D=deletion. The algorithm finds $j^{*}=4$ that minimizes
the cost of the last row, with $C(m,j^{*})=3$.

Consider the example where in the middle of decoding of an utterance, the user
has said “Rosalie how are you”. The causal model, being smaller and less
accurate, has incorrectly decoded it as “_ro za ee _how _are _you” ($n=6$ word
pieces111We use word-pieces Sainath et al. (2020) to decompose text into
tokens even but the approach can also use Unicode codepoints as tokens), while
the cascaded is more accurate but has a delay, and thus has only decoded “_ro
sa l ie _how” ($m=5$ word pieces) at the same frame. As expected $m<n$ since
the cascaded model uses a right context, but it is not true that the $n-m=1$
word piece “_you” corresponds to amount of audio in the right context. Indeed,
if we were to naively copy the first $m$ word pieces from the cascaded model
and then append the $n-m=1$ last word pieces from the causal model, we would
get an incorrect transcript: “_ro sa l ie _how _you” because the word “_are”
is missing.

$\texttt{latest\\_partial\\_used\\_for\\_rewriting}\leftarrow\texttt{""}$

$\texttt{latest\\_cascaded\\_partial}\leftarrow\texttt{""}$

while AudioStillAvailable() do

partial_result $\leftarrow$ PullPartialResult()

if partial_result.origin == CASCADED then

latest_cascaded_partial $\leftarrow$ partial_result.text

elseOutputPartial(RewriteResult(partial_result.text))

end if

end while

Algorithm 1 Streaming algorithm that recomposes results.

procedure RewriteResult(causal_partial)

$\texttt{cost},\texttt{composite}$ $\leftarrow$
CreateComposite(causal_partial, latest_cascaded_partial)

if $\texttt{cost}<\texttt{cost\\_threshold}$ then

$\texttt{latest\\_partial\\_used\\_for\\_rewriting}\leftarrow\texttt{latest\\_cascaded\\_partial}$
return composite

else

$\\_,\texttt{composite}$ $\leftarrow$ CreateComposite(causal_partial,
latest_partial_used_for_rewriting) return composite

end if

end procedure

Algorithm 2 Algorithm that attempts rewriting with a fall-back.

Instead, we need a way to blend the two partial results correctly. Every time
we merge two partial results, we should correctly estimate how many extra
tokens to copy from the causal results.

We propose to use a Levenshein alignment Cormen et al. (2001) of the two
sequences, as shown in figure 1. By aligning the two sequences, we are able to
compute the alignment costs. We then modify the Levenshtein alignment
algorithm in a similar fashion as Bruguier et al. (2022) by allowing a
variable reference length. Since we want to consume the entire cascaded
sequence, we must have a final best alignment path on the last row (shown in a
red box on figure 1). However, we do not want to consume all the causal
sequence because it is likely to have more token due to the emission delay
from the cascaded model. In other words, the deletions due to the time delay
should be ignored. Thus, instead of using the bottom right cell as the end-
path of our alignment, we sweep all the costs on the bottom row and use the
cell with the lowest cost. In essence, the procedure allows us to discover how
many causal tokens correspond to the time delay. We are now able to create a
composite transcript: First, we copy all the tokens from the cascaded model,
namely “_ro sa l ie _how”. Then, we append the remaining causal tokens, namely
“_are _you” to get a composite transcript “ _ro sa l ie _how _are _you”.

Mathematically, if the cascaded transcript $x$ has length $m$ and the causal
transcript $y$ has length $n$, then the Levenshtein algorithm will compute the
cost $C(i,j)$ of aligning substrings $x[1,\ldots,i]$ and $y[1,\ldots,j]$ for
$i\in[1,m]$ and $j\in[1,n]$. We then compute:

$j^{*}=\operatorname*{arg\,min}_{j\in[1,n]}\left(C(m,j)\right)$ (1)

and the composite transcript is (where $\oplus$ is a string concatenation):

$z\triangleq x[1,\ldots,m]\oplus y[(j^{*}+1),\ldots,n]$ (2)

###  3.2 Cropping partial results for non-quadratic growth

Because we run the algorithm on streaming audio, we must make sure it doesn’t
delay the appearance of results. Further, on device applications require a low
power consumption. One of the issue with the naive approach of section 3.1 is
that its run time is quadratic. For short audio segments, this is not a
problem, but when decoding longer duration segments, the cost can become
prohibitive.

procedure CreateComposite(causal_partial,cascaded_partial)

$\texttt{causal\\_tok}\leftarrow\texttt{Split(causal\\_partial)}$

$\texttt{cascaded\\_tok}\leftarrow\texttt{Trim(Split(cascaded\\_partial))}$

$\texttt{composite\\_tok}\leftarrow\texttt{\\{\\}}$

$\texttt{P}\leftarrow\texttt{max(min(causal\\_tok.size, cascaded\\_tok.size) -
M, 1)}$

$\texttt{codes}\leftarrow\texttt{LevAlign(causal\\_tok[P..],
cascaded\\_tok[P..])}$

for $\texttt{j}\leftarrow\texttt{1}$ to P do
composite_tok.append(cascaded_tok[j])

end for

$\texttt{i}\leftarrow\texttt{P}$

$\texttt{m}\leftarrow 1$

$\texttt{cost}\leftarrow 0.0$

for $\texttt{code}\leftarrow\texttt{codes}$ do

if code == SUBSTITUTE || code == CORRECT then
composite_tok.append(cascaded_tok[j])

if j > cascaded_tok.size - K then

if code == SUBSTITUTE then

$\texttt{c}\leftarrow\texttt{c + 1.0}$

end if

$\texttt{m}\leftarrow\texttt{m + 1}$

end if

$\texttt{i, j}\leftarrow\texttt{i + 1, j + 1}$

end if

if code == INSERT then composite_tok.append(cascaded_tok[j])

if j > cascaded_tok.size - K then

$\texttt{c}\leftarrow\texttt{c + 1.0}$

$\texttt{m}\leftarrow\texttt{m + 1}$

end if

$\texttt{j}\leftarrow\texttt{j + 1}$

end if

if code == DELETE then

if j > cascaded_tok.size - K then

$\texttt{c}\leftarrow\texttt{c + 1.0}$

$\texttt{m}\leftarrow\texttt{m + 1}$

end if

$\texttt{i}\leftarrow\texttt{i + 1}$

end if

end for

while $\texttt{i}<\texttt{causal\\_tok.size}$ do
composite_tok.append(causal_tok[i])

end whilereturn c / m, composite_tok

end procedure

Algorithm 3 Algorithm that creates a composite result.

This can be alleviated by capping the length of the sequences so that the
shorter sequence is at most of length $M$. Thus, we remove the first
$P=\max(\min(m,n)-M,1)$ tokens of each string and only align $x[P,\ldots,m]$
against $y[P,\ldots,n]$. Then, the composite transcript is:

$z\triangleq x[1,\dots,P]\oplus x[P+1,\ldots,m]\oplus y[(P+j^{*}+1),\ldots,n]$
(3)

The intuition is that we only to have at most the last $M$ tokens of both
transcripts to align them properly. Using the approach, we can make the
algorithm linear without much reduction in its quality.

###  3.3 Trimming cascaded input transcript

In order to reduce the amount of flickering, we can trim the cascaded
transcript. Instead of simply using the full $x[1,\dots,m]$ word pieces as an
input, we can trim $T$ word pieces and only use $x[1,\ldots,\max(m-T,1)]$.
While this does not create an additional latency (since we still use the full
causal transcript), this may cause the composite transcript to have lower
quality.

###  3.4 Alignment cost bailing and adding hysteresis

If the causal and cascaded transcripts differ too much, alignment may not
produce desirable results. Misalignment reduce the quality of the composite
result and when they get corrected cause flickering. We can use the alignment
cost to determine the distance between the two transcripts, and if it is too
high, we simply do not create a composite transcript and keep the causal
transcript. We defined two costs measurements. The first uses the full
sequence, namely:

$\rho_{f}\triangleq\frac{C(m,n)}{m}$ (4)

We can also limit ourselves to the end of the alignment and only consider the
last $K$ tokens:

$\rho_{r}(K)\triangleq\frac{C\left(m,n\right)-C\left(\max(m-K,0),\max(n-K,0)\right)}{\min(K,m)}$
(5)

We only rewrite the transcript if the cost is below a settable threshold.
However, if we had previously used a cascaded transcript to create a composite
transcript, we fall back to the previously used one, so that we do not
suddenly stop rewriting transcript.

###  3.5 Overall algorithm

We decompose the overall approach in three algorithms. In algorithm 1, we show
the entry point of our approach. The code keeps pulling for new results as
long as there is still audio to be decoded. In case a partial result comes
from the cascaded model, it is suppressed and we only store the text that
would have been outputted. In case a partial result comes from the causal
model, we rewrite it and show it to the user.

In algorithm 2, we show the hysteresis that allows to fall back on previously
used cascaded partial results. We attempt to rewrite the causal partial using
the latest cascaded partial. If the cost of rewriting such partial is low
enough, we record the latest cascaded result and return the composite
transcript. If the cost is too high, then we fall back to unconditionally
rewriting the causal transcript using the latest cascaded result used. Thus,
if the two models completely disagree, we fall back on the last time they
agreed.

Finally, algorithm 3 shows how we create a composite partial result. It
computes the alignment on a subset of the tokens and then outputs the
composite tokens. The cost is computed during the reconstruction. For brevity,
the algorithm for trimming of the tokens is omitted. In order to reduce the
flickering of both the causal and cascaded models themselves, we applied the
same deflickering algorithm as Bruguier et al. (2022) with $\alpha=0.2$ for
our test model.

##  4 Results

Test set |  |  | UPWR |  | PWER
---|---|---|---|---|---
|  | part. | trans. | all |
Dict- | Base | 0.04 | 0.12 | 0.15 | 3.89
ation | Test | 0.07 | 0.06 | 0.13 | 3.49
| $\Delta$ | 75% | -50% | -13% | -10%
Voice | Base | 0.05 | 0.14 | 0.19 | 5.10
Search | Test | 0.08 | 0.09 | 0.16 | 4.99
| $\Delta$ | 60% | -36% | -16% | -2%
TTS | Base | 0.03 | 0.51 | 0.54 | 3.64
Audio | Test | 0.16 | 0.02 | 0.18 | 2.94
book | $\Delta$ | 433% | -96% | -67% | -19%
Libri- | Base | 0.05 | 0.31 | 0.36 | 7.47
speech | Test | 0.17 | 0.05 | 0.22 | 6.18
| $\Delta$ | 240% | -84% | -39% | -17%
Tele- | Base | 0.08 | 0.34 | 0.42 | 17.63
phony | Test | 0.23 | 0.11 | 0.34 | 15.51
| $\Delta$ | 197% | -68% | -19% | -12%
Table 1: Results of our proposed approach. As described in section 2.3, we
measure UPWR for all three subsets (for only the partials, only the
transition, and for all results) and PWER for our base and test models. We
also show the change ($\Delta$) either in percentage or milliseconds. The
dictation test set are longer-form utterances corresponding to messages. The
voice search test set are shorter queries seeking web results. We used text-
to-speech (TTS) to synthesize long-form audio to produce long utterances. The
librispeech test set is from Panayotov et al. (2015). Finally, we also
reported on a telephony set. All data was handled abiding by Google AI
Principles aip .

###  4.1 Base model

In order to evaluate the quality of our algorithm, we reused the model of
Sainath et al. (2021).

It uses a streaming comformer-transducer where the encoder consists of 12
causal comformer layers Yu et al. (2021), each with 23 frames of left context,
a self-attention with 8 heads, and a convolution kernel size of 15. We use an
embedding prediction network with 2 previous labels as input Botros et al.
(2021) and an embedding dimension of 320. The base model used fast emit Yu et
al. (2021) in order to reduce the latency of the output.

We used a frame rate of 30ms, but the encoder stacks two frames and then
downsamples, the effective frame rate is 60ms. Further, because the cascaded
model looks ahead 15 stacked frames, it means that its output roughly
corresponds to what was said about 900ms in the past. We reused the model
Narayanan et al. (2019) which was trained on 400k hours of multidomain data
with a one-hot domain ID Yu et al. (2021) indicating the type of utterance.
All data are deidentified and the collection and handling abide by Google AI
Principles aip .

###  4.2 Results

The results are shown on table 1. We do not report the regular WER as we
verified experimentally that our algorithm indeed does not change the final
results. We measure the UPWR, PWER, and partial latency on four test sets
using the metrics described in section 2.3. The test model is the result of a
sweep of the parameters that gives a good balance between increased quality
and flickering ($alpha=0.2$, $\rho_{f}=\infty$, $\rho_{r}=0.5$, $K=10$, $P=25$
and $T=1$). We can see that for all sets, the PWER is reduced significantly.
The smaller reduction is for the voice search test set; this is explained by
the average duration of the utterances. Since they are shorter in this test
set, the 900ms delay of the cascaded model has greater impact. There is not as
much time to rewrite the partial results. For the other test sets, the PWER is
reduced by at least 10%.

The picture for flickering is more mixed. Because we want to improve the
quality of partial results, we must, by definition, change the decoded words
and because we do not want to increase the latency, we must therefore change
already decoded words and thus cause flicker. We do see a large increase in
the flickering of the partial results (left sub-column of the UPWR section)
for all sets, with a larger increase for test sets for longer utterances.
However, the UPWR for the transition from the last partial result to the final
result (middle sub-column) is significantly lower. This is explained by the
fact that our algorithm pulls in the transition from causal results to
cascaded results earlier. Therefore, at the end of the utterance, the partial
results are closer to the final results. Overall, we see that the total UPWR
(right sub-column) is lower.

As described in section 2.3, only the change in latency metric is meaningful.
For all tests sets, the change was less than 10ms, well below human
perception. We also ran measurements on a Pixel6a phone and the time spent in
each rewriting is on average below $0.1ms$ meaning that our algorithm has very
low computation requirements and power consumption.

The reader can observe the effect of the algorithm in videos sup  that show
the causal, cascaded, and merged partial results when decoding Librispeech
Panayotov et al. (2015) utterances.

###  4.3 Hyperparameter sweep analysis

Figure 2: PWER and UPWER as a function of $T$ Figure 3: PWER and UPWER as a
function of $\rho_{r}$ Figure 4: PWER and UPWER as a function of $K$

We plotted the PWER and the UPWR metrics according to the hyperparameter of
the model for the dictation test set in figures 2, 3, and 4. Note that the
parameters we chose for the results of table 1 are not the best according to
the plots below. This is because we chose a set of values that had acceptable
results for the many test sets. This is a product judgment call, but we found
that the parameters we chose work well for almost all our test sets. This
suggests that the hyperparameters were not overfitted to the dictation test
set.

We see on figure 2 that as $T$ increases (the number of words we remove from
the causal partial), the PWER goes up. This is expected because if we have
fewer causal words (which are usually more correct), we cannot improve the
rewritten transcript as much. As $T$ increases, we see that the UPWR for only
the partials goes down (we rewrite less during decoding, thus we flicker
less), and the UPWR for the transition goes up (by the time the final is
shown, we have rewritten less and thus the transition has more flickering).
Overall, the flickering is neutral.

We sweep the $\rho_{r}$ parameter in figure 3 where a lower value means that
we want a smaller edit distance. When the parameter is 0.0, we recover the
value of the base model of table 1 (i.e. rewrite nothing). As it goes up, the
PWER goes down because we rewrite more, but the flickering of the partial goes
up (because we merge transcripts that are more different). As for the previous
parameter, the transition UPWR goes down because we are closer to the final
result.

We sweep the parameter $K$ in figure 4, which means how many words we use for
the edit distance hysteresis. If we set it to 0, we recover the case where
$\rho_{r}=1$. As K goes up, we are more strict when accepting rewrites, and
thus the PWER goes up while the flickering goes down.

##  5 Conclusions

Until recently Bruguier et al. (2022), not much attention was paid to the
quality of partial results of streaming ASR systems. It was assumed that
improving final results would automatically improve partial results.
Separately, more recent approaches for streaming decoding used multiple models
Narayanan et al. (2021). We presented an algorithm that doesn’t require a re-
architecturing of the model like in Mahadeokar et al. (2022) but instead
relies on a simple, more general, rule to combine partials from multiple
models. Doing so, we are able to significantly reduce partial word error rate
and flickering without impacting partial latency. The computational cost of
our approach is extremely low and therefore well-suited for on-device
recognition. While we used a cascaded architecture to validate our approach,
the proposed algorithm is applicable to any multi-decoder approach; For
example, it can be applied to merging a stream of high quality but high
latency partial results from a server recognizer into a stream of low-latency
partial results from an on-device recognizer.

Our approach also opens a more flexible allocation of the model weights. We
could re-allocate weights away from the causal model and towards the cascaded
one. This could improve the quality of the final result, with limited impact
on the partial results, precisely because we have rewriting.

Further work will focus on expanding the applicability of our approach. One
area of further research is lowering the computational requirements of our
systems by trading memory and computation away from the causal model towards
the cascaded model without impacting the PWER too much. For example, the
causal model could be run greedily with a single beam, or we could re-allocate
the some weights to the cascaded model.

Another area of further research could be multi-stage and intermittent model.
The approach described is easily expanded to 3 or more models, where the
smallest model could be run more often, while the subsequent larger ones only
as needed.

## References

  * Yu and Deng [2015] Dong Yu and Li Deng.  _Automatic Speech Recognition_.  Springer London, 2015.
  * Hinton et al. [2012] Geoffrey Hinton, Li Deng, Dong Yu, George Dahl, Abdel-Rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara Sainath, and Brian Kingsbury.  Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups.  _IEEE Signal Processing Magazine_ , 2012.
  * Brown et al. [1982] Peter Brown, James Spohrer, Peter Hochschild, and James Baker.  Partial traceback and dynamic programming.  _ICASSP_ , 1982.
  * Selfridge et al. [2011] Ethan Selfridge, Iker Arizmendi, Peter Heeman, and Jason Williams.  Stability and accuracy in incremental speech recognition.  _SIGDIAL_ , 2011.
  * Fink et al. [1998] Gernot Fink, Christoph Schillo, Franz Kummert, and Gerhard Sagerer.  Incremental speech recognition for multimodal interfaces.  _Interspeech_ , 1998.
  * Saraclar et al. [2002] Murat Saraclar, Michael Riley, Enrico Bocchieri, and Vincent Goffin.  Towards automatic closed captioning: low latency real time broadcast news transcription.  _Interspeech_ , 2002.
  * Li et al. [2019] Jinyu Li, Rui Zhao, Hu Hu, and Yifan Gong.  Improving RNN transducer modeling for end-to-end speech recognition.  _ASRU_ , 2019.
  * Yeh et al. [2019] Ching-Feng Yeh, Jay Mahadeokar, Kaustubh Kalgaonkar, Yongqiang Wang, Duc Le, Kjell Schubert Mahaveer Jain, Christian Fuegen, and Michael Seltzer.  Transformer-transducer: End-to-end speech recognition with self-attention.  _CoRR_ , 2019.
  * Sainath et al. [2020] Tara Sainath, Yanzhang He, Bo Li, Arun Narayanan, Ruoming Pang, Antoine Bruguier, Shuo yiin Chang, Wei Li, Raziel Alvarez, Zhifeng Chen, Chung-Cheng Chiu, David Garcia, Alex Gruenstein, Ke Hu, Minho Jin, Anjuli Kannan, Qiao Liang, Ian McGraw, Cal Peyser, Rohit Prabhavalkar, Golan Pundak, David Rybach, Yuan Shangguan, Yash Sheth, Trevor Strohman, Mirko Visontai, Yonghui Wu, Yu Zhang, and Ding Zhao.  A streaming on-device end-to-end model surpassing server-side conventional model quality and latency.  _ICASSP_ , 2020.
  * He et al. [2019] Yanzhang He, Tara Sainath, Rohit Prabhavalkar, Ian McGraw, Raziel Alvarez, Ding Zhao, David Rybach, Anjuli Kannan, Yonghui Wu, Ruoming Pang, Qiao Liang, Deepti Bhatia, Yuan Shangguan, Bo Li, Golan Pundak, Khe Chai Sim, Tom Bagby, Kanishka Rao Shuo-yiin Chang, and Alexander Gruenstein.  Streaming end-to-end speech recognition for mobile device.  _ICASSP_ , 2019.
  * Narayanan et al. [2021] Arun Narayanan, Tara Sainath, Ruoming Pang, Jiahui Yu, Chung-Cheng Chiu, Rohit Prabhavalkar, Ehsan Variani, and Trevor Strohman.  Cascaded encoders for unifying streaming and non-streaming ASR.  _ICASPP_ , 2021.
  * Sainath et al. [2021] Tara N Sainath, Yanzhang He, and Arun Narayanan et al.  An efficient streaming non-recurrent on-device end-to-end model with improvements to rare-word modeling.  _Interspeech_ , 2021.
  * Sainath et al. [2022] Tara N. Sainath, Yanzhang He, Arun Narayanan, Rami Botros, Weiran Wang, David Qiu, Chung-Cheng Chiu, Rohit Prabhavalkar, Alexander Gruenstein, Anmol Gulati, Bo Li, David Rybach, Emmanuel Guzman, Ian McGraw, James Qin, Krzysztof Choromanski, Qiao Liang, Robert David, Ruoming Pang, Shuo yiin Chang, Trevor Strohman, W. Ronny Huang, Wei Han, Yonghui Wu, and Yu Zhang.  Improving the latency and quality of cascaded encoders.  _ICASSP_ , 2022.
  * Mahadeokar et al. [2022] Jay Mahadeokar, Yangyang Shi, Ke Li, Duc Le, Jiedan Zhu, Vikas Chandra, Ozlem Kalinli, and Michael Seltzer.  Streaming parallel transducer beam search with fast-slow cascaded encoders.  _CoRR_ , 2022.
  * Li et al. [2023] Ke Li, Jay Mahadeokar, Jinxi Guo, Yangyang Shi, Gil Keren, Ozlem Kalinli, Michael L. Seltzer, and Duc Le.  Improving fast-slow encoder based transducer with streaming deliberation.  _ICASSP_ , 2023.
  * Noble et al. [2022] Steve Noble, Jason White, Scott Hollier, Janina Sajka, and Joshue O’Conno.  Synchronization accessibility user requirements.  https://www.w3.org/TR/saur/#caption-synchronization-thresholds, 2022.  [Online; accessed 23-July-2022].
  * Muller et al. [2016] Markus Muller, Sarah Funfer, Sebastian Stuker, and Alex Waibel.  Evaluation of the kit lecture translation system.  _LREC_ , 2016.
  * Iranzo-Sánchez et al. [2020] Javier Iranzo-Sánchez, Adrià Giménez Pastor, Joan Albert Silvestre-Cerdà, Pau Baquero-Arnal, Jorge Civera Saiz, and Alfons Juan.  Streaming cascade-based speech translation leveraged by a direct segmentation model.  _EMNLP_ , 2020.
  * Chang et al. [2020] Shuo-Yiin Chang, Bo Li, David Rybach, Yanzhang He, Wei Li, Tara Sainath, and Trevor Strohman.  Low latency speech recognition using end-to-end prefetching.  _Interspeech_ , 2020.
  * Shangguan et al. [2020] Yuan Shangguan, Kate Knister, Yanzhang He, Ian McGraw, and Françoise Beaufays.  Analyzing the quality and stability of a streaming end-to-end on-device speech recognizer.  _Interspeech_ , 2020.
  * Bruguier et al. [2022] Antoine Bruguier, David Qiu, Trevor Strohman, and Yanzhang He.  Flickering reduction with partial hypothesis reranking for streaming ASR.  In _SLT_ , 2022.
  * Cormen et al. [2001] Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein.  _Introduction to Algorithms_.  The MIT Press, 2nd edition, 2001.  ISBN 0262032937.
  * Panayotov et al. [2015] Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur.  Librispeech: an ASR corpus based on public domain audio books.  In _ICASSP_ , pages 5206–5210. IEEE, 2015.
  * [24] Google, Artificial Intelligence at Google: Our Principles.  https://ai.google/principles/.
  * Yu et al. [2021] Jiahui Yu, Chung-Cheng Chiu, Bo Li, Shuo yiin Chang, Tara N. Sainath, Yanzhang He, Arun Narayanan, Wei Han, Anmol Gulati, Yonghui Wu, and Ruoming Pang.  Fastemit: Low-latency streaming ASR with sequence-level emission regularization.  _ICASSP_ , 2021.
  * Botros et al. [2021] Rami Botros, Tara N Sainath, Robert David, Emmanuel Guzman, Wei Li, and Yanzhang He.  Tied & reduced RNN-T decoder.  _arXiv preprint arXiv:2109.07513_ , 2021.
  * Narayanan et al. [2019] Arun Narayanan, Rohit Prabhavalkar, Chung-Cheng Chiu, David Rybach, Tara Sainath, and Trevor Strohman.  Recognizing long-form speech using streaming end-to-end models.  _ASRU_ , 2019.
  * [28] Supplementary material.  Contains three videos from utterances from Librispeech. A vertical bar shows the boundary between the text coming from the causal model and the cascaded one. This vertical border is added artificially in order to understand better the effect of the algorithm and is not present in the production code. We show an utterance where the PWER is improved greatly. In particular, we can see that in the base model, the correction only happens at the very end, when the final result comes from the cascaded model, rather than being corrected mid-decoding. We also show an example where the PWER is degraded. While the cascaded model has better quality on average, it sometimes performs worse than the causal model. Finally, we also show an example of high increase in flickering.