# Some properties of normal subgroups determined from character tables

Z. Akhlaghi1,2 ,  M. J. Felipe 3 and  M. K. Jean-Philippe4 1 Faculty of
Mathematics and Computer Science, Amirkabir University of Technology (Tehran
Polytechnic), 15914 Tehran, Iran.  2 School of Mathematics, Institute for
Research in Fundamental Science (IPM) P.O. Box:19395-5746, Tehran, Iran.  3
Instituto Universitario de Matemática Pura y Aplicada (IUMPA-UPV), Universitat
Politècnica de València, Valencia, Spain. ORCID: 0000-0002-6699-313.  4
Departamento de Matemáticas, Instituto Tecnológico de Santo Domingo (INTEC)
and Universidad Autónoma de Santo Domingo (UASD), Santo Domingo, República
Dominicana.
(Z. Akhlaghi<EMAIL_ADDRESS>
(M. J. Felipe<EMAIL_ADDRESS>
(M. K. Jean-Philippe<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract.

$G$-character tables of a finite group $G$ were defined in [6]. These tables
can be very useful to obtain certain structural information of a normal
subgroup from the character table of $G$. We analyze certain structural
properties of normal subgroups which can be determined using their
$G$-character tables. For instance, we prove an extension of the Thompson’s
theorem from minimal $G$-invariant characters of a normal subgroup. We also
obtain a variation of Taketa’s theorem for hypercentral normal subgroups
considering their minimal $G$-invariant characters. This generalization allows
us to introduce a new class of nilpotent groups, the class of $nMI$-groups,
whose members verify that its nilpotency class is bounded by the number of
irreducible character degrees of the group.

Mathematics Subject Classification (2022): 20C15, 20E45, 20E15.

Keywords Finite groups $\cdot$ Irreducible characters $\cdot$ Normal subgroups
$\cdot$ Minimal $G$-invariant characters $\cdot$ Hypercentral subgroup

The first author is supported by a grant from IPM (No. 1402200112) and the
second author is supported by Proyecto CIAICO/2021/163, Generalitat Valenciana
(Spain).

##  1\. Introduction

In general, if $N$ is a normal subgroup of a finite group $G$, the character
table of $N$ can not be determined from the character table of $G$. In [6],
the $G$-character table of $N$ obtained from the character table of $G$ is
defined (see Section 2). They are non-singular matrices and provide certain
arithmetical relations among significant integers associated to the
irreducible characters of $N$ such as their character degrees, indices of
inertia subgroups and ramification numbers. The character table of $G$
determines those irreducible characters of $G$ that are over the same minimal
$G$-invariant character of $N$, which are defined as the sum of characters of
$N$ belonging in the same orbit determined by the conjugation action of $G$ on
the set of irreducible characters of $N$. These characters of $N$ form a basis
of the $\mathbb{C}$-vector space of the $G$-invariant class functions of $N$,
and they become relevant to $G$-character tables of $N$, playing a similar
role that irreducible characters do to the character table of a group $G$.

The nilpotency of a normal subgroup $N$ of a group $G$ can be recognized from
the character table $G$, but the nilpotency class of $N$ cannot be recognized.
In fact, we can not deduce if a normal subgroup $N$ is abelian (see [4] and
[16]). However, the commutator $[G,N]$ can be computed from $G$-character
tables of $N$ as the intersection of kernels of linear $G$-invariant
characters of $N$ and this information can be read from $G$-character tables
of $N$ (Corollary 3.3). In particular, we can read from the character table of
$G$ if a normal subgroup $N$ is hypercentral and we can obtain its lower
central $G$-series.

We can not compute prime divisors of irreducible character degrees of $N$ from
the character table of $G$. However, prime divisors of the minimal
$G$-invariant character degrees of a normal subgroup $N$ are known,
considering some relations that arise from $G$-character tables of $N$
(Corollary 2.3). We present a generalization of the well-known Thompson’s
theorem (see 12.2 in [11]) applied to minimal $G$-invariant character degrees
of $N$ (Theorem 3.10). This generalization gives us a sufficient condition to
deduce whether a normal subgroup $N$ has a normal $p$-complement from the
character table of $G$.

Finally, we present a new class of groups which are determined by the
following property: for every irreducible character
$\chi\in{\operatorname{Irr}}(G)$ there exists a normal subgroup $H$ of $G$
such that $\\{\chi\\}={\operatorname{Irr}}(G|\lambda)$, with $\lambda$ a
linear $G$-invariant character of $H$. These groups are called $nMI$-groups. A
variation of Taketa’s theorem (Theorem 3.18) (see 5.13 in [11]), considering
minimal $G$-invariant character degrees of certain hypercentral normal
subgroups of $G$, allows us to prove that if $G$ is a $nMI$-group, then $G$ is
nilpotent and the nilpotency class of $G$ is bounded by the number of distinct
irreducible character degrees of $G$ (Theorem 3.19). Moreover, if $G$ is a
$nMI$-group, all normal subgroups of $G$ are $G$-invariant $nMI$-subgroups
(Definition 3.13).

In the sequel, all groups considered are finite. The notation and terminology
used are standard, and they are taken mainly from the books [11], [15] and
[10].

##  2\. Preliminaries

The group $G$ acts by conjugation on the set ${\operatorname{Irr}}(N)$ of
irreducible complex characters of $N$. If $\theta\in{\operatorname{Irr}}(N)$,
then $\theta^{g}\in{\operatorname{Irr}}(N)$ and
$\widehat{\theta}=\sum_{i=1}^{t}\theta^{g_{i}}$ is a minimal $G$-invariant
character of $N$, where $\\{\theta^{g_{i}}\mid
i=1,\ldots,t\\}=\\{\theta^{g}\mid g\in G\\}$ is the orbit of $\theta$ under
the action by conjugation of $G$ on ${\operatorname{Irr}}(N)$, while the set
$\\{g_{i}\in G\mid i=1,\ldots,t\\}$ is a right transversal in $G$ of
$I_{G}(\theta)=\\{g\in G\mid\theta^{g}=\theta\\}$, the inertia subgroup of
$\theta$ in $G$. Clifford’s theorem (see [11, Theorem (6.2)]) states that
irreducible characters $\chi$ of $G$ whose restrictions $\chi_{N}$ to $N$ have
$\theta$ as constituent satisfy $\chi_{N}=e\,\widehat{\theta}$ for suitable
integers $e$, known as ramification numbers. This fact introduces an
equivalence relation (with respect to $N$) on the set
${\operatorname{Irr}}(G)$ of irreducible complex characters of $G$, being two
elements equivalent if their restrictions to the normal subgroup $N$ have a
common irreducible constituent.

We recall the main notation used in Notation 4.5 of [6] about character
G-tables.

######  Notation 2.1.

Let $N$ be a normal subgroup of a group $G$. Denote:

  * •

$\\{n_{1}^{G},\ldots,n_{k}^{G}\\}$ the set of $G$-conjugacy classes of $N$;

  * •

$\textsf{D}=(d_{ij})$ a diagonal matrix with entries
$d_{ij}=\delta_{ij}|n_{i}^{G}|$, where $\delta_{ij}$ is the Kronecker delta
function, for $i,j\in\\{1,\ldots,k\\}$;

  * •

$\Delta=\\{\chi_{1}=1_{G},\chi_{2},\ldots,\chi_{k}\\}$ a representative system
of the equivalence classes in the equivalence relation with respect to $N$,
defined on ${\operatorname{Irr}}(G)$;

  * •

$\Omega=\\{\theta_{i}\in{\operatorname{Irr}}(N)\mid 1\leq i\leq k\\}$ such
that $\chi_{i}\in{\operatorname{Irr}}(G|\theta_{i})$, for each
$i\in\\{1,\ldots,k\\}$;

  * •

$t_{i}=|G:I_{G}(\theta_{i})|$, $e_{i}=[(\chi_{i})_{N},\theta_{i}]\neq 0$,
$1\leq i\leq k$.

  * •

$\textsf{X}=(x_{ij})\in\textup{M}_{k}(\mathbb{C})$, with entries
$x_{ij}=\chi_{i}(n_{j})$ for $1\leq i,j\leq k$, is the $G$-character table of
$N$ constructed from $\Delta$, and $\overline{\textsf{X}}^{t}$ its transposed
conjugate matrix; i.e if $\textsf{X}=(x_{ij})$, then
$\overline{\textsf{X}}^{t}=(y_{ij})$ being $y_{ij}=\overline{x_{ji}}$ the
complex conjugate of $x_{ji}$, for $1\leq i,j\leq k$.

  * •

$\Lambda_{\textsf{X}}=\text{diag}(\lambda_{1},\ldots,\lambda_{k})$ the
diagonal matrix with entries $\lambda_{i}=\left|N\right|t_{i}e_{i}^{2}$,
$1\leq i\leq k.$

######  Theorem 2.2.

With Notation 2.1, it holds that
$\Lambda_{\textsf{X}}=\textsf{X}\textsf{D}\overline{\textsf{X}}^{t}.$ In
particular, X is non-singular.

As a consequence of Theorem 2.2, it is obtained that the equivalence classes
of irreducible characters of $G$ are in one-to-one correspondence with the
orbits of irreducible characters of $N$ under the action of $G$ by
conjugation, by Clifford’s theorem, and we can recognize the sets
${\operatorname{Irr}}(G|\theta)$, for every
$\theta\in{\operatorname{Irr}}(N)$, from the character table of $G$. Moreover,
the following arithmetical relations between the aforementioned integers
$t_{i}$, $e_{i}$ and $\theta_{i}(1)$.

######  Corollary 2.3.

With Notation 2.1, the next integer relations hold, where the corresponding
right sides can be computed from the $G$-character table X of $N$:

($A_{i}$) $\displaystyle e_{i}^{2}t_{i}$
$\displaystyle=\dfrac{\lambda_{i}}{\left|N\right|}$ ($B_{i}$) $\displaystyle
t_{i}\theta_{i}(1)^{2}$
$\displaystyle=\dfrac{|N|\chi_{i}(1)^{2}}{\lambda_{i}}$

for $1\leq i\leq k$.

##  3\. Main results

In general, the nilpotency and solvability of a normal subgroup $N$ of a group
$G$ can be obtained from the character table of $G$. However, it is not
possible to know the derived length when $N$ is solvable or the nilpotency
class when $N$ is nilpotent. In fact, we are not able to find out if $N$ is
abelian by the character table of $G$. (it is well-known that Problem 10 of
Brauer’s famous list in [3] has a negative answer).

######  Proposition 3.1.

Let $N$ be a normal subgroup of $G$. The character table of $G$ determines if
$N$ is nilpotent or solvable.

Proof. Let $p$ be a prime divisor of the order of $N$. By Higman’s theorem
(see 8.21 of [11]), the character table of $G$ determines the sets of prime
divisors of orders of elements of $G$. In fact, an element $x\in N$ is a
$p$-element if and only if

$\chi(x)^{|G|_{p}}\equiv\chi(1)\text{ (mod $p$)},$

for all $\chi\in{\operatorname{Irr}}(G)$ (see Corollary 5.3 of [2]).
Therefore, from the character table of $G$ we can realize which $G$-conjugacy
classes of $N$ contain elements with $p$-power order. Let $P$ be a Sylow
$p$-subgroup of $N$ and $T=\dot{\bigcup}_{x\in P}x^{G},$ the disjoint union of
$G$-conjugacy clases of $p$-elements of $N$. The $G$-conjugacy class size
$|x^{G}|$, for $x\in P$, can be obtained from the character table of $G$. Then
$P$ is normal in $N$ if and only if $|T|+1=|P|$. Hence, the character table of
$G$ allows us to know the nilpotency of $N$.

We observe that $N$ is solvable if and only if there exists a series

$1=N_{0}\unlhd N_{1}\unlhd N_{2}\unlhd\ldots\unlhd N_{r}=N$

such that $N_{i}\unlhd G$, for $i\in\\{0,\ldots,r\\}$, and $N_{i}/N_{i-1}$ is
nilpotent, for $i\in\\{1,\ldots,r\\}$. Since we can check if the quotient
groups $N_{i}/N_{i-1}$ are nilpotent from the character table of $G/N_{i-1}$
(obtained from the character table of $G$), then we can deduce if $N$ is
solvable or not from the character table of $G$.

We consider the following set of irreducible characters of $N$

$\operatorname{Lin}_{G}(N)=\\{\theta\in{\operatorname{Irr}}(N)\text{ }|\text{
}\theta\text{ is a linear }G\text{-invariant character of }N\\}.$

######  Proposition 3.2.

Let $N$ be a normal subgroup of a group $G$,
$\theta\in{\operatorname{Irr}}(N)$ and
$\chi\in{\operatorname{Irr}}(G|\theta)$. Then $[N,G]\subseteq\ker(\chi)$ if
and only if $\theta\in\operatorname{Lin}_{G}(N)$. In particular,
$|N:[N,G]|=|\operatorname{Lin}_{G}(N)|$.

Proof. Firstly, we prove that
$\operatorname{Lin}_{G}(N)=\\{\theta\in{\operatorname{Irr}}(N)\text{ }|\text{
}[G,N]\subseteq\ker(\theta)\\}.$ Let $[n,g]$ be a generator of $[N,G]=[G,N]$,
with $g\in G$ and $n\in N$. Let $\theta\in\operatorname{Lin}_{G}(N)$, then

$\theta([g,n])=\theta(g^{-1}n^{-1}gn)=\theta^{g^{-1}}(n^{-1})\theta(n)=\theta(n^{-1})\theta(n)=\theta(n^{-1}n)=\theta(1)=1.$

Then $[N,G]\subseteq\ker(\theta).$

Reciprocally, if $\theta\in{\operatorname{Irr}}(N)$ and
$[N,G]\subseteq\ker(\theta)$, then $\theta$ is a linear character of $N$ since
$N^{\prime}\subseteq\ker(\theta)$. Moreover, if $g\in G$ and $n\in N$, then

$1=\theta(1)=\theta(g^{-1}n^{-1}gn)=\theta(g^{-1}n^{-1}g)\theta(n).$

Therefore

$\theta^{g^{-1}}(n^{-1})\theta(n)=1$

for every $n\in N$ and $g\in G$. Then $\theta^{g}(n)=\theta(n),$ for every
$n\in N$ and $g\in G$ and $\theta$ is a $G$-invariant character of $N$; that
is $\theta\in\operatorname{Lin}_{G}(N)$. In particular, if
$[N,G]\subseteq\ker(\chi)$ for some $\chi\in{\operatorname{Irr}}(G|\theta)$
with $\theta\in{\operatorname{Irr}}(N)$, then $[G,N]\subset\ker(\theta)$ and
$\theta\in\operatorname{Lin}_{G}(N)$.

Moreover,

${\operatorname{Irr}}(N/[G,N])=\\{\theta\in{\operatorname{Irr}}(N)\mid[G,N]\subseteq\ker(\theta)\\}=\operatorname{Lin}_{G}(N)$

and $|N:[N,G]|=|\operatorname{Lin}_{G}(N)|$, since $N/[G,N]$ is abelian.

######  Corollary 3.3.

Let $N$ be a normal subgroup of a group $G$, then the subgroup $[N,G]$ and the
sets ${\operatorname{Irr}}(G|\theta)$, with
$\theta\in\operatorname{Lin}_{G}(N)$, are determined from the character table
of $G$.

Proof. Let $\Delta$ be a representative system of the equivalence classes in
the equivalence relation with respect to $N$ defined on
${\operatorname{Irr}}(G)$. By Corollary 2.3$(B_{i})$, we have
$\theta\in\operatorname{Lin}_{G}(N)$ if and only if

$\lambda_{i}=|N|\chi_{i}(1)^{2}$

where $\lambda_{i}$ is an element of the main diagonal of the matrix
$\Lambda_{X}$ obtained from a $G$-character table $X$ of $N$, with
$\chi_{i}\in\Delta$ such that $[(\chi_{i})_{N},\theta]\neq 0$. As a
consequence, we can read from the correspondent $G$-character table $X$ of $N$
if a constituent of $(\chi_{i})_{N}$ is a linear $G$-invariant character of
$N$. Observe that

$[N,G]=\bigcap_{\theta\in\operatorname{Lin}_{G}(N)}\ker(\theta)=\bigcap_{\scriptsize\begin{array}[]{c}\theta\in\operatorname{Lin}_{G}(N)\\\
\chi\in{\operatorname{Irr}}(G|\theta)\cap\Delta\end{array}}\ker(\chi)\cap N.$

Therefore, the subgroup $[N,G]$ can also be determined from the character
table of $G$.

Next, we introduce some notation and some elementary results on hypercentral
subgroups that we will use later. Let $N$ be a normal subgroup of $G$. A
$G$-series of $N$ is a normal series of $N$

$1=N_{0}\trianglelefteq N_{1}\trianglelefteq\cdots\trianglelefteq N_{s}=N$

with $N_{i}\trianglelefteq G$, for $i\in\\{1,\ldots,s\\}$. In particular, we
said that a $G$-series of $N$ is a chief $G$-series if $N_{i}/N_{i-1}$ is a
chief factor of $G$, for every $i\in\\{1,\ldots,s\\}$.

######  Definition 3.4.

Let $N$ be a normal subgroup of a group $G$. The descending $G$-series

$\Gamma_{G}^{1}(N)=N\trianglerighteq\Gamma_{G}^{2}(N)=[\Gamma_{G}^{1}(N),G]\trianglerighteq\Gamma_{G}^{3}(N)=[\Gamma_{G}^{2}(N),G]\trianglerighteq\ldots$

is called lower central $G$-series of $N$.

If there exists some integer $r$ such that $\Gamma_{G}^{r}(N)=1$, then $N$ is
a hypercentral subgroup of $G$. As a consequence of Corollary 3.3 we obtain
the following.

######  Corollary 3.5.

Let $N$ be a normal subgroup of a group $G$, then the character table of $G$
determines the $G$-hypercentre of $N$. In particular, if $N$ is a hypercentral
subgroup of $G$, the character table of $G$ determines lower central
$G$-series of $N$.

In the literature, there exist different equivalent descriptions and several
results relating to the hypercenter of a group (see, for instance, [1],
Section 6 in IV of [5], [13], [14] and Section 6 in Appendix C of [17]). With
notation in [14], we consider the hypercenter $H(G)$ of $G$ as the upper
ascending central series of $G$.

######  Definition 3.6.

We consider the ascending $G$-series of a normal subgroup $N$:

$1=\textbf{Z}_{G}^{0}(N)\trianglelefteq\textbf{Z}_{G}^{1}(N)=\textbf{Z}(G)\cap
N\trianglelefteq\textbf{Z}_{G}^{2}(N)\trianglelefteq\ldots$

such that

$\textbf{Z}_{G}^{i+1}(N)/\textbf{Z}_{G}^{i}(N)=\textbf{Z}(G/\textbf{Z}_{G}^{i}(N))\cap
N/\textbf{Z}_{G}^{i}(N),$

for $i=1,2,\ldots$. This series is called the upper central $G$-series of $N$.
We define the $G$-hypercenter of $N$ as
$H_{G}(N)=\bigcup\limits_{i}\textbf{Z}_{G}^{i}(N)$. We have that $H(G)\cap
N=H_{G}(N)\subseteq H(N)$.

The following result is elementary and can be proven by induction easily.

######  Proposition 3.7.

Let $N$ be a subgroup of $G$. If there exists an integer $r\geq 1$ such that
$\Gamma^{r}_{G}(N)=1$ in the lower central $G$-series of $N$, then
$\textbf{Z}_{G}^{r-1}(N)=N$ in the upper central $G$-series of $N$.
Reciprocally, if $\textbf{Z}_{G}^{r}(N)=N$, for some integer $r\geq 1$, then
$\Gamma^{r+1}_{G}(N)=1$.

The above proposition allows us to define the concept of the hypercentral
$G$-length of a normal subgroup of a group $G$.

######  Definition 3.8.

Let $N$ be a hypercentral normal subgroup of a finite group $G$. The least
integer $k$ such that $\Gamma^{k+1}_{G}(N)=1$ is equal to the least integer
$k$ such that $\textbf{Z}_{G}^{k}(N)=N$ and $k$ is called the hypercentral
$G$-length of $N$. We denote this integer by $\textbf{l}_{G}(N)$.

In [8], it is proved that the Frattini subgroup of a solvable group can be
determined from the character table; however, this fact does not happen when
the group is not solvable. If $N$ is a hypercentral normal subgroup in $G$,
then $N$ is nilpotent and $[N,N]\leq\Phi(N)\leq\Phi(G)\cap N$, where $\Phi(G)$
is the Frattini subgroup of $G$, but the subgroups $[N,N]$ and $\phi(N)$ can
not be read from the character table of $G$. Using induction on the
hypercentral $G$-length of hypercentral normal subgroups of a group $G$, we
obtain the following result.

######  Corollary 3.9.

Let $N$ be a hypercentral subgroup in $G$. Then $[G,N]\leq\Phi(G)\cap N$.

Proof. We argue by induction on the hypercentral $G$-length of $N$, which is
donated by $r$. If $r=1$, then $[G,N]=1$ and the result is trivial. Suppose
that $N_{1}=[G,N]\neq 1$. Since $N$ is a hypercentral subgroup in $G$, then
$N_{1}<N$ and $N_{1}$ is a hypercentral subgroup in $G$. The hypercentral
$G$-length of $N_{1}$ is less than or equal to $r-1$. By induction,
$[G,N_{1}]\leq\Phi(G)\cap N_{1}\leq\Phi(G)\cap N.$

Let $M$ be a maximal subgroup of $G$, we have $[G,N_{1}]\leq M\cap N$. We
claim that $N_{1}\leq M$. Assume that $G=N_{1}M$, then $N=N_{1}(M\cap N)$ and
$[G,N]=[G,N_{1}][G,M\cap N]$. Since $M\cap N$ is normal in $M$, we have
$[M,M\cap N]$ is a subgroup of $M\cap N$. Moreover, $[N_{1},M\cap
N]\leq[N_{1},G]\leq M\cap N$. Therefore, $[G,M\cap N]=[M,M\cap N][N_{1},M\cap
N]\leq M\cap N$. As a consequence, $N_{1}=[G,N]=[G,N_{1}][G,M\cap N]\leq M\cap
N$, a contradiction, and the proof is complete.

We point out that it is not difficult to find examples that show that the
reciprocal implication of the above result is not true (for instance,
$G=D_{18}$ and $N=\Phi(G)$).

Now, we denote by

$\operatorname{Min}_{G}(N)=\\{\widehat{\theta}\mid\theta\in{\operatorname{Irr}}(N)\\}=\\{\widehat{\theta}_{i}\mid
i=1,\dots,k\\}$

the set of minimal $G$-invariant characters of $N$. By Corollary 2.3$(B_{i})$,
we have

$\widehat{\theta_{i}}(1)\theta_{i}(1)=\dfrac{|N|\chi_{i}(1)^{2}}{\lambda_{i}},$

which is obtained from a $G$-character table $X$ of $N$. Then, prime divisors
of $\widehat{\theta}(1)$ are known from this equation, although we can not
find out the prime divisors of $\theta(1)$. In particular, if a prime $p$ does
not divide $\widehat{\theta}(1)$ for every minimal $G$-invariant character
$\widehat{\theta}$ of $N$, then we can deduce that $N$ has an abelian normal
Sylow $p$-subgroup by Itô-Michler’s theorem (see Corollary 4.11 of [6]).

However, we consider the opposite situation. Assume there exists a prime $p$
divides $\widehat{\theta}(1)$ for every
$\widehat{\theta}\in\operatorname{Min}_{G}(N)\setminus\operatorname{Lin}_{G}(N)$.
We observe that this fact does not imply that $p$ divides every irreducible
character degree of $N$. In this situation, we obtain the following
generalization of the well-known Thompson’s theorem for minimal $G$-invariant
characters of $N$.

We keep the notation of Theorem 12.1 of [11]. Let

$\mathscr{S}(N)=\\{\theta\in{\operatorname{Irr}}(N)\text{ }|\text{
}p\nmid\theta(1)\text{ and }p\nmid o(\theta)\\}.$

and $s(N)=\sum_{\theta\in\mathscr{S}(N)}\theta(1)^{2}$. By Theorem 12.1 of
[11], we have

$|{\bf O}^{p}(N)|\equiv s(N)\text{ }(\text{mod }p).$

We consider the set
$\mathscr{S}_{G}(N)=\\{\theta\in\operatorname{Lin}_{G}(N)|\text{ }p\nmid
o(\theta)\\}\subseteq\mathscr{S}(N).$

######  Theorem 3.10.

Let $p$ be a prime integer. Suppose $p$ divides $\widehat{\theta}(1)$ for
every
$\widehat{\theta}\in\operatorname{Min}_{G}(N)\setminus\operatorname{Lin}_{G}(N)$,
then $N$ has a normal $p$-complement.

Proof. We have

$s(N)=\sum_{\theta\in\mathscr{S}(N)}\theta(1)^{2}=\sum_{\theta\in\mathscr{S}_{G}(N)}\theta(1)^{2}+\sum_{\theta\in\mathscr{S}(N)\setminus\mathscr{S}_{G}(N)}\theta(1)^{2}.$

If $\theta\in\mathscr{S}(N)\setminus\mathscr{S}_{G}(N)$, then $p$ divides the
index of the inertia group $I_{G}(\theta)$ of $\theta$ in $G$. Moreover, every
$G$-conjugate character of $\theta$ is also in the set
$\mathscr{S}(N)\setminus\mathscr{S}_{G}(N)$. Therefore,

$\sum_{\theta\in\mathscr{S}(N)\setminus\mathscr{S}_{G}(N)}\theta(1)^{2}\equiv
0\text{ }(\text{mod }p)$

and then

$s(N)\equiv\sum_{\theta\in\mathscr{S}_{G}(N)}\theta(1)^{2}\text{ }(\text{mod
}p).$

If $\theta\in\mathscr{S}_{G}(N)$, then $\theta\in\operatorname{Lin}_{G}(N)$
and $p\nmid o(\theta)=|N:\ker(\theta)|$. By Proposition 3.2 we have
$[G,N]\subseteq\ker(\theta)$. Then $[G,N]{\bf
O}^{p^{\prime}}(N)\subseteq\ker(\theta)$ and
$\theta\in{\operatorname{Irr}}(N/[G,N]{\bf O}^{p^{\prime}}(N))$. Reciprocally,
if $\theta\in{\operatorname{Irr}}(N/[G,N]{\bf O}^{p^{\prime}}(N))$, then
$N^{\prime}\subseteq[G,N]\subseteq\ker(\theta)$ and ${\bf
O}^{p^{\prime}}(N)\subseteq\ker(\theta)$. We conclude that
$\theta\in\operatorname{Lin}_{G}(N)$ and $p\nmid o(\theta)$, and then
$\theta\in\mathscr{S}_{G}(N)$. Therefore,

$\sum_{\theta\in\mathscr{S}_{G}(N)}\theta(1)^{2}=|N:[G,N]{\bf
O}^{p^{\prime}}(N)|$

and this integer is a $p^{\prime}$-number. By Theorem 12.1 of [11],

$|{\bf O}^{p}(N)|\equiv s(N)\text{ }(\text{mod
}p)\equiv\sum_{\theta\in\mathscr{S}_{G}(N)}\theta(1)^{2}\text{ }(\text{mod
}p).$

Therefore, $p$ does not divide $|{\bf O}^{p}(N)|$ and ${\bf O}^{p}(N)$ is a
normal $p$-complement of $N$.

######  Example 3.11.

It is not difficult to find examples satisfying the previous theorem. For
instance, using GAP, the group $G=SmallGroup(120,3)$ has a cyclic normal
subgroup $N$ of order 20 and we can check from Corollary 2.3$(B_{i})$, that
$p=2$ divides $\widehat{\theta}(1)$ for every
$\widehat{\theta}\in\operatorname{Min}_{G}(N)\setminus\operatorname{Lin}_{G}(N)$.
Then, we deduce that $N$ has a normal $2$-complement by Theorem 3.10. In fact,
the unique prime number dividing a degree of a minimal $G$-invariant character
of $N$ is $2$. Therefore, a minimal $G$-invariant character of $N$ has either
degree 1 (which means it is a linear $G$-invariant character of $N$) or
$2$-power degree. However, in relation to Thompson’s theorem, $p=2$ does not
divide every irreducible character degree of $N$ since $N$ is abelian.

In [12], using the classification of nonabelian simple groups, it is proved
that $MI$-groups are solvable. A group $G$ is said to be a $MI$-group if every
nonlinear irreducible character of $G$ is a multiply imprimitive character or
m.i. character for short (that is, for every irreducible character
$\chi\in{\operatorname{Irr}}(G)$ there exists a proper subgroup $U$ of $G$ and
an irreducible character $\lambda\in{\operatorname{Irr}}(U)$ such that
$\lambda^{G}=m\chi$ for some nonnegative integer $m$). We introduce the
following definition.

######  Definition 3.12.

Let $G$ be a finite group and $\chi\in{\operatorname{Irr}}(G)$. We say that
$(H_{\chi},\lambda_{\chi})$ is a linear character pair with respect to $\chi$
if $H_{\chi}$ is a normal subgroup of $G$,
$\lambda_{\chi}\in\operatorname{Lin}_{G}(H_{\chi})$ and
${\operatorname{Irr}}(G|\lambda_{\chi})=\\{\chi\\}$ (so $\chi$ is m.i.
character with notation in [12]).

Recall that a group $G$ is a monomial group or $M$-group if for every
irreducible character $\chi\in{\operatorname{Irr}}(G)$ there exists a subgroup
$H$ and a linear irreducible character $\varphi\in{\operatorname{Irr}}(H)$
such that $\chi=\varphi^{G}$. By Taketa’s theorem (theorem 5.12 and 5.13 in
[11]), every $M$-group is solvable and the derived length of $G$ is bounded by
the number of irreducible character degrees of $G$. Structural properties of
$nM$-groups (and $sM$-groups) were introduced in [9] as a generalization of
$M$-groups. A group $G$ is said to be a $nM$-group if for every irreducible
character $\chi\in{\operatorname{Irr}}(G)$ there exists a normal subgroup $H$
and a linear irreducible character $\varphi\in{\operatorname{Irr}}(H)$ such
that $\chi=\varphi^{G}$. Observe that in this situation, we have
$\chi(1)=|G:H|$ and, using the Frobenius reciprocity, the ramification number
of $\chi$ with respect to $N$ is equal to $1$.

Now, we consider the following class of groups as a variation of $nM$-groups
and $MI$-groups.

######  Definition 3.13.

A group $G$ is said to be a $nMI$-group if for every irreducible character
$\chi\in{\operatorname{Irr}}(G)$ there exists $(H_{\chi},\lambda_{\chi})$ a
linear character pai rwith respect to $\chi$.

Trivially, every abelian group $G$ is an $nMI$-group considereing
$\lambda_{\chi}^{G}=\chi$ for every irreducible character
$\chi\in{\operatorname{Irr}}(G)$, and reciprocally. Moreover, the character
table determines if a group $G$ is a $nMI$-group.

######  Corollary 3.14.

A finite group $G$ is $nMI$-group if and only if for every
$\chi\in{\operatorname{Irr}}(G)$ there exists a normal subgroup $N$ of $G$
such that $[G,N]\subseteq\ker(\chi)$ and the respective element of the
principal diagonal $\Lambda_{X}$ from a $G$-character table $X$ of $N$
associated to $\chi$ is equal to $|G|$.

Proof. It is a consequence of Corollary 2.3$(B_{i})$ and Proposition 3.2.

In a more general context, we present the following definitions for normal
subgroups.

######  Definition 3.15.

Let $N$ be a normal subgroup of a group $G$ and
$\theta\in{\operatorname{Irr}}(N)$. We say that
$(H_{\theta},\lambda_{\theta})$ is a linear $G$-character pair with respect to
$\theta$ if $H_{\theta}$ is a normal subgroup of $G$ contained in $N$,
$\lambda_{\theta}\in\operatorname{Lin}_{G}(H_{\theta})$ and
${\operatorname{Irr}}(G|\theta)={\operatorname{Irr}}(G|\lambda_{\theta})$.

Notice that the last condition above is equivalent to
$(\lambda_{\theta})^{N}=m\widehat{\theta}$ (and then the restriction
$\theta_{H_{\theta}}=m\lambda_{\theta}$) for some positive interger $m$.

######  Definition 3.16.

A normal subgroup $N$ of a group $G$ is said to be a $G$-invariant
$nMI$-subgroup if for every $\theta\in{\operatorname{Irr}}(N)$, there exists
$(H_{\theta},\lambda_{\theta})$ a linear $G$-invariant character pai withr
respect to $\theta$. In particular, when $N=G$ we have that $G$ is a
$nMI$-group.

######  Corollary 3.17.

The character table of $G$ determines if a normal subgroup $N$ of $G$ is a
$G$-invariant $nMI$-subgroup. In this case, if
$\chi\in{\operatorname{Irr}}(G|\theta)$, with
$\theta\in{\operatorname{Irr}}(N)$ and $(H_{\theta},\lambda_{\theta})$ is a
linear $G$-character pair respect to $\theta$, then
$[G,H_{\theta}]\subseteq\ker(\chi)$ and the respective element of the
principal diagonal $\Lambda_{X}$ from a $G$-character table $X$ of $N$
associated to $\chi$ is equal to $|H_{\theta}|\chi(1)^{2}$.

Proof. We obtain the result by using Corollary 2.3$(B_{i})$ and Proposition
3.2 and the fact that the equivalence classes in the equivalence relation with
respect to $N$, defined on ${\operatorname{Irr}}(G)$, are detected in the
character table of the group $G$ (Corollary 4.6 of [6]).

Let

$\operatorname{Min}_{G}(N)=\\{\widehat{\theta}\mid\theta\in{\operatorname{Irr}}(N)\\}=\\{\widehat{\theta}_{i}\mid
i=1,\dots,k\\}$

be the set of minimal $G$-invariant characters of $N$. For convenience, we
make the following notation

$\operatorname{Mcd}_{G}(N)=\\{\widehat{\theta}(1)\mid\widehat{\theta}\in\operatorname{Min}_{G}(N)\\}=\\{\widehat{\theta}_{i}(1)\mid
i=1,\dots,k\\}$

and by

$\operatorname{Lcd}_{G}(N)=\\{\theta(1)\widehat{\theta}(1)\mid\theta\in{\operatorname{Irr}}(N)\\}=\\{\theta_{i}(1)\widehat{\theta}_{i}(1)\mid
i=1,\dots,s\\}.$

The next theorem is a variation of Taketa’s theorem for $G$-invariant
$nMI$-subgroups.

######  Theorem 3.18.

Let $N$ be a $G$-invariant $nMI$-subgroup of a group $G$ and let
$1=f_{1}<f_{2}<\cdots<f_{s}$ be the distinct elements of
$\operatorname{Lcd}_{G}(N)$. Then

$\Gamma^{i}_{G}(N)\subseteq\ker(\widehat{\theta_{i}})$

with $\theta_{i}(1)\widehat{\theta_{i}}(1)=f_{i}$, for $i\in\\{1,\ldots,s\\}$.
In particular, $N$ is a hypercentral subgroup of $G$ and
$\textbf{l}_{G}(N)\leq s=|\operatorname{Lcd}_{G}(N)|$.

Proof. We prove the theorem, using induction on $G$. Let $N_{0}$ be any normal
subgroup of $G$ contained in $N$. We claim that $N/N_{0}$ satisfies the
hypothesis of the theorem. Let $\theta\in{\operatorname{Irr}}(N/N_{0})$, then
there exists a $G$-character pair with respect to $\theta$, say
$(H_{\theta},\lambda_{\theta})$. Note that $\lambda_{\theta}\in{\rm
Lin}_{G}(H_{\theta})$ and $\lambda_{\theta}^{N}=m\widehat{\theta}$, for some
integer $m$. Hence,

$\ker\widehat{\theta}=\ker(\lambda_{\theta})^{N}=\bigcap\limits_{x\in
N}(\ker\lambda_{\theta})^{x}=\bigcap\limits_{x\in
N}\ker\lambda_{\theta}^{x}=\ker\lambda_{\theta}.$

Then $N_{0}\leq\ker\widehat{\theta}=\ker\lambda_{\theta}\leq H_{\theta}$,
which means that $\lambda_{\theta}\in\operatorname{Lin}_{G}(H_{\theta}/N_{0})$
and so our claim is proved.

Suppose that $\ker\widehat{\theta}\not=1$ and we work on
$\bar{G}=G/\ker\widehat{\theta}$ by adopting bar for the subgroups. If
$\theta(1)\widehat{\theta}(1)=f_{i}$, for some $1\leq i\leq s$, then
$\theta(1)\widehat{\theta}(1)$ is j-th smallest degree of ${\rm
Lcd}_{\bar{G}}(\bar{N})$, for some $j\leq i$. By induction, we have

$\Gamma^{i}_{G}(N)\ker\widehat{\theta}/\ker\widehat{\theta}\leq\Gamma^{j}_{G}(N)\ker\widehat{\theta}/\ker\widehat{\theta}=\Gamma^{j}_{\bar{G}}(\bar{N})\leq\ker\widehat{\theta}/\ker\widehat{\theta}=1.$

Therefore, $\Gamma^{i}_{G}(N)\subseteq\ker(\widehat{\theta_{i}})$ as wanted.

We may assume $\widehat{\theta}$ is faithful. In this case, we claim that
$H_{\theta}={\bf Z}(G)\cap N$. Since
$\ker\lambda_{\theta}=\ker\lambda_{\theta}^{N}=\ker\widehat{\theta}=1$, then

$[H_{\theta},G]=\bigcap\limits_{\lambda\in{\rm
Lin}_{G}(H_{\theta})}\ker\lambda\leq\ker\lambda_{\theta}=1.$

Thus, $H_{\theta}\leq{\bf Z}(G)\cap N$. On the other hand,
$\widehat{\theta}|_{{\bf Z}(G)\cap N}=m\beta$, for some
$\beta\in{\operatorname{Irr}}({\bf Z}(G)\cap N)$. If $x\in{\bf Z}(G)\cap
N\setminus H_{\theta}$, then
$\lambda_{\theta}^{N}(x)=0=\widehat{\theta}(x)=m\beta(x)$, which means that
$\beta(x)=0$ and this is not possible. Therefore, ${\bf Z}(G)\cap
N=H_{\theta}$. Moreover,

$m\widehat{\theta}(1)=\lambda_{\theta}^{N}(1)=|N:{\bf Z}(G)\cap N|$

and $m=\theta(1)$. Adobting tilde for convention $\tilde{G}=G/({\bf Z}(G)\cap
N)$, we have $\theta(1)\widehat{\theta}(1)=|\tilde{N}|$. On the other side,

$|\tilde{N}|=\sum\limits_{\chi\in{\operatorname{Irr}}(\tilde{N})}\chi^{2}(1)=\sum\limits_{\widehat{\chi}\in{\rm
Min}_{\tilde{G}}(\tilde{N})}\chi(1)\widehat{\chi}(1).$

This implies that $\theta(1)\widehat{\theta}(1)>\chi(1)\widehat{\chi}(1)$ for
all $\chi\in{\operatorname{Irr}}(\tilde{N})$. Let $t=|{\rm
Lcd}_{\tilde{G}}(\tilde{N})|$, it follows that $\theta(1)\widehat{\theta}(1)$
is k-th smallest number in $\operatorname{Lcd}_{G}(N)$, for some integer
$k\geq t+1$. By induction, we have

$({\bf Z}(G)\cap N)\Gamma^{t}_{G}(N)/{\bf Z}(G)\cap
N=\Gamma^{t}_{\tilde{G}}(\tilde{N})\leq\bigcap\limits_{\widehat{\chi}\in{\rm
Min}_{\tilde{G}}(\tilde{N})}\ker\widehat{\chi}=({\bf Z}(G)\cap N)/({\bf
Z}(G)\cap N).$

Hence, $\Gamma^{t+1}_{G}(N)=1=\ker\widehat{\theta}$ as wanted. So the proof is
complete.

######  Corollary 3.19.

If $G$ is a $nMI$-group and $1=\chi_{i}(1)<\chi_{2}(1)<\cdots<\chi_{n}(1)$ are
the distinct degrees of irreducible characters of $G$, then

$\Gamma_{i}(G)\leq Ker(\chi_{i}),$

for $i\in\\{1,\ldots,n\\}$. In particular, $G$ is nilpotent and the nilpotence
class of $G$ is bounded by $n$. Moreover, every normal subgroup of $G$ is a
$G$-invariant $nMI$-subgroup of $G$.

Proof. To prove this corollary we just need to replace $N$ by $G$ in Theorem
3.18. Clearly, in this case $\widehat{\chi}=\chi$ for
$\chi\in{\operatorname{Irr}}(G)$ and $|{\rm Lcd_{G}(G)}|=|{\rm cd}(G)|$. Thus
the first part of the theorem is done.

Let $N$ be a normal subgroup of $G$, we claim $N$ is a $G$-invariant
$nMI$-group of $G$. Let $\theta\in{\operatorname{Irr}}(N)$ and
$\chi\in{\operatorname{Irr}}(G|\theta)$. By hypothesis, there exists
$H\trianglelefteq G$ and a linear $G$-invariant character
$\varphi\in{\operatorname{Irr}}(H)$ such that $\varphi^{G}=m\chi$ and
$\chi_{H}=m\varphi$, for some positive integer $m$. It follows that
$\chi\in{\operatorname{Irr}}(N|\varphi)$.

Firstly, we assume $H\subseteq N$. If
$\theta_{1}\in{\operatorname{Irr}}(N|\varphi)$, then
$[\theta_{1}^{G},\chi]\neq 1$ and $[\chi_{N},\theta_{1}]\neq 1$, thus
$\widehat{\theta_{1}}=\widehat{\theta}$. Then, $\varphi^{N}=s\widehat{\theta}$
and $\theta_{H}=s\varphi$, for some positive integer $s$, and the proposition
is proved. Now, assume $H$ is not contained in $N$. We have
$\chi_{HN}=k\widehat{\lambda}$, with
$\lambda\in{\operatorname{Irr}}(HN|\varphi)$ and some integer $k\geq 0$.
Hence, $\lambda_{H}=r\varphi$ and $\varphi^{HN}=r\widehat{\lambda}$, for some
positive integer $r$. Moreover,
$\chi_{N}=e\widehat{\theta}=k\widehat{\lambda}_{N}$, for some integer $e\geq
0$. Then,

$\chi_{H\cap N}=e\widehat{\theta}_{H\cap N}=k\widehat{\lambda}_{H\cap
N}=kl\beta,$

for some positive integer $l$. Therefore,
$\theta\in{\operatorname{Irr}}(N|\beta)$ and $\theta_{H\cap N}=s\beta$, for
some $\beta\in{\operatorname{Irr}}(H\cap N)$ and some integer $s\geq 0$ .
Moreover, by Frobenius reciprocity, we have

$\beta^{N}=(\varphi_{H\cap
N})^{N}=(\varphi^{HN})_{N}=r\widehat{\lambda}_{N}=s\widehat{\theta},$

and the proof is complete.

######  Example 3.20.

Regarding the Corollary 3.19, we note that not all nilpotent groups verify
that the class of nilpotency is bounded by the number of degrees of
irreducible characters. For instance, using GAP, we can check that the group
$G=SmallGroup(32,9)$ has nilpotency class 3 and only has two irreducible
character degrees.

######  Example 3.21.

It is also not difficult to find groups that are $nMI$-groups and we can check
easily this property from the character table. For example, we consider in GAP
the character table of the group $G=SmallGroup(32,8)$, which has 11
irreducible characters. For every irreducible character
$\chi_{i}\in{\operatorname{Irr}}(G)$, for $i=\\{1,\ldots,11\\}$, there exists
a normal subgroup $N_{i}$ of $G$ and a linear character $G$-invariant
$\lambda_{i}\in{\operatorname{Irr}}(N_{i})$ such that
${\operatorname{Irr}}(G|\lambda_{i})=\\{\chi_{i}\\}$. Indeed, for linear
characters $\chi_{i}$ of $G$, for $i=1,\ldots,8$, this fact is trivial
considering $N_{i}=G$. The group $G$ has also two irreducible characters,
$\chi_{9}$ and $\chi_{10}$, of degree 2. Both characters are, respectively,
the unique irreducible constituent determined by induction in $G$ of certain
irreducible character $\lambda_{i}$, for $i\in\\{9,10\\}$, of the normal
subgroup

$N=N_{9}=N_{10}=\bigcap\limits_{i=1}^{4}\ker(\chi_{i}).$

We can check that $\lambda_{i}$ is a linear $G$-invariant character of $N$
taking acount the relations of Corollary 2.3($B_{i}$) obtained from
considering the corresponding $G$-character table of $N$ associated to
$\chi_{i}$, for $i\in\\{9,10\\}$. Finally, the irreducible character
$\chi_{11}$ of $G$ has degree $4$. For this character, we consider the normal
subgroup

$N_{11}=\bigcap\limits_{i=1}^{4}\ker(\chi_{i})\bigcap\ker(\chi_{10}).$

We have that $\chi_{11}$ is the unique irreducible constituent determined by
induction in $G$ of an irreducible character $\lambda_{11}$ of $N_{11}$. Using
the corresponding $G$-character table of $N_{11}$ associated to $\chi_{11}$,
and the relation Corollary 2.3$(B_{i})$, we can also check that $\lambda_{11}$
is a linear $G$-invariant character of $N_{11}$. Therefore,
$(N_{i},\lambda_{i})$ is a linear character pair with respect to $\chi_{i}$,
for $i\in\\{1,\ldots,11\\}$, and thus $G=SmallGroup(32,8)$ is an $nMI$-group.
In relation to Corollary 3.19, the nilpotency class of $G$ is exactly equal to
the number of irreducible character degrees of $G$ (that is equal to $3$).

######  Example 3.22.

The bound in the Theorem 3.18 is sharp. In fact, looking at group $G=Q_{16}$
and its normal subgroup $N=Q_{8}$, we can similarly check from the character
table of $G$ that $N$ is a $G$-invariant $nMI$-subgroup of $G$ but $G$ is not
$nMI$-group. In this case, we have that $\Gamma^{4}_{G}(N)=1$ and
$\Gamma^{i}_{G}(N)\not=1$ for $i<4$. Therefore, the hypercentral $G$-length of
$N$ is $\textbf{l}_{G}(N)=3$. Moreover, it is worth mentioning that
$2=|\operatorname{Mcd}_{G}(N)|=|\\{1,2\\}|<|{\rm Lcd_{G}(N)}|=|\\{1,2,4\\}|=3$
and so the $G$-hypercentral lenght of $N$ is not bounded by
$|\operatorname{Mcd}_{G}(N)|$.

Acknowledgment. This work was done during a visit of the first author at
Universitat Politècnica de València(UPV) and Universitat de València(UV). She
wishes to thank them for their hospitality. The results in this paper are part
of the third author’s Ph.D. thesis, and he acknowledges the support of
Instituto Tecnológico de Santo Domingo (INTEC) and Universidad Autónoma de
Santo Domingo (UASD), Santo Domingo, (Dominican Republic).

## References

  * [1] Baer R.: Group elements of prime power order, Trans. Amer. Math. Soc. 75, 20–47 (1953).
  * [2] Beltrán, A., Felipe, M.J., Malle G., Moretó, A., Navarro, G., Sanus, L., Solomon, R., Tiep, P.H.: Nilpotent and abelian Hall subgroups in finite groups, Trans. Amer. Math. Soc. 368 (4), 2497–2513 (2016).
  * [3] Brauer, R.: Representations of Finite Groups. Lectures on Modern Mathematics, Vol. I, 133–175, New York (1963). Reprinted in “R. Brauer: _Collected papers_ , Vol. II, 183–225, MIT Press, Cambridge (1980).
  * [4] Dockx,K., Igodt, P.: Character tables and commutativity of normal subgroups. ACM SIGSAM bulletin 25.1, 28–31 (1991).
  * [5] Doerk, K., Hawkes, T.: Finite soluble groups. De Gruyter (1992).
  * [6] Felipe, M.J., Pérez-Ramos, M.D., Sotomayor, V.: On $G$-character tables for normal subgroups. Quaest. Math. DOI: 102989/16073606/ 16073606.2022.2040633.
  * [7] The GAP Group: GAP – Groups, Algorithms, and Programming. Version 4.11.1 (2021). https://www.gap-system.org.
  * [8] Garrison, S.: Determining the Frattini subgroup from character table, Can. J. Math, Vol. XXVIII, No 3, 560-567 (1976).
  * [9] How, G.A.: Some classes of monomial groups, Bull. Austral. Math. Soc. 22, 477-478 (1980).
  * [10] Huppert, B.: Endliche Gruppen I. Springer-Verlag, Berlin-Heidelberd-New York-Tokyo (1983).
  * [11] Isaacs, I.M.: Character theory of finite groups. Academic Press Inc. London (1976).
  * [12] Le, T., Moori, J., Tong-Viet, H.P.: On a generalization of $M$-group, J. Algebra, 374, 27-41 (2013).
  * [13] Peng, T.A.: The hypercenter of a finite group, J. Algebra, 48, 56-46 (1977).
  * [14] Peng, T.A.: Hypercentral subgroups of finite groups, J. Algebra, 78, 431-436 (1982).
  * [15] Rose, J.S.: A course on group theory. Cambridge University Press, Cambridge (1978).
  * [16] Saksonov, A.I.: An answer to a question of R. Brauer. Vesci. Akad. Navuk. BSSR, Ser. Fiz.-Mat. (Russian). 1 129–130 (1967).
  * [17] Weinstein, W.: Between Nilpotent and solvable. Polygonal Publishing House, Passaic, NJ, USA (1982).


Again by (<ref>), we also have that

\begin{align*}
T^{s} Z
= ( x^{m' - c_1} T^{s + c_2 - c_1} A_1 + y^{m' - c_2} T^{s} A_2 )
\in H_{l' , m'} (\mathbfe{\alpha})
\end{align*}

for all $0 \leq s \leq m' - c_2 -1$. But then, using the second result in Lemma <ref> for the first relation below, we have that

\begin{align*}
\in H_{l' + (m' - c_2) , m' - (m' - c_2)} (\mathbfe{\alpha})
= H_{c_1 , c_2} (\mathbfe{\alpha})
= \Big\{ X A_1 : \substack{X \in \mathcal{A_1} \\ \degree X \leq c_2 - c_1 - 1} \Big\} .
\end{align*}

Recalling the definition of $Z$, we can see that this implies

\begin{align*}
\in \Big\{ X A_1 : \substack{X \in \mathcal{A} \\ \degree X \leq c_2 - c_1} \Big\} ,
\end{align*}

which contradicts (<ref>).

We now prove Theorem <ref>, which will be required for the proof of Theorem <ref> afterwards.

For what follows, we define $\mathbfe{\alpha}' := (\alpha_0 , \alpha_1 , \ldots , \alpha_{2 d_1})$.

Now, consider the matrix $H_{d_1 -1 , n+3 - d_1 } (\mathbfe{\alpha})$. It has kernel

\begin{align*}
\kernel H_{d_1 -1 , n+3 - d_1 } (\mathbfe{\alpha})
= \bigg\{ B_1 A_1 + B_2 A_2 : \substack{B_1 , B_2 \in \mathcal{A} \\ \degree B_1 \leq n+3 - 2 d_1 \\ \degree B_2 \leq 0} \bigg\} .
\end{align*}

Recall that $\degree A_1 = d_1$ and $\degree A_2 = d_2 < d_1$. In particular, the vectors associated to $A_1 , A_2$ in $H_{d_1 -1 , n+3 - d_1 } (\mathbfe{\alpha})$ have at least $n+2 - 2d_1$ zero entries at the end. Therefore, consider the matrix that we obtain by removing the last $n+2 - 2d_1$ columns of $H_{d_1 -1 , n+3 - d_1 } (\mathbfe{\alpha})$, which is $H_{d_1 -1 , d_1 +1} (\mathbfe{\alpha}')$. From the above we can see that

\begin{align*}
\bigg\{ B_1 A_1 + B_2 A_2 : \substack{B_1 , B_2 \in \mathcal{A} \\ \degree B_1 \leq 0 \\ \degree B_2 \leq 0} \bigg\}
\subseteq \kernel H_{d_1 -1 , d_1 +1} (\mathbfe{\alpha}') .
\end{align*}

Given that $H_{d_1 , d_1 } (\mathbfe{\alpha}') = H_{r , r } (\mathbfe{\alpha}')$ is invertible, we can see that the dimension of $\break \kernel H_{d_1 -1 , d_1 +1} (\mathbfe{\alpha}')$ is $2$ and so we must have equality above:

\begin{align} \label{statement, Theorem Hankel matrices incorporate Euclidean algorithm proof, kernel of H_(d_1 -1 , d_1 +1) (alpha')}
\kernel H_{d_1 -1 , d_1 +1} (\mathbfe{\alpha}')
= \bigg\{ B_1 A_1 + B_2 A_2 : \substack{B_1 , B_2 \in \mathcal{A} \\ \degree B_1 \leq 0 \\ \degree B_2 \leq 0} \bigg\} .
\end{align}

Now, we have $\degree A_2 = d_2 < d_1$, and so the polynomial associated to $A_2$ in $\kernel H_{d_1 -1 , d_1 +1} (\mathbfe{\alpha}')$ has $d_1 - d_2$ zeros entries at the end. What we do next depends on the size of $d_2$.

Case 1: If $d_2 \geq 2$, then $d_1 - d_2 < d_1 -1$. Thus, we can remove the last $d_1 - d_2$ columns from $\kernel H_{d_1 -1 , d_1 +1} (\mathbfe{\alpha}')$. This gives the matrix $\kernel H_{d_2 -1 , d_1 +1} (\mathbfe{\alpha}^{(2)} )$, and we can see that it has kernel

\begin{align*}
\kernel H_{d_2 -1 , d_1 +1} (\mathbfe{\alpha}^{(2)} )
= \bigg\{ B_1 A_1 + B_2 A_2 : \substack{B_1 , B_2 \in \mathcal{A} \\ \degree B_1 \leq 0 \\ \degree B_2 \leq d_1 - d_2} \bigg\} .
\end{align*}

The fact that the left side is a subset of the right side follows from (<ref>) and successive applications of Lemma <ref>. Equality then follows by noting that removing a row will increase the dimension by (at most) $1$. Also, we can see that $A_3$ is in the kernel of $H_{d_2 -1 , d_1 +1} (\mathbfe{\alpha}^{(2)} )$ and we can replace $A_1$ with $A_3$:

\begin{align*}
\kernel H_{d_2 -1 , d_1 +1} (\mathbfe{\alpha}^{(2)} )
= \bigg\{ B_2 A_2 + B_3 A_3: \substack{B_2 , B_3 \in \mathcal{A} \\ \degree B_2 \leq d_1 - d_2 \\ \degree B_3 \leq 0 } \bigg\} .
\end{align*}

We can now deduce that the characteristic polynomials of $\mathbfe{\alpha}^{(2)}$ are $A_2 , A_3$, and that $\mathbfe{\alpha}^{(2)} \in \mathscr{L}_{d_1 + d_2 -2} (d_2 , d_2 , 0)$.

Case 2: If $d_2 = 1$, then $d_1 - d_2 = d_1 -1$. Thus, we can only remove the last $d_1 - d_2 -1 = d_1 -2$ columns from $\kernel H_{d_1 -1 , d_1 +1} (\mathbfe{\alpha}')$. This gives the matrix $\kernel H_{d_2  , d_1 +1} (\mathbfe{\alpha}^{(2)} ) = \kernel H_{1  , d_1 +1} (\mathbfe{\alpha}^{(2)} )$ (recall that the definition of $\mathbfe{\alpha}^{(2)}$ differs between the cases), and we can see by similar means as in Case 1 that it has kernel

\begin{align*}
\kernel H_{1 , d_1 +1} (\mathbfe{\alpha}^{(2)} )
= \bigg\{ B_1 A_1 + B_2 A_2 : \substack{B_1 , B_2 \in \mathcal{A} \\ \degree B_1 \leq 0 \\ \degree B_2 \leq d_1 - d_2 -1} \bigg\} .
\end{align*}

Note that $B_2$ cannot have degree equal to $d_1 - d_2$, and so $A_3$ is not in the kernel above. Thus, we have that the characteristic polynomials are $A_2 , A_1$ (recall that the order matters). The fact that $B_2$ cannot have degree equal to $d_1 - d_2$ also tells us that $\pi (\mathbfe{\alpha}^{(2)} ) = 1$. We deduce that $\mathbfe{\alpha}^{(2)} \in \mathscr{L}_{d_1 } (2 , 1 , 1)$.

Case 3: This case is very similar to Case 2.

We now prove the uniqueness claim that is made in the theorem. We do so for Case 1, and the remaining cases are almost identical and only need to take into account the difference in definition of $\mathbfe{\alpha}^{(2)}$. To this end, consider the sequences

\begin{align*}
\mathbfe{\alpha}^{(2)} = &(\alpha_0 , \alpha_1 , \ldots , \alpha_{d_1 + d_2 -1}) \\
\mathbfe{\alpha}' = &(\alpha_0 , \alpha_1 , \ldots , \alpha_{d_1 + d_2 -1} , \alpha_{d_1 + d_2 } , \ldots , \alpha_{2 d_1} ) .
\end{align*}

Note that $\alpha_{d_1 + d_2 } , \ldots , \alpha_{2 d_1}$ form the last $d_1 - d_2 +1$ entries in the last column of the matrix $H_{d_1 -1 , d_1 +1} (\mathbfe{\alpha}')$, and by assumption we must have that $A_1$ is in the kernel of this matrix. Thus, for $d_1 + d_2 \leq i \leq 2 d_1$, the $i$-th row of $H_{d_1 -1 , d_1 +1} (\mathbfe{\alpha}')$ is orthogonal to the vector associated with $A_1$. Since this vector has non-zero final entry, we can express the last entry in the $i$-th row in terms of the previous entries. An inductive argument proves that the entries $\alpha_{d_1 + d_2 } , \ldots , \alpha_{2 d_1}$ can be uniquely determined in terms of the entries $\alpha_0 , \alpha_1 , \ldots , \alpha_{d_1 + d_2 -1}$.

Now consider the sequence

\begin{align*}
\mathbfe{\alpha} = &(\alpha_0 , \alpha_1 , \ldots , \alpha_{2 d_1} , \alpha_{2 d_1 +1} , \ldots , \alpha_{n} ) ,
\end{align*}

and note that $\alpha_{2 d_1 +1} , \ldots , \alpha_{n}$ form the last $n - 2 d_1$ entries of the final row of $H_{d_1 -1 , n+3 - d_1 } (\mathbfe{\alpha})$. By the assumptions made in the uniqueness claim, we must have that the polynomials

\begin{align*}
A_1 , T A_1 , \ldots , T^{n - 2 d_1 -1} A_1
\end{align*}

are in the kernel of $H_{d_1 -1 , n+3 - d_1 } (\mathbfe{\alpha})$ (and thus orthogonal to its final row). Hence, a similar inductive argument as above will show that the entries $\alpha_{2 d_1 +1} , \ldots , \alpha_{n}$ can be uniquely determined in terms of the entries $\alpha_0 , \alpha_1 , \ldots , \alpha_{2 d_1}$, thus concluding the proof of the uniqueness claim.

All that remains is to prove the final claim made in the theorem, which we do for the case $r \geq 2$. We have that

\begin{align*}
\mathbfe{\alpha} = (\alpha_0 , \alpha_1 , \ldots , \alpha_n ) \in \mathscr{L}_n (r , \rho_1 , \pi_1 ) .
\end{align*}

By considering the $(\rho , \pi )$-form of the matrix $H_{n_1 , n_2} (\mathbfe{\alpha})$ and removing last $\pi_1$ columns, we can deduce that $\mathbfe{\alpha}^{(1)}$ is in $\mathscr{L}_{n - \pi_1 } ( \rho_1 , \rho_1 , 0)$. Given that $A_1$ is in the kernel of $H_{n+1-r , r+1} (\mathbfe{\alpha})$, and that the associated vector has $\pi_1$ number of zero entries at the end, we can see that $A_1$ is in the kernel of matrix obtained by removing the last $\pi_1$ columns of $H_{n+1-r , r+1} (\mathbfe{\alpha})$; that is, the matrix $H_{n+1-r , \rho_1 +1} (\mathbfe{\alpha}^{(1)} )$. This tells us that $A_1$ is the first characteristic polynomial. Similarly, since $A_2$ is in the kernel of $H_{r-1 , n+3-r} (\mathbfe{\alpha})$, we can see that it is in the kernel of the matrix obtained by removing the last $\pi_1$ rows of $H_{r-1 , n+3-r} (\mathbfe{\alpha})$; that is, the matrix $H_{\rho_1 -1 , n+3-r} (\mathbfe{\alpha}^{(1)} )$. This tells us that $A_2$ is the second characteristic polynomial.

We now prove Theorem <ref>.

The construction of the sequences $\mathbfe{\alpha}$ in Claims 1 and 2 are not difficult, and so we proceed to Claims 3 and 4.

Claim 3: Let us write $A_2'$ to be the unique polynomial satisfying

\begin{align*}
A_2 = R A_1 + A_2' \hspace{3em} \degree A_2' < \degree A_1 .
\end{align*}

That is, $A_2'$ is the smallest representative of $A_2$ modulo $A_1$. Recalling our discussion on uniqueness in Definition <ref>, it is equivalent to prove that there exists a sequence $\mathbfe{\alpha} \in \mathscr{L}_n (r,r,0)$ with characteristic polynomials $A_1 , A_2'$. Now, let us define $d_1 := \degree A_1$ and $d_2 := \degree A_2'$, and we apply the Euclidean algorithm to $A_1, A_2'$:

\begin{align*}
A_1 = &R_2 A_2' + A_3 			&&d_3 := \degree A_3 < \degree A_2' , \\
A_2' = &R_3 A_3 + A_4 			&&d_4 := \degree A_4 < \degree A_3 , \\
A_3 = &R_4 A_4 + A_5 			&&d_5 := \degree A_5 < \degree A_4 , \\
&\vdots 					&&\vdots \\
A_t = &R_{t+1} A_{t+1} + A_{t+2} 		&&d_{t+2} := \degree A_{t+2} < \degree A_{t+1} ,
\end{align*}

where $t$ is such that $\degree A_{t} \geq 2 > \degree A_{t+1} \geq 0$. Also, let

\begin{align*}
\mathbfe{\alpha}^{(t+1)} := (\alpha_0 , \alpha_1 , \ldots , \alpha_{d_{t+1}} ) .
\end{align*}

Corollary <ref> tells us that our desired $\mathbfe{\alpha}$ exists if and only if we have

\begin{align*}
\mathbfe{\alpha}^{(t+1)}
\in \begin{cases}
\mathscr{L}_{d_{t}} (2 , 1 , 1) &\text{ if $d_{t+1} =1$,} \\
\mathscr{L}_{d_{t}} (2 , 0 , 2) &\text{ if $d_{t+1} =0$,}
\end{cases}
\end{align*}

with characteristic polynomials $A_{t+1} , A_{t}$.

Suppose $d_{t+1} = 1$. Then, it is equivalent to find $\mathbfe{\alpha}^{(t+1)}$ such that

\begin{align} \label{statement, all corpime characteristic polynomial have a Hankel matrix theorem proof, claim 2, d_(t+1)=1, A_(t+2) mults in kernel}
A_{t+1} \hspace{1em} , \hspace{1em} T A_{t+1} \hspace{1em} , \hspace{1em} \ldots \hspace{1em} , \hspace{1em} T^{d_{t} -2} A_{t+1}
\in \kernel (\alpha_0 , \alpha_1 , \ldots , \alpha_{d_{t}} )
\end{align}

\begin{align} \label{statement, all corpime characteristic polynomial have a Hankel matrix theorem proof, claim 2, d_(t+1)=1, A_(t+1) in kernel}
\in \kernel (\alpha_0 , \alpha_1 , \ldots , \alpha_{d_{t}} ) .
\end{align}

To this end, we let $\alpha_0$ take any value in $\mathbb{F}_q$. Since $\degree A_{t+1} = 1$, we can see that (<ref>) uniquely determines the values of $\alpha_1 , \ldots , \alpha_{d_{t} -1}$ in terms of $\alpha_0$. Similarly, since $\degree A_{t} = d_{t}$, we can see that (<ref>) uniquely determines the value of $\alpha_{d_{t}}$ in terms of $\alpha_0$. Ultimately, we have shown that our desired $\mathbfe{\alpha}$ exists. The fact that it is unique up to multiplication by elements in $\mathbb{F}_q^*$ follows from the following four facts:

* We could let $\alpha_0$ take any value in $\mathbb{F}_q$.

* The entries $\alpha_1 , \ldots , \alpha_{d_{t}}$ can be expressed uniquely in terms of $\alpha_0$. In fact, each such $\alpha_i$ can be expressed as a linear function of $\alpha_0$ that passes through the origin.

* The sequence $\mathbfe{\alpha}$ is uniquely determined by the sequence $\mathbfe{\alpha}^{(t+1)}$, which follows from the uniqueness claim at the end of Corollary <ref>. In fact, we can show that for all $i \leq n$ the entry $\alpha_i$ can be expressed as a linear function of $\alpha_0$ that passes through the origin.

* Finally, we note from the above that if $\alpha_0 = 0$, then $\mathbfe{\alpha} = \mathbf{0}$. We dismiss this case as it does not give us $\mathbfe{\alpha}^{(t+1)} \in \mathscr{L}_{d_{t}} (2 , 1 , 1)$.

Now suppose that we have $d_{t+1} = 0$ instead. We can use a similar argument as above. The main difference will be that $\alpha_0 , \ldots , \alpha_{d_{t} -2} = 0$, while $\alpha_{d_{t} -1}$ will be free to take any value in $\mathbb{F}_q^*$, and for all $d_{t} \leq i \leq n$ the entry $\alpha_i$ can be expressed as a linear function of $\alpha_{d_{t} -1}$ that passes through the origin.

Claim 4: We know by Claim 2 that there is a sequence $\mathbfe{\alpha}^{(1)} = (\alpha_0 , \alpha_1 , \ldots , \alpha_{n - \pi_1} ) \in \mathscr{L}_{n-\pi_1 } (\rho_1 , \rho_1 , 0)$ with characteristic polynomials $A_1 , A_2$. Note that

\begin{align} \label{statement, all corpime characteristic polynomial have a Hankel matrix theorem proof, claim 3, kernel H_(rho_1 -1 , n+3-r) (alpha^(1))}
\kernel H_{\rho_1 -1 , n+3 - r} (\mathbfe{\alpha}^{(1)})
= \bigg\{ B_1 A_1 + B_2 A_2 : \substack{B_1 , B_2 \in \mathcal{A} \\ \degree B_1 \leq n+2 - r - \rho_1 \\ \degree B_2 \leq 0} \bigg\} .
\end{align}

We now define the extension $\mathbfe{\alpha} = (\alpha_0 , \alpha_1 , \ldots , \alpha_n )$ of $\mathbfe{\alpha}^{(1)}$ by the following property:

\begin{align} \label{statement, all corpime characteristic polynomial have a Hankel matrix theorem proof, claim 3, kernel H_(r -1 , n+3-r) (alpha)}
\kernel H_{r -1 , n+3 - r} (\mathbfe{\alpha})
\subseteq \bigg\{ B_1 A_1 + B_2 A_2 : \substack{B_1 , B_2 \in \mathcal{A} \\ \degree B_1 \leq n+2 - 2r \\ \degree B_2 \leq 0} \bigg\} .
\end{align}

Note that removing the last $\pi_1$ rows of $H_{r -1 , n+3 - r} (\mathbfe{\alpha})$ will leave us with the matrix $\break H_{\rho_1 -1 , n+3 - r} (\mathbfe{\alpha}^{(1)})$ from (<ref>). Hence, by successive applications of Lemma <ref>, regardless of the way we extended $\mathbfe{\alpha}^{(1)}$, we certainly have that

\begin{align*}
\kernel H_{r -1 , n+3 - r} (\mathbfe{\alpha})
\supseteq \Big\{ B_1 A_1 : \substack{B_1 \in \mathcal{A} \\ \degree B_1 \leq n+2 - 2r} \bigg\} .
\end{align*}

The requirement that $A_2$ is in the kernel above will actually uniquely determine the entries $\alpha_{n - \pi_1 +1} , \ldots , \alpha_{n}$ (which form the extended part of $\mathbfe{\alpha}$) in terms of $\alpha_0 , \ldots , \alpha_{n - \pi_1 }$. This follows from the fact that $\alpha_{n - \pi_1 +1} , \ldots , \alpha_{n}$ appear as the final entries in the last $\pi_1$ rows of $H_{r -1 , n+3 - r} (\mathbfe{\alpha})$, and the fact that $A_2$ has degree $n+2-r$.

We will now show that $\mathbfe{\alpha}$ is in $\mathscr{L}_n (r , \rho_1 , \pi_1 )$. This is a condition of Claim 3, but it also allows us to determine the dimension of the left side of (<ref>) by using Corollary <ref>. By comparing this to the dimension of the right side of (<ref>), we see that we must have equality, and thus allowing us to deduce that $A_1 , A_2$ are the characteristic polynomials of $\mathbfe{\alpha}$. All that will be left to prove is that $\mathbfe{\alpha}$ is unique up to multiplication by  elements in $\mathbb{F}_q^*$.

Consider the sequence $\mathbfe{\alpha}' := (\alpha_0 , \alpha_1 , \ldots , \alpha_{n - \pi_1 +1} )$ and the associated matrix $H' := \break H_{\rho_1 , n+3 - r} (\mathbfe{\alpha}')$. Note that if we remove the last row from this matrix then we are left with the matrix $H := H_{\rho_1 -1 , n+3 - r} (\mathbfe{\alpha}^{(1)})$ from (<ref>). Hence, by Lemma <ref> we have that

\begin{align*}
\Big\{ B_1 A_1 : \substack{B_1 \in \mathcal{A} \\ \degree B_1 \leq n+1 - r - \rho_1 } \Big\}
\subseteq \kernel H' .
\end{align*}

By our construction of $\mathbfe{\alpha}$,we also have that $A_2$ is in this kernel, and thus

\begin{align*}
\bigg\{ B_1 A_1 + B_2 A_2 : \substack{B_1 , B_2 \in \mathcal{A} \\ \degree B_1 \leq n+1 - r - \rho_1 \\ \degree B_2 \leq 0} \bigg\}
\subseteq \kernel H' .
\end{align*}

In fact, we must have equality above. This follows from the fact that the dimension of $\kernel H'$ is one less than that of $\kernel H$. Indeed, the number of columns remains the same, but the rank of $H'$ is one more than the rank of $H$, which follows by the fact that $H'$ has full row rank since $\rho (\mathbfe{\alpha}')  \geq \rho (\mathbfe{\alpha}^{(1)}) = \rho_1$. Now, because $T^{n+2 - r - \rho_1} A_1$ is not in the kernel of $H'$, we can see that $\pi (H') = 1$, and thus $\pi (\mathbfe{\alpha}') = 1$. By considering $(\rho , \pi )$-forms, we can see that this forces $\pi (\mathbfe{\alpha}') = \pi_1$ (indeed, extending the sequence by one entry will increase the $\pi$-characteristic by one), and thus $\mathbfe{\alpha} \in \mathscr{L}_n (r , \rho_1 , \pi_1 )$.

Finally, for the uniqueness claim, this follows from the uniqueness claim in Claim 2 and the way we formed our extension $\mathbfe{\alpha}$ of $\mathbfe{\alpha}^{(1)}$. Technically, we should also prove a converse result; that any sequence $\mathbfe{\alpha}$ with the properties in Claim 3 has a truncation $\mathbfe{\alpha}^{(1)} \in \mathscr{L}_{n-\pi_1 } (\rho_1 , \rho_1 , 0)$ with characteristic polynomials $A_1 , A_2$. This is to ensure that our construction above actually addresses all possibilities for $\mathbfe{\alpha}$. This is not difficult to prove, and we have done something similar in the proof of Theorem <ref>.

Finally, we prove Theorem <ref>.

Claim 1: The cases $r \leq 1$ are considerably easier than the cases $r \geq 2$; we consider only the latter.

Let $H :=H_{n_1 , n_2} (\mathbfe{\alpha})$. The $(\rho , \pi )$-form of $H$ is

\begin{align*}
\begin{pmatrix}
H [r , r] & \vline & H [r , -(n_2 -r)] \\
\hline
\mathbf{0} & \vline &
\begin{NiceMatrix}
0 & \Cdots & \Cdots & 0 \\
\Vdots &  &  & \Vdots \\
0 & \Cdots & \Cdots & 0 \\
0 & \Cdots & \Cdots & 0
\end{NiceMatrix}
\end{pmatrix} .
\end{align*}

By recalling the definition of $(\rho , \pi )$-form, we can see that the $(\rho , \pi )$-form of $H' :=H_{n_1' , n_2'} (\mathbfe{\alpha}')$ is

\begin{align*}
\begin{pmatrix}
H [r , r] & \vline & H [r , -(n_2' -r)] \\
\hline
\mathbf{0} & \vline &
\begin{NiceMatrix}
0 & \Cdots & \Cdots & 0 \\
\Vdots &  &  & \Vdots \\
0 & \Cdots & \Cdots & 0 \\
0 & \Cdots & 0 & \gamma
\end{NiceMatrix}
\end{pmatrix} ,
\end{align*}

where $\gamma \in \mathbb{F}_q$, and there is a one-to-one correspondence between $\gamma$ and $\alpha_{n+1}$. If $\gamma = 0$, then we have $\mathbfe{\alpha}' \in \mathscr{L}_{n+1} (r,r,0)$, while if $\gamma \neq 0$, then we have $\mathbfe{\alpha}' \in \mathscr{L}_{n+1} (r+1,r,1)$. The claims on the characteristic degrees follow by definition.

Let us now consider the characteristic polynomials. Consider the case $\gamma = 0$ first. By definition, we have that $A_1$ spans the kernel of $H_{c_2 -1 , c_1 +1} (\mathbfe{\alpha})$. By comparing the $(\rho , \pi )$-form of $H_{c_2 -1 , c_1 +1} (\mathbfe{\alpha})$ to the $(\rho , \pi )$-form of $H_{c_2' -1 , c_1' +1} (\mathbfe{\alpha}') = H_{c_2  , c_1 +1} (\mathbfe{\alpha}')$ (the latter having an extra row of zeros at the bottom, compared to the former), we see that $A_1$ spans the kernel of $H_{c_2' -1 , c_1' +1} (\mathbfe{\alpha}')$. Thus, $A_1' = A_1$.

Regarding $A_2'$, by definition it is the polynomial in the kernel of $H_{c_1' -1 , c_2' +1} (\mathbfe{\alpha}')$ that is not a multiple of $A_1'$. Now, let $\mathbf{a}_2$ be the vector in the kernel of $H_{c_1 -1 , c_2 +1} (\mathbfe{\alpha})$ that is associated to $A_2$. We can see that $(\mathbf{a}_2 \mid 0)$ is in the kernel of $H_{c_1' -1 , c_2' +1} (\mathbfe{\alpha}') = H_{c_1 -1 , c_2 +2} (\mathbfe{\alpha}')$ (the latter matrix has an extra column compared to the former). In terms of polynomials, this means $A_2$ is in the kernel of $H_{c_1' -1 , c_2' +1} (\mathbfe{\alpha}')$; and since it is not a multiple of $A_1' = A_1$, we have that $A_2' = A_2$.

Now consider the case where $\gamma \neq 0$. Let us write $\mathbf{a}_1$ for the vector associated to $A_1$ in the kernel of $H_{c_2 -1 , c_1 +1} (\mathbfe{\alpha})$. Similar to above, we compare the $(\rho , \pi )$-form of $H_{c_2 -1 , c_1 +1} (\mathbfe{\alpha})$ to the $(\rho , \pi )$-form of $H_{c_2' -1 , c_1' +1} (\mathbfe{\alpha}') = H_{c_2 -1  , c_1 +2} (\mathbfe{\alpha}')$ (the latter having an additional column compared to the former, with a non-zero entry in the bottom-right), and we we see that $(\mathbf{a}_1 \mid 0)$ spans the kernel of $H_{c_2' -1 , c_1' +1} (\mathbfe{\alpha}')$. Thus, $A_1' = A_1$.

For $A_2'$, as above, it is the polynomial in the kernel of $H_{c_1' -1 , c_2' +1} (\mathbfe{\alpha}') = H_{c_1 , c_2 +1} (\mathbfe{\alpha}')$ that is not a multiple of $A_1'$. Since $A_1' = A_1$ is the first characteristic polynomial, we have

\begin{align*}
\kernel H_{c_1' -1 , c_2' +1} (\mathbfe{\alpha}')
\supseteq \Big\{ B_1 A_1' : \substack{B_1 \in \mathcal{A} \\ \degree B_1 \leq c_2' - c_1' } \Big\}
= \Big\{ B_1 A_1 : \substack{B_1 \in \mathcal{A} \\ \degree B_1 \leq c_2 - c_1 -1 } \Big\} .
\end{align*}

We also have

\begin{align} \label{lemma, kernel structure subsection, extending sequence by one, T^(c_1 - c_1) A_1 not in kernel}
T^{c_2 - c_1} A_1 \not\in \kernel H_{c_1' -1 , c_2' +1} (\mathbfe{\alpha}').
\end{align}

Otherwise, by Lemma <ref>, we would have that $A_1$ is in the kernel of $H_{c_2' -2 , c_1'} (\mathbfe{\alpha}')$, which is a contradiction. Note also that $H_{c_1 -1 , c_2 +1} (\mathbfe{\alpha})$ is the matrix we obtain after removing the last row from $H_{c_1' -1 , c_2' +1} (\mathbfe{\alpha}') = H_{c_1 , c_2 +1} (\mathbfe{\alpha}')$. In particular, the kernel of the latter is a subspace of the former. That is,

\begin{align*}
\kernel H_{c_1' -1 , c_2' +1} (\mathbfe{\alpha}')
\subseteq \kernel H_{c_1 -1 , c_2 +1} (\mathbfe{\alpha})
= \bigg\{ B_1 A_1 + B_2 A_2 : \substack{B_1 , B_2 \in \mathcal{A} \\ \degree B_1 \leq c_2 - c_1 \\ \degree B_2 \leq 0 } \bigg\} .
\end{align*}

Thus, we have established that

\begin{align*}
\Big\{ B_1 A_1 : \substack{B_1 \in \mathcal{A} \\ \degree B_1 \leq c_2 - c_1 -1 } \Big\}
\subseteq \kernel H_{c_1' -1 , c_2' +1} (\mathbfe{\alpha}')
\subseteq \bigg\{ B_1 A_1 + B_2 A_2 : \substack{B_1 , B_2 \in \mathcal{A} \\ \degree B_1 \leq c_2 - c_1 \\ \degree B_2 \leq 0 } \bigg\} .
\end{align*}

Corollary <ref> tells us that the dimensions of adjacent vector spaces above differ by $1$. Now, $A_2'$ is the vector in $\kernel H_{c_1' -1 , c_2' +1} (\mathbfe{\alpha}')$ that is not a multiple of $A_1' = A_1$, and recall that by Theorem <ref> the degree of $A_2'$ must be equal to $c_2'$. Thus, we can deduce

\begin{align*}
A_2' = \beta A_2 + T^{c_2 - c_1} A_1
\end{align*}

for some $\beta \in \mathbb{F}_q^*$ ($\beta$ cannot be $0$ by (<ref>)). It is not difficult to see that if we change the value of $\alpha_{n+1}$ (which appears in the last row of $H_{c_1' -1 , c_2' +1} (\mathbfe{\alpha}')$), then the value of $\beta$ will have to change to ensure that the vector associated to $A_2'$ is orthogonal to the last row of $H_{c_1' -1 , c_2' +1} (\mathbfe{\alpha}')$, and thus in the kernel of $H_{c_1' -1 , c_2' +1} (\mathbfe{\alpha}')$.( Note that this makes use of the fact that $A_2$ is not in the kernel of $H_{c_1' -1 , c_2' +1} (\mathbfe{\alpha}')$; indeed, otherwise, $A_2$ would be orthogonal to the last row and altering the value of $\beta$ would have no effect). Hence, there is a one-to-one correspondence between $\alpha_{n+1}$ and $\beta$.

Claim 2: Let $H := H_{n_1 , n_2} (\mathbfe{\alpha})$. The $(\rho , \pi )$-form of $H$ is of the form

\begin{align} \label{statement, kernel structure subsection, sequence extension theorem, claim 2, H rho, pi matrix}
\begin{pmatrix}
H [\rho_1 , \rho_1] & \vline & H [\rho_1 , -(n_2 - \rho_1)] \\
\hline
\mathbf{0} & \vline &
\begin{NiceMatrix}
0 & \Cdots &  &  &  &  & 0 \\
\Vdots &  &  &  &  &  & \Vdots \\
0 & \Cdots &  &  &  &  & 0 \\
0 & \Cdots &  &  &  & 0 & 1 \\
0 & \Cdots &  &  & 0 & 1 & * \\
\Vdots &  &  & \Iddots & \Iddots & \Iddots & \Vdots \\
0 & \Cdots & 0 & 1 & * & \Cdots & *
\end{NiceMatrix}
\end{pmatrix} ,
\end{align}

where there are $\pi_1$ number of $1$s in the bottom right submatrix (recall, $1$ represents an element in $\mathbb{F}_q^*$, $0$ represents $0$ as usual, and $*$ represents an element in $\mathbb{F}_q$). Now consider the matrix $H' = H_{n_1' , n_2'} (\mathbfe{\alpha}')$, and note that $H'$ is the same matrix as $H$ but it has either an additional row or column at the end (depending on whether $n$ is even or odd). Thus, we can deduce that the $(\rho , \pi)$-form of $H'$ is the same as (<ref>) but with an additional row or column. This additional row or column contributes an additional $1$ in the bottom-right submatrix, and thus we have $\pi (\mathbfe{\alpha}') = \pi_1 +1$. Clearly $\rho (\mathbfe{\alpha}') = \rho_1$, and thus $\mathbfe{\alpha}' \in \mathscr{L}_n (r+1, \rho_1, \pi_1 +1 )$, as required. The claims on the characteristic degrees follow by definition.

Now, the proof that $A_1' = A_1$ is similar as in the second case of Claim 1. The proof for the second characteristic polynomial is also similar, but slightly different, and so we give an outline of the proof.

By definition of the first characteristic polynomial, we have that $A_1' = A_1$ spans the kernel of $H_{c_2' -1 , c_1' +1} (\mathbfe{\alpha})$. Thus, using Lemma <ref> for the second relation below, we have that

\begin{align*}
\Big\{ B_1 A_1 : \substack{B_1 \in \mathcal{A} \\ \degree B_1 \leq c_2 - c_1 -1} \Big\}
= \Big\{ B_1 A_1' : \substack{B_1 \in \mathcal{A} \\ \degree B_1 \leq c_2' - c_1'} \Big\}
\subseteq \kernel H_{c_1'-1 , c_2'+1} (\mathbfe{\alpha}') ,
\end{align*}

\begin{align} \label{lemma, kernel structure subsection, extending sequence by one, pi_1 geq 1 case, T^(c_2 -c_1) A_1 not in kernel}
T^{c_2 - c_1} A_1
\not\in \kernel H_{c_1'-1 , c_2'+1} (\mathbfe{\alpha}') .
\end{align}

We also have that

\begin{align*}
\kernel H_{c_1'-1 , c_2'+1} (\mathbfe{\alpha}')
\subseteq \kernel H_{c_1'-2 , c_2'+1} (\mathbfe{\alpha})
= \kernel H_{c_1-1 , c_2+1} (\mathbfe{\alpha})
= \bigg\{ B_1 A_1 + B_2 A_2 : \substack{B_1 , B_2 \in \mathcal{A} \\ \degree B_1 \leq c_2 - c_1 \\ \degree B_2 \leq 0 } \bigg\} .
\end{align*}

Thus, we have

\begin{align*}
\Big\{ B_1 A_1 : \substack{B_1 \in \mathcal{A} \\ \degree B_1 \leq c_2 - c_1 -1} \Big\}
\subseteq \kernel H_{c_1'-1 , c_2'+1} (\mathbfe{\alpha}')
\subseteq \bigg\{ B_1 A_1 + B_2 A_2 : \substack{B_1 , B_2 \in \mathcal{A} \\ \degree B_1 \leq c_2 - c_1 \\ \degree B_2 \leq 0 } \bigg\} ,
\end{align*}

and, by Corollary <ref>, the dimensions of adjacent vector spaces above differ by $1$. Thus, by (<ref>), and the fact that Theorem <ref> tells us that $A_2'$ must be of degree $c_2'$, we have that

\begin{align*}
A_2' = \beta T^{c_2 - c_1} A_1 + A_2
\end{align*}

for some $\beta \in \mathbb{F}_q$. The one-to-one correspondence between $\alpha_{n+1}$ and $\beta$ follows by similar reasoning as in Claim 2.

Claim 3:  Consider the first case given in Claim 3. The fact that $\mathbfe{\alpha} \in \mathscr{L}_{n+1} (n_1, n_1, 0 )$, and the claims on the characteristic degrees, are not difficult to deduce. Thus, we restrict our attention to the characteristic polynomials. By definition, $A_1'$ is the polynomial that spans the kernel of $H_{n_1 , n_1 +1} (\mathbfe{\alpha}')$, and Theorem <ref> tells us that $\degree A_1' = n_1$. Now, the matrix $H_{n_1 , n_1 +1} (\mathbfe{\alpha}')$ is the same as the matrix $\kernel H_{n_1 -1 , n_1 +1} (\mathbfe{\alpha})$ but with an additional row, and we know that

\begin{align*}
\kernel H_{n_1 -1 , n_1 +1} (\mathbfe{\alpha})
= \{ B_1 A_1 + B_2 A_2 : B_1 , B_2 \in \mathbb{F}_q \},
\end{align*}

and so

\begin{align*}
\kernel H_{n_1 , n_1 +1} (\mathbfe{\alpha}')
\subseteq \{ B_1 A_1 + B_2 A_2 : B_1 , B_2 \in \mathbb{F}_q \}.
\end{align*}

Since we know $\degree A_1' = n_1$, and that $\degree A_1 = n_1$ and $\degree A_2 < n_1$, we must have that $A_1' = \beta A_2 + A_1$ for some $\beta \in \mathbb{F}_q$. The one-to-one correspondence between $\alpha_{n+1}$ and $\beta$ follows similarly as previously. The second characteristic polynomial is, by definition, the polynomial in the kernel of $H_{n_1 -1 , n_1 +2} (\mathbfe{\alpha}')$ that is not a multiple of $A_1'$. Similar to the previous claims, we note that $H_{n_1 -1 , n_1 +2} (\mathbfe{\alpha}')$ is the same as the matrix $H_{n_1 -1 , n_1 +1} (\mathbfe{\alpha}')$ but with an additional column. So, because $A_2$ is in the kernel of the latter, we can see that it must be in the kernel of the former. Hence $A_2' = A_2$.

The second case of Claim 3 is very similar to Claim 1. However, for the second bullet point, one should note that, because $\mathbfe{\alpha}$ is quasi-regular (unlike in the analogous result in Claim 1), we require that $\degree A_1' = n_1 +1$, and this is why we take $A_1' = \beta A_2 + T A_1$ (as opposed to $A_2' = \beta A_2 + T A_1 = \beta A_2 + T^{c_2 - c_1} A_1$ which would be completely analogous to Claim 1). However, this “swapping” of $A_1'$ and $A_2'$ is important and natural as it plays a role in the manifestation of the Euclidean algorithm that we saw in Theorem <ref> and its corollaries.

The third case of Claim 3 is very similar to Claim 2. Again, there is a similar “swapping” of $A_1'$ and $A_2'$.

Recall that in Subsection <ref> we considered the generalisation of Theorem <ref> to higher moments such as the third, and we described that we would need to determine how many $\mathbfe{\alpha} , \mathbfe{\beta} , \mathbfe{\gamma}$ there are such that $H_{l+1 , m+1}(\mathbfe{\alpha})$, $H_{l+1 , m+1}(\mathbfe{\beta})$, and $H_{l+1 , m+1}(\mathbfe{\gamma})$ have certain given ranks, and $\mathbfe{\alpha} + \mathbfe{\beta} + \mathbfe{\gamma} = \mathbfe{0}$. Now that we have established various results on Hankel matrices, we can reduce this problem to certain special cases.

Let us write

\begin{align*}
\mathbfe{\alpha} = &(\alpha_0 , \alpha_1 , \ldots , \alpha_n ) \\
\mathbfe{\beta} = &(\beta_0 , \beta_1 , \ldots , \beta_n ) \\
\mathbfe{\gamma} = &(\gamma_0 , \gamma_1 , \ldots , \gamma_n )
\end{align*}

\begin{align*}
\mathbfe{\alpha}^- = &(\alpha_0 , \alpha_1 , \ldots , \alpha_{n-1} ) \\
\mathbfe{\beta}^- = &(\beta_0 , \beta_1 , \ldots , \beta_{n-1} ) \\
\mathbfe{\gamma}^- = &(\gamma_0 , \gamma_1 , \ldots , \gamma_{n-1} )
\end{align*}

Suppose we know how many $\mathbfe{\alpha}^- , \mathbfe{\beta}^- , \mathbfe{\gamma}^-$ there are with certain given $(\rho , \pi )$-characteristics and $\mathbfe{\alpha}^- + \mathbfe{\beta}^- + \mathbfe{\gamma}^- = \mathbfe{0}$. If at least one of these sequences $\mathbfe{\alpha}^- , \mathbfe{\beta}^- , \mathbfe{\gamma}^-$ has non-zero $\pi$-characteristic (that is, not all are quasi-regular) then we can determine the $(\rho , \pi )$-characteristics of $\mathbfe{\alpha} , \mathbfe{\beta} , \mathbfe{\gamma}$, even when we impose the condition that $\mathbfe{\alpha} + \mathbfe{\beta} + \mathbfe{\gamma} = \mathbfe{0}$.

Indeed, without loss of generality, suppose that $\pi (\mathbfe{\gamma}^- ) \geq 1$. Theorem <ref> allows us to determine the $(\rho , \pi )$-characteristics of $\mathbfe{\alpha} , \mathbfe{\beta}$ based on the $(\rho , \pi )$-characteristics of $\mathbfe{\alpha}^- , \mathbfe{\beta}^-$. However, if, for example, $\mathbfe{\alpha}^-$ is quasi-regular, then the $(\rho , \pi )$-characteristic of $\mathbfe{\alpha}$ would depend on $\alpha_n$. Theorem <ref> tells us how many values $\alpha_n$ can take so that the $(\rho , \pi )$-characteristic of $\mathbfe{\alpha}$ takes a specific value. However, it does not tell us exactly what those values are, and this is why it is important that $\pi (\mathbfe{\gamma}^- ) \geq 1$: Regardless of the values of $\alpha_n$ and $\beta_n$, we know there exists $\gamma_n$ such that $\alpha_n + \beta_n + \gamma_n = 0$, and we still know what the $(\rho , \pi )$-characteristic of $\mathbfe{\gamma}$ is because it is independent of the value of $\gamma_n$ (which uses the fact that $\pi (\mathbfe{\gamma}^- ) \geq 1$).

Thus we have reduce our problem to the cases where $\mathbfe{\alpha}^- , \mathbfe{\beta}^- , \mathbfe{\gamma}^-$ are all quasi-regular.

§ THE VARIANCE OF THE DIVISOR FUNCTION

We now prove Theorem <ref>. In this section, we work with a prime $p$, and not a prime power $q$. That is, $\mathcal{A} = \mathbb{F}_p [T]$.

In what follows, $n \geq 4$.

We have that

\begin{align} \label{statement, div var section, variance as Delta in terms of noncentred d(B)}
\frac{1}{p^n} \sum_{A \in \mathcal{M}_n} \lvert \Delta (A;h) \rvert^2
= \frac{1}{p^n} \sum_{A \in \mathcal{M}_n} \bigg( \sum_{B \in I (A;h)} d(B) \bigg)^2 - p^{2h} (n+1)^2 .
\end{align}

So, we will consider the first term on the right side.

\begin{align*}
&\sum_{A \in \mathcal{M}_n} \bigg( \sum_{B \in I (A;h)} d(B) \bigg)^2 \\
= &\sum_{A \in \mathcal{M}_n} \bigg( \sum_{\substack{B \in \mathcal{M}_n \\ \degree (B-A) < h}} d(B) \bigg)^2 \\
= &\sum_{A \in \mathcal{M}_n}
\bigg( \sum_{\substack{B \in \mathcal{M}_n \\ \degree (B-A) < h}}
\sum_{\substack{0 \leq l,m \leq n \\ l+m=n}}
\sum_{E \in \mathcal{M}_l} \sum_{F \in \mathcal{M}_m} \mathbbm{1}_{EF=B} \bigg)^2 \\
= &\sum_{A \in \mathcal{A}_{\leq n }}
\bigg( \sum_{\substack{B \in \mathcal{M}_n \\ \degree (B-A) < h}}
\sum_{\substack{0 \leq l,m \leq n \\ l+m=n}}
\sum_{E \in \mathcal{M}_l} \sum_{F \in \mathcal{M}_m} \mathbbm{1}_{EF=B} \bigg)^2 .
\end{align*}

For the last line, we changed the first summation range from $A \in \mathcal{M}_n$ to all $A \in \mathcal{A}$ with $\degree A \leq n$. This does not change the result because the conditions on $E$ and $F$ force $A$ to be in $\mathcal{M}_n$. Continuing, we have

\begin{align*}
&\sum_{A \in \mathcal{M}_n} \bigg( \sum_{B \in I (A;h)} d(B) \bigg)^2 \\
= &\sum_{A \in \mathcal{A}_{\leq n }}
\bigg( \sum_{\substack{B \in \mathcal{M}_n \\ \degree (B-A) < h}}
\sum_{\substack{0 \leq l,m \leq n \\ l+m=n}}
\sum_{E \in \mathcal{M}_l} \sum_{F \in \mathcal{M}_m}
\prod_{i=0}^{n} \mathbbm{1}_{\{ EF \}_i = b_i } \bigg)^2 .
\end{align*}

Here, for a polynomial $C$, we let $\{ C \}_i$ denote its $i$-ith coefficient, which is convenient when working with products of polynomials such as $EF$. We also  denote $i$-th coefficient of $A,B,E,F$ by $a_i , b_i , e_i , f_i$, respectively.

We will now express the sums over $A,B,E,F$ by sums over their coefficients. For example $\sum_{A \in \mathcal{A}_{\leq n }}$ will be expressed as $\sum_{a_0 , \ldots , a_n \in \mathbb{F}_p}$. We also note that if $A'$ satisfies $\degree (A-A') < h$, then (due to the sum over $B$) both $A$ and $A'$ give the same contribution. For a given $A$, there are $p^h$ such $A'$. Thus, we can multiply the whole expression by $p^h$ and consider only the $A$ that are of the form

\begin{align*}
A = 0 + 0 T + 0 T^2 + \ldots + 0 T^{h-1} + a_h T^h + a_{h+1} T^{h+1} + \ldots + a_{n} T^{n} .
\end{align*}

Furthermore, this means that $B$ is of the form

\begin{align*}
B = b_0 + b_1 T + b_2 T^2 + \ldots + b_{h-1} T^{h-1} + a_h T^h + a_{h+1} T^{h+1} + \ldots + a_{n} T^{n} .
\end{align*}

Thus, using the above and applying (<ref>) to the terms $\mathbbm{1}_{\{ EF \}_i = b_i }$, we obtain

\begin{align}
\begin{split} \label{statement, div var section, variance as exp sums before alpha_i = 0 for i < h}
&\sum_{A \in \mathcal{M}_n} \bigg( \sum_{B \in I (A;h)} d(B) \bigg)^2 \\
= &\frac{p^{h}}{p^{2n+2}} \sum_{a_h , \ldots , a_n \in \mathbb{F}_p }
\Bigg( \sum_{b_0 , \ldots , b_{h-1} \in \mathbb{F}_p}
\sum_{\substack{0 \leq l,m \leq n \\ l+m=n}}
\sum_{\substack{e_0 , \ldots , e_{l-1} \in \mathbb{F}_p \\ e_l =1 }}
\sum_{\substack{f_0 , \ldots , f_{m-1} \in \mathbb{F}_p \\ f_m =1 }} \\
&\hspace{15em} \prod_{i=0}^{h-1} \sum_{\alpha_i \in \mathbb{F}_p }
\exp \bigg( \frac{2 \pi i}{p} \alpha_i \Big( b_i - \sum_{\substack{0 \leq l_1 \leq l \\ 0 \leq m_1 \leq m \\ l_1 + m_1 = i }} e_{l_1} f_{m_1} \Big) \bigg) \\
&\hspace{15em} \cdot \prod_{i=h}^{n} \sum_{\alpha_i \in \mathbb{F}_p }
\exp \bigg( \frac{2 \pi i}{p} \alpha_i \Big( a_i - \sum_{\substack{0 \leq l_1 \leq l \\ 0 \leq m_1 \leq m \\ l_1 + m_1 = i }} e_{l_1} f_{m_1} \Big) \bigg) \Bigg)^2 .
\end{split}
\end{align}

For any element in $\mathbb{F}_p$ that appears in the exponentials above, we take it as an integer in $\{0 , 1 , \ldots , p-1 \}$. Now, consider the sum over one of the coefficients $b_i$, and apply (<ref>). We obtain,

\begin{align*}
\frac{1}{p} \sum_{b_i \in \mathbb{F}_p } \exp \Big( \frac{2 \pi i}{p} \alpha_i b_i \Big)
= \begin{cases}
0 &\text{ if $\alpha_i \in \mathbb{F}_p^*$,} \\
1 &\text{ if $\alpha_i = 0$.}
\end{cases}
\end{align*}

Thus, we require $\alpha_i = 0$ in order to have a non-zero contribution to (<ref>). Applying this for $i = 0 , \ldots , h-1$, we obtain

\begin{align}
\begin{split} \label{statement, div var section, variance as exp sums after alpha_i = 0 for i < h}
&\sum_{A \in \mathcal{M}_n} \bigg( \sum_{B \in I (A;h)} d(B) \bigg)^2 \\
= &\frac{p^{3h}}{p^{2n+2}} \sum_{a_h , \ldots , a_n \in \mathbb{F}_p }
\Bigg( \sum_{\substack{0 \leq l,m \leq n \\ l+m=n}}
\sum_{\substack{e_0 , \ldots , e_{l-1} \in \mathbb{F}_p \\ e_l =1 }}
\sum_{\substack{f_0 , \ldots , f_{m-1} \in \mathbb{F}_p \\ f_m =1 }}
\prod_{i=h}^{n} \sum_{\alpha_i \in \mathbb{F}_p }
\exp \bigg( \frac{2 \pi i}{p} \alpha_i \Big( a_i - \sum_{\substack{0 \leq l_1 \leq l \\ 0 \leq m_1 \leq m \\ l_1 + m_1 = i }} e_{l_1} f_{m_1} \Big) \bigg) \Bigg)^2 \\
= &\frac{p^{3h}}{p^{2n+2}} \sum_{a_h , \ldots , a_n \in \mathbb{F}_p }
\Bigg( \sum_{\substack{0 \leq l,m \leq n \\ l+m=n}}
\sum_{\substack{e_0 , \ldots , e_{l-1} \in \mathbb{F}_p \\ e_l =1 }}
\sum_{\substack{f_0 , \ldots , f_{m-1} \in \mathbb{F}_p \\ f_m =1 }}
\prod_{i=h}^{n} \sum_{\alpha_i \in \mathbb{F}_p }
\exp \bigg( \frac{2 \pi i}{p} \alpha_i \Big( a_i - \sum_{\substack{0 \leq l_1 \leq l \\ 0 \leq m_1 \leq m \\ l_1 + m_1 = i }} e_{l_1} f_{m_1} \Big) \bigg) \Bigg) \\
&\hspace{6em} \cdot \Bigg( \sum_{\substack{0 \leq l' , m' \leq n \\ l' + m' = n}}
\sum_{\substack{g_0 , \ldots , g_{l' -1} \in \mathbb{F}_p \\ g_{l'} =1 }}
\sum_{\substack{h_0 , \ldots , h_{m' -1} \in \mathbb{F}_p \\ h_{m'} =1 }}
\prod_{i=h}^{n} \sum_{\beta_i \in \mathbb{F}_p }
\exp \bigg( \frac{2 \pi i}{p} \beta_i \Big( a_i - \sum_{\substack{0 \leq l_1' \leq l' \\ 0 \leq m_1' \leq m' \\ l_1' + m_1' = i }} g_{l_1'} h_{m_1'} \Big) \bigg) \Bigg) .
\end{split}
\end{align}

We now consider the sum over one of the coefficients $a_i$. Unlike the $b_i$ which appeared within the largest parentheses, the $a_i$ appear outside. Thus, we must simultaneously consider the terms within each pair of parentheses whose product forms the square, and that is why we have written them explicitly in the last line above. Again, we apply (<ref>) to obtain

\begin{align*}
\sum_{a_i \in \mathbb{F}_p } \exp \Big( \frac{2 \pi i}{p} (\alpha_i + \beta_i) a_i \Big)
= \begin{cases}
0 &\text{ if $\alpha_i \neq \beta_i$,} \\
p &\text{ if $\alpha_i = - \beta_i$.}
\end{cases}
\end{align*}

Applying this to (<ref>) gives

\begin{align*}
&\sum_{A \in \mathcal{M}_n} \bigg( \sum_{B \in I (A;h)} d(B) \bigg)^2 \\
= &p^{2h-n-1} \sum_{\alpha_{h} , \alpha_{h+1} , \ldots , \alpha_{n} \in \mathbb{F}_p}
\Bigg( \sum_{\substack{0 \leq l,m \leq n \\ l+m=n}}
\sum_{\substack{e_0 , \ldots , e_{l-1} \in \mathbb{F}_p \\ e_l =1 }}
\sum_{\substack{f_0 , \ldots , f_{m-1} \in \mathbb{F}_p \\ f_m =1 }}
\prod_{i=h}^{n} \exp \bigg( - \frac{2 \pi i}{p} \alpha_i \sum_{\substack{0 \leq l_1 \leq l \\ 0 \leq m_1 \leq m \\ l_1 + m_1 = i }} e_{l_1} f_{m_1} \bigg) \Bigg) \\
&\hspace{6em} \cdot \Bigg( \sum_{\substack{0 \leq l' , m' \leq n \\ l' + m' = n}}
\sum_{\substack{g_0 , \ldots , g_{l' -1} \in \mathbb{F}_p \\ g_{l'} =1 }}
\sum_{\substack{h_0 , \ldots , h_{m' -1} \in \mathbb{F}_p \\ h_{m'} =1 }}
\prod_{i=h}^{n} \exp \bigg( \frac{2 \pi i}{p} \alpha_i \sum_{\substack{0 \leq l_1' \leq l' \\ 0 \leq m_1' \leq m' \\ l_1' + m_1' = i }} g_{l_1'} h_{m_1'} \bigg) \Bigg) \\
= &p^{2h-n-1} \sum_{\mathbfe{\alpha} \in \mathscr{L}_n^h }
\Bigg( \sum_{\substack{0 \leq l,m \leq n \\ l+m=n}}
\sum_{\substack{e_0 , \ldots , e_{l-1} \in \mathbb{F}_p \\ e_l =1 }}
\sum_{\substack{f_0 , \ldots , f_{m-1} \in \mathbb{F}_p \\ f_m =1 }}
\prod_{i=h}^{n} \exp \bigg( - \frac{2 \pi i}{p} \alpha_i \sum_{\substack{0 \leq l_1 \leq l \\ 0 \leq m_1 \leq m \\ l_1 + m_1 = i }} e_{l_1} f_{m_1} \bigg) \Bigg) \\
&\hspace{6em} \cdot \Bigg( \sum_{\substack{0 \leq l' , m' \leq n \\ l' + m' = n}}
\sum_{\substack{g_0 , \ldots , g_{l' -1} \in \mathbb{F}_p \\ g_{l'} =1 }}
\sum_{\substack{h_0 , \ldots , h_{m' -1} \in \mathbb{F}_p \\ h_{m'} =1 }}
\prod_{i=h}^{n} \exp \bigg( \frac{2 \pi i}{p} \alpha_i \sum_{\substack{0 \leq l_1' \leq l' \\ 0 \leq m_1' \leq m' \\ l_1' + m_1' = i }} g_{l_1'} h_{m_1'} \bigg) \Bigg) \\
= &p^{2h-n-1} \sum_{\mathbfe{\alpha} \in \mathscr{L}_n^h }
\Bigg( \sum_{\substack{0 \leq l,m \leq n \\ l+m=n}}
\sum_{\substack{\mathbf{e} \in {\mathbb{F}_p}^l \times \{ 1 \} \\ \mathbf{f} \in {\mathbb{F}_p}^m \times \{ 1 \} }}
\exp \bigg( - \frac{2 \pi i}{p} \mathbf{e}^T H_{l+1 , m+1} (\mathbfe{\alpha}) \mathbf{f} \bigg) \Bigg) \\
&\hspace{6em} \cdot \Bigg( \sum_{\substack{0 \leq l' , m' \leq n \\ l' + m' = n}}
\sum_{\substack{\mathbf{g} \in {\mathbb{F}_p}^{l'} \times \{ 1 \} \\ \mathbf{h} \in {\mathbb{F}_p}^{m'} \times \{ 1 \} }}
\exp \bigg( \frac{2 \pi i}{p} \mathbf{g}^T H_{l' + 1 , m' + 1} (\mathbfe{\alpha}) \mathbf{h} \bigg) \Bigg) \\
= &p^{2h+n} \sum_{r=0, h+1 , h+2 , \ldots , n_1 -1} \sum_{\mathbfe{\alpha}^- \in \mathscr{L}_{n-1}^h (r,r,0) } (n+1-2r)^2 p^{-2r} \\
= &(p-1) p^{h+n-1} \sum_{r=h+1}^{n_1 -1} (n+1-2r)^2
\hspace{2em} + p^{2h+n} (n+1)^2
\end{align*}

For the second equality, $\mathbfe{\alpha} = (\alpha_0 , \alpha_1 , \ldots , \alpha_n)$; of course, since $\mathbfe{\alpha} \in \mathscr{L}_n^h$, we have $\alpha_0 , \alpha_1 , \ldots , \break \alpha_{h-1} = 0$, which is consistent with the previous line. For the third equality, we are writing $\mathbf{e} = (e_1 , e_2 , \ldots , e_l )^T$ and similarly for $\mathbf{f}, \mathbf{g}, \mathbf{h}$. The fourth equality uses Lemma <ref> below, and the fifth equality uses Claims 2 and 3 from Theorem <ref>. So, applying this to (<ref>) gives

\begin{align*}
\frac{1}{p^n} \sum_{A \in \mathcal{M}_n} \lvert \Delta (A;h) \rvert^2
= \begin{cases}
(p-1) p^{h-1} \frac{(n-2h-1)(n-2h)(n-2h+1)}{6} &\text{ for $h \leq n_1 -2$,} \\
0 &\text{ for $h \geq n_1 -1$.}
\end{cases}
\end{align*}

In what follows we have

\begin{align*}
\mathbfe{\alpha} = (\alpha_0 , \alpha_1 , \ldots , \alpha_n ) \in \mathbb{F}_p^{n+1}
\end{align*}

and we define

\begin{align*}
\mathbfe{\alpha}^- := (\alpha_0 , \alpha_1 , \ldots , \alpha_{n-1} ) .
\end{align*}

The two claims below address all possible values that $\mathbfe{\alpha}$ could take.

Claim 1: Suppose $\mathbfe{\alpha}$ is such that $\mathbfe{\alpha}^{-} \in \mathscr{L}_{n-1} (r , \rho_1 , \pi_1 )$, where $0 \leq r \leq n_1 -1$ and $\pi_1 \geq 1$. For all $l,m \geq 0$ with $l+m=n$, we have

\begin{align} \label{statement, variance section, evaluation of exp sums of vectors and matrices lemma, pi geq 2 eq}
\sum_{\substack{\mathbf{e} \in {\mathbb{F}_p}^l \times \{ 1 \} \\ \mathbf{f} \in {\mathbb{F}_p}^m \times \{ 1 \} }}
\exp \bigg( - \frac{2 \pi i}{p} \mathbf{e}^T H_{l+1 , m+1} (\mathbfe{\alpha}) \mathbf{f} \bigg)
= 0 .
\end{align}

Now suppose $n$ is odd and $\mathbfe{\alpha}$ is such that $\mathbfe{\alpha}^{-} \in \mathscr{L}_{n-1} (n_1 , 0 , 0 )$. For all $l,m \geq 0$ with $l+m=n$, we have

\begin{align*}
\sum_{\substack{\mathbf{e} \in {\mathbb{F}_p}^l \times \{ 1 \} \\ \mathbf{f} \in {\mathbb{F}_p}^m \times \{ 1 \} }}
\exp \bigg( - \frac{2 \pi i}{p} \mathbf{e}^T H_{l+1 , m+1} (\mathbfe{\alpha}) \mathbf{f} \bigg)
= 0 .
\end{align*}

Claim 2: Suppose $\mathbfe{\alpha}$ is such that $\mathbfe{\alpha}^- \in \mathscr{L}_{n-1} (r,r,0)$, where $0 \leq r \leq n_1 -1$. We will fix $\mathbfe{\alpha}^-$ and consider all possible extensions $\mathbfe{\alpha}$; that is, we let $\alpha_n$ vary in $\mathbb{F}_p$. We have

\begin{align*}
&\sum_{\alpha_n \in \mathbb{F}_p }
\Bigg( \sum_{\substack{0 \leq l,m \leq n \\ l+m=n}}
\sum_{\substack{\mathbf{e} \in {\mathbb{F}_p}^l \times \{ 1 \} \\ \mathbf{f} \in {\mathbb{F}_p}^m \times \{ 1 \} }}
\exp \bigg( - \frac{2 \pi i}{p} \mathbf{e}^T H_{l+1 , m+1} (\mathbfe{\alpha}) \mathbf{f} \bigg) \Bigg) \\
&\hspace{6em} \cdot \Bigg( \sum_{\substack{0 \leq l' , m' \leq n \\ l' + m' = n}}
\sum_{\substack{\mathbf{g} \in {\mathbb{F}_p}^{l'} \times \{ 1 \} \\ \mathbf{h} \in {\mathbb{F}_p}^{m'} \times \{ 1 \} }}
\exp \bigg( \frac{2 \pi i}{p} \mathbf{g}^T H_{l' + 1 , m' + 1} (\mathbfe{\alpha}) \mathbf{h} \bigg) \Bigg) \\
= &p^{2n - 2r +1} (n+1-2r)^2 .
\end{align*}

In what follows, $e_i$ and $f_i$ are the $i$-th entries of $\mathbf{e}$ and $\mathbf{f}$, respectively.

Claim 1: Consider the first result in this claim. Since $\pi_1 \geq 1$, we can see from Theorem <ref> tells us that for $m \leq n-r-1$ there is no monic polynomial of degree $m$ in the kernel of $H_{l , m+1} (\mathbfe{\alpha}^-)$. That is, there is no vector $\mathbf{f} \in {\mathbb{F}_p}^m \times \{ 1 \}$ such that $H_{l , m+1} (\mathbfe{\alpha}^-) \mathbf{f} = \mathbf{0}$. Therefore, for any $\mathbf{f} \in {\mathbb{F}_p}^m \times \{ 1 \}$ we can find a row, $R_i$, of $H_{l , m+1} (\mathbfe{\alpha})$ such that $R_i \mathbf{f} \neq 0$. Now, consider only the sum and terms involving $e_i$ on the left side of (<ref>); we have

\begin{align*}
\sum_{e_i \in \mathbb{F}_p} \exp \bigg( - \frac{2 \pi i}{p} e_i R_i \mathbf{f} \bigg)
= 0 ,
\end{align*}

where we have used (<ref>) and the fact that $R_i \mathbf{f} \neq 0$. Thus, the left side of (<ref>) is indeed $0$.

Let us now consider when $m \geq n-r$. Let $c_1^- := r$ and $c_2^- := n+1-r$. Theorem <ref> tells us that

\begin{align} \label{statement, variance section, evaluation of exp sums of vectors and matrices lemma proof, kernel smaller matr}
\kernel H_{l,m+1} (\mathbfe{\alpha}^-)
= \bigg\{ B_1 A_1 + B_2 A_2 : \substack{B_1 , B_2 \in \mathcal{A} \\ \degree B_1 \leq m - c_1^- \\ \degree B_2 \leq m - c_2^- } \bigg\} ,
\end{align}

for some $A_1 \in \mathcal{A}_{\rho_1}$ and $A_2 \in \mathcal{A}_{c_2^-}$. Note that in this case, there are monic polynomials of degree $m$ in the kernel of $H_{l,m+1} (\mathbfe{\alpha}^-)$. That is, there are vectors $\mathbf{f} \in {\mathbb{F}_p}^m \times \{ 1 \}$ such that $H_{l , m+1} (\mathbfe{\alpha}^-) \mathbf{f} = \mathbf{0}$. By similar reasoning as above, these are the only candidates for non-zero contributions to the left side of (<ref>).

Now, $A_1 , A_2$ are the characteristic polynomials of $\mathbfe{\alpha}^-$. Theorem <ref> tells us that

\begin{align*}
\mathbfe{\alpha}
\in \begin{cases}
\mathscr{L}_{n} (n_1, n_1 , 0 ) &\text{ if $n$ is even and $r= n_1 -1$,} \\
\mathscr{L}_{n} (r+1 , \rho_1 , \pi +1)  &\text{ otherwise.}
\end{cases}
\end{align*}

Consider the latter case. Theorem <ref> also tells us that the characteristic polynomials of $\mathbfe{\alpha}$ are $A_1$ and $\beta T^{c_2^- - c_1^- } A_1 + A_2$, and so Theorem <ref> tells us that

\begin{align*}
\kernel H_{l+1 , m+1} (\mathbfe{\alpha})
= \bigg\{ B_1 A_1 + B_2 (\beta T^{c_2^- - c_1^- } A_1 + A_2 ) : \substack{B_1 , B_2 \in \mathcal{A} \\ \degree B_1 \leq m - c_1^- -1 \\ \degree B_2 \leq m - c_2^- } \bigg\} .
\end{align*}

Comparing this to (<ref>), we see that

\begin{align} \label{statement, variance section, evaluation of exp sums of vectors and matrices lemma proof, kernel smaller matr in terms of larger matr}
\kernel H_{l,m+1} (\mathbfe{\alpha}^-)
=\kernel H_{l+1 , m+1} (\mathbfe{\alpha})
\hspace{1em} \oplus \hspace{1em} \{ \gamma T^{m-c_1^- } A_1 : \gamma \in \mathbb{F}_p \} .
\end{align}

(This is not surprising as the dimension of $\kernel H_{l,m+1} (\mathbfe{\alpha}^-) $ is just one more than the dimension of $\kernel H_{l+1 , m+1} (\mathbfe{\alpha})$). So, for $\mathbf{f} \in {\mathbb{F}_p}^m \times \{ 1 \}$ that are in $\kernel H_{l,m+1} (\mathbfe{\alpha}^-) $, we can write $\mathbf{f} = \mathbf{f}^- + \gamma \mathbf{g}_m$; where $\mathbf{f}^- \in \kernel H_{l+1 , m+1} (\mathbfe{\alpha})$, $\gamma \in \mathbb{F}_p$, and $\mathbf{g}_m$ is the polynomial associated with $T^{m-c_1^- } A_1$. Hence,

\begin{align*}
\mathbf{e}^T H_{l+1 , m+1} (\mathbfe{\alpha}) \mathbf{f}
= \gamma R_{l+1} \mathbf{g}_m ,
\end{align*}

where $R_{l+1}$ is the last row of $H_{l+1 , m+1} (\mathbfe{\alpha})$. Thus, we have

\begin{align*}
&\sum_{\substack{\mathbf{e} \in {\mathbb{F}_p}^l \times \{ 1 \} \\ \mathbf{f} \in {\mathbb{F}_p}^m \times \{ 1 \} }}
\exp \bigg( - \frac{2 \pi i}{p} \mathbf{e}^T H_{l+1 , m+1} (\mathbfe{\alpha}) \mathbf{f} \bigg) \\
= &\sum_{\substack{\mathbf{e} \in {\mathbb{F}_p}^l \times \{ 1 \} \\ \mathbf{f}^- \in ({\mathbb{F}_p}^m \times \{ 1 \}) \cap \kernel H_{l+1 , m+1} (\mathbfe{\alpha}) \\ \gamma \in \mathbb{F}_p }}
\exp \bigg( - \frac{2 \pi i}{p} \mathbf{e}^T H_{l+1 , m+1} (\mathbfe{\alpha}) (\mathbf{f}^- +\gamma \mathbf{g}_m ) \bigg) \\
= &\sum_{\substack{\mathbf{e} \in {\mathbb{F}_p}^l \times \{ 1 \} \\ \mathbf{f}^- \in ({\mathbb{F}_p}^m \times \{ 1 \}) \cap \kernel H_{l+1 , m+1} (\mathbfe{\alpha}) \\ \gamma \in \mathbb{F}_p }}
\exp \bigg( - \frac{2 \pi i}{p} \gamma R_{l+1} \mathbf{g}_m \bigg) \\
= &p^l C \sum_{\gamma \in \mathbb{F}_p} \exp \bigg( - \frac{2 \pi i}{p} \gamma R_{l+1} \mathbf{g}_m \bigg) \\
= &0 ,
\end{align*}

where for the last equality we have used (<ref>) and the fact that $R_{l+1} \mathbf{g} \neq 0$ (since $\mathbf{g} \not\in \break \kernel H_{l+1 , m+1} (\mathbfe{\alpha})$), and $C$ is the number of $\mathbf{f}^-$ in $({\mathbb{F}_p}^m \times \{ 1 \}) \cap \kernel H_{l+1 , m+1} (\mathbfe{\alpha})$ (which we can calculate explicitly but do not need to).

The case where $n$ is even and $r=n_1 -1$ follows similarly as above.

The second result of Claim 1 is also proved similarly to the above.

Claim 2: By similar means as in Claim 1, we can show that

\begin{align*}
\sum_{\substack{\mathbf{e} \in {\mathbb{F}_p}^l \times \{ 1 \} \\ \mathbf{f} \in {\mathbb{F}_p}^m \times \{ 1 \} }}
\exp \bigg( - \frac{2 \pi i}{p} \mathbf{e}^T H_{l+1 , m+1} (\mathbfe{\alpha}) \mathbf{f} \bigg)
= 0 ,
\end{align*}

for $m \leq r-1$ and $m \geq n+1-r$.

So, suppose that $r \leq m \leq n-r$. Consider

\begin{align*}
\sum_{\substack{\mathbf{e} \in {\mathbb{F}_p}^l \times \{ 1 \} \\ \mathbf{f} \in {\mathbb{F}_p}^m \times \{ 1 \} }}
\exp \bigg( - \frac{2 \pi i}{p} \mathbf{e}^T H_{l+1 , m+1} (\mathbfe{\alpha}) \mathbf{f} \bigg) ;
\end{align*}

As previously, a non-zero contribution requires that $\mathbf{f} \in \kernel H_{l,m+1} (\mathbfe{\alpha}^-)$. Let $R_{l+1} (\mathbfe{\alpha})$ be the last row of $H_{l+1,m+1} (\mathbfe{\alpha})$. We can deduce the following two points from Theorem <ref>:

* There are $p-1$ values of $\alpha_{n}$ such that

\begin{align*}
\mathbfe{\alpha}
\in \begin{cases}
\mathscr{L}_{n} (n_1 , n_1 , 0) &\text{ if $n$ is even and $r=n_1 -1$,} \\
\mathscr{L}_{n} (r+1 , r , 1) &\text{ otherwise.}
\end{cases}
\end{align*}

Theorem <ref> tells us that

\begin{align*}
\kernel H_{l,m+1} (\mathbfe{\alpha}^-)
= \Big\{ B_1 A_1 : \substack{ B_1 \in \mathcal{A} \\ \degree B_1 \leq m-r} \Big\}
\end{align*}

\begin{align*}
\kernel H_{l+1,m+1} (\mathbfe{\alpha})
= \Big\{ B_1 A_1 : \substack{ B_1 \in \mathcal{A} \\ \degree B_1 \leq m-1-r} \Big\} ,
\end{align*}

for some $A_1 \in \mathcal{M}_{r}$. Thus, any $\mathbf{f} \in ({\mathbb{F}_p}^m \times \{ 1 \}) \cap \kernel H_{l,m+1} (\mathbfe{\alpha}^-)$ can be written as $\mathbf{f} = \mathbf{f}^- + \mathbf{a}_m$ for some $\mathbf{f}^- \in \kernel H_{l+1,m+1} (\mathbfe{\alpha})$, and $\mathbf{a}_{m}$ is the polynomial associated with $T^{m-r} A_1$. This gives

\begin{align*}
\mathbf{e}^T H_{l+1 , m+1} (\mathbfe{\alpha}) \mathbf{f} = R_{l+1} (\mathbfe{\alpha}) \mathbf{a}_m \neq 0.
\end{align*}

Note that $R_{l+1} (\mathbfe{\alpha}) \mathbf{a}_m$ is independent of the values of $l,m$ (so long as $r \leq m \leq n-r$) This is because the non-zero entries of $\mathbf{a}_m$ occur in the last $r+1$ entries, and the last $r+1$ entries $R_{l+1} (\mathbfe{\alpha})$ are independent of the value of $l$.

Similar statements hold for the sums over $\mathbf{g} , \mathbf{h}$, but we should keep in mind that there is no negative sign in the exponential in these sums. Hence, we have

\begin{align*}
&\Bigg( \sum_{\substack{0 \leq l,m \leq n \\ l+m=n}}
\sum_{\substack{\mathbf{e} \in {\mathbb{F}_p}^l \times \{ 1 \} \\ \mathbf{f} \in {\mathbb{F}_p}^m \times \{ 1 \} }}
\exp \bigg( - \frac{2 \pi i}{p} \mathbf{e}^T H_{l+1 , m+1} (\mathbfe{\alpha}) \mathbf{f} \bigg) \Bigg) \\
&\hspace{6em} \cdot \Bigg( \sum_{\substack{0 \leq l' , m' \leq n \\ l' + m' = n}}
\sum_{\substack{\mathbf{g} \in {\mathbb{F}_p}^{l'} \times \{ 1 \} \\ \mathbf{h} \in {\mathbb{F}_p}^{m'} \times \{ 1 \} }}
\exp \bigg( \frac{2 \pi i}{p} \mathbf{g}^T H_{l' + 1 , m' + 1} (\mathbfe{\alpha}) \mathbf{h} \bigg) \Bigg) \\
= &\Bigg( \sum_{\substack{ l+m=n \\ r \leq m \leq n-r}}
\sum_{\substack{\mathbf{e} \in {\mathbb{F}_p}^l \times \{ 1 \} \\ \mathbf{f}^- \in \kernel H_{l+1,m+1} (\mathbfe{\alpha}) }}
\exp \bigg( - \frac{2 \pi i}{p} R_{l+1} (\mathbfe{\alpha}) \mathbf{a}_m \bigg) \Bigg) \\
&\hspace{6em} \cdot \Bigg( \sum_{\substack{l' + m' = n \\ r \leq m' \leq n-r}}
\sum_{\substack{\mathbf{g} \in {\mathbb{F}_p}^{l'} \times \{ 1 \} \\ \mathbf{h}^- \in \kernel H_{l' +1,m' +1} (\mathbfe{\alpha}) }}
\exp \bigg( \frac{2 \pi i}{p} R_{l+1} (\mathbfe{\alpha}) \mathbf{a}_m \bigg) \Bigg) \\
= &\Bigg( \sum_{\substack{ l+m=n \\ r \leq m \leq n-r}}
\sum_{\substack{\mathbf{e} \in {\mathbb{F}_p}^l \times \{ 1 \} \\ \mathbf{f}^- \in \kernel H_{l+1,m+1} (\mathbfe{\alpha}) }} 1 \Bigg)
\Bigg( \sum_{\substack{l' + m' = n \\ r \leq m' \leq n-r}}
\sum_{\substack{\mathbf{g} \in {\mathbb{F}_p}^{l'} \times \{ 1 \} \\ \mathbf{h}^- \in \kernel H_{l' +1,m' +1} (\mathbfe{\alpha}) }} 1 \Bigg) \\
=& \big( (n+1-2r) p^{n-r} \big)^2 ,
\end{align*}

where we have used the fact that $\lvert {\mathbb{F}_p}^l \times \{ 1 \} \rvert = p^l$ and $\lvert \kernel H_{l+1,m+1} (\mathbfe{\alpha}) \rvert = p^{(m+1) - (r+1)}$.

* There is one value of $\alpha_{n+1}$ such that $\mathbfe{\alpha} \in \mathscr{L}_{n}^h (r,r,0)$. In which case, all $\mathbf{f}$ in $\kernel H_{l,m+1} (\mathbfe{\alpha}^-)$ are also in $\kernel H_{l+1,m+1} (\mathbfe{\alpha})$, thus giving $R_{l+1} (\mathbfe{\alpha}) \mathbf{f} = 0$. A similar statement holds for the sums over $\mathbf{g} , \mathbf{h}$. Hence, we have

\begin{align*}
&\Bigg( \sum_{\substack{0 \leq l,m \leq n \\ l+m=n}}
\sum_{\substack{\mathbf{e} \in {\mathbb{F}_p}^l \times \{ 1 \} \\ \mathbf{f} \in {\mathbb{F}_p}^m \times \{ 1 \} }}
\exp \bigg( - \frac{2 \pi i}{p} \mathbf{e}^T H_{l+1 , m+1} (\mathbfe{\alpha}) \mathbf{f} \bigg) \Bigg) \\
&\hspace{6em} \cdot \Bigg( \sum_{\substack{0 \leq l' , m' \leq n \\ l' + m' = n}}
\sum_{\substack{\mathbf{g} \in {\mathbb{F}_p}^{l'} \times \{ 1 \} \\ \mathbf{h} \in {\mathbb{F}_p}^{m'} \times \{ 1 \} }}
\exp \bigg( \frac{2 \pi i}{p} \mathbf{g}^T H_{l' + 1 , m' + 1} (\mathbfe{\alpha}) \mathbf{h} \bigg) \Bigg) \\
= &\Bigg( \sum_{\substack{ l+m=n \\ r \leq m \leq n-r}}
\sum_{\substack{\mathbf{e} \in {\mathbb{F}_p}^l \times \{ 1 \} \\ \mathbf{f}^- \in \kernel H_{l+1,m+1} (\mathbfe{\alpha}) }} 1 \Bigg)
\Bigg( \sum_{\substack{l' + m' = n \\ r \leq m' \leq n-r}}
\sum_{\substack{\mathbf{g} \in {\mathbb{F}_p}^{l'} \times \{ 1 \} \\ \mathbf{h}^- \in \kernel H_{l' +1,m' +1} (\mathbfe{\alpha}) }} 1 \Bigg) \\
=& \big( (n+1-2r) p^{n-r} \big)^2 .
\end{align*}

Thus, we have

\begin{align*}
&\sum_{\alpha_n \in \mathbb{F}_p }
\Bigg( \sum_{\substack{0 \leq l,m \leq n \\ l+m=n}}
\sum_{\substack{\mathbf{e} \in {\mathbb{F}_p}^l \times \{ 1 \} \\ \mathbf{f} \in {\mathbb{F}_p}^m \times \{ 1 \} }}
\exp \bigg( - \frac{2 \pi i}{p} \mathbf{e}^T H_{l+1 , m+1} (\mathbfe{\alpha}) \mathbf{f} \bigg) \Bigg) \\
&\hspace{6em} \cdot \Bigg( \sum_{\substack{0 \leq l' , m' \leq n \\ l' + m' = n}}
\sum_{\substack{\mathbf{g} \in {\mathbb{F}_p}^{l'} \times \{ 1 \} \\ \mathbf{h} \in {\mathbb{F}_p}^{m'} \times \{ 1 \} }}
\exp \bigg( \frac{2 \pi i}{p} \mathbf{g}^T H_{l' + 1 , m' + 1} (\mathbfe{\alpha}) \mathbf{h} \bigg) \Bigg) \\
= &\sum_{\alpha_n \in \mathbb{F}_p } \big( (n+1-2r) p^{n-r} \big)^2 \\
= &p^{2n - 2r +1} (n+1-2r)^2 .
\end{align*}

Acknowledgements: This research was conducted during a postdoctoral fellowship funded by the Leverhulme Trust research project grant “Moments of $L$-functions in Function Fields and Random Matrix Theory” (grant number RPG-2017-320) secured by Julio Andrade. The author is most grateful for this support.

[1]
J. C. Andrade, L. Bary-Soroker and Z. Rudnick.
Shifted Convolution and the Titchmarsh Divisor Problem over
$\mathbb{F}_q [T]$.
Phil. Trans. R. Soc. A, 373 (2015).

[2]
G. Coppola and S. Salerno.
On the Symmetry of the Divisor Function in Almost All Short
Acta Arith., 113(2) (2004), 189–201.

[3]
H. Cramér.
Über Zwei Sätze von Herrn G. H. Hardy.
Math. Z., 15 (1922), 201–210.

[4]
M. García-Armas, S. R. Ghorpade and S. Ram.
Relatively Prime Polynomials and Nonsingular Hankel Matrices over
Finite Fields.
J. Comb. Theory Ser. A, 118 (2011), 819–828.

[5]
D. R. Heath-Brown.
The Distribution and Moments of the Error Term in the Dirichlet
Divisor Problem.
Acta Arith., 60(4) (1992), 389–415.

[6]
G. Heinig.
Kernel Structure of Toeplitz-plus-Hankel Matrices.
Linear Algebra Appl., 340(1) (2002), 1–13.

[7]
G. Heinig and P. Jankowski.
Kernel Structure of Block Hankel and Toeplitz Matrices and Partial
Linear Algebra Appl., 175 (1992), 1–30.

[8]
G. Heinig and K. Rost.
Algebraic Methods for Toeplitz-like Matrices and Operators.
Birkhäuser Basel, Basel (1984).

[9]
A. Ivić.
On the Divisor Function and the Riemann Zeta-function in Short
Ramanujan J., 19 (2009), 207–224.

[10]
M. Jutila.
On the Divisor Function for Short Intervals. Studies in Honour of
Arto Kustaa Salomaa on the Occasion of his Fiftieth Birthday.
Ann. Univ. Turku. Ser. A, I(186) (1984), 23–30.

[11]
J. P. Keating, E. Roditty-Gershon, B. Rodgers and Z. Rudnick.
Sums of Divisor Functions in $\mathbb{F}_q [t]$ and Matrix Integrals.
Math. Z., 288 (2018), 167–198.

[12]
S. Lester.
On the Variance of Sums of Divisor Functions in Short Intervals.
Proc. Amer. Math. Soc., 144(12) (2016), 5015–5027.

[13]
S. Lester and N. Yesha.
On the Distribution of the Divisor Function and Hecke Eigenvalues.
Israel J. Math., 212(1) (2016), 443–472.

[14]
R. Meshulam.
Spaces of Hankel Matrices over Finite Fields.
Linear Algebra Appl., 218 (1995), 73–76.

[15]
M. B. Milinovich and C. L. Turnage-Butterbaugh.
Moments of Products of Automorphic $L$-functions.
J. Number Theory, 139 (2014), 175–204.

[16]
O. Rojo.
A New Algebra of Toeplitz-plus-Hankel Matrices and Applications.
Comput. Math. with Appl., 55(12) (2008), 2856–2869.

[17]
E. C. Titchmarsh.
The Theory of the Riemann Zeta-Function (Oxford Science
Oxford University Press, 2nd edition (1987).
Revised by D. R. Heath-Brown.

[18]
K.-C. Tong.
On Divisor Problems III.
Acta Math. Sin., 6 (1956), 515–541.

[19]
K.-M. Tsang.
Higher-power Moments of $\Delta (x)$, $E (t)$ and $P (x)$.
Proc. London Math. Soc., 65(3) (1992), 65–84.