# Polyhedral geometry of refined $q,t$-Catalan numbers

Matthias Beck, Mitsuki Hanada, Max Hlavacek, John Lentfer,
Andrés R. Vindas-Meléndez, Katie Waddle  Department of Mathematics, San
Francisco State University
https://matthbeck.github.io/<EMAIL_ADDRESS>Department of Mathematics,
University of California, Berkeley
https://math.berkeley.edu/~mhanada<EMAIL_ADDRESS>Department of
Mathematics & Statistics, Pomona College
https://math.berkeley.edu/~mhlava<EMAIL_ADDRESS>Department of
Mathematics, University of California, Berkeley
https://math.berkeley.edu/~jlentfer<EMAIL_ADDRESS>Department of
Mathematics, Harvey Mudd College, https://math.hmc.edu/arvm<EMAIL_ADDRESS>Department of Mathematics, University of Michigan
https://sites.google.com/view/katie-waddle<EMAIL_ADDRESS>
(Date: 30 July 2024)

###### Abstract.

We study a refinement of the $q,t$-Catalan numbers introduced by Xin and Zhang
(2022, 2023) using tools from polyhedral geometry. These refined $q,t$-Catalan
numbers depend on a vector of parameters $\vec{k}$ and the classical
$q,t$-Catalan numbers are recovered when $\vec{k}=(1,\ldots,1)$. We interpret
Xin and Zhang’s generating functions by developing polyhedral cones arising
from constraints on $\vec{k}$-Dyck paths and their associated area and bounce
statistics. Through this polyhedral approach, we recover Xin and Zhang’s
theorem on $q,t$-symmetry of the refined $q,t$-Catalan numbers in the cases
where $\vec{k}=(k_{1},k_{2},k_{3})$ and $(k,k,k,k)$, give some extensions,
including the case $\vec{k}=(k,k+m,k+m,k+m)$, and discuss relationships to
other generalizations of the $q,t$-Catalan numbers.

###### 1991 Mathematics Subject Classification:

05A15 (Primary), 52B20 (Secondary)

##  1\. Introduction

The $q,t$-Catalan numbers are a two-parameter deformation of the well-known
Catalan numbers $\frac{1}{n+1}\binom{2n}{n}$. They were first introduced by
Haiman in [10] as the bigraded Hilbert series of the sign representation of a
certain $S_{n}$-module called the diagonal coinvariants. An elegant way to
write down the $q,t$-Catalan numbers, due to Garsia and Haglund [4], is

(1.1)
$C_{n}(q,t)=\sum_{D\in\mathcal{D}_{n}}q^{\operatorname{area}(D)}\,t^{\operatorname{bounce}(D)},$

where $\mathcal{D}_{n}$ is the set of Dyck paths of length $n$ and area and
bounce are certain statistics on Dyck paths; we give details below in Section
2. We can immediately observe the symmetry of $q$ and $t$ from Haiman’s
original definition of $C_{n}(q,t)$. A famous open problem asks for a
bijection on Dyck paths that interchanges area and bounce, which would provide
a combinatorial proof of $q,t$-symmetry. For more on the history and
equivalent definitions of $q,t$-Catalan numbers, see [8, Chapter 3] or [13,
Section 1.5].

We study a refinement $C_{\vec{k}}(q,t)$ of the $q,t$-Catalan numbers
introduced by Xin and Zhang [18]. Here $\vec{k}$ is a vector with positive
integer entries that sum to $n$, and we take the sum in (1.1) only over those
Dyck paths whose north steps are precisely the entries of $\vec{k}$. We
recover $C_{n}(q,t)$ as the special case where $\vec{k}$ has dimension $n$ and
all of its entries are 1. It is a natural question to ask in which cases
$C_{\vec{k}}(q,t)$ is symmetric in $q$ and $t$. Xin and Zhang proved symmetry
when $k$ is two- or three-dimensional, while giving a counterexample in
dimension four [18, 17]. They also proved symmetry for the case that $\vec{k}$
has four equal entries. Their proofs are constructive and involved: they
compute, via MacMahon’s Omega operator, the rational generating function of
$C_{\vec{k}}(q,t)$, from which the symmetry is plainly visible; for
$\vec{k}\in\mathbb{Z}_{>0}^{3}$, they also give an intricate bijective proof.
One of our goals is to interpret (and re-derive) Xin–Zhang’s generating
functions

(1.2)
$\operatorname{F}(x_{1},x_{2},x_{3},q,t):=\sum_{\vec{k}\in\mathbb{Z}_{\geq
0}^{3}}C_{\vec{k}}(q,t)\,x_{1}^{k_{1}}x_{2}^{k_{2}}x_{3}^{k_{3}}$

and

(1.3) $\operatorname{H}(x,q,t):=\sum_{k\in\mathbb{Z}_{\geq
0}}C_{(k,k,k,k)}(q,t)\,x^{k}$

through the use of polyhedral geometry: we develop polyhedral cones and
certain subdivisions which appear organically from the (linear) constraints
defining Dyck paths and their area and bounce statistics. It turns out these
cones are arithmetically nice: their integer lattice-point structure are
either unimodular or close to unimodular, and so their generating functions
are manageable to compute.

Xin and Zhang were primarily interested in the $q,t$-symmetry of

$C_{\lambda}(q,t)=\sum_{\mu(\vec{k})=\lambda}C_{\vec{k}}(q,t),$

where $\mu(\vec{k})=\lambda$ means that rearranging the entries of $\vec{k}$
in decreasing order gives the partition $\lambda$. In general, if all
$C_{\vec{k}}(q,t)$ are $q,t$-symmetric for a fixed $\mu(\vec{k})=\lambda$,
then $C_{\lambda}(q,t)$ is $q,t$-symmetric. It is also possible for
$C_{\lambda}(q,t)$ to be $q,t$-symmetric even if some of the
$C_{\vec{k}}(q,t)$ summands used in its definition are not; for example,

$C_{\lambda=(1,1,1,2)}(q,t)=C_{\vec{k}=(1,1,1,2)}(q,t)+C_{\vec{k}=(1,1,2,1)}(q,t)+C_{\vec{k}=(1,2,1,1)}(q,t)+C_{\vec{k}=(2,1,1,1)}(q,t)$

is $q,t$-symmetric, but individually, $C_{\vec{k}=(1,1,2,1)}(q,t)$ and
$C_{\vec{k}=(1,2,1,1)}(q,t)$ are not. It is unclear how common this symmetry
phenomenon is; starting with partitions $\lambda$ of length 4 many
$C_{\lambda}$ are not symmetric in $q,t$.

After giving background on both $q,t$-Catalan numbers and the integer-point
structure of polyhedral cones in Section 2, we compute
$\operatorname{F}(x_{1},x_{2},x_{3},q,t)$ in Section 3 and
$\operatorname{H}(x,q,t)$ in Section 4. Naturally, this geometric _ansatz_ can
be used in other situations, as we exhibit this in Section 5 by computing

(1.4) $\operatorname{G}(x,y,q,t):=\sum_{k,m\in\mathbb{Z}_{\geq
0}}C_{(k,k+m,k+m,k+m)}(q,t)\,x^{k}\,y^{m},$

realizing symmetry again along the way. We conjecture that
$C_{(k,a,\ldots,a)}(q,t)$ is $q,t$-symmetric for any positive $k$ and $a$. We
conclude in Section 6 with various extensions of our work, including a proof
that

$C_{\vec{k}=(k_{1},\ldots,k_{j},m)}(q,t)=C_{\vec{k}=(k_{1},\ldots,k_{j},l)}(q,t)\,.$

Finally, we discuss the relationship between the refined $q,t$-Catalan numbers
and other generalizations of the $q,t$-Catalan numbers.

##  2\. Background

###  2.1. $q,t$-Catalan numbers

The _Catalan numbers_ $C_{n}=\frac{1}{n+1}\binom{2n}{n}$ are well known to
have numerous combinatorial interpretations (see, for example, [16]) and
satisfy the recurrence relation

$C_{n}=\sum_{k=1}^{n}C_{k-1}C_{n-k}\,,$

for $n\geq 1$, with the initial condition $C_{0}=1$. There is not always a
canonical way to create a $q$-analogue of a combinatorial sequence, and
indeed, the Catalan numbers have two well-known $q$-analogues. One of them is
the _Carlitz–Riordan $q$-Catalan numbers_ $C_{n}(q)$, which are defined by the
recurrence relation

$C_{n}(q)=\sum_{k=1}^{n}q^{k-1}\,C_{k-1}(q)\,C_{n-k}(q)\,,$

for $n\geq 1$, with the initial condition $C_{0}(q)=1$. The second is the
_MacMahon $q$-Catalan numbers_ $D_{n}(q)$, which are defined via the usual
$q$-integers and $q$-binomial coefficients by

$D_{n}(q)=\frac{1}{[n+1]_{q}}\genfrac{[}{]}{0.0pt}{0}{2n}{n}_{q}\,.$

Based on computational data, Haiman first observed that the bigraded Hilbert
series of the sign representation of the diagonal coinvariants was a
$q,t$-deformation of the Catalan numbers [10]. Haiman first defined the
$q,t$-Catalan numbers by $C_{n}(q,t):=\langle e_{n},\nabla e_{n}\rangle$,
where $\nabla$ is a certain Macdonald eigenoperator for the Macdonald
symmetric functions. These $q,t$-Catalan numbers $C_{n}(q,t)$ were further
studied in [5], where the first combinatorial formula was presented. Several
conjectured properties were then proven in [11, 12].

Garsia and Haiman [5] showed that their combinatorial construction of the
$q,t$-Catalan numbers $C_{n}(q,t)$ included both the Carlitz-Riordan and
MacMahon $q$-Catalan numbers as specializations:

$C_{n}(q)=C_{n}(q,1)=C_{n}(1,q)\qquad\text{ and }\qquad
D_{n}(q)=C_{n}(q,\tfrac{1}{q})\,q^{\binom{n}{2}}\,.$

Garsia and Haiman’s combinatorial formula for $C_{n}(q,t)$ was not obviously a
polynomial in $\mathbb{Z}[q,t]$, but a rational function of polynomials.
Later, an elegant characterization of the $q,t$-Catalan numbers, which made
polynomiality clear, was conjectured by Haglund [7] and proven by Garsia and
Haglund [4] in terms of two statistics on Dyck paths, namely, area and bounce.

######  Definition 2.1.

A _Dyck path_ is a lattice path on the lattice $\mathbb{Z}^{2}$ that starts at
$(0,0)$, uses only $(0,1)$ and $(1,0)$ as steps (called north and east,
respectively), ends at $(n,n)$, and never goes below the line $y=x$. Denote
the set of all Dyck paths from $(0,0)$ to $(n,n)$ by $\mathcal{D}_{n}$. For a
Dyck path $D\in\mathcal{D}_{n}$ we define $\operatorname{area}(D)$ to be the
number of complete cells between the Dyck path and the line $y=x$.

######  Definition 2.2.

For $D\in\mathcal{D}_{n}$, the statistic $\operatorname{bounce}(D)$ is due to
Haglund [7] and is defined by the following algorithm. Begin at $(0,0)$ and
travel north until an east step of $D$ is encountered. Turn at a right angle
to the east and travel east until the line $y=x$ is reached. Then turn north
and travel until an east step of $D$ is encountered and repeat until $(n,n)$
is reached. This defines the bounce path of $D$. The bounce path “bounces” off
the line $y=x$ at positions
$(0,0),(j_{1},j_{1}),(j_{2},j_{2}),\ldots,(j_{b},j_{b})=(n,n)$. Then define

$\operatorname{bounce}(D):=\sum_{i=1}^{b-1}n-j_{i}\,.$

For the purposes of this paper, we take the following as the definition of the
$q,t$-Catalan numbers [4]:

######  Theorem 2.3 (Garsia–Haglund).

The $q,t$-Catalan numbers are given by

$C_{n}(q,t)=\sum_{D\in\mathcal{D}_{n}}q^{\operatorname{area}(D)}\,t^{\operatorname{bounce}(D)}.$

The symmetry of $C_{n}(q,t)$ follows immediately from Haiman’s original
definition of the $q,t$-Catalan numbers. Thus, we obtain the following as a
consequence.

######  Corollary 2.4.

For any nonnegative integer $n$,

$\sum_{D\in\mathcal{D}_{n}}q^{\operatorname{area}(D)}\,t^{\operatorname{bounce}(D)}=\sum_{D\in\mathcal{D}_{n}}q^{\operatorname{bounce}(D)}\,t^{\operatorname{area}(D)}.$

A notable open problem [8, Open Problem 3.11] is to find a bijection on Dyck
paths that interchanges area and bounce, which would prove the $q,t$-symmetry
in a combinatorial way. Figure 1 provides an example of the calculation of
$C_{3}(q,t)$, highlighting the aforementioned terminology and $q,t$-symmetry.

Figure 1. The five Dyck paths on a $3\times 3$ grid, used to calculate
$C_{3}(q,t)=q^{3}+q^{2}t+qt+qt^{2}+t^{3}$. The red boxes contribute to area.
The dashed blue path represents the bounce path. From left to right, the area
statistic is $3,2,1,1,0$ and the bounce statistic is $0,1,1,2,3$.

###  2.2. $\vec{k}$-Dyck paths

Xin and Zhang refined the $q,t$-Catalan numbers by modifying Haglund’s area-
bounce formula [18]. They provided three equivalent models which can be used
to define a $\vec{k}$-Dyck path. Here, we recall their first model.

######  Definition 2.5.

Let $\vec{k}=(k_{1},\dots,k_{m})$ be a vector of positive integers which sum
to $n$. A $\vec{k}$-Dyck path is a lattice path in $\mathbb{Z}^{2}$ from
$(0,0)$ to $(n,n)$, which never goes below the diagonal $y=x$, such that the
north steps are of length $k_{i}$, for $1\leq i\leq m$, from bottom to top.
The east steps are of length 1 without additional restriction. Denote the set
of all $\vec{k}$-Dyck paths by $D_{\vec{k}}$. Define the rank of a point
$(x,y)$ of a $\vec{k}$-Dyck path as $y-x$.

######  Remark 2.6.

In general, a $\vec{k}$-Dyck path encodes more information than a standard
Dyck path because it also includes information about how long the north steps
are. For example, consider a $\vec{k}$-Dyck path that contains a north step of
length $k_{i}$ immediately followed by a north step of length $k_{i+1}$: if
one considers this as a standard Dyck path by sending north steps of length
$k_{i}$ to $k_{i}$ north steps of length 1, we cannot then recover the lengths
$k_{i},k_{i+1}$.

We recover the $k$-Dyck paths studied by Loehr [13] as a special case by using
the vector $\vec{k}=(k,k,\ldots,k)$ (and shearing the $k$-Dyck path as
necessary so that the slope of the main diagonal becomes $1$). We can also
recover the standard Dyck paths as a special case by using the vector
$\vec{k}=(1,1,\ldots,1)$. Xin and Zhang also extended the definition of the
area and bounce of a Dyck path to $\vec{k}$-Dyck paths (such that they are
compatible with the definitions of area and bounce for $k$-Dyck paths). To
describe area for a $\vec{k}$-Dyck path, we need an auxiliary definition [18,
Section 2.1].

######  Definition 2.7.

Let $D\in D_{\vec{k}}$ be a $\vec{k}$-Dyck path. Define the _red ranks_ of a
$\vec{k}$-Dyck path to be the sequence $(r_{1},\ldots,r_{n})$, where $r_{i}$
is the rank of the starting point of the north step corresponding to $k_{i}$.

The red ranks can also be computed by counting complete lattice cells between
the path and the line $y=x$. Specifically, $r_{i}$ is the number of such cells
in the row whose bottom left corner is the south end of the start of the
$k_{i}$ set of north steps. A $\vec{k}$-Dyck path is uniquely determined by
its red ranks since they determine the location of all north steps. Note that
the first red rank $r_{1}$ is always equal to $0$.

######  Definition 2.8.

For $D\in D_{\vec{k}}$, define the statistic area to be the sum of its red
ranks, that is,

$\operatorname{area}(D):=r_{1}+\cdots+r_{n}\,.$

We recall from [18, Section 2.3] the algorithm to compute bounce for a
$\vec{k}$-Dyck path. For most of this paper, with the exception of Section 6,
it suffices use a few special cases.

######  Algorithm 2.9 (Xin-Zhang).

Input: A $\vec{k}$-Dyck path $D\in D_{\vec{k}}$.

  * •

Set $i=0$.

  * •

Let $R$ be a diagram with $k_{j}+1$ cells in column $j$.

  * •

Set $P_{0}:=(0,0)$ and define $R^{0}:=R$ to be the empty diagram.

  * •

While $P_{i}\neq(|\vec{k}|,|\vec{k}|)$, do the following:

    1. (1)

Start a vertical path from $P_{i}$ towards the north, and let $v_{i}$ be the
number of north steps traversed before reaching the starting point
$Q_{i}=(x_{i},y_{i})$ of an east step;

    2. (2)

To obtain $R^{i+1}$, write $i$ in the first row of the next $v_{i}$ unfilled
columns of $R^{i}$. To complete these $v_{i}$ columns, increment by $1$ going
down;

    3. (3)

Let $h_{i}$ be the number of cells in $R^{i+1}$ filled by $i+1$, and set
$P_{i+1}:=(x_{i}+h_{i},y_{i})$;

    4. (4)

Increment $i$ to $i+1$.

Output: The bounce statistic $\operatorname{bounce}(D):=\sum_{i\geq 0}iv_{i}$.

Xin and Zhang call the final tableau created during the algorithm the rank
tableau and they note that $\operatorname{bounce}(D)$ is equal to the sum of
the entries in its first row. For an example showing the construction of the
rank tableau, see [18, Figure 3]. The bounce path is a $\vec{k}$-Dyck path,
distinct from the original Dyck path $D$, where each $Q_{i}$ is the start of
an east step of $D$. Note that the bounce path for $D$ does not necessarily
touch the diagonal line $y=x$ (or “bounce” off the line) after each sequence
of consecutive east steps like it does in the classical $\vec{k}=(1,\ldots,1)$
case.

For small cases, one can give a piecewise closed-form formula for bounce. In
general, this becomes more difficult as the number of entries in $\vec{k}$
increases. For example, we extensively use the following characterization of
bounce when $\vec{k}$ has three parts [18, p. 9].

######  Lemma 2.10 (Xin–Zhang).

For a $(k_{1},k_{2},k_{3})$-Dyck path $D$,

(2.1)
$\operatorname{bounce}(D)=\begin{cases}2(k_{1}-r_{2})+r_{2}+k_{2}-r_{3}-\min(r_{2},k_{2})&\text{\rm
if }r_{2}+k_{2}-r_{3}\geq 2\min(r_{2},k_{2}),\\\
2(k_{1}-r_{2})+\lceil\frac{r_{2}+k_{2}-r_{3}}{2}\rceil&\text{\rm
otherwise.}\end{cases}$

In Section 4, we also state and use an involved formula for bounce when
$\vec{k}=(k,k,k,k)$ due to Niu [15].

######  Definition 2.11.

Let $\mathcal{D}_{\vec{k}}$ be the set of all $\vec{k}$-Dyck paths and define
the refined $q,t$-Catalan number by

$C_{\vec{k}}(q,t)=\sum_{D\in\mathcal{D}_{\vec{k}}}q^{\operatorname{area}(D)}\,t^{\operatorname{bounce}(D)}.$

Arranging the entries of $\vec{k}$ in decreasing order gives a partition
$\mu(\vec{k})$ and we further define

$C_{\lambda}(q,t)=\sum_{\mu(\vec{k})=\lambda}\sum_{D\in\mathcal{D}_{\vec{k}}}q^{\operatorname{area}(D)}\,t^{\operatorname{bounce}(D)}.$

Xin and Zhang proved the $q,t$-symmetry of $C_{(k_{1},k_{2})}(q,t)$,
$C_{(k_{1},k_{2},k_{3})}(q,t)$, and $C_{(k,k,k,k)}(q,t)$ for any positive
integers $k_{1},k_{2},k_{3}$, and $k$ [18, 17]. For an example, see Figure 2.
They also investigated the symmetry of $C_{\lambda}(q,t)$, observing that
symmetry holds for partitions of length 2. They infer symmetry in the case
that $\lambda$ is a partition of length 3 as a corollary to the $q,t$-symmetry
of $C_{(k_{1},k_{2},k_{3})}(q,t)$ and exhibit examples showing that symmetry
does not hold in general for partitions of length 4. Furthermore, Xin and
Zhang conjectured that symmetry of $C_{\lambda}(q,t)$ holds given
$\lambda=((a+1)^{s},a^{n-s})$ with $0\leq s\leq n$. Niu [15] proves this
conjecture in the case that $n=4$ using similar partition analysis methods as
those employed by Xin and Zhang [17].

######  Notation 2.12.

When $\vec{k}$ or $\lambda$ are given specific values in $C_{\vec{k}}(q,t)$ or
$C_{\lambda}(q,t)$, respectively, it can be ambiguous which definition is
being referred to. In this paper, we adopt the convention that if otherwise
unlabelled by $\vec{k}$ or $\lambda$, we are referring to $C_{\vec{k}}(q,t)$
when we use an expression like $C_{(1,2,3)}(q,t)$.

Figure 2. Top: The $(1,2)$-Dyck paths used to calculate $C_{(1,2)}(q,t)$.
Bottom: The $(2,1)$-Dyck paths used to calculate $C_{(2,1)}(q,t)$. The red
boxes contribute to Xin-Zhang’s area and their bounce path is drawn in dashed
blue. The $(1,2)$-Dyck paths have area $1$ and $0$ and bounce $0$ and $1$. The
$(2,1)$-Dyck paths have area $2$, $1$, and $0$ and bounce $0$, $1$, and $2$.
Thus, $C_{(1,2)}(q,t)=q+t$ and $C_{(2,1)}(q,t)=q^{2}+qt+t^{2}$. Summing, we
obtain $C_{\lambda=(2,1)}(q,t)=q^{2}+qt+t^{2}+q+t$.

###  2.3. Integer-point generating functions of polyhedral cones

As mentioned in the introduction, our computations employ the arithmetic of
integer points in polyhedral cones, and we sketch how to compute their
generating functions here, following [1, Section 4.8]. A _polyhedral cone_ is
a nonnegative linear combination of a finite set of vectors in
$\mathbb{R}^{d}$. The cone is _simplicial_ if its generators are linearly
independent, and _rational_ if we can choose its generators to be in
$\mathbb{Z}^{d}$. To a rational cone $C$ we associate its _integer-point
transform_

$\sigma_{C}(\vec{z}):=\sum_{\vec{m}\in
C\cap\mathbb{Z}^{d}}z_{1}^{m_{1}}z_{2}^{m_{2}}\cdots z_{d}^{m_{d}}.$

We abbreviate the monomial in the sum as ${\vec{z}}^{\vec{m}}$. We compute
integer-point transforms of simplicial rational cones, with a twist, namely,
some of its facets (faces of codimension 1) removed. To be precise, fix
linearly independent vectors
$\vec{v}_{1},\vec{v}_{2},\dots,\vec{v}_{k}\in\mathbb{Z}^{d}$ and let

$C:=\mathbb{R}_{\geq 0}\vec{v}_{1}+\dots+\mathbb{R}_{\geq
0}\vec{v}_{m-1}+\mathbb{R}_{>0}\vec{v}_{m}+\dots+\mathbb{R}_{>0}\vec{v}_{k}\,,$

which we call a _half-open cone_ ; here $1\leq m\leq k$. Because $C$ is
simplicial, each facet of $C$ is opposite one of its generators, and so we may
think of $C$ as having the facets opposite
$\vec{v}_{m},\vec{v}_{m+1},\dots,\vec{v}_{k}$ removed. We call the vectors
$\vec{v}_{1},\dots,\vec{v}_{k}$ _generators_ of this cone. A standard tiling
argument and geometric series yield (see for example [1, proof of Theorem
4.8.1]),

$\sigma_{C}(\vec{z})=\frac{\sigma_{\Pi(C)}(\vec{z})}{\prod_{j=1}^{k}(1-{\vec{z}}^{\vec{v}_{j}})},$

where

$\Pi(C):=[0,1)\vec{v}_{1}+\dots+[0,1)\vec{v}_{m-1}+(0,1]\vec{v}_{m}+\dots+(0,1]\vec{v}_{k}\,$

denotes the _fundamental parallelepiped_ of $C$. In particular,
$\sigma_{C}(\vec{z})$ is a rational function and so (because we can
triangulate any cone into simplicial cones) is the integer-point transform of
any rational cone. Now, we are ready to apply these methods to our first
application.

##  3\. The case $\vec{k}=(k_{1},k_{2},k_{3})$

Throughout this section, let $\vec{k}=(k_{1},k_{2},k_{3})$ be a vector of
three positive integers that sum to $n$. For a $\vec{k}$-Dyck path, denote its
red ranks (as defined in Definition 2.7) $r_{1},r_{2},r_{3}$, recalling that
$r_{1}$ is 0. As discussed in Section 2, the bounce of a $\vec{k}$-Dyck path
$D$ is given by equation (2.1), and

$\text{area}(D)=r_{2}+r_{3}\,.$

Our goal in this section is to replicate the following formula due to Xin and
Zhang [17, Section 2.3] for the generating function (1.2), i.e.,

$\operatorname{F}(x_{1},x_{2},x_{3},q,t)=\sum_{k_{1},k_{2},k_{3}\geq
0}x_{1}^{k_{1}}x_{2}^{k_{2}}x_{3}^{k_{3}}\sum_{D\in\mathcal{D}_{(k_{1},k_{2},k_{3})}}q^{\text{area}(D)}\,t^{\text{bounce}(D)}.$

######  Theorem 3.1.

The generating function $\operatorname{F}{}(x_{1},x_{2},x_{3},q,t)$ equals

$\frac{(1-x_{1}x_{2}qt^{2})(1-x_{1}x_{2}q^{2}t)}{(1-x_{2}q)(1-x_{2}t)(1-x_{1}qt)(1-x_{1}t^{2})(1-x_{1}q^{2})(1-x_{1}x_{2}qt)(1-x_{3})}\,.$

Note that this formula is $q,t$-symmetric, and so the $q,t$-symmetry of
$C_{(k_{1},k_{2},k_{3})}(q,t)$ follows immediately. We represent the
$(k_{1},k_{2},k_{3})$-Dyck paths as integer points of cones and then compute
$\operatorname{F}{}(x_{1},x_{2},x_{3},q,t)$ via integer-point transforms.

Xin and Zhang [17] split their computation of this generating function into
two parts, each divided into two cases based on numerical relationships
between the quantities $k_{2}$, $r_{2}$, and $r_{3}$ that are relevant in the
computation of bounce. We consider both the conditions in the bounce formula
and the geometry of our cones to form a coarser decomposition than that of Xin
and Zhang.

Each $(k_{1},k_{2},k_{3})$-Dyck path corresponds to a vector of the form
$(k_{1},k_{2},k_{3},r_{2},r_{3})$ where we omit $r_{1}$, as it always equals
$0$. By definition the following conditions hold for $k_{1},k_{2},k_{3}$-Dyck
paths:

(3.1) $\displaystyle k_{1},k_{2},k_{3}\geq 0,$ (3.2) $\displaystyle k_{1}\geq
r_{2}\geq 0,$ (3.3) $\displaystyle r_{2}+k_{2}\geq r_{3}\geq 0.$

$\left[\begin{smallmatrix}1\\\ 0\\\ 0\\\ 1\\\
0\end{smallmatrix}\right]$$\left[\begin{smallmatrix}1/2\\\ 1/2\\\ 0\\\ 1/2\\\
0\end{smallmatrix}\right]$$\left[\begin{smallmatrix}0\\\ 1\\\ 0\\\ 0\\\
0\end{smallmatrix}\right]$$\left[\begin{smallmatrix}0\\\ 1\\\ 0\\\ 0\\\
1\end{smallmatrix}\right]$$\left[\begin{smallmatrix}1/2\\\ 1/2\\\ 0\\\ 1/2\\\
1\end{smallmatrix}\right]$$\left[\begin{smallmatrix}1\\\ 0\\\ 0\\\ 1\\\
1\end{smallmatrix}\right]$$C_{1}$$C_{2}$$C_{3}$ Figure 3. Decomposition of the
$k_{1},k_{2},k_{3}$-case illustrated via projection onto
$k_{3}=0,k_{1}+k_{2}=1,\text{ and }k_{1}=r_{1}.$ Solid and dashed lines
indicate included and missing faces of the respective cones.

These inequalities define a 5-dimensional cone with six generators: four of
them depicted in Figure 3 plus the unit vectors in direction $k_{1}$ and
$k_{3}$. This cone naturally decomposes into three simplicial cones, also
shown in Figure 3. For each of these subcones, we write its integer-point
transform

$\displaystyle\sigma_{C_{i}}(z_{1},z_{2},z_{3},w_{2},w_{3})=\sum_{(\vec{k},\vec{r})\in
C_{i}\cap\mathbb{Z}^{5}}z_{1}^{k_{1}}z_{2}^{k_{2}}z_{3}^{k_{3}}w_{2}^{r_{2}}w_{3}^{r_{3}}.$

We now compute these.

###  3.1. Cone $C_{1}$

Let $C_{1}$ be the cone defined by the inequalities (3.1)-(3.3) along with

$r_{2}+k_{2}-r_{3}\geq 2k_{2}\quad\text{ and }\quad r_{2}\geq k_{2}\,.$

Thus, $C_{1}$ has the generators

$\begin{bmatrix}1\\\ 0\\\ 0\\\ 0\\\ 0\end{bmatrix},\begin{bmatrix}0\\\ 0\\\
1\\\ 0\\\ 0\end{bmatrix},\begin{bmatrix}1\\\ 0\\\ 0\\\ 1\\\
0\end{bmatrix},\begin{bmatrix}1\\\ 0\\\ 0\\\ 1\\\
1\end{bmatrix},\begin{bmatrix}1\\\ 1\\\ 0\\\ 1\\\ 0\end{bmatrix}$

and is _unimodular_ : these generators are a lattice basis for
$\mathbb{Z}^{5}$. The corresponding integer-point transform is

$\displaystyle\sigma_{C_{1}}(z_{1},z_{2},z_{3},w_{2},w_{3})$
$\displaystyle=\frac{1}{(1-z_{1})(1-z_{3})(1-z_{1}w_{2})(1-z_{1}w_{2}w_{3})(1-z_{1}z_{2}w_{2})}\,.$

Any $\vec{k}$-Dyck path corresponding to an integer point in $C_{1}$ satisfies
both

$\min(r_{2},k_{2})=k_{2}\text{ and }r_{2}+k_{2}-r_{3}\geq 2\min(r_{2},k_{2}),$

so its bounce is computed by

$\text{bounce}(D)=2k_{1}-r_{2}-r_{3}$

and thus, the contribution of $C_{1}$ to (1.2) is the generating function

$\displaystyle\operatorname{F}_{1}(x_{1},x_{2},x_{3},q,t)$
$\displaystyle=\sum_{(\vec{k},\vec{r})\in
C_{1}\cap\mathbb{Z}^{5}}x_{1}^{k_{1}}x_{2}^{k_{2}}x_{3}^{k_{3}}q^{r_{2}+r_{3}}t^{2k_{1}-r_{2}-r_{3}}.$

Writing down the generating function $\operatorname{F}_{1}$ is equivalent to
replacing each monomial term
$z_{1}^{k_{1}}z_{2}^{k_{2}}z_{3}^{k_{3}}w_{2}^{r_{2}}w_{3}^{r_{3}}$ of
$\sigma_{C_{1}}$ with the corresponding term
$x_{1}^{k_{1}}x_{2}^{k_{2}}x_{3}^{k_{3}}q^{r_{2}+r_{3}}t^{2k_{1}-r_{2}-r_{3}}$,
and so

$\displaystyle\operatorname{F}_{1}(x_{1},x_{2},x_{3},q,t)$
$\displaystyle=\sum_{(\vec{k},\vec{r})\in
C_{1}\cap\mathbb{Z}^{5}}(x_{1}t^{2})^{k_{1}}\left(x_{2}\right)^{k_{2}}\left(x_{3}\right)^{k_{3}}\left(qt^{-1}\right)^{r_{2}}(qt^{-1})^{r_{3}}$
$\displaystyle=\sigma_{C_{1}}(x_{1}t^{2},x_{2},x_{3},qt^{-1},qt^{-1})$
$\displaystyle=\frac{1}{(1-x_{1}t^{2})(1-x_{3})(1-x_{1}qt)(1-x_{1}q^{2})(1-x_{1}x_{2}qt)}\,.$

###  3.2. Cone $C_{2}$

Let $C_{2}$ be the cone defined by the inequalities (3.1)-(3.3) along with

$r_{2}+k_{2}-r_{3}\geq 2k_{2}\quad\text{ and }\quad r_{2}\leq k_{2}\,.$

The cone $C_{2}$ is also unimodular, generated by

$\begin{bmatrix}1\\\ 0\\\ 0\\\ 0\\\ 0\end{bmatrix},\begin{bmatrix}0\\\ 1\\\
0\\\ 0\\\ 0\end{bmatrix},\begin{bmatrix}0\\\ 0\\\ 1\\\ 0\\\
0\end{bmatrix},\begin{bmatrix}1\\\ 1\\\ 0\\\ 1\\\
0\end{bmatrix},\begin{bmatrix}0\\\ 1\\\ 0\\\ 0\\\ 1\end{bmatrix},$

and the corresponding integer-point transform is

$\displaystyle\sigma_{C_{2}}(z_{1},z_{2},z_{3},w_{2},w_{3})$
$\displaystyle=\frac{1}{(1-z_{1})(1-z_{2})(1-z_{3})(1-z_{1}z_{2}w_{2})(1-z_{2}w_{3})}\,.$

For any $\vec{k}$-Dyck path $D$ corresponding to an integer point in this
cone,

$\text{bounce}(D)=2(k_{1}-r_{2})+k_{2}-r_{3}\,,$

and so the contribution of $C_{2}$ to (1.2) is the generating function

$\displaystyle\operatorname{F}_{2}(x_{1},x_{2},x_{3},q,t)$
$\displaystyle=\sum_{(\vec{k},\vec{r})\in
C_{2}\cap\mathbb{Z}^{5}}x_{1}^{k_{1}}x_{2}^{k_{2}}x_{3}^{k_{3}}q^{r_{2}+r_{3}}t^{2(k_{1}-r_{2})+k_{2}-r_{3}}$
$\displaystyle=\sum_{(\vec{k},\vec{r})\in
C_{2}\cap\mathbb{Z}^{5}}(x_{1}t^{2})^{k_{1}}\left(x_{2}t\right)^{k_{2}}\left(x_{3}\right)^{k_{3}}\left(qt^{-2}\right)^{r_{2}}(qt^{-1})^{r_{3}}$
$\displaystyle=\sigma_{C_{2}}(x_{1}t^{2},x_{2}t,x_{3},qt^{-2},qt^{-1})$
$\displaystyle=\frac{1}{(1-x_{1}t^{2})(1-x_{2}t)(1-x_{3})(1-x_{1}x_{2}qt)(1-x_{2}q)}\,.$

###  3.3. Cone $C_{3}$

Let $C_{3}$ be the cone defined by the inequalities (3.1)-(3.3) along with

$r_{2}+k_{2}-r_{3}<2k_{2}\quad\text{ and }\quad r_{2}+k_{2}-r_{3}<2r_{2}\,.$

The cone $C_{3}$ is generated by

$\begin{bmatrix}1\\\ 0\\\ 0\\\ 0\\\ 0\end{bmatrix},\begin{bmatrix}0\\\ 0\\\
1\\\ 0\\\ 0\end{bmatrix},\begin{bmatrix}1\\\ 1\\\ 0\\\ 1\\\
0\end{bmatrix},\begin{bmatrix}1\\\ 0\\\ 0\\\ 1\\\
1\end{bmatrix},\begin{bmatrix}0\\\ 1\\\ 0\\\ 0\\\ 1\end{bmatrix}$

and thus, not unimodular. The fundamental parallelepiped of $C_{3}$ is

$\Pi(C_{3})=\left\\{\lambda_{1}\begin{bmatrix}1\\\ 0\\\ 0\\\ 0\\\
0\end{bmatrix}+\lambda_{2}\begin{bmatrix}0\\\ 0\\\ 1\\\ 0\\\
0\end{bmatrix}+\lambda_{3}\begin{bmatrix}1\\\ 1\\\ 0\\\ 1\\\
0\end{bmatrix}+\lambda_{4}\begin{bmatrix}1\\\ 0\\\ 0\\\ 1\\\
1\end{bmatrix}+\lambda_{5}\begin{bmatrix}0\\\ 1\\\ 0\\\ 0\\\ 1\end{bmatrix}\
\middle|\ 0\leq\lambda_{1},\lambda_{2},\lambda_{3}<1\text{ and
}0<\lambda_{4},\lambda_{5}\leq 1\right\\},$

with $\Pi(C_{3})\cap\mathbb{Z}^{5}=\\{(1,1,0,1,2),(1,1,0,1,1)\\}$. Thus,

(3.4) $\displaystyle\sigma_{C_{3}}(z_{1},z_{2},z_{3},w_{2},w_{3})=$
$\displaystyle\frac{z_{1}z_{2}w_{2}w^{2}_{3}+z_{1}z_{2}w_{2}w_{3}}{(1-z_{1})(1-z_{3})(1-z_{1}z_{2}w_{2})(1-z_{1}w_{2}w_{3})(1-z_{2}w_{3})}\,.$

For the cones $C_{1}$ and $C_{2}$, we used the linearity of the bounce
formulas to write $\operatorname{F}_{1},\operatorname{F}_{2}$ using a change
of variables in $\sigma_{C_{1}},\sigma_{C_{2}}$. However, for a $\vec{k}$-Dyck
path $D$ corresponding to an integer point in $C_{3}$,

$\text{bounce}(D)=2(k_{1}-r_{2})+\left\lceil\frac{r_{2}+k_{2}-r_{3}}{2}\right\rceil.$

In order to obtain a rational formula for $\operatorname{F}_{3}$, we can
partition the integer points in $C_{3}$ into two subsets, such that bounce is
a linear function on each subset. This partition corresponds to the two
summands in the numerator of (3.4). Let

$A=\left\\{\alpha_{1}\begin{bmatrix}1\\\ 0\\\ 0\\\ 0\\\
0\end{bmatrix}+\alpha_{2}\begin{bmatrix}0\\\ 0\\\ 1\\\ 0\\\
0\end{bmatrix}+\alpha_{3}\begin{bmatrix}1\\\ 1\\\ 0\\\ 1\\\
0\end{bmatrix}+\alpha_{4}\begin{bmatrix}1\\\ 0\\\ 0\\\ 1\\\
1\end{bmatrix}+\alpha_{5}\begin{bmatrix}0\\\ 1\\\ 0\\\ 0\\\ 1\end{bmatrix}\\\
\ \middle|\ \alpha_{1},\alpha_{2},\alpha_{3}\in\mathbb{Z}_{\geq 0}\text{ and
}\alpha_{4},\alpha_{5}\in\mathbb{Z}_{>0}\right\\}.$

For the points in $A$, area is computed by
$\alpha_{3}+2\alpha_{4}+\alpha_{5}$, and (since $A$ falls under the second
bounce condition) bounce is computed by

$\displaystyle
2(k_{1}-r_{2})+\left\lceil\frac{r_{2}+k_{2}-r_{3}}{2}\right\rceil$
$\displaystyle=2(\alpha_{1}+\alpha_{3}+\alpha_{4}-\alpha_{3}-\alpha_{4})+\left\lceil\frac{\alpha_{3}+\alpha_{4}+\alpha_{3}+\alpha_{5}-\alpha_{4}-\alpha_{5}}{2}\right\rceil$
$\displaystyle=2\alpha_{1}+\alpha_{3}\,.$

Hence, $A$ contributes the generating function

$\displaystyle\operatorname{F}_{3A}(x_{1},x_{2},x_{3},q,t)$
$\displaystyle=\sum_{\alpha_{i}\in\mathbb{Z}_{\geq
0},\,\alpha_{4},\alpha_{5}\neq
0}(x_{1}t^{2})^{{\alpha_{1}}}(x_{3})^{\alpha_{2}}(x_{1}x_{2}qt)^{\alpha_{3}}(x_{1}q^{2})^{\alpha_{4}}(x_{2}q)^{\alpha_{5}}$
$\displaystyle=\frac{x_{1}x_{2}q^{3}}{(1-x_{1}t^{2})(1-x_{3})(1-x_{1}x_{2}qt)(1-x_{2}q)(1-x_{1}q^{2})}\,.$

The second set of the partition of integer points in $C_{3}$ is

$B=\left\\{\begin{bmatrix}1\\\ 1\\\ 0\\\ 1\\\
1\end{bmatrix}+\alpha_{1}\begin{bmatrix}1\\\ 0\\\ 0\\\ 0\\\
0\end{bmatrix}+\alpha_{2}\begin{bmatrix}0\\\ 0\\\ 1\\\ 0\\\
0\end{bmatrix}+\alpha_{3}\begin{bmatrix}1\\\ 1\\\ 0\\\ 1\\\
0\end{bmatrix}+\alpha_{4}\begin{bmatrix}1\\\ 0\\\ 0\\\ 1\\\
1\end{bmatrix}+\alpha_{5}\begin{bmatrix}0\\\ 1\\\ 0\\\ 0\\\ 1\end{bmatrix}\
\middle|\ \alpha_{i}\in\mathbb{Z}_{\geq 0}\right\\}.$

Area is computed by $\alpha_{3}+2\alpha_{4}+\alpha_{5}+2$, by summing the
fourth and fifth coordinates corresponding to $r_{2}$ and $r_{3}$, and the
constant 2 represents the contribution from the special point $(1,1,0,1,1)$.
Since $B$ falls under the second bounce condition, bounce is computed by

$\displaystyle
2(k_{1}-r_{2})+\left\lceil\frac{r_{2}+k_{2}-r_{3}}{2}\right\rceil=2\alpha_{1}+\alpha_{3}+1\,,$

and so the generating function enumerating points in set $B$ is

$\operatorname{F}_{3B}(x_{1},x_{2},x_{3},q,t)=\frac{x_{1}x_{2}q^{2}t}{(1-x_{1}t^{2})(1-x_{3})(1-x_{1}x_{2}qt)(1-x_{2}q)(1-x_{1}q^{2})}\,.$

Lastly, we combine the generating functions for sets $A$ and $B$ to obtain a
generating function for $(k_{1},k_{2},k_{3})$-Dyck paths in cone $C_{3}$:

$\operatorname{F}_{3}(x_{1},x_{2},x_{3},q,t)=\frac{x_{1}x_{2}q^{3}+x_{1}x_{2}q^{2}t}{(1-x_{1}t^{2})(1-x_{3})(1-x_{1}x_{2}qt)(1-x_{2}q)(1-x_{1}q^{2})}\,.$

###  3.4. Summing over the full cone

By construction, any $(k_{1},k_{2},k_{3})$-Dyck path corresponds to a point in
either $C_{1},C_{2}$ or $C_{3}$. The only points that appear in more than one
cone are points in the cone generated by $\left[\begin{smallmatrix}1\\\ 0\\\
0\\\ 0\\\ 0\end{smallmatrix}\right],\left[\begin{smallmatrix}0\\\ 0\\\ 1\\\
0\\\ 0\end{smallmatrix}\right],\left[\begin{smallmatrix}1\\\ 1\\\ 0\\\ 1\\\
0\end{smallmatrix}\right]$ (represented by the point labeled by
$\left[\begin{smallmatrix}1/2\\\ 1/2\\\ 0\\\ 1/2\\\ 0\end{smallmatrix}\right]$
in Figure 3), which appear in both cones $C_{1}$ and $C_{2}$ and are
enumerated by the generating function

$\sum_{(\vec{k},\vec{r})\in(C_{1}\cap
C_{2})\cap\mathbb{Z}^{5}}x_{1}^{k_{1}}x_{2}^{k_{2}}x_{3}^{k_{3}}q^{r_{2}+r_{3}}t^{2k_{1}-r_{2}-r_{3}}=\frac{1}{(1-x_{1}t^{2})(1-x_{3})(1-x_{1}x_{2}qt)}\,.$

Hence,

$\displaystyle\operatorname{F}{}(x_{1},x_{2},x_{3},q,t)$
$\displaystyle=\operatorname{F}_{1}+\operatorname{F}_{2}+\operatorname{F}_{3}-\frac{1}{(1-x_{1}t)(1-x_{3})(1-x_{1}x_{2}qt)}$
$\displaystyle=\frac{(1-x_{1}x_{2}qt^{2})(1-x_{1}x_{2}q^{2}t)}{(1-x_{2}q)(1-x_{2}t)(1-x_{1}qt)(1-x_{1}t^{2})(1-x_{1}q^{2})(1-x_{1}x_{2}qt)(1-x_{3})}\,,$

and this finishes our proof of Theorem 3.1.

##  4\. The case $\vec{k}=(k,k,k,k)$

We turn our attention to our second application, where we revisit ideas from
the previous section, as well as introduce some new ones. Our goal in this
section is to compute (1.3), i.e.,

$\operatorname{H}(x,q,t):=\sum_{k\in\mathbb{Z}_{\geq
0}}C_{(k,k,k,k)}(q,t)\,x^{k},$

re-establishing the following formula of Xin and Zhang [17].

######  Theorem 4.1.

The generating function $\operatorname{H}(x,q,t)$ equals

$\frac{N}{(1-q^{3}tx)(1-qt^{3}x)(1-q^{2}t^{2}x)(1-q^{6}x)(1-t^{6}x)}\,,$

where

$\displaystyle N=1+$
$\displaystyle\left(q^{5}t+qt^{5}+q^{4}t^{2}+q^{2}t^{4}+q^{4}t+qt^{4}+q^{3}t^{2}+q^{2}t^{3}+q^{3}t^{3}\right)x$
$\displaystyle{}+\left(-q^{7}t^{3}-q^{3}t^{7}+q^{6}t^{5}+q^{5}t^{6}-q^{6}t^{4}-q^{4}t^{6}-q^{5}t^{5}-q^{5}t^{4}-q^{4}t^{5}\right)x^{2}$
$\displaystyle{}-\left(q^{8}t^{8}+q^{9}t^{6}+q^{6}t^{9}+q^{8}t^{7}+q^{7}t^{8}\right)x^{3}.$

Note again the $q,t$-symmetry of $\operatorname{H}$. Throughout this section,
let $\vec{k}=(k,k,k,k)$, where $k$ is a positive integer. Recall that a
$(k,k,k,k)$-Dyck path $D$ can be uniquely identified with the vector
$(k,a,b,c)$ based on the red ranks $(0,r_{2},r_{3},r_{4})$, where

$r_{2}=k-a,\quad r_{3}=2k-a-b,\quad\text{ and }r_{4}=3k-a-b-c.$

We have from [15] that $\operatorname{area}(D)=6k-3a-2b-c$ and

$\displaystyle\text{bounce}(D)$ $\displaystyle=\begin{cases}6a+3b+c-4k&\text{
if }b\geq 2k-2a\text{ and }c\geq 4k-2a-2b,\\\
5a+2b+\lceil\frac{c}{2}\rceil-2k&\text{ if }b\geq 2k-2a\text{ and
}c<4k-2a-2b,\\\ 4a+2b+c-2k&\text{ if }b<2k-2,\,b\text{ is even, and }c\geq
3k-a-\frac{3b}{2},\\\
2a+\frac{b}{2}+k+\left\lceil\frac{3a+\frac{3b}{2}+c-3k}{2}\right\rceil&\text{
if }b<2k-2,\,b\text{ is even, and }\\\ &\quad\quad\quad 3k-3a-\frac{3b}{2}\leq
c<3k-a-\frac{3b}{2},\\\ 3a+b+\frac{c}{3}&\text{ if }b<2k-2a,\,b\text{ is even,
and }c<3k-3a-\frac{3b}{2},\\\ 4a+2b+c-2k+1&\text{ if }b<2k-2a,\,b\text{ is
odd, and }\\\ &\quad\quad\quad c\geq 3k-a-\frac{3(b+1)}{2}+1,\\\
2a+\frac{b+1}{2}+k+&\text{ if }b<2k-2a,\,b\text{ is odd, and }\\\
\quad\quad\frac{3a+\frac{3(b+1)}{2}+c-3k-1}{2}&3k-3a-\frac{3(b+1)}{2}+1\leq
c<3k-a-\frac{3(b+1)}{2}+1,\\\ 3a+b+1+\lceil\frac{c-1}{3}\rceil&\text{ if
}b<2k-2,\,b\text{ is odd, and }\\\ &\quad\quad\quad
c<3k-3a-\frac{3(b+1)}{2}+1.\end{cases}$

We now use the techniques from Section 3 to compute the generating function

$\operatorname{H}(x,y_{1},y_{2},y_{3},q,t)=\sum_{k\geq
0}x^{k}\sum_{(k,a,b,c)\in\mathcal{D}_{(k,k,k,k)}}y_{1}^{a}\ y_{2}^{b}\
y_{3}^{c}\ q^{\text{area}(D)}\ t^{\text{bounce}(D)},$

as a sum of generating functions derived from cones. By definition, all
vectors $(k,a,b,c)$ corresponding to $(k,k,k,k)$-Dyck paths must satisfy the
following inequalities:

(4.1) $\displaystyle 0$ $\displaystyle\leq a\leq k\,,$ (4.2) $\displaystyle 0$
$\displaystyle\leq b\leq 2k-a\,,$ (4.3) $\displaystyle 0$ $\displaystyle\leq
c\leq 3k-a-b\,.$

This system of inequalities corresponds to the cone $C$ generated by

$\displaystyle\begin{bmatrix}1\\\ 0\\\ 2\\\ 0\end{bmatrix},\begin{bmatrix}1\\\
0\\\ 0\\\ 0\end{bmatrix},\begin{bmatrix}1\\\ 1\\\ 0\\\
0\end{bmatrix},\begin{bmatrix}1\\\ 1\\\ 1\\\
0\end{bmatrix},\begin{bmatrix}1\\\ 1\\\ 1\\\
1\end{bmatrix},\begin{bmatrix}1\\\ 0\\\ 2\\\
1\end{bmatrix},\begin{bmatrix}1\\\ 0\\\ 0\\\
3\end{bmatrix},\begin{bmatrix}1\\\ 1\\\ 0\\\ 2\end{bmatrix}.$

We denote these cone generators by $v_{1},v_{2},\dots,v_{8}$, respectively.
Figure 4 depicts a projection of them to the hyperplane $k=1$.

$v_{1}$$v_{2}$$v_{3}$$v_{4}$$v_{5}$$v_{6}$$v_{7}$$v_{8}$$a$$b$$c$ Figure 4.
All generators have first coordinate 1, so we visualize them by projecting to
the last three coordinates.

To compute the integer-point transform

$\sigma_{C}(y,z_{1},z_{2},z_{3})=\sum\limits_{(k,a,b,c)\in
C}y^{k}z_{1}^{a}z_{2}^{b}z_{3}^{c}\,,$

we triangulate $C$ depending on the bounce cases listed above. We follow the
notation in [17] and denote

Part 1 $\displaystyle:b\geq 2k-2a,$ Part 2 $\displaystyle:b<2k-2a\text{
and~{}$b$ even},$ Part 3 $\displaystyle:b<2k-2a\text{ and~{}$b$ odd}.$

Part 1 is split into two cases, Part 2 into three cases where $b$ is always
even, and Part 3 into three cases where $b$ is always odd.

### Part 1

We divide the points satisfying $b\geq 2k-2a$ into two cases:

(4.4) Case 1: $\displaystyle c\geq 4k-2a-2b,$ (4.5) Case 2: $\displaystyle
c<4k-2a-2b.$

Let $C_{1.1}$ be the cone defined by the inequalities (4.1)-(4.3) and (4.4);
it has generators $v_{1},v_{4},v_{5},v_{6},v_{8}$ and is not simplicial, but
can be triangulated into two simplicial cones:

  * •

Cone A generated by $v_{1},v_{4},v_{6},v_{8}$,

  * •

Cone B generated by $v_{4},v_{5},v_{6},v_{8}.$

We remove the facet opposite $v_{5}$ in cone $B$; it equals the facet opposite
$v_{1}$ in cone $A$ which is included there (see Figure 5).

$c$$v_{7}$$v_{1}$$v_{2}$$v_{3}$$v_{4}$$v_{6}$$v_{8}$$v_{5}$$a$$b$BA

$c$$v_{7}$$v_{1}$$v_{3}$$v_{4}$$v_{5}$$v_{6}$$v_{7}$$v_{8}$$a$$b$$c$

Figure 5. Part 1, Case 1 (left) and Case 2 (right). In Part 1, Case 1, the
shared facet for cones A and B is missing in cone B and included in cone A.
The facet shared by the Part 1, Case 1 cone A and Part 1, Case 2 is included
in cone A and missing in Part 1, Case 2.

Both Cone A and Cone B are unimodular, so we can directly compute the integer-
point transforms

$\displaystyle\sigma_{C_{1.1A}}(y,z_{1},z_{2},z_{3})$
$\displaystyle=\frac{1}{(1-yz_{2}^{2})(1-yz_{1}z_{2})(1-yz_{2}^{2}z_{3})(1-yz_{1}z_{3}^{2})}\,,$
$\displaystyle\sigma_{C_{1.1B}}(y,z_{1},z_{2},z_{3})$
$\displaystyle=\frac{yz_{1}z_{2}z_{3}}{(1-yz_{1}z_{2})(1-yz_{1}z_{2}z_{3})(1-yz_{2}^{2}z_{3})(1-yz_{1}z_{3}^{2})}\,,$

which in turn gives

$\displaystyle\sigma_{C_{1.1}}(y,z_{1},z_{2},z_{3})=\sigma_{C_{1A}}+\sigma_{C_{1B}}=\frac{1-y^{2}z_{1}z_{2}^{3}z_{3}}{(1-yz_{2}^{2})(1-yz_{1}z_{2})(1-yz_{2}^{2}z_{3})(1-yz_{1}z_{3}^{2})(1-yz_{1}z_{2}z_{3})}\,.$

Using methods similar to the ones for first two cones in the Section 3, we
compute the generating function

$\displaystyle\operatorname{H}_{1.1}(x,y_{1},y_{2},y_{3},q,t)=\frac{1-x^{2}y_{1}y_{2}^{3}y_{3}q^{2}t^{8}}{(1-xy_{2}^{2}q^{2}t^{2})(1-xy_{1}y_{2}qt^{5})(1-xy_{2}^{2}y_{3}qt^{3})(1-xy_{1}y_{3}^{2}qt^{4})(1-xy_{1}y_{2}y_{3}t^{6})}\,.$

Next, define $C_{1.2}$ to be the cone defined by inequalities (4.1)-(4.3) and
(4.5); it has generators $v_{1},v_{3},v_{4},v_{8}$, and we remove the facet
opposite $v_{3}$. This cone is simplicial (though not unimodular), with
integer-point transform

$\displaystyle\sigma_{C_{1.2}}(y,z_{1},z_{2},z_{3})$
$\displaystyle=\frac{yz_{1}+yz_{1}z_{3}}{(1-yz_{2}^{2})(1-yz_{1})(1-yz_{1}z_{2})(1-yz_{1}z_{3}^{2})}\,,$

as $\Pi(C_{1.2})\cap\mathbb{Z}^{4}=\\{(1,1,0,0),(1,1,0,1)\\}$. We apply the
techniques of Section 3.3 to compute

$\displaystyle\operatorname{H}_{1.2}(x,y_{1},y_{2},y_{3},q,t)=\frac{xy_{1}q^{3}t^{3}+xy_{1}y_{3}q^{2}t^{4}}{(1-xy_{2}^{2}q^{2}t^{2})(1-xy_{1}q^{3}t^{3})(1-xy_{1}y_{2}qt^{5})(1-xy_{1}y_{3}^{2}qt^{4})}\,.$

### Part 2

We divide the points satisfying $b<2k-2a$ with $b$ even into three cases:

(4.6) Case 1: $\displaystyle c\geq 3k-a-\frac{3b}{2}\,,$ (4.7) Case 2:
$\displaystyle 3k-3a-\frac{3b}{2}\leq c<3k-a-\frac{3b}{2}\,,$ (4.8) Case 3:
$\displaystyle c<3k-3a-\frac{3b}{2}\,.$

### Part 3

Similarly, we divide the points satisfying $b<2k-2a$ with $b$ odd into three
cases:

(4.9) Case 1: $\displaystyle c\geq 3k-a-\frac{3(b+1)}{2}+1\,,$ (4.10) Case 2:
$\displaystyle 3k-3a-\frac{3(b+1)}{2}+1\leq c<3k-a-\frac{3(b+1)}{2}+1\,,$
(4.11) Case 3: $\displaystyle c<3k-3a-\frac{3(b+1)}{2}+1\,.$

We treat each case for Parts 2 and 3 in parallel. We start by disregarding the
parity condition on $b$ and consider all points that satisfy the given
inequalities. Once we determine the relevant cone generating function, we
extract the points with the correct parity condition for $b$.

### Case 1

We start with Part 2, Case 1. Let $C_{2.1}$ be the cone defined by the
inequalities (4.1)-(4.3) and (4.6); it has generators
$v_{1},v_{6},v_{7},v_{8}$, and the facet opposite $v_{7}$ is missing.

$c$$v_{7}$$v_{1}$$v_{2}$$v_{3}$$v_{4}$$v_{5}$$v_{6}$$v_{7}$$v_{8}$$a$$b$$c$
Figure 6. Part 2, Case 1 and Part 3, Case 1 have the same generators but a
different apices.

Its fundamental parallelepiped contains the two integer points
$\Pi(C_{2.1})\cap\mathbb{Z}^{4}=\\{(1,0,0,3),(1,0,1,2)\\}$ and so the
corresponding integer-point transform is

$\displaystyle\sigma_{C_{2.1}}(y,z_{1},z_{2},z_{3})$
$\displaystyle=\frac{yz_{3}^{3}+yz_{2}z_{3}^{2}}{(1-yz_{2}^{2})(1-yz_{2}^{2}z_{3})(1-yz_{3}^{3})(1-yz_{1}z_{3}^{2})}\,.$

We are interested in extracting all points $(k,a,b,c)$ enumerated by this
generating function such that $b$ is even. Since the coordinate $b$ of a given
point corresponds to the power of $z_{2}$ in the monomial, this is equivalent
to considering monomial terms where $z_{2}$ is raised to an even power. Since
$z_{2}$ appears only with even powers in the denominator, the two numerator
terms of $\sigma_{C_{2.1}}$ correspond to generating functions enumerating
points $(k,a,b,c)$ where $b$ is even and odd, respectively. Since we are only
interested in the case that $b$ is even, we simply disregard the term
$yz_{2}z_{3}^{2}$ in the numerator. It remains to apply the techniques of
Sections 3.1 and 3.2 to compute the generating function

$\displaystyle\operatorname{H}_{2.1}(x,y_{1},y_{2},y_{3},q,t)$
$\displaystyle=\frac{xy_{3}^{3}q^{3}t}{(1-xy_{2}^{2}q^{2}t^{2})(1-xy_{2}^{2}y_{3}qt^{3})(1-xy_{3}^{3}q^{3}t)(1-xy_{1}y_{3}^{2}qt^{4})}\,.$

Next, we consider Part 3, Case 1, where we eventually require that $b$ is odd.
The inequalities (4.1)-(4.3) and (4.9) reduce to the following system of
inequalities:

(4.12) $\displaystyle 0\leq a,$ (4.13) $\displaystyle 0\leq b,$ (4.14)
$\displaystyle 0\leq c,$ (4.15) $\displaystyle c\leq 3k-a-b,$ (4.16)
$\displaystyle b<2k-2a,$ (4.17) $\displaystyle 3k-a-\frac{3(b+1)}{2}+1\leq c.$

The last inhomogeneous inequality makes computations more subtle. For example,
(4.12), (4.16), and (4.17) imply

$-\frac{1}{2}\leq 2a-\frac{1}{2}\leq 3k-3k-a+3a-\frac{1}{2}\leq
3k-a-\frac{3}{2}b-\frac{1}{2}\leq c,$

but since $c\in\mathbb{Z}$ the implication is the redundant assumption that
$0\leq c$.

Inequalities (4.15) and (4.17) imply $-1\leq b$, which is weaker than (4.13).
To simplify our computations, we first assume $-1\leq b$ and compute a cone
generating function, and then impose the condition $0\leq b$ by eliminating
the points where $b=-1$. Hence, we consider the cone $C_{3.1}$ defined by the
following minimal hyperplane description:

$\displaystyle 0\leq a,$ $\displaystyle c\leq 3k-a-b,$ $\displaystyle
b<2k-2a,$ $\displaystyle 3k-a-\frac{3(b+1)}{2}+1\leq c.$

The cone $C_{3.1}$ has generators $v_{1},v_{6},v_{7},v_{8}$, and the facet
opposite $v_{7}$ is missing. Though these are the same generators as of
$C_{2.1}$, the cone $C_{3.1}$ is a translation of $C_{2.1}$ such that its apex
is at $(-\frac{1}{2},0,-1,-\frac{1}{2})$ rather than the origin. With
$\Pi(C_{3.1})\cap\mathbb{Z}^{4}=\\{(0,0,-1,1),(1,0,0,2)\\}$, we compute

$\displaystyle\sigma_{C_{3.1}}(y,z_{1},z_{2},z_{3})$
$\displaystyle=\frac{z_{2}^{-1}z_{3}+yz_{3}^{2}}{(1-yz_{2}^{2})(1-yz_{2}^{2}z_{3})(1-yz_{3}^{3})(1-yz_{1}z_{3}^{2})}\,.$

To impose the condition $0\leq b$, we first subtract all terms where $b=-1$,
i.e., $z_{2}^{-1}$:

$\displaystyle\sigma_{C_{3.1}}(y,z_{1},z_{2},z_{3})-\frac{z_{2}^{-1}z_{3}}{(1-yz_{3}^{3})(1-yz_{1}z_{3}^{2})}$
$\displaystyle=\frac{yz_{3}^{2}-y^{2}z_{2}^{3}z_{3}^{2}+yz_{2}z_{3}+yz_{2}z_{3}^{2}}{(1-yz_{2}^{2})(1-yz_{2}^{2}z_{3})(1-yz_{3}^{3})(1-yz_{1}z_{3}^{2})}.$

Finally, we need to extract the terms for odd $b$, i.e., the monomials with
odd powers of $z_{2}$. Again, the denominator contributes only even powers of
$z_{2}$, and so we simply need to disregard the first term in the numerator,
giving rise to the resulting generating function

$\displaystyle\frac{-y^{2}z_{2}^{3}z_{3}^{2}+yz_{2}z_{3}+yz_{2}z_{3}^{2}}{(1-yz_{2}^{2})(1-yz_{2}^{2}z_{3})(1-yz_{3}^{3})(1-yz_{1}z_{3}^{2})}\,.$

From this, we obtain the contribution

$\displaystyle\operatorname{H}_{3.1}(x,y_{1},y_{2},y_{3},q,t)$
$\displaystyle=\frac{xy_{2}y_{3}q^{2}t^{2}(-xy_{2}^{2}y_{3}q^{2}t^{3}+q+y_{3}t)}{(1-xy_{2}^{2}q^{2}t^{2})(1-xy_{2}^{2}y_{3}qt^{3})(1-xy_{3}^{3}q^{3}t)(1-xy_{1}y_{3}^{2}qt^{4})}\,.$

### Case 2

We next consider Part 2, Case 2. Let $C_{2.2}$ be the cone defined by the
inequalities (4.1)-(4.3) and (4.7); it has generators
$v_{1},v_{3},v_{7},v_{8}$, and we remove the facets opposite to $v_{3},v_{7}$.

$c$$v_{7}$$v_{2}$$v_{1}$$v_{3}$$v_{4}$$v_{5}$$v_{6}$$v_{7}$$v_{8}$$a$$b$$c$
Figure 7. Part 2, Case 2 and Part 3, Case 2 have the same generators, but
different apices; both visible facets are missing.

Here, $\Pi(C_{2.2})$ contains four integer points, and the integer-point
transform is

$\displaystyle\sigma_{C_{2.2}}(y,z_{1},z_{2},z_{3})=\frac{y^{2}z_{1}z_{3}^{3}+y^{2}z_{1}z_{3}^{4}+y^{2}z_{1}z_{2}z_{3}+y^{2}z_{1}z_{2}z_{3}^{2}}{(1-yz_{2}^{2})(1-yz_{1})(1-yz_{3}^{3})(1-yz_{1}z_{3}^{2})}\,.$

Imposing the condition that $b$ is even means disregarding the last two terms
in the numerator, from which we obtain the generating function

$\displaystyle\operatorname{H}_{2.2}(x,y_{1},y_{2},y_{3},q,t)$
$\displaystyle=\frac{x^{2}y_{1}y_{3}^{3}q^{6}t^{4}+x^{2}y_{1}y_{3}^{4}q^{5}t^{5}}{(1-xy_{2}^{2}q^{2}t^{2})(1-xy_{1}q^{3}t^{3})(1-xy_{3}^{3}q^{3}t)(1-xy_{1}y_{3}^{2}qt^{4})}\,.$

Part 3, Case 2 proceeds in a similar fashion. We define the cone $C_{3.2}$ by
the following system of inequalities reduced from inequalities (4.1)-(4.3) and
(4.10):

$\displaystyle 0$ $\displaystyle\leq b,$ $\displaystyle b$
$\displaystyle<2k-2a,$ $\displaystyle c$ $\displaystyle\geq
3k-3a-\frac{3(b+1)}{2}+1,$ $\displaystyle c$
$\displaystyle<3k-a-\frac{3(b+1)}{2}+1.$

The cone $C_{3.2}$ is a translation of the cone $C_{2.2}$ such that its apex
is at $(0,0,0,-\frac{1}{2})$. Again, $\Pi(C_{3.2})$ contains four integer
points, and the resulting integer-point transform is

$\displaystyle\sigma_{C_{3.2}}(y,z_{1},z_{2},z_{3})$
$\displaystyle=\frac{y^{2}z_{1}z_{2}z_{3}+y^{2}z_{1}z_{2}z_{3}^{2}+y^{2}z_{1}z_{3}^{3}+y^{2}z_{1}z_{3}^{4}}{(1-yz_{2}^{2})(1-yz_{1})(1-yz_{3}^{3})(1-yz_{1}z_{3}^{2})}\,.$

To impose the condition that $b$ is odd, we disregard the last two terms in
the numerator, from which we obtain the generating function

$\displaystyle\operatorname{H}_{3.2}(q,t,x,y_{1},y_{2},y_{3})$
$\displaystyle=\frac{x^{2}y_{1}y_{2}y_{3}q^{5}t^{5}(q+y_{3}t)}{(1-xy_{2}^{2}q^{2}t^{2})(1-xy_{1}q^{3}t^{3})(1-q^{3}txy_{3}^{3})(1-xy_{1}y_{3}^{2}qt^{4})}\,.$

### Case 3

Let $C_{2.3}$ be the cone defined by inequalities (4.1)-(4.3) and (4.8); it
has generators $v_{1},v_{2},v_{3},v_{7}$, and we discard the facet opposite to
$v_{2}$.

$c$$v_{7}$$v_{2}$$v_{2}$$v_{1}$$v_{3}$$v_{4}$$v_{5}$$v_{6}$$v_{7}$$v_{8}$$a$$b$$c$
Figure 8. Part 2, Case 3 and Part 3, Case 3 have the same generators but
different apices; the visible face is missing.

Since

$\Pi(C_{2.3})\cap\mathbb{Z}^{4}=\\{(1,0,0,0),(1,0,1,0),(1,0,0,1),(1,0,1,1),(1,0,0,2),(2,0,1,2)\\}\,,$

the integer-point transform is

$\displaystyle\sigma_{C_{2.3}}(y,z_{1},z_{2},z_{3})$
$\displaystyle=\frac{y+yz_{2}+yz_{3}+yz_{2}z_{3}+yz_{3}^{2}+y^{2}z_{2}z_{3}^{2}}{(1-yz_{2}^{2})(1-y)(1-yz_{1})(1-yz_{3}^{3})}\,.$

We are only interested in the points where $b$ is even, and so we disregard
the three terms in the numerator with $z_{2}$, arriving at the generating
function

$\displaystyle\operatorname{H}_{2.3}(q,t,x,y_{1},y_{2},y_{3})$
$\displaystyle=\frac{xq^{4}(q^{2}+y_{3}qt+y_{3}^{2}t)}{(1-xy_{2}^{2}q^{2}t^{2})(1-xq^{6})(1-xy_{1}q^{3}t^{3})(1-xy_{3}^{3}q^{3}t)}\,.$

For Part 3, Case 3, let $C_{3.3}$ be the cone defined by inequalities
(4.1)-(4.3) and (4.11); it has the same generators as $C_{2.3}$, but is
translated to place its apex at $(\frac{1}{6},0,0,0)$ rather than the origin.
Since

$\Pi(C_{3.3})\cap\mathbb{Z}^{4}=\\{(1,0,0,0),(1,0,1,0),(1,0,0,1),(2,0,1,1),(1,0,0,2),(2,0,1,2)\\}\,,$

the integer-point transform is

$\displaystyle\sigma_{C_{3.3}}(y,z_{1},z_{2},z_{3})$
$\displaystyle=\frac{y+yz_{2}+yz_{3}+y^{2}z_{2}z_{3}+yz_{3}^{2}+y^{2}z_{2}z_{3}^{2}}{(1-yz_{2}^{2})(1-y)(1-yz_{1})(1-yz_{3}^{3})}\,.$

We are now only interested in the points where $b$ is odd and so we disregard
the three terms in the numerator without $z_{2}$; the resulting generating
function is

$\displaystyle\operatorname{H}_{3.3}(q,t,x,y_{1},y_{2},y_{3})$
$\displaystyle=\frac{xy_{2}q^{4}t^{2}+x^{2}y_{2}y_{3}q^{9}t^{2}+x^{2}y_{2}y_{3}^{2}q^{8}t^{3}}{(1-xy_{2}^{2}q^{2}t^{2})(1-xq^{6})(1-xy_{1}q^{3}t^{3})(1-xy_{3}^{3}q^{3}t)}\,.$

Adding the eight generating functions yields

$\displaystyle\operatorname{H}(q,t,x,y_{1},y_{2},y_{3})=$
$\displaystyle\frac{1-q^{2}t^{8}x^{2}y_{1}y_{2}^{3}y_{3}}{(1-q^{2}t^{2}xy_{2}^{2})(1-qt^{5}xy_{1}y_{2})(1-qt^{3}xy_{2}^{2}y_{3})(1-qt^{4}xy_{1}y_{3}^{2})(1-t^{6}xy_{1}y_{2}y_{3})}$
$\displaystyle{}+\frac{xy_{1}q^{2}t^{3}(q+ty_{3})}{(1-q^{2}t^{2}xy_{2}^{2})(1-q^{3}t^{3}xy_{1})(1-qt^{5}xy_{1}y_{2})(1-qt^{4}xy_{1}y_{3}^{2})}$
$\displaystyle{}+\frac{q^{3}txy_{3}^{3}}{(1-q^{2}t^{2}xy_{2}^{2})(1-q^{3}txy_{3}^{3})(1-qt^{3}xy_{2}^{2}y_{3})(1-qt^{4}xy_{1}y_{3}^{2})}$
$\displaystyle{}+\frac{q^{5}t^{4}x^{2}y_{1}y_{3}^{3}(q+ty_{3})}{(1-q^{2}t^{2}xy_{2}^{2})(1-q^{3}txy_{3}^{3})(1-q^{3}t^{3}xy_{1})(1-qt^{4}xy_{1}y_{3}^{2})}$
$\displaystyle{}+\frac{q^{4}x(q^{2}+ty_{3}^{2}+qty_{3})}{(1-q^{2}t^{2}xy_{2}^{2})(1-q^{6}x)(1-q^{3}txy_{3}^{3})(1-q^{3}t^{3}xy_{1})}$
$\displaystyle{}+\frac{q^{2}t^{2}xy_{2}y_{3}(q+ty_{3}-q^{2}t^{3}xy_{2}^{2}y_{3})}{(1-q^{2}t^{2}xy_{2}^{2})(1-q^{3}txy_{3}^{3})(1-qt^{3}xy_{2}^{2}y_{3})(1-qt^{4}xy_{1}y_{3}^{2})}$
$\displaystyle{}+\frac{x^{2}y_{1}y_{2}y_{3}q^{5}t^{5}(q+y_{3}t)}{(1-q^{2}t^{2}xy_{2}^{2})(1-q^{3}txy_{3}^{3})(1-q^{3}t^{3}xy_{1})(1-qt^{4}xy_{1}y_{3}^{2})}$
$\displaystyle{}+\frac{q^{4}t^{2}xy_{2}(1+q^{5}xy_{3}+q^{4}txy_{3}^{2})}{(1-q^{2}t^{2}xy_{2}^{2})(1-q^{6}x)(1-q^{3}txy_{3}^{3})(1-q^{3}t^{3}xy_{1})}\,.$

Setting $y_{2}=y_{3}=y_{4}=1$ we obtain

$\operatorname{H}(x,1,1,1,q,t)=\frac{N}{(1-q^{3}tx)(1-qt^{3}x)(1-q^{2}t^{2}x)(1-q^{6}x)(1-t^{6}x)}\,,$

where

$\displaystyle N=1+$
$\displaystyle\left(q^{5}t+qt^{5}+q^{4}t^{2}+q^{2}t^{4}+q^{4}t+qt^{4}+q^{3}t^{2}+q^{2}t^{3}+q^{3}t^{3}\right)x$
$\displaystyle{}+\left(-q^{7}t^{3}-q^{3}t^{7}+q^{6}t^{5}+q^{5}t^{6}-q^{6}t^{4}-q^{4}t^{6}-q^{5}t^{5}-q^{5}t^{4}-q^{4}t^{5}\right)x^{2}$
$\displaystyle{}-\left(q^{8}t^{8}+q^{9}t^{6}+q^{6}t^{9}+q^{8}t^{7}+q^{7}t^{8}\right)x^{3},$

and this completes the proof of Theorem 4.1.

##  5\. The case $\vec{k}=(k,a,\dots,a)$

Based on computational data, we make the following conjecture about
$q,t$-symmetry of the refined $q,t$-Catalan numbers $C_{\vec{k}}(q,t)$.

######  Conjecture 5.1.

If $\vec{k}=(k,a,\ldots,a)$, for positive integers $k,a$, then the refined
$q,t$-Catalan numbers $C_{\vec{k}}(q,t)$ are $q,t$-symmetric.

We settle this conjecture in the four-dimensional case when $k\leq a$. As in
the case of $\vec{k}=(k,k,k,k)$, our current understanding of the bounce
formula makes further progress daunting. Our goal in this section is to
compute (1.4), i.e.,

$\operatorname{G}(x,y,q,t):=\sum_{k,m\in\mathbb{Z}_{\geq
0}}C_{\overrightarrow{k}=(k,k+m,k+m,k+m)}(q,t)\,x^{k}\,y^{m}.$

######  Theorem 5.2.

If $k,m\geq 0$, then

$\displaystyle\operatorname{G}(x,y,q,t)=\frac{M}{(1-xq^{6})(1-xt^{6})(1-xq^{3}t)(1-xqt^{3})(1-xq^{2}t^{2})(1-yq^{3})(1-yt^{3})(1-yqt)}\,,$

where

$\displaystyle M=(q^{13}t^{7}$
$\displaystyle+q^{7}t^{13}+q^{12}t^{8}+q^{8}t^{12}+q^{9}t^{12}+q^{12}t^{9}+q^{11}t^{11}+q^{10}t^{11}+q^{11}t^{10}+q^{9}t^{11}+q^{11}t^{9}+q^{10}t^{10})x^{3}y^{2}$
$\displaystyle-(q^{11}t^{5}+q^{5}t^{11}+q^{10}t^{6}+q^{6}t^{10}+q^{8}t^{9}+q^{9}t^{8}+q^{7}t^{9}+q^{9}t^{7}+q^{8}t^{8}-q^{7}t^{8}-q^{8}t^{7}+q^{7}t^{7})x^{2}y^{2}$
$\displaystyle-(q^{7}t^{4}+q^{4}t^{7}+q^{6}t^{5}+q^{5}t^{6})xy^{2}$
$\displaystyle-(q^{12}t^{6}+q^{6}t^{12}+q^{8}t^{11}+q^{11}t^{8}+q^{10}t^{8}+q^{8}t^{10}+q^{7}t^{11}+q^{11}t^{7}+q^{7}t^{10}+q^{10}t^{7}+2q^{9}t^{9}+q^{8}t^{9}+q^{9}t^{8})x^{3}y$
$\displaystyle-(q^{10}t^{3}+q^{3}t^{10}+q^{5}t^{7}+q^{7}t^{5}-q^{5}t^{9}-q^{9}t^{5}+q^{4}t^{9}+q^{9}t^{4}+q^{4}t^{8}+q^{8}t^{4}+q^{5}t^{6}+q^{6}t^{5}-q^{8}t^{6}-q^{6}t^{8})x^{2}y$
$\displaystyle+(qt^{8}+q^{8}t+qt^{7}+q^{7}t+q^{6}t^{3}+q^{3}t^{6}+q^{4}t^{5}+q^{5}t^{4}+q^{2}t^{5}+q^{5}t^{2}+2q^{4}t^{4}+q^{5}t^{3}+q^{3}t^{5}+2q^{3}t^{4}+2q^{4}t^{3}$
$\displaystyle\qquad\;\;\;+q^{7}t^{2}+q^{2}t^{7}+q^{6}t^{2}+q^{2}t^{6})xy$
$\displaystyle-(q^{2}t+qt^{2})y$
$\displaystyle+(q^{9}t^{6}+q^{6}t^{9}+q^{8}t^{7}+q^{7}t^{8}+q^{8}t^{8})x^{3}+(q^{7}t^{3}+q^{3}t^{7}-q^{5}t^{6}-q^{6}t^{5}+q^{4}t^{6}+q^{6}t^{4}+q^{5}t^{5}+q^{4}t^{5}+q^{5}t^{4})x^{2}$
$\displaystyle-(qt^{5}+q^{5}t+q^{2}t^{4}+q^{4}t^{2}+qt^{4}+q^{4}t+q^{3}t^{3}+q^{2}t^{3}+q^{3}t^{2})x\;-\;1\,.$

Note once more the $q,t$-symmetry in this formula.

A $(k,k+m,k+m,k+m)$-Dyck path $D$ can be uniquely identified with the vector
$(k,m,a,b,c)$ based on the red ranks $(0,r_{2},r_{3},r_{4})$ where

$r_{2}=k-a,\quad r_{3}=2k+m-a-b,\quad r_{4}=3k+2m-a-b-c.$

We compute $\text{area}(D)=6k+3m-3a-2b-c$ and

$\text{bounce(D)}=\left\\{\begin{array}[]{ll}3a&b,c=0,\\\
3a+\left\lceil\frac{c}{3}\right\rceil&b=0,\,0<c\leq 3(k-a),\\\
2a+k+\left\lceil\frac{c-3(k-a)}{2}\right\rceil&b=0,\,c>3(k-a),\\\
3a+2\left\lceil\frac{b}{2}\right\rceil&0<b\leq 2(k-a),\,c=0,\\\
3a+2\left\lceil\frac{b}{2}\right\rceil+\left\lceil\frac{c}{3}\right\rceil&0<b\leq
2(k-a),\,b\text{ is even and }\\\ &\quad\quad 0<c\leq
3\left(k-a-\left\lceil\frac{b}{2}\right\rceil\right),\\\
3a+2\left\lceil\frac{b}{2}\right\rceil+\left\lceil\frac{c-1}{3}\right\rceil&0<b\leq
2(k-a),\,b\text{ is odd and }\\\ &\quad\quad 0\leq c-1\leq
3\left(k-a-\left\lceil\frac{b}{2}\right\rceil\right),\\\
2a+\left\lceil\frac{b}{2}\right\rceil+k+\left\lceil\frac{c-3(k-a-\left\lceil\frac{b}{2}\right\rceil)}{2}\right\rceil&0<b\leq
2(k-a),\,b\text{ is even and }\\\ &\quad\quad
3\left(k-a-\left\lceil\frac{b}{2}\right\rceil\right)<c\leq
3(k-\left\lceil\frac{b}{2}\right\rceil)-a+2m,\\\
2a+\left\lceil\frac{b}{2}\right\rceil+k+&0<b\leq 2(k-a),\,b\text{ is odd and
}\\\
\quad\quad\left\lceil\frac{c-1-3\left(k-a-\left\lceil\frac{b}{2}\right\rceil\right)}{2}\right\rceil&\quad\quad
3\left(k-a-\left\lceil\frac{b}{2}\right\rceil\right)<c-1\leq
3(k-\left\lceil\frac{b}{2}\right\rceil)-a+2m,\\\
c-2k+4a+4\left\lceil\frac{b}{2}\right\rceil-m&0<b\leq 2(k-a),\,b\text{ is even
and }\\\ &\quad\quad c>3(k-\left\lceil\frac{b}{2}\right\rceil)-a+2m,\\\
c-1-2k+4a+4\left\lceil\frac{b}{2}\right\rceil-m&0<b\leq 2(k-a),\,b\text{ is
odd and }\\\ &\quad\quad c-1>3(k-\left\lceil\frac{b}{2}\right\rceil)-a+2m,\\\
5a+2b-2k+\left\lceil\frac{c}{2}\right\rceil&b>2(k-a),\,0\leq c\leq
2(2k-a+m-b),\\\ 6a+3b-4k+c-m&b>2(k-a),\,c>2(2k-a+m-b).\end{array}\right.$

By definition, all vectors $(k,ma,b,c)$ corresponding to
$(k,k+m,k+m,k+m)$-Dyck paths must satisfy the following inequalities:

(5.1) $\displaystyle 0$ $\displaystyle\leq a\leq k,$ (5.2) $\displaystyle 0$
$\displaystyle\leq m,$ (5.3) $\displaystyle 0$ $\displaystyle\leq b\leq
2k+m-a,$ (5.4) $\displaystyle 0$ $\displaystyle\leq c\leq 3k+2m-a-b.$

This system of inequalities corresponds to the cone $C$ generated by

$\displaystyle\begin{bmatrix}1\\\ 0\\\ 0\\\ 2\\\
0\end{bmatrix},\begin{bmatrix}1\\\ 0\\\ 1\\\ 0\\\
0\end{bmatrix},\begin{bmatrix}1\\\ 0\\\ 0\\\ 0\\\
0\end{bmatrix},\begin{bmatrix}0\\\ 1\\\ 0\\\ 0\\\
0\end{bmatrix},\begin{bmatrix}0\\\ 1\\\ 0\\\ 1\\\
0\end{bmatrix},\begin{bmatrix}1\\\ 0\\\ 1\\\ 1\\\
0\end{bmatrix},\begin{bmatrix}1\\\ 0\\\ 1\\\ 1\\\
1\end{bmatrix},\begin{bmatrix}1\\\ 0\\\ 0\\\ 2\\\
1\end{bmatrix},\begin{bmatrix}1\\\ 0\\\ 1\\\ 0\\\
2\end{bmatrix},\begin{bmatrix}1\\\ 0\\\ 0\\\ 0\\\
3\end{bmatrix},\begin{bmatrix}0\\\ 1\\\ 0\\\ 0\\\
2\end{bmatrix},\begin{bmatrix}0\\\ 1\\\ 0\\\ 1\\\ 1\end{bmatrix}.$

We denote these cone generators by $v_{1},\dots,v_{12}$. Similarly to the
$(k,k,k,k)$ case we triangulate $C$ depending on the bounce cases listed
above. We denote

(5.5) Part 1: $\displaystyle b=0,$ (5.6) Part 2: $\displaystyle 0<b\leq
2(k-a)\,\text{and}\,c=0,$ (5.7) Part 3: $\displaystyle 0<b\leq 2(k-a),\,c\neq
0,\,\text{and $b$ is even},$ (5.8) Part 4: $\displaystyle 0<b\leq
2(k-a),\,c\neq 0,\,\text{ and $b$ is odd},$ (5.9) Part 5: $\displaystyle
b>2(k-a).$

Some parts are further split into subcases. We do not exhibit the details of
the ensuing computations, as the generating-function methods are similar to
those for the $\vec{k}=(k_{1},k_{2},k_{3})$ and $\vec{k}=(k,k,k,k)$ cases
above. However, we do give the result for each subcase.

### Part 1

We divide the points with $b=0$ into three cases:

(5.10) Case 1: $\displaystyle c=0,$ (5.11) Case 2: $\displaystyle 0<c\leq
3(k-a),$ (5.12) Case 3: $\displaystyle c>3(k-a).$

Case 1 gives rise to the unimodular cone $C_{1.1}$ defined by the inequalities
5.1-5.4, 5.5, and 5.10, with generating function

$\operatorname{G}_{1.1}(x,y,z_{1},q,t)=\frac{1}{(1-xz_{1}q^{3}t^{3})(1-yq^{3})(1-xq^{6})}\,.$

Case 2 gives the simplicial cone $C_{1.2}$ defined by the inequalities
5.1-5.4, 5.5 and 5.11, with generating function

$\operatorname{G}_{1.2}(x,y,z_{1},z_{3},q,t)=\frac{xz_{3}q^{3}t(q^{2}+z_{3}q+z_{3}^{2})}{(1-xz_{3}^{3}q^{3}t)(1-xq^{6})(1-xz_{1}q^{3}t^{3})(1-yq^{3})}\,.$

Case 3 gives the cone $C_{1.3}$ defined by the inequalities 5.1-5.4, 5.5 and
5.12. It needs to be triangulated, giving rise to the generating function

$\displaystyle\operatorname{G}_{1.3}(x,y,z_{1},z_{3},q,t)$
$\displaystyle=\frac{yz_{3}q^{2}t+yz_{3}^{2}qt}{(1-xz_{3}^{3}q^{3}t)(1-yq^{3})(1-xz_{1}q^{3}t^{3})(1-yz_{3}^{2}qt)}$
$\displaystyle\qquad+\frac{xz_{1}z_{3}q^{2}t^{4}+xz_{1}z_{3}^{2}qt^{4}}{(1-xz_{3}^{3}q^{3}t)(1-xz_{1}q^{3}t^{3})(1-xz_{1}z_{3}^{2}qt^{4})(1-yz_{3}^{2}qt)}\,.$

### Part 2

Let $C_{2}$ be the simplicial cone defined by the inequalities 5.1-5.4 and
5.6. It gives rise to the generating function

$\operatorname{G}_{2}(x,y,z_{1},z_{2},z_{3},q,t)=\frac{xz_{2}q^{4}t^{2}+xz_{2}^{2}q^{2}t^{2}}{(1-xz_{1}q^{3}t^{3})(1-yq^{3})(1-xz_{2}^{2}q^{2}t^{2})(1-xq^{6})}\,.$

### Part 3

We divide the points with $b$ even, $c\neq 0$, and $0<b\leq 2(k-a)$ into three
cases:

(5.13) Case 1: $\displaystyle 0<c\leq 3\left(k-a-\frac{b}{2}\right)\,,$ (5.14)
Case 2: $\displaystyle 3\left(k-a-\frac{b}{2}\right)<c\leq
3\left(k-\frac{b}{2}\right)-a+2m\,,$ (5.15) Case 3: $\displaystyle
c>3\left(k-\frac{b}{2}\right)-a+2m\,.$

Case 1 gives the simplicial cone $C_{3.1}$ defined by the inequalities
5.1-5.4, 5.7 and 5.13; we need only consider the points with even $b$ values.
This gives the generating function

$\operatorname{G}_{3.1}(x,y,z_{1},z_{2},z_{3},q,t)=\frac{x^{2}z_{2}^{2}z_{3}q^{7}t^{3}+x^{2}z_{2}^{2}z_{3}^{2}q^{6}t^{3}+x^{2}z_{2}^{2}z_{3}^{3}q^{5}t^{3}}{(1-yq^{3})(1-xz_{2}^{2}q^{2}t^{2})(1-xz_{1}q^{3}t^{3})(1-xz_{3}^{3}q^{3}t)(1-xq^{6})}\,.$

Case 2 gives the cone $C_{3.2}$ defined by inequalities 5.1-5.4, 5.7 and 5.14;
again we need only consider the points with even $b$ value. After a
triangulation, we compute the generating function

$\displaystyle\operatorname{G}_{3.2}(x,y,z_{1},z_{2},z_{3},q,t)$
$\displaystyle=\frac{xyz_{2}^{2}z_{3}q^{4}t^{3}+xyz_{2}^{2}z_{3}^{2}q^{3}t^{3}}{(1-yq^{3})(1-yz_{3}^{2}qt)(1-xz_{3}^{3}q^{3}t)(1-xz_{2}^{2}q^{2}t^{2})(1-xz_{1}q^{3}t^{3})}$
$\displaystyle\qquad+\frac{x^{2}z_{1}z_{2}^{2}z_{3}q^{4}t^{6}+x^{2}z_{1}z_{2}^{2}z_{3}^{2}q^{3}t^{6}}{(1-yz_{3}^{2}qt)(1-xz_{3}^{3}q^{3}t)(1-xz_{2}^{2}q^{2}t^{2})(1-xz_{1}q^{3}t^{3})(1-xz_{1}z_{3}^{2}qt^{4})}\,.$

Case 3 gives the simplicial cone $C_{3.3}$ defined by the inequalities 5.1-5.4
, 5.7 and 5.15; once more we need only consider the points with even $b$
value. Its generating function is

$\operatorname{G}_{3.3}(x,y,z_{1},z_{2},z_{3},q,t)=\frac{xz_{2}^{2}z_{3}qt^{3}}{(1-xz_{3}^{3}q^{3}t)(1-yz_{3}^{2}qt)(1-xz_{1}z_{3}^{2}qt^{4})(1-xz_{2}^{2}q^{2}t^{2})(1-xz_{2}^{2}z_{3}qt^{3})}\,.$

### Part 4

We similarly divide the points with $b$ odd and $0<b\leq 2(k-a)$ into three
cases:

(5.16) Case 1: $\displaystyle 0\leq c-1\leq
3\left(k-a-\frac{b+1}{2}\right)\,,$ (5.17) Case 2: $\displaystyle
3\left(k-a-\frac{b+1}{2}\right)<c-1\leq 3\left(k-\frac{b+1}{2}\right)-a+2m\,,$
(5.18) Case 3: $\displaystyle 3\left(k-\frac{b+1}{2}\right)-a+2m<c-1\,.$

For Case 1, we consider the points with odd $b$ value of the cone $C_{4.1}$
defined by the inequalities 5.1-5.4, 5.8 and 5.16, the last of which in this
context simplifies to

$0<c<3k-3a-\frac{3b}{2}\,.$

It comes with the generating function

$\operatorname{G}_{4.1}(x,y,z_{1},z_{2},z_{3},q,t)=\frac{xz_{2}z_{3}q^{3}t^{2}+x^{2}z_{2}z_{3}^{2}q^{8}t^{3}+x^{2}z_{2}z_{3}^{3}q^{7}t^{3}}{(1-yq^{3})(1-xz_{2}^{2}q^{2}t^{2})(1-xz_{1}q^{3}t^{3})(1-xz_{3}^{3}q^{3}t)(1-xq^{6})}\,.$

Similarly, Case 2 is captured by the points with odd $b$ value of the cone
$C_{4.2}$ defined by the inequalities 5.1-5.4, 5.8, and 5.17, the last of
which in this context simplifies to

$3k-3a-\frac{3b}{2}\leq c<3k-\frac{3b}{2}-a+2m\,.$

After a triangulation, its generating function is

$\displaystyle\operatorname{G}_{4.2}(x,y,z_{1},z_{2},z_{3},q,t)$
$\displaystyle=\frac{xyz_{2}z_{3}^{2}q^{5}t^{3}+xyz_{2}z_{3}^{3}q^{4}t^{3}}{(1-xz_{3}^{3}q^{3}t)(1-xz_{2}^{2}q^{2}t^{2})(1-yz_{3}^{2}qt)(1-xz_{1}q^{3}t^{3})(1-yq^{3})}$
$\displaystyle\qquad{}+\frac{x^{2}z_{1}z_{2}z_{3}^{2}q^{5}t^{6}+x^{2}z_{1}z_{2}z_{3}^{3}q^{4}t^{6}}{(1-xz_{3}^{3}q^{3}t)(1-xz_{2}^{2}q^{2}t^{2})(1-yz_{3}^{2}qt)(1-xz_{1}q^{3}t^{3})(1-xz_{1}z_{3}^{2}qt^{4})}\,.$

For Case 3, we consider the points with odd $b$ value of the cone $C_{4.3}$
defined by the inequalities 5.1-5.4, 5.8, and 5.18, the last of which
simplifies to

$c\geq 3k-\frac{3b}{2}-a+2m\,.$

The corresponding generating function is

$\operatorname{G}_{4.3}(x,y,z_{1},z_{2},z_{3},q,t)=\frac{xz_{2}z_{3}^{2}q^{2}t^{3}}{(1-xz_{3}^{3}q^{3}t)(1-yz_{3}^{2}qt)(1-xz_{1}z_{3}^{2}qt^{4})(1-xz_{2}^{2}q^{2}t^{2})(1-xz_{2}^{2}z_{3}qt^{3})}\,.$

### Part 5

We divide the points with $b>2(k-a)$ into two cases:

(5.19) Case 1: $\displaystyle 0\leq c\leq 2(2k-a+m-b)\,,$ (5.20) Case 2:
$\displaystyle c>2(2k-a+m-b)\,.$

For Case 1, we consider the cone $C_{5.1}$ defined by the inequalities
5.1-5.4, 5.9, and 5.19. After a triangulation, we compute its generating
function as

$\displaystyle\operatorname{G}_{5.1}(x,y,z_{1},z_{2},z_{3},q,t)$
$\displaystyle=\frac{yz_{2}qt^{2}+y^{2}z_{2}z_{3}q^{3}t^{3}}{(1-yq^{3})(1-yz_{3}^{2}qt)(1-yz_{2}qt^{2})(1-xz_{2}^{2}q^{2}t^{2})(1-xz_{1}q^{3}t^{3})}$
$\displaystyle\qquad{}+\frac{xyz_{1}z_{2}z_{3}q^{3}t^{6}+xyz_{1}z_{2}z_{3}^{2}q^{2}t^{6}}{(1-yz_{3}^{2}qt)(1-yz_{2}qt^{2})(1-xz_{2}^{2}q^{2}t^{2})(1-xz_{1}q^{3}t^{3})(1-xz_{1}z_{3}^{2}qt^{4})}$
$\displaystyle\qquad{}+\frac{xz_{1}z_{2}qt^{5}+x^{2}z_{1}^{2}z_{2}z_{3}q^{3}t^{9}}{(1-yz_{2}qt^{2})(1-xz_{2}^{2}q^{2}t^{2})(1-xz_{1}q^{3}t^{3})(1-xz_{1}z_{3}^{2}qt^{4})(1-xz_{1}z_{2}qt^{5})}\,.$

Finally, for Case 2, let $C_{5.2}$ be the cone defined by the inequalities
5.1-5.4, 5.9 and 5.20. With another triangulation, we compute the generating
function

$\displaystyle\operatorname{G}_{5.2}(x,y,z_{1},z_{2},z_{3},q,t)$
$\displaystyle=\frac{yz_{2}z_{3}t^{3}}{(1-yz_{3}^{2}qt)(1-yz_{2}qt^{2})(1-yz_{2}z_{3}t^{3})(1-xz_{2}^{2}q^{2}t^{2})(1-xz_{1}z_{3}^{2}qt^{4})}$
$\displaystyle\qquad{}+\frac{xyz_{2}^{3}z_{3}^{2}qt^{6}}{(1-yz_{3}^{2}qt)(1-yz_{2}z_{3}t^{3})(1-xz_{2}^{2}q^{2}t^{2})(1-xz_{2}^{2}z_{3}qt^{3})(1-xz_{1}z_{3}^{2}qt^{4})}$
$\displaystyle\qquad{}+\frac{xyz_{1}z_{2}^{2}z_{3}qt^{8}}{(1-yz_{2}qt^{2})(1-yz_{2}z_{3}t^{3})(1-xz_{2}^{2}q^{2}t^{2})(1-xz_{1}z_{3}^{2}qt^{4})(1-xz_{1}z_{2}qt^{5})}$
$\displaystyle\qquad{}+\frac{x^{2}z_{1}z_{2}^{3}z_{3}q^{2}t^{8}}{(1-yz_{2}z_{3}t^{3})(1-xz_{2}^{2}q^{2}t^{2})(1-xz_{2}^{2}z_{3}qt^{3})(1-xz_{1}z_{3}^{2}qt^{4})(1-xz_{1}z_{2}qt^{5})}$
$\displaystyle\qquad{}+\frac{xz_{1}z_{2}z_{3}t^{6}}{(1-yz_{2}z_{3}t^{3})(1-xz_{2}^{2}z_{3}qt^{3})(1-xz_{1}z_{3}^{2}qt^{4})(1-xz_{1}z_{2}qt^{5})(1-xz_{1}z_{2}z_{3}t^{6})}\,.$

Setting $z_{1},z_{2},z_{3}=1$ and adding all the generating functions yields
Theorem 5.2.

##  6\. Further directions

###  6.1. Revisiting Xin and Zhang’s conjecture

As previously mentioned, Xin and Zhang [17] conjectured that if
$\lambda=((a+1)^{s},a^{n-s})$ with $0\leq s\leq n$, then $C_{\lambda}(q,t)$ is
$q,t$-symmetric. Since $C_{\lambda}(q,t)$ involves taking a sum over possibly
multiple $C_{\vec{k}}(q,t)$, it is interesting to further ask for which
$\vec{k}$ we have $q,t$-Catalan symmetry. For example, Xin and Zhang note that
$C_{\lambda=(1,1,1,3)}(q,t)$ is not $q,t$-symmetric. Breaking this up into the
constituent $C_{\vec{k}}(q,t)$, we find that

$C_{\vec{k}=(1,1,1,3)}(q,t)\qquad\text{ and }\qquad
C_{\vec{k}=(3,1,1,1)}(q,t)$

are $q,t$-symmetric (see Figure 9), but

$C_{\vec{k}=(1,1,3,1)}(q,t)\qquad\text{ and }\qquad
C_{\vec{k}=(1,3,1,1)}(q,t)$

are not $q,t$-symmetric (see Figure 10). Since the asymmetries present in
$C_{\vec{k}=(1,1,3,1)}(q,t)$ and $C_{\vec{k}=(1,3,1,1)}(q,t)$ do not cancel
out, $C_{\lambda=(1,1,1,3)}(q,t)$ is not $q,t$-symmetric.

$c$$v_{7}$$v_{2}$$v_{2}$11111111111111$q^{0}$$q^{1}$$q^{2}$$q^{3}$$q^{4}$$q^{5}$$q^{6}$$t^{0}$$t^{1}$$t^{2}$$t^{3}$$t^{4}$$t^{5}$$t^{6}$

$c$$v_{7}$$v_{2}$$v_{2}$111111111111111111111111233333211111$q^{0}$$q^{1}$$q^{2}$$q^{3}$$q^{4}$$q^{5}$$q^{6}$$q^{7}$$q^{8}$$q^{9}$$q^{10}$$q^{11}$$q^{12}$$t^{0}$$t^{1}$$t^{2}$$t^{3}$$t^{4}$$t^{5}$$t^{6}$$t^{7}$$t^{8}$$t^{9}$$t^{10}$$t^{11}$$t^{12}$

Figure 9. Coefficients of $C_{\vec{k}=(1,1,1,3)}(q,t)$ (left) and of
$C_{\vec{k}=(3,1,1,1)}(q,t)$ (right). Both are $q,t$-symmetric. Reading the
tables: the lower left box is $(0,0)$, the coefficient in box $(i,j)$
corresponds to the summand $q^{i}t^{j}$.

$c$$v_{7}$$v_{2}$$v_{2}$11111111111122112211$q^{0}$$q^{1}$$q^{2}$$q^{3}$$q^{4}$$q^{5}$$q^{6}$$q^{7}$$q^{8}$$t^{0}$$t^{1}$$t^{2}$$t^{3}$$t^{4}$$t^{5}$$t^{6}$$t^{7}$$t^{8}$

$c$$v_{7}$$v_{2}$$v_{2}$111111111111111121112321111110$q^{0}$$q^{1}$$q^{2}$$q^{3}$$q^{4}$$q^{5}$$q^{6}$$q^{7}$$q^{8}$$q^{9}$$q^{10}$$t^{0}$$t^{1}$$t^{2}$$t^{3}$$t^{4}$$t^{5}$$t^{6}$$t^{7}$$t^{8}$$t^{9}$$t^{10}$

Figure 10. Coefficients of $C_{\vec{k}=(1,1,3,1)}(q,t)$ (left) and of
$C_{\vec{k}=(1,3,1,1)}(q,t)$ (right). Both are not $q,t$-symmetric, with
asymmetries highlighted in red.

This contrasts with examples which fall within Xin and Zhang’s conjecture,
such as $C_{\lambda=(1,1,1,2)}(q,t)$ which is $q,t$-symmetric. There,

$C_{\vec{k}=(1,1,1,2)}(q,t)\qquad\text{ and }\qquad
C_{\vec{k}=(2,1,1,1)}(q,t)$

are $q,t$-symmetric (see Figure 11), but

$C_{\vec{k}=(1,1,2,1)}(q,t)\qquad\text{ and }\qquad
C_{\vec{k}=(1,2,1,1)}(q,t)$

are not $q,t$-symmetric (see Figure 12). However, this time the asymmetries
present in $C_{\vec{k}=(1,1,2,1)}(q,t)$ and $C_{\vec{k}=(1,2,1,1)}(q,t)$ do
cancel out, so $C_{\lambda=(1,1,1,2)}(q,t)$ is $q,t$-symmetric.

$c$$v_{7}$$v_{2}$$v_{2}$11111111111111$q^{0}$$q^{1}$$q^{2}$$q^{3}$$q^{4}$$q^{5}$$q^{6}$$t^{0}$$t^{1}$$t^{2}$$t^{3}$$t^{4}$$t^{5}$$t^{6}$

$c$$v_{7}$$v_{2}$$v_{2}$111111111111111111222211$q^{0}$$q^{1}$$q^{2}$$q^{3}$$q^{4}$$q^{5}$$q^{6}$$q^{7}$$q^{8}$$q^{9}$$t^{0}$$t^{1}$$t^{2}$$t^{3}$$t^{4}$$t^{5}$$t^{6}$$t^{7}$$t^{8}$$t^{9}$

Figure 11. Coefficients of $C_{\vec{k}=(1,1,1,2)}(q,t)$ (left) and of
$C_{\vec{k}=(2,1,1,1)}(q,t)$ (right). Both are $q,t$-symmetric. Reading the
tables: the lower left box is $(0,0)$, the coefficient in box $(i,j)$
corresponds to the summand $q^{i}t^{j}$.

$c$$v_{7}$$v_{2}$$v_{2}$11111111111211211$q^{0}$$q^{1}$$q^{2}$$q^{3}$$q^{4}$$q^{5}$$q^{6}$$q^{7}$$t^{0}$$t^{1}$$t^{2}$$t^{3}$$t^{4}$$t^{5}$$t^{6}$$t^{7}$

$c$$v_{7}$$v_{2}$$v_{2}$1111111111111111221110$q^{0}$$q^{1}$$q^{2}$$q^{3}$$q^{4}$$q^{5}$$q^{6}$$q^{7}$$q^{8}$$t^{0}$$t^{1}$$t^{2}$$t^{3}$$t^{4}$$t^{5}$$t^{6}$$t^{7}$$t^{8}$

Figure 12. Coefficients of $C_{\vec{k}=(1,1,2,1)}(q,t)$ (left) and of
$C_{\vec{k}=(1,2,1,1)}(q,t)$ (right). Both are individually not
$q,t$-symmetric, with asymmetries highlighted in red. Upon combining them,
their sum is $q,t$-symmetric.

###  6.2. Varying the last parameter

We start with a result which shows that the final entry in the vector
$\vec{k}$ does not affect its area or bounce.

######  Proposition 6.1.

Let

$(k_{1},\dots,k_{j})\in\mathbb{Z}_{>0}^{j}\quad\text{ and
}\quad(a_{1},\dots,a_{j})\in\mathbb{Z}_{\geq 0}^{j}$

be such that the $a_{i}$ are the number of sequential east steps of a valid
$(k_{1},\dots,k_{j})$-Dyck path (i.e. take $a_{1}$ east steps after the size
$k_{1}$ north steps, etc.). Let $l,m\in\mathbb{Z}_{>0}$. The bounce and area
formulas for the $(k_{1},\dots,k_{j},l)$-Dyck path and
$(k_{1},\dots,k_{j},m)$-Dyck path with east steps $(a_{1},\dots,a_{j})$ are
identical.

###### Proof.

In the course of applying the bounce algorithm (2.9), bounce can be computed
as soon as the top row of the rank tableau is complete. We claim that all the
numbers in the top row of the rank tableau are entered without knowledge of
the length of the final north step.

The initial numbers in the top row are entered while the bounce path lies
below the final north steps. Eventually we arrive at a point $P_{i}$ such that
traveling north from $P_{i}$ the next east steps are along the top boundary of
the $n\times n$ square containing the Dyck path. We count $v_{i}$, the number
of north steps this last leg of the bounce path traverses, and enter this many
copies of $i$ in the rank tableau—completing the top row, and arriving at the
moment we can compute bounce. Therefore, the values of $l$ and $m$ are
immaterial to the bounce computation, only the fact that they are non-zero
matters.

For either Dyck path we have

$\displaystyle r_{1}$ $\displaystyle=k_{1}-a_{1}\,,$ $\displaystyle r_{2}$
$\displaystyle=k_{1}-a_{1}+k_{2}-a_{2}\,.$ $\displaystyle\vdots$
$\displaystyle r_{j}$ $\displaystyle=k_{1}-a_{1}+\dots+k_{j}-a_{j}\,,$

and since area for either path is the sum of these $r_{i}$, area also does not
depend on $l,m$. ∎

By combining Proposition 6.1 with the definition of $C_{\vec{k}}(q,t)$, we
obtain the following result.

######  Corollary 6.2.

For any positive integers $k_{1},\ldots,k_{j},m,l$,

$C_{\vec{k}=(k_{1},\ldots,k_{j},m)}(q,t)=C_{\vec{k}=(k_{1},\ldots,k_{j},l)}(q,t)\,.$

###### Proof.

Without loss of generality, assume $m<l$. Given a $(k_{1},\dots,k_{j},m)$-Dyck
path, we can add $l-m$ units to the final north step, and $l-m$ east steps
after the final east step. The result is a valid $(k_{1},\dots,k_{j},l)$-Dyck
path.

On the other hand, given a $(k_{1},\dots,k_{j},l)$-Dyck path, we can subtract
$l-m$ units from the final north step. We need to delete $l-m$ east steps from
the end in order to create a valid $(k_{1},\dots,k_{j},m)$-Dyck path. The
total number of east steps that can occur in the original
$(k_{1},\dots,k_{j},l)$-Dyck path prior to the final north step is
$k_{1}+\dots+k_{j}$, and the final total is $k_{1}+\dots+k_{j}+l$, so at least
$l$ east steps must occur after the final north step. Therefore, we can delete
$l-m$ east steps from the end and still have a valid Dyck path.

We have described a bijection on $(k_{1},\dots,k_{j},m)$\- and
$(k_{1},\dots,k_{j},l)$-Dyck paths, and by Proposition 6.1 this bijection
preserves area and bounce, proving the claim. ∎

######  Remark 6.3.

We can see Corollary 6.2 in action in the following examples:

  * •

For $m\in\mathbb{Z}_{>0}$ the bounce and area formulas for $(k,k,k,m)$-Dyck
paths are the same as the bounce and area formulas for $(k,k,k,k)$-Dyck paths.

  * •

The formula for all $(k_{1},k_{2},m)$-Dyck paths are also the same regardless
of $m\in\mathbb{Z}_{>0}$, the formula for bounce in the
$(k_{1},k_{2},k_{3})$-Dyck path case does not depend on $k_{3}$.

  * •

Similarly, Niu points out in [15] that the formula for $(k,k,k,k-1)$-Dyck
paths is the same as that of $(k,k,k,k)$-Dyck paths.

  * •

If Conjecture 5.1 holds, Corollary 6.2 implies that for positive integers
$k,a,l$, that $C_{\vec{k}}(q,t)$ with $\vec{k}=(k,a,\ldots,a,l)$ is also
$q,t$-symmetric.

Note Corollary 6.2 does not allow us to extend the bounce formula for a
$(k_{1},\ldots,k_{m})$-Dyck path to a bounce formula for a
$(k_{1},\ldots,k_{m},k_{m+1})$-Dyck path; the fact that we have an additional
north step of size $k_{m+1}$ significantly changes the computation.

###  6.3. Other generalized $q,t$-Catalan numbers

We briefly discuss the relationship between Xin-Zhang’s refined $q,t$-Catalan
numbers and other generalizations which have appeared in the literature. We
already saw that with $\vec{k}=(\underbrace{1,\ldots,1}_{n})$, we recover the
classical $q,t$-Catalan numbers $C_{n}(q,t)$. Taking
$\vec{k}=(\underbrace{m,\ldots,m}_{n})$, we recover the higher $q,t$-Catalan
numbers $C_{n}^{(m)}(q,t)$ [5], which are a special case of the rational
$q,t$-Catalan numbers. Combinatorial formulas for $C_{n}^{(m)}(q,t)$ were
conjectured by Haglund et al. [9] and Loehr [13], and proven as a consequence
of the $(m,n)$-shuffle theorem, conjectured by Haglund et al. [9], refined by
Bergeron et al. [2], and proven by Mellit [14]. Note that in both of these
cases, there is no difference between $C_{\vec{k}}(q,t)$ and
$C_{\lambda}(q,t)$ since there is only one unique rearrangement of the vector.

In 2020, Gorsky et al. [6] introduced a generalized $q,t$-Catalan number,
which was further studied by Blasiak et al. [3], where it is defined as

$C_{\textbf{b}}:=\langle s_{(|\textbf{b}|)},\omega(D_{\textbf{b}}\cdot
1)\rangle\,,$

$s_{(|\textbf{b}|)}$ is a single-row Schur function, $\omega$ is an involution
on symmetric functions, and $D_{\mathbf{b}}$ is a certain operator in the
Schiffmann algebra introduced by Negut. It is interesting to speculate on the
relationship between $C_{\textbf{b}}$ and the $q,t$-Catalan numbers of Xin-
Zhang, since they depend on a vector of integers, or positive integers,
respectively, and generalize the higher $q,t$-Catalan numbers. For example,
the generalized $q,t$-Catalan number

$C_{\mathbf{b}=(1,2,1,0)}=q^{4}+q^{3}t+q^{2}t^{2}+qt^{3}+t^{4}+q^{2}t+qt^{2}$

equals the refined $q,t$-Catalan number

$C_{\vec{k}=(1,2,1)}=q^{4}+q^{3}t+q^{2}t^{2}+qt^{3}+t^{4}+q^{2}t+qt^{2}$

(and neither is a higher $q,t$-Catalan number). Furthermore, Gorsky et al.
showed that the generalized $q,t$-Catalan number does not depend on $b_{1}$,
the first entry of the vector $\mathbf{b}$. This is similar to our remark in
Section 6.2 that the refined $q,t$-Catalan number does not depend on the final
entry of $\vec{k}$ (the way the diagrams are drawn is flipped between these
settings).

This might suggest that $C_{\mathbf{b}}(q,t)$ and $C_{\vec{k}}(q,t)$ are equal
when $\mathbf{b}=(b_{1},\ldots,b_{\ell},0)=(b_{\ell},\ldots,b_{1})=\vec{k}$,
however, the following example shows that, in general, this is not the case:

$C_{\mathbf{b}=(2,1,2,2,0)}(q,t)-C_{\vec{k}=(2,2,1,2)}(q,t)=q^{2}t^{7}+q^{4}t^{4}+q^{3}t^{5}-q^{4}t^{3}-q^{3}t^{4}-q^{2}t^{5}\,.$

Perhaps understanding when $C_{\mathbf{b}}(q,t)=C_{\vec{k}}(q,t)$ would allow
for some progress on a conjecture of Negut [6] (and extended by Blasiak et al.
[3, Conjecture 7.1.1]).

Gorsky et al. [6, Remark 2.4] give an example where the coefficients of the
generalized $q,t$-Catalan number $C_{\textbf{b}}$ are negative. Some of Xin-
Zhang’s $q,t$-Catalan numbers are not $q,t$-symmetric, while all of the
generalized $q,t$-Catalan numbers are $q,t$-symmetric by construction from
Negut’s operator (see [3, Equation (3)]). As a result, neither generalization
is contained within the other.

## Acknowledgements & Funding

This work was initiated at the 2023 GEMS of Combinatorics Workshop held at the
American Institute of Mathematics (AIM). The authors thank the workshop
organizers and AIM for fostering a fruitful research atmosphere. We thank
Carolina Benedetti, Laura Colmenarejo, and Sophie Rehberg for conversations at
the start of the project. We also thank Mark Haiman for helpful conversations.
Lentfer is supported by the National Science Foundation Graduate Research
Fellowship DGE-2146752. Vindas-Meléndez was partially supported by the
National Science Foundation under Award DMS-2102921. Waddle is partially
supported by the National Science Foundation RTG grant DMS-1840234.

## References

  * [1] Matthias Beck and Raman Sanyal, _Combinatorial reciprocity theorems: An invitation to enumerative geometric combinatorics_ , Grad. Stud. Math., vol. 195, Providence, RI: American Mathematical Society (AMS), 2018.
  * [2] Francois Bergeron, Adriano Garsia, Emily Sergel Leven, and Guoce Xin, _Compositional $(km,kn)$-shuffle conjectures_, International Mathematics Research Notices 2016 (2015), no. 14, 4229–4270.
  * [3] Jonah Blasiak, Mark Haiman, Jennifer Morse, Anna Pun, and George H. Seelinger, _A shuffle theorem for paths under any line_ , Forum of Mathematics, Pi 11 (2023), e5.
  * [4] Adriano M. Garsia and James Haglund, _A proof of the $q,t$-Catalan positivity conjecture_, Discrete Math. 256 (2002), no. 3, 677–717.
  * [5] Adriano M. Garsia and Mark Haiman, _A remarkable $q,t$-Catalan sequence and $q$-Lagrange inversion_, J. Algebraic Combin. 5 (1996), no. 3, 191–244.
  * [6] Eugene Gorsky, Graham Hawkes, Anne Schilling, and Julianne Rainbolt, _Generalized $q,t$-Catalan numbers_, Algebraic Combinatorics 3 (2020), no. 4, 855–886.
  * [7] James Haglund, _Conjectured statistics for the $q,t$-Catalan numbers._, Adv. Math. 175 (2003), no. 2, 319–334.
  * [8] by same author, _The $q,t$-Catalan numbers and the space of diagonal harmonics: With an appendix on the combinatoric of Macdonald polynomials_, American Mathematical Society, 2008.
  * [9] James Haglund, Mark Haiman, Nicholas A. Loehr, Jeffrey B. Remmel, and Alexander Ulyanov, _A combinatorial formula for the character of the diagonal coinvariants_ , Duke Mathematical Journal 126 (2005), no. 2, 195–232.
  * [10] Mark Haiman, _Conjectures on the quotient ring by diagonal invariants_ , J. Algebr. Comb. 3 (1994), no. 1, 17–76.
  * [11] by same author, _$t,q$ -Catalan numbers and the Hilbert scheme._, Discrete Math. 193 (1998), no. 1-3, 201–224.
  * [12] by same author, _Vanishing theorems and character formulas for the Hilbert scheme of points in the plane_ , Invent. Math. 149 (2002), no. 2, 371–407.
  * [13] Nicholas A. Loehr, _Conjectured statistics for the higher $q,t$-Catalan sequences_, Electron. J. Comb. 12 (2005), no. Research paper R9, 54 pages.
  * [14] Anton Mellit, _Toric braids and $(m,n)$-parking functions_, Duke Mathematical Journal 170 (2021), no. 18, 4123–4169.
  * [15] Wanmeng Niu, _The $q,t$-symmetry of the generalized $q,t$-Catalan number $c_{\lambda=(k^{s},(k-1)^{n-s})}(q,t)$ when $n=4$_, 2022, M.S. Thesis, Capital Normal University (Written in Chinese).
  * [16] Richard P. Stanley, _Catalan numbers_ , Cambridge: Cambridge University Press, 2015.
  * [17] Guoce Xin and Yingrui Zhang, _The $q,t$-symmetry of the generalized $q,t$-Catalan number $c_{(k_{1},k_{2},k_{3})}(q,t)$ and $c_{(k,k,k,k)}(q,t)$_, 2022, arXiv:2104.04338.
  * [18] by same author, _Dinv, area, and bounce for $\overrightarrow{k}$-Dyck paths_, Adv. in Appl. Math. 143 (2023), Paper No. 102455, 19 pages.