# Enumerating Staircase Diagrams and Smooth Schubert Varieties over type $E$
Dynkin Diagrams

Andean Medjedovic<EMAIL_ADDRESS>and  William Slofstra
<EMAIL_ADDRESS>
###### Abstract.

We enumerate the number of staircase diagrams over classically finite $E$-type
Dynkin diagrams, extending the work of Richmond and Slofstra (Staircase
Diagrams and Enumeration of smooth Schubert varieties) and completing the
enumeration of staircase diagrams over finite type Dynkin diagrams. The
staircase diagrams are in bijection to smooth and rationally smooth Schubert
varieties over $E$-type thereby giving an enumeration of these varieties.

##  1\. Introduction

###  1.1. Staircase Diagrams

The goal of this paper is to complete the enumeration done by Slofstra and
Richmond in [5]. It was known from [5] and [1] that it is possible to
enumerate rationally smooth Schubert varieties through what is known as a
Billey-Postnikov decomposition. There is a bijection, derived in one of the
authors previous works, between staircase diagrams, and these smooth
varieties.

Informally, a staircase diagram is a construction of “blocks” over some
underlying graph. Each block is some subset of the vertices of the graph, with
some ordering of the blocks. We also require each block to “see” both up and
down. This corresponds to a minimal and maximal in the poset for every given
block. To illustrate the idea, what follows is a staircase diagram on the line
graph of length $9$, another diagram on a star graph with $3$ leaves and a
non-example.

, ,

Let $E_{Disc.}(x)=\sum_{n=1}^{\infty}e_{n}x^{n}$, where $e_{n}$ is the number
of smooth Schubert varieties of finite classical type $E_{n}$. The main result
of the paper is

######  Theorem 1.1.

$E_{Disc.}(x)=\frac{P_{Disc.}(x)+Q_{Disc.}(x)\sqrt{1-4x}}{R_{Disc.}(x)}.$

Where

$P_{Disc.}(x)=1536x^{16}-9586x^{15}+20762x^{14}-19750x^{13}+10942x^{12}-15139x^{11}+27760x^{10}-28954x^{9}$
$+16898x^{8}-4690x^{7}-173x^{6}+689x^{5}-454x^{4}+238x^{3}-73x^{2}+8x$
$Q_{Disc.}(x)=3224x^{16}-13230x^{15}+21016x^{14}-15930x^{13}+4800x^{12}+3759x^{11}-10616x^{10}+13958x^{9}$
$-10482x^{8}+4200x^{7}-695x^{6}-95x^{5}+168x^{4}-140x^{3}+57x^{2}-8x$

and

$R_{Disc.}(x)=(-1+x)^{4}(-1+6x-8x^{2}+4x^{3}).$

The growth rate of the coefficients of a generating series is determined by
the singularity of smallest modulus. This result can be found in [3]. In this
case it is one of the roots of $\left(-1+6x-8x^{2}+4x^{3}\right)$,

$\alpha:=\frac{1}{6}\left(4-\sqrt[3]{17+3\sqrt{33}}+\sqrt[3]{-17+3\sqrt{33}}\right)\approx
0.228155.$

Then $e_{n}\sim\frac{1}{\alpha^{n+1}}$. This agree with the growth rate of all
enumerations over classical finite type Dynkin diagrams [5].

###  1.2. Outline

We begin by introducing formally the notion of a staircase diagram and
definitions corresponding to it. We then formalize tools to dissect these
diagrams into parts that are easier to enumerate combinatorially, giving
plenty of examples along they way to build intuition. We prove a few lemmas
relating to the possible endings staircase diagrams can have and the form
these endings take. We define sight-sets of a partial staircase diagram which
encodes how close to being a valid staircase diagram it is. In section $4$ we
work on the connected $E$-type case using generating function techniques
related to dyck paths and the theory so far developed. In the penultimate
section we deal with the more complicated disconnected case using the now
complete classification for connected staircase diagrams. Finally, we leave
with an explanation of how staircase diagrams relate to smooth Schubert
varieties and the Billey-Postnikov decomposition [1].

###  1.3. Acknowledgements

We would like to thank NSERC for providing the funding for this research, and
the members of IQC for helpful discussions. Finally we would like to thank
Jang Soo Kim, for TikZ plane partition code which was modified to make the
staircase diagrams within the paper.

##  2\. Preliminary Definitions

We use the definition of staircase diagram as in [5]. Let $\Gamma$ be a graph
and $S$ be the set of vertices in $\Gamma$. If $s,t\in S$ we write $s$ adj $t$
to mean that $s$ is adjacent to $t$. Let $\mathcal{D}\subseteq 2^{S}$ and
$\prec$, a poset on $\mathcal{D}$. Elements $\mathcal{B}\in\mathcal{D}$ are
called _blocks_ of $\mathcal{D}$. Let $\mathcal{B}\in\mathcal{D}$. We say
$\mathcal{B}$ is connected if the induced subgraph of $\mathcal{B}$ is
connected in $\Gamma$. Recall that $\mathcal{B}$ covers
$\mathcal{B}^{\prime}$, in the partial order under $\prec$, if
$\mathcal{B}\succ\mathcal{B}^{\prime}$ and there is no
$\mathcal{B}^{\prime\prime}\in\mathcal{D}$ so that
$\mathcal{B}\succ\mathcal{B}^{\prime\prime}\succ\mathcal{B}^{\prime}$. A
subset $C\subset\mathcal{D}$ is a chain if it is totally ordered. A subset
$C\subset\mathcal{D}$ is saturated if there is no
$\mathcal{B}^{\prime\prime}\in\mathcal{D}\setminus C$ with
$\mathcal{B}\succ\mathcal{B}^{\prime\prime}\succ\mathcal{B}^{\prime}$ for any
$\mathcal{B},\mathcal{B}^{\prime}\in\mathcal{D}$.

Given some vertex in $\Gamma$ we define $\mathcal{D}_{s}$ to be the blocks
over $S$:

$\mathcal{D}_{s}:=\\{\mathcal{B}\in\mathcal{D}\ |\ s\in\mathcal{B}\\}.$

We now define the staircase diagram:

######  Definition 2.1.

Let $\mathcal{D}=(\mathcal{D},\preceq)$ be a partially ordered subset of
$2^{S}$ not containing the empty set. We say that $\mathcal{D}$ is a
_staircase diagram_ if the following are true:

  1. (1)

Every $\mathcal{B}\in\mathcal{D}$ is connected, and if $\mathcal{B}$ covers
$\mathcal{B}^{\prime}$ then $\mathcal{B}\cup\mathcal{B}^{\prime}$ is
connected.

  2. (2)

The subset $\mathcal{D}_{s}$ is a chain for every $s\in S$.

  3. (3)

If $s\operatorname{adj}t$, then $\mathcal{D}_{s}\cup\mathcal{D}_{t}$ is a
chain, and $\mathcal{D}_{s}$ and $\mathcal{D}_{t}$ are saturated subchains of
$\mathcal{D}_{s}\cup\mathcal{D}_{t}$.

  4. (4)

If $\mathcal{B}\in\mathcal{D}$, then there is some $s\in S$ (resp.
$s^{\prime}\in S$) such that $\mathcal{B}$ is the minimum element of
$\mathcal{D}_{s}$ (resp. maximum element of $\mathcal{D}_{s^{\prime}}$).

In later parts of the paper we deal with type $E$ Dynkin diagrams and
introduce other constructions on staircase diagrams over $E$-type diagrams. It
is useful to have the following definition:

######  Definition 2.2.

A star graph is defined to be a tree with the property that only $1$ node
(called the branching vertex) has degree greater than $2$. A branch of a star
graph is the path from a vertex of degree $1$ to the branching vertex and the
length of a branch is the length of the corresponding path. We let
$\Gamma(a_{0},a_{1},\cdots,a_{k})$ denote a graph with branches of length
$a_{0},\cdots,a_{k}$. We say the branch of length $a_{0}$ is the main branch
of the graph. Let $v_{b}$ denote the branching vertex.

For example, the following is a star graph $\Gamma(6,4,2)$.

######  Example 2.3.

65432178910

And the following is a staircase diagram over that graph.

######  Definition 2.4.

[5] We define $\operatorname{S}(\mathcal{D})$ to be the set of vertices in the
support of a staircase diagram.

$\operatorname{S}(\mathcal{D}):=\bigcup_{\mathcal{B}\in\mathcal{D}}\mathcal{B}.$

We say $\mathcal{D}$ is _connected_ if the support is a connected subset of
the base graph. A subset $\mathcal{D}^{\prime}\subset\mathcal{D}$ is a
_subdiagram_ if $\mathcal{D}^{\prime}$ is a saturated subset of $\mathcal{D}$.

######  Definition 2.5.

A staircase diagram, $D$, is fully supported (on the graph $\Gamma$) if for
each $\gamma\in\Gamma$, $D_{\gamma}$ is non-empty.

We now introduce tools to decompose staircase diagrams. Along each branch we
will have $2$ parts, the broken tower and the regular part.

######  Definition 2.6.

A broken tower over a vertex $s$ is the set $\mathcal{D}_{s}$. Unless
otherwise stated, we consider broken tower diagrams over the branch vertex
$v_{b}$. We say that a restriction to a broken tower diagram over a vertex
$s_{1}$ is a poset of $2^{S}$ containing only sets of the form
$s_{1},\cdots,s_{l}$ and satisfying axioms $(1)-(3)$ of the staircase diagram
definition. In the case that the poset also satisfies axiom $(1)-(4)$ then it
is said to be a tower diagram.

######  Definition 2.7.

Given a subgraph $H\subset G$ and a staircase diagram $\mathcal{D}$ over $G$
the restriction of $\mathcal{D}$ to $H$ is the set
$\mathcal{D}|_{H}=\\{\mathcal{B}\cap H|\mathcal{B}\in\mathcal{D}\\}$. The
restriction of a subgraph to a broken tower diagram of $\mathcal{D}$ over a
branch $H\subset\Gamma$, $\Gamma$ being a star graph (with $v_{b}$ being the
branch vertex) is the set $\mathcal{D}_{v_{b}}|_{H}=\\{\mathcal{B}\cap
H|\mathcal{B}\in\mathcal{D}_{v_{b}}\\}$. Define $\mathcal{B}_{l}$ to be the
maximum length block in the restriction to a broken tower, that is, so that
$|\mathcal{B}_{i}|$ is maximal. In the case where there are $2$ or more blocks
with maximum length we define $\mathcal{B}_{l}$ to be the set of all maximum
length blocks.

The following is an immediate consequence of the above definition.

######  Lemma 2.8.

The $2$ definitions of a restriction to a broken tower are equivalent.

###### Proof.

Over any path blocks containing $s_{1}$ and $s_{m}$ must contain $s_{i}$ for
all $1\leq i\leq m$ by axiom $(1)$. ∎

######  Lemma 2.9.

For any staircase diagram $\mathcal{D}$, the restriction to an broken tower
diagram is a broken tower diagram.

###### Proof.

$\mathcal{D}_{v_{b}}|H$ satisfies axioms $(1)-(3)$ of the staircase diagram
definition and each $\mathcal{B}\in\mathcal{D}_{v_{b}}$ has
$v_{b}\in\mathcal{B}$. ∎

######  Definition 2.10.

Let $\mathcal{D}_{v_{b}}|_{H}$ be a restriction to a broken tower diagram over
a branch $H$ of $\Gamma$. We define the regular part to be the set of blocks
$Reg(\mathcal{D})=\\{\mathcal{B}\in\mathcal{D}|\mathcal{B}\notin\mathcal{D}_{v_{b}}|_{H}\\}\cup(\mathcal{B}_{l}\setminus\mathcal{B}_{l+1})$
or
$Reg(\mathcal{D})=\\{\mathcal{B}\in\mathcal{D}|\mathcal{B}\notin\mathcal{D}_{v_{b}}|_{H}\\}\cup(\mathcal{B}_{l}\setminus\mathcal{B}_{l-1})$
(where $\mathcal{B}_{l}$ is the maximum block in the broken tower and
$\mathcal{B}_{l\pm 1}$ is the block $1$ above or below $\mathcal{B}_{l}$ in
the poset) with either $(\mathcal{B}_{l}\setminus\mathcal{B}_{l\pm
1})\prec\mathcal{B}_{1}$ or $(\mathcal{B}_{l}\setminus\mathcal{B}_{l\pm
1})\succ\mathcal{B}_{1}$ (with $\mathcal{B}_{1}$ being the first block along
the branch $\Gamma$).

######  Lemma 2.11.

The regular part of a staircase diagram is always a staircase diagram over
Dynkin diagrams of type $A_{n}$.

###### Proof.

Since the regular part is along a path it must be of type $A_{n}$.
Furthermore, all blocks of $Reg(\mathcal{D})$ satisfies axioms $(1)-(4)$ of 2
because they are a connected subset of a diagram $\mathcal{D}$. ∎

We can decompose the previous example to obtain the following broken tower
diagram (which can be further decomposed into restrictions to broken towers
along the $3$ branches) as well as $3$ regular parts (in this case, one
regular part is empty).

######  Example 2.12.

The broken tower diagram of 2.3 is:

With the $3$ regular parts being the empty-set and:

The restriction to a broken tower diagram on each branch would be the
following $3$ diagrams. You’ll notice we can piece together the entire
staircase from this data.

We now define the idea of a join and glue of $2$ diagrams. The idea being we
can take the join of a regular part and a restriction to a broken tower to get
back to a branch of the original staircase diagram after decomposing it into
the $2$ parts. Similarly, gluing takes multiple branches of a diagram
$\mathcal{D}$ and returns the staircase diagram with the branches “glued”
along the branching vertex.

######  Definition 2.13.

Given a restriction of broken tower diagram $\mathcal{D}_{1}$ (over some
branch) and a regular part $\mathcal{D}_{2}$ over the same branch we define
the join of the diagrams to be the diagram obtained by joining the largest
block of $\mathcal{D}_{1}$ and the first block of $\mathcal{D}_{2}$, under the
condition that $\mathcal{B}_{l}\setminus\mathcal{B}_{l\pm 1}\setminus
S(\mathcal{D}_{2}\setminus\mathcal{B}_{2})\neq\emptyset$,
$\mathcal{B}_{1}=\mathcal{B}_{l}\setminus\mathcal{B}_{l\pm 1}$ and
$S(\mathcal{D}_{1})\cap S(\mathcal{D}_{2})\setminus\mathcal{B}_{1}=\emptyset$
(where $\mathcal{B}_{1}$ is the first block of $\mathcal{D}_{2}$ and
$\mathcal{B}_{2}$ the second, along the branch $\Gamma$).

That is, $Join(\mathcal{D}_{1},\mathcal{D}_{2})$ is the poset
$\mathcal{D}_{1}\cup(\mathcal{D}_{2}\setminus\mathcal{B}_{1})$ with
$\mathcal{B}_{2}\prec\mathcal{B}_{l}$ or
$\mathcal{B}_{l}\prec\mathcal{B}_{2}$. In the case that one of the conditions
is not satisfied the join does not exist and it is said to be not valid.

######  Example 2.14.

The join of 2.3 along the main branch ($6$-nodes) is

the “union” of the restricted tower diagram and the regular part.

######  Definition 2.15.

Given $2$ staircase diagrams (or broken tower/ restriction broken tower
diagrams) containing $v_{b}$, $\mathcal{D}_{1}$ and $\mathcal{D}_{2}$, we
denote $Glue(\mathcal{D}_{1},\mathcal{D}_{2})$ to be the diagram where the
diagrams $\mathcal{D}_{1}$ and $\mathcal{D}_{2}$ are glued together at the
vertex $v_{b}$. If $\mathcal{B}_{1},\cdots,\mathcal{B}_{n}$ are blocks (in
poset order) over the restriction to a broken tower diagram of
$\mathcal{D}_{1}$ and $b_{1},\cdots,b_{m}$, over $\mathcal{D}_{2}$, then

$Glue(\mathcal{D}_{1},\mathcal{D}_{2})=\bigcup_{i\in\mathbb{Z}}(\mathcal{B}_{i}\cup
b_{\rho(i)})\cup Reg(\mathcal{D}_{1})\cup Reg(\mathcal{D}_{2}).$

With the order induced from $\mathcal{D}1$ and $\mathcal{D}2$,and
$\rho(\alpha)=\alpha+x$, for some $x$ with $(-m\leq x\leq n)$ and
$\mathcal{B}_{i}=\emptyset$ or $b_{i}=\emptyset$ for values of $i$ not between
$1$ and $m$ or $n$, respectively. Given $n$ diagrams $\mathcal{D}_{i}$,
$Glue_{i}(\mathcal{D}_{i})$ is defined to be
$Glue(\mathcal{D}_{1},Glue(\mathcal{D}_{2},\cdots
Glue(\mathcal{D}_{n-1},\mathcal{D}_{n})\cdots)$.

For example, gluing the $3$ restricted tower diagrams above gives the broken
tower diagram:

######  Lemma 2.16.

Note the following:

$Glue(\mathcal{D}_{1},Glue(\mathcal{D}_{2},\mathcal{D}_{3}))=Glue(Glue(\mathcal{D}_{1},\mathcal{D}_{2}),\mathcal{D}_{3}).$

Thus $Glue(\mathcal{D}_{1},\cdots,\mathcal{D}_{n})$ is well-defined.

##  3\. Decomposing Staircases

We now prove that every staircase can be decomposed into a broken tower and
regular part along each branch (with a choice of arrow for each branch).
Furthermore, $n$ triples of regular branch, arrow and broken tower uniquely
determine a star graph on with $n$ branches.

######  Theorem 3.1.

There is a bijection between staircase diagram over a star graph $G$ with $n$
branches and $n$ triples of regular part, restriction to a broken tower
diagrams, choice of $\uparrow,\downarrow$ of $G$ with the join of the regular
parts and broken tower diagrams valid.

###### Proof.

Any connected staircase diagram $\mathcal{D}$ over a star graph $G$ may be
decomposed uniquely into $n$ triple of regular part, restriction to a broken
tower diagram, and choice of arrow as follows: Let the branches be $A_{i}$
with branch vertex $v_{b}$. For each branch ($1\leq i\leq n$)we use the
following triple: The broken tower diagram is $\mathcal{D}_{v_{b}}|_{A_{i}}$.
The regular part is $Reg(\mathcal{D}|_{A_{i}})$. The choice of arrow is
defined by whether $(\mathcal{B}_{l}\setminus\mathcal{B}_{l\pm
1})\prec\mathcal{B}_{1}$ or $(\mathcal{B}_{l}\setminus\mathcal{B}_{l\pm
1})\succ\mathcal{B}_{1}$. Since each of these are defined uniquely this is
indeed unique and every connected staircase diagram can be decomposed this
way.

For the other direction assume we are given $n$ triples of regular part
$S_{i}$, restriction to broken tower diagrams $T_{i}$ and an arrow. Then

$Glue_{i=1}^{n}(Join(S_{i},T_{i}))$

is a unique staircase diagram. Here each $\uparrow\downarrow$ denotes whether
$\mathcal{B}_{2}\prec\mathcal{B}_{l}$ or $\mathcal{B}_{l}\prec\mathcal{B}_{2}$
in the join. ∎

The advantage of this is that we can focus on enumerating broken parts and
regular parts almost individually. The following lemma determines the form of
all broken tower diagrams. The blocks must form and increasing then decreasing
sequence in length. An example of a potential broken tower diagram is

while a non-example (as disproved in 3.2) is

######  Lemma 3.2.

Let $\mathcal{D}$ be a restriction of a broken tower diagram on the graph
$\Gamma$ with vertices $s_{1},s_{2},\cdots s_{p}$ (the branch vertex is
$v_{b}$) blocks
$\mathcal{B}_{1}\prec\mathcal{B}_{2}\cdots\prec\mathcal{B}_{n}\in\mathcal{D}$
each with cardinality $|\mathcal{B}_{i}|=l_{i}$. Then there exist
$i,j\in\mathbb{N}$ such that $l_{1}\leq
l_{2}\leq\cdots<l_{i}=l_{i+1},\ldots,l_{i+j}>\ldots\geq l_{n-1}\geq l_{n}$.

###### Proof.

We proceed by induction on $n$. For $n=1$ the statement holds. Now suppose we
have a restriction of broken tower staircase diagram of with $n$ blocks
ordered by
$\mathcal{B}_{1}\prec\mathcal{B}_{2}\cdots\prec\mathcal{B}_{n}\in\mathcal{D}$.
And suppose we insert another block, $\mathcal{B}_{q}$ with
$|\mathcal{B}_{q}|=l_{q}$ into the poset between $2$ blocks
$\mathcal{B}_{k-1}$ and $\mathcal{B}_{k}$. Recall that $i$ and $j$ give the
boundaries for the maximum sized blocks from the inductive assumption. We have
$3$ cases, $k<i$, $i\leq k\leq i+j+1$ or $k>i+j+1$ and we will now show that
in each case we can find new $i,j$ to satisfy the theorem if $\mathcal{D}$ is
still a restriction of broken tower staircase diagram.

Case 1: If $k<i$ then $l_{k-1}\leq l_{q}\leq l_{k}$, otherwise we have
$l_{k-1}>l_{q}$ or $l_{n-1}>l_{k}$ the former of which implies
$\mathcal{D}_{s_{l_{q}+1}}$ is not a saturated subchain of
$\mathcal{D}_{s_{l_{q}}}\cup\mathcal{D}_{s_{l_{q}+1}}$ (since
$\mathcal{D}_{s_{l_{q}}}\notin\mathcal{D}_{s_{l_{q}+1}}$ meaning that
$\mathcal{B}_{s_{k}}\prec\mathcal{B}_{s_{l_{q}}}\prec\mathcal{B}_{s_{k+1}}$) ,
breaking axiom $(3)$ of the broken tower staircase diagram definition. The
latter implies $\mathcal{D}_{s_{l_{k}+1}}$ is not a saturated subchain of
$\mathcal{D}_{s_{l_{k}+1}}\cup\mathcal{D}_{s_{l_{k}}}$ (since
$\mathcal{D}_{s_{l_{k}}}\notin\mathcal{D}_{s_{l_{k}+1}}$ but
$\mathcal{D}_{s_{l_{q}}}\in\mathcal{D}_{s_{l_{k}+1}}$ meaning that
$\mathcal{B}_{s_{k}}\prec\mathcal{B}_{s_{l_{q}}}\prec\mathcal{B}_{s_{k+1}}$),
again breaking axiom $(3)$ of the broken tower staircase diagram definition.

Case 2: If $i\leq k\leq i+j+1$ then $l_{q}\geq\min(l_{k},l_{k-1})$. Otherwise
we have $l_{q}<l_{k}$ and $l_{q}<l_{k+1}$. Then $\mathcal{D}_{s_{l_{q}+1}}$ is
not a saturated subchain of
$\mathcal{D}_{s_{l_{q}+1}}\cup\mathcal{D}_{s_{l_{q}}}$ since
$\mathcal{B}_{q}\in\mathcal{D}_{s_{l_{q}}}$ and
$\mathcal{B}_{k-1}\prec\mathcal{B}_{q}\prec\mathcal{B}_{k}$ but
$\mathcal{B}_{q}\notin\mathcal{D}_{s_{l_{q}+1}}$. Again, breaking axiom $(3)$
or the broken tower staircase diagram definition.

Case 3: If $k>i+j+1$ then we may repeat Case $1$, mutatis mutandis, by
symmetry. ∎

######  Definition 3.3.

For each block in the restriction to a broken tower diagram we assign an
element of the set $\\{\emptyset,u,d,u/d\\}$ depending on whether the block
has a maximal element, minimal element, both or neither ($u,d,u/d$ or
$\emptyset$ respectively ). We call the ordered set of such elements the
sight-set of the particular broken tower (over the branching node of that
tower). If a sight-set consists of only $u/d$ elements then we say it is full
and corresponds to a valid tower over the branch node (a tower that is also a
staircase diagram of definition 2.1).

######  Corollary 3.4.

The sight-sets over the branch vertex of a restriction to a broken tower
diagram is of form $u\\{u,\emptyset\\}^{*}\\{u/d\\}\\{d,\emptyset\\}^{*}d$,
$u\\{u,\emptyset\\}^{*}\\{d,\emptyset\\}^{*}d$,
${\\{u/d\\}\\{d,\emptyset\\}^{*}d}$, $u\\{u,\emptyset\\}^{*}\\{u/d\\}$ or
$\\{u/d\\}$. The notation used here is formal language notation for strings
containing the $4$ symbols $\\{u,d,u/d,\emptyset\\}$. We call any particular
form the gluing data of broken tower diagram.

###### Proof.

From lemma 3.2 we have that the length or cardinality of the blocks form an
increasing then decreasing sequence. For any block $\mathcal{B}_{k}$ if
$|\mathcal{B}_{k-1}|<|\mathcal{B}_{k}|$ then the $k^{th}$ term of the sight-
set has a $u$ since $\mathcal{B}_{k}$ has a maximal element. Similarly, if
$|\mathcal{B}_{k}|>|\mathcal{B}_{k+1}|$ then the $k^{th}$ term of the sight-
set has a $d$ since $\mathcal{B}_{k}$ has a minimal element. The above forms
are a consequence of the increasing then decreasing sequence with $\emptyset$
occurring when $|\mathcal{B}_{k-1}|=|\mathcal{B}_{k}|=|\mathcal{B}_{k+1}|$ ∎

######  Theorem 3.5.

Blocks added onto a broken tower diagram (via a join to a regular part) don’t
change the sight set on the branching vertex.

###### Proof.

Firstly, we cannot increase the sight set since none of the added blocks are
blocks on the branching vertex. Suppose (W.L.O.G) that adding a block onto a
broken tower diagram caused a $u$ to be removed from the sight set. The $u$
removed cannot be from $\mathcal{B}_{1}$ since that would imply the branching
vertex was covered. Suppose $u$ was removed from $\mathcal{B}_{i}$, then this
would imply that the new block (say, $\mathcal{B}_{\nu}$) is adjacent to
$\mathcal{B}_{i-1}$ in poset order (otherwise there is some chain with an
element in $\mathcal{B}_{\nu}$ that is not saturated ). From the valid join
condition we have that
$\mathcal{B}_{i}\nsubseteq\mathcal{B}_{\nu}\cup\mathcal{B}_{i-1}$ thus
$\mathcal{B}_{i}$ still has a maximal block and the $u$ is preserved.
Contradiction, the join cannot be valid and sight-set reducing. ∎

######  Theorem 3.6.

$|\mathcal{D}_{v_{b}}|$ is bounded as $n$ varies. Furthermore,
$|\mathcal{D}_{v_{b}}|\leq\sum_{i}a_{i}+1$.

###### Proof.

Let $\mathcal{D}_{v_{b}}=\\{\mathcal{B}_{1},\cdots,\mathcal{B}_{h}\\}$ with
$\mathcal{B}_{i}\succ\mathcal{B}_{j}$ if $i<j$. Since $\mathcal{D}$ is a
staircase diagram each block is minimal over some vertex and maximal over some
vertex. Consider $\mathcal{D}$ restricted to the subgraph consisting of the
segment from the vertex $v_{b}$ to the end of the branch of size $n$. Consider
the sight-set of each restriction to broken tower. By lemma 3.5 we can further
restrict ourselves to the case where $\mathcal{D}$ is now a restricted broken
tower diagram since adding blocks to broken tower diagrams does not change the
sight-set over the branching vertex. The form of the sight set will be one of
the forms as described in 3.4 (where the size of the ordered set is $h$).
Since $\mathcal{D}$ is a restriction to a broken tower diagram, we must have
$h-1$ symbols (that is, either $u$ or $d$) added when we glue (at $v_{b}$) the
broken tower diagram of $\mathcal{D}$ restricted to the other branches of
$\Gamma$.

For each branch of $\Gamma$ we can restrict to the broken tower diagram by
lemma 3.5 and for each vertex $v$ in the other branches we can have $2$
symbols from the minimal and maximal elements in $\mathcal{D}_{v}$. However,
from the $2$ symbols coming from $\mathcal{D}_{v}$ only $1$ such symbol will
be able to change the sight-set of $\mathcal{D}$ restricted, as if we had both
begin used this would imply that the minimal block is some $\mathcal{B}_{i}$,
the maximal block is some $\mathcal{B}_{j}$ with
$\mathcal{B}_{i}\succ\mathcal{B}_{j}$ since all the blocks containing $u$ are
greater in the poset than blocks containing $d$ (in $\mathcal{D}$ restricted).
This implies to add $h-1$ symbols we need $\sum_{i}a_{i}\geq h-1$ or

$h\leq\sum_{i}a_{i}+1.$

∎

##  4\. The fully supported $E$-type case

We now move on to prove the main results of the paper, the complete
enumeration of rationally smooth Schubert varieties over type $E$ Dynkin
diagrams.

######  Definition 4.1.

We let $E_{n}$ denote the Dynkin diagram of the corresponding Coxeter group
(the finite type Dynkin diagram of type $E$ with $n$ nodes). Let
$E(x)=\sum_{i}e_{n}x^{n}$ where $e_{n}$ is the number of fully supported
connected staircase diagrams over the graph $E_{n}$.

######  Lemma 4.2.

Suppose we have $2$ restriction to a broken tower diagram $\mathcal{D}_{1}$
and $\mathcal{D}_{2}$ with the same sight-set over a graph with $n$ branches.
Then the set of all $n-1$ tuples of pairs of restricted broken towers and
regular parts so that the glue and join of them all with the first restriction
to a broken tower diagram, $\mathcal{D}_{1}$ (so that a staircase diagram is
formed, as described by theorem 3.1) is the same as that for the second,
$\mathcal{D}_{2}$.

###### Proof.

Suppose we have $2$ different restrictions to a broken tower with the same
sight-set but a $n-1$ tuple of pairs of diagrams that gives a full sight-set
over the branching node of only $\mathcal{D}_{1}$. Then there is some
$\mathcal{B}\in\mathcal{D}_{2}$ so that when glued with the pairs does not
give a $u/d$. Since $\mathcal{D}_{1}$ has the same sight-set as
$\mathcal{D}_{2}$ that means that there is some broken tower in the set of
$n-1$ with a different sight-set. Contradiction. ∎

######  Definition 4.3.

Fix a gluing data $t$ across $1$ branch of the star graph $\Gamma$ with $n$
branches. For any gluing data $t$, we define $Cont(t)$ to be the number of
$n-1$ tuples consisting of pairs of restrictions of broken towers along the
other branches and regular parts so that the join across each of the pairs and
glue between them all has a full sight-set over the branch vertex.

######  Corollary 4.4.

As a direct corollary of 3.6, we have that $|\mathcal{D}_{v_{b}}|\leq 4$. From
now on we let $V_{1},V_{2},V_{3},V_{4}$ denote these blocks from greatest to
least in poset order, in the case that less than $4$ are needed, define the
extra blocks to be empty sets.

######  Lemma 4.5.

$E(x)=x^{3}\sum_{t\in\mathcal{T}}Cont(t)\sum_{BT_{t}}SD_{BT_{t}}(x)$

Here $\mathcal{T}$ is the set of all possible gluing data, $BT_{t}$ is any
broken tower diagram of gluing data $t$ and $SD_{BT_{t}}(x)$ is the generating
function for joins of regular part and broken tower diagrams of gluing data
$t$ (with exponent indicating length of the main branch).

###### Proof.

We have that any staircase diagram on $E$ can be decomposed into a triple of
pairs of regular part, broken tower along each branch. Fix some gluing data
$t$ for the broken tower along the main branch, by lemma 4.2 we have that the
number of pairs of regular part, broken tower for the other $2$ branches is
fixed, thus we only concern ourselves with pairs of regular part and broken
tower diagrams. Given a generating function for the number of such pairs of
length $n-3$ we simply introduce a $x^{3}$ term for the $3$ additional
vertices past the branch. Summing $t$ over all gluing data gives the required.
∎

######  Lemma 4.6.

The number of dyck paths from $(k,1)$ to $(n,n)$ is given by

$\frac{\binom{2n-k}{n-k}k}{2n-k}.$

This is a consequence of the ballot theorem, for a proof see p. 73 of [2].

######  Definition 4.7.

A broken staircase diagram is partially ordered set collection of subsets
$\mathcal{B}$ (over $A_{n}$) so that $\mathcal{B}=\\{\mathcal{B}\cap
S_{n}|\mathcal{B}\in\mathcal{D}\\}$ where $\mathcal{D}$ is a increasing or
decreasing staircase diagram.

######  Lemma 4.8.

There is a bijection between Dyck paths from $(k,1)$ to $(n,n)$ and increasing
connected staircase diagrams starting with a block of length $k$. An
increasing staircase diagram has the property that
$\mathcal{B}_{1}\prec\mathcal{B}_{2}\prec\cdots\mathcal{B}_{n}$ if
$\mathcal{B}_{i}$ is the $i^{th}$ block along the branch. This is the same
bijection as in [5].

###### Proof.

Let $r_{i}$ be a sequence of integers that form a dyck path with $i^{th}$ step
being of length $r_{i}$ with the property that $\sum_{i}r_{2i}=n$ and
$\sum_{i}r_{2i-1}=n$ (since this is a path to $(n,n)$) and $r_{1}=k$. Define
the blocks $\mathcal{B}_{i}$ over the vertices $s_{1},s_{2},\cdots$ to be

$\mathcal{B}_{i}=\\{s_{\sum_{k=1}^{i-1}r_{2k}+1},\cdots,s_{\sum_{k=1}^{i}r_{2k-1}+1}\\}.$

The poset order on these blocks is
$\mathcal{B}_{1}\prec\mathcal{B}_{2}\cdots$. This is indeed a staircase
diagram as one can easily see conditions $(1)-(3)$ from the staircase diagram
definition are satisfied. For the final condition we see that both
${\sum_{k=1}^{i-1}r_{2k}+1}$ and ${\sum_{k=1}^{i}r_{2k-1}+1}$ are increasing
with $i$ so all blocks have minimal and maximal elements. Also, note that the
diagonal condition on Dyck paths makes sure that $\mathcal{B}_{i}$ is always
defined, since this forces
${\sum_{k=1}^{i-1}r_{2k}+1}\leq{\sum_{k=1}^{i}r_{2k-1}+1}$. This map from Dyck
paths to staircase diagrams is injective (suppose that $2$ different Dyck
paths differ at the $2i^{th}$ or $2i-1^{th}$ step, then the corresponding
block $\mathcal{B}_{i}$ is different).

Finally, every staircase diagram starting with a block of length $k$ can be
represented as a Dyck path (with $r_{1}=k$) by simply taking the difference of
successive vertices in $\mathcal{B}_{i}$ to get a sequence of $r_{i}$. The
Dyck path constraint that $\sum_{i}r_{2i-1}\geq\sum_{i}r_{2i}$ is satisfied
because in any increasing staircase diagram $\mathcal{D}$ we can consider the
subdiagram of the first $i$ blocks. The subdiagram has $\sum_{i}r_{2i-1}$
minimal elements and at least $\sum_{i}r_{2i}$ maximal elements (exactly
$\sum_{i}r_{2i}$ if we only consider maximal elements that are maximal within
$\mathcal{D}$ as well). Since the number of maximal and minimal elements on a
diagram must be equal ($1$ of each for every vertex) we have that

$\sum_{i}r_{2i-1}\geq\sum_{i}r_{2i}.$

∎

######  Lemma 4.9.

Let $I_{k}(x)$ be the generating function for increasing connected staircase
diagrams starting with a block of length $k$. Let $IB_{k}(x)$ be the
generating function for increasing broken staircase diagrams starting with a
block of length $k$. Then:

  1. (1)

$I_{k}(x)=\sum_{n}\frac{\binom{2n-k}{n-k}k}{2n-k}x^{n}$

  2. (2)

$IB_{k}(x)=\frac{(1-x)I_{k}(x)}{x}-x^{k-1}+x^{k}$

###### Proof.

The generating function for $I_{k}(x)$ come directly from 4.6 and 4.8. To find
the generating function for $IB_{k}(x)$ note that the restriction to
$\\{s_{1},\cdots,s_{n-1}\\}$ of any increasing staircase diagram starting with
a block of length $k$ on $\\{s_{1},\cdots,s_{n}\\}$ is either an increasing
broken staircase diagram or another increasing staircase diagram starting with
a block of length $k$ except for the special case where the staircase diagram
is just a block of length $k$. This information gives the following:

$x^{k}+xIB_{k}(x)+xI_{k}(x)=I_{k}(x)+x^{k+1}$

Yielding the required. ∎

######  Lemma 4.10.

The generating function for $B_{k}(x)$, that is, staircase diagrams starting
with a first block of length $k$ and second block greater than the first in
poset order ($\mathcal{B}_{1}\prec\mathcal{B}_{2}$) on the graph $A_{n}$ is
given by

$B_{k}(x)=\frac{I(x)IB_{k}(x)}{1-Br(x)^{2}}.$

Here $I(x)$ and $Br(x)$ are the generating functions for increasing staircase
diagrams and increasing broken staircase diagrams as found in [5].

$I(x)=\frac{1-2x\sqrt{1-4x}}{2x}$ $Br(x)=\frac{(1-x)I(x)}{x}-1$

###### Proof.

Let $\mathcal{D}$ be a broken staircase diagram of type $\mathcal{B}_{k}$. We
define a critical point of $\mathcal{D}$ as an node $s\in\Gamma$ in the
underlying graph so that $s$ is both maximal and minimal in the chain, or
equivalently, that $|\mathcal{D}_{s}|=1$.

######  Example 4.11.

5410976432987321

Has critical points at $s_{1},s_{5},s_{8}$ and $s_{10}$.

Suppose we break diagram $\mathcal{D}$ into parts
$\mathcal{D}_{1},\mathcal{D}_{2},\cdots\mathcal{D}_{2n}$ so that
$\bigcup_{i=1}^{2n}\mathcal{D}_{i}=\mathcal{D}$ where $\mathcal{D}_{1}$ is an
increasing diagram starting with a first block of length $k$,
$\mathcal{D}_{2n}$ is a decreasing broken staircase diagram (including the
empty staircase diagram) and $\mathcal{D}_{i}$ for $2\leq i\leq 2n-1$ is a
broken staircase diagram.We may represent the generating function for such
diagrams as $I(x)IB_{k}(x)Br(x)^{2n-2}$. Summing over $n$ gives:

$\sum_{n=1}^{\infty}I(x)IB_{k}(x)Br(x)^{2n-2}=\frac{I(x)IB_{k}(x)}{1-Br(x)^{2}}.$

∎

######  Definition 4.12.

We use $SD_{\mathcal{D}}(x)$ to denote the generating function (with respect
to length of the branch) enumerating the number of joins of staircase diagrams
with restriction to a broken tower, $\mathcal{D}$.

######  Lemma 4.13.

Suppose $\mathcal{D}_{1}$ is an broken tower diagram with maximum block length
$|\mathcal{B}_{l}|=k$ and suppose that $|\mathcal{B}_{<l}|=x_{1}$ and
$|\mathcal{B}_{>l}|=x_{2}$. Here $\mathcal{B}_{<l}$ denote the block
$\mathcal{B}_{i}\succ\mathcal{B}_{l}$ and $|\mathcal{B}_{i}|<k$. Similarly,
$\mathcal{B}_{>l}$ denote the block $\mathcal{B}_{i}\prec\mathcal{B}_{l}$ and
$|\mathcal{B}_{i}|<k$. Then

$SD_{\mathcal{D}_{1}}(x)=x^{k}+x^{x_{1}}B_{k-x_{1}}(x)+x^{x_{2}}B_{k-x_{2}}.$

###### Proof.

Suppose that we have some join of a regular part and a restriction to a broken
tower. As before we let $\mathcal{B}_{1}$ and $\mathcal{B}_{2}$ be the first
$2$ blocks of the regular part. We have two cases:
$\mathcal{B}_{l}\prec\mathcal{B}_{2}$ or
$\mathcal{B}_{l}\succ\mathcal{B}_{2}$, each case corresponds to a term (either
$x^{x_{1}}B_{k-x_{1}}(x)$ or $x^{x_{2}}B_{k-x_{2}}(x)$). W.L.O.G. assume
$\mathcal{B}_{1}\prec\mathcal{B}_{2}$. Assume we have a join of a regular part
and a restriction to a broken tower, $\mathcal{D}$, over a path of $n$
vertices. From the definition of join we have that
$\mathcal{B}_{1}=\mathcal{B}_{l}\setminus\mathcal{B}_{l-1}$ giving
$|\mathcal{B}_{1}|=k-x_{1}$. Apart from this there are no more conditions on
the regular part to be a valid join to $\mathcal{D}$. Since the first $x_{1}$
vertices of the path on $n$ are taken up solely by the restriction to a broken
tower the number of valid regular parts is the $x^{n-x_{1}}$ coefficient of
$B_{k-x_{1}}(x)$. Summing over $n$ gives the desired generating function (and
mutatis mutandis for $\mathcal{B}_{l}\succ\mathcal{B}_{2}$). We also add an
$x^{k}$ term for no join:

$SD_{D,\uparrow}(x)=\sum_{n}[x^{n-x_{1}}]B_{k-x_{1}}(x)=\sum_{n}[x^{n}]x^{x_{1}}B_{k-x_{1}}(x)=x^{x_{1}}B_{k-x_{1}}(x)$
$SD_{D,\downarrow}(x)=\sum_{n}[x^{n-x_{2}}]B_{k-x_{2}}(x)=\sum_{n}[x^{n}]x^{x_{2}}B_{k-x_{2}}(x)=x^{x_{2}}B_{k-x_{2}}(x)$
$SD_{D}(x)=SD_{D,\uparrow}(x)+SD_{D,\downarrow}(x)+x^{k}=x^{k}+x^{x_{1}}B_{k-x_{1}}(x)+x^{x_{2}}B_{k-x_{2}}$

∎

######  Lemma 4.14.

$\sum_{BT_{t}}SD_{BT_{t}}(x)=\sum_{k}\sum_{(V_{1},V_{2},V_{3},V_{4})\in
BT_{t,k}}x^{x_{1}}B_{k-x_{1}}(x)+x^{x_{2}}B_{k-x_{2}}.$

The quadruple $(V_{1},V_{2},V_{3},V_{4})$ is in $BT_{t,k}$ if the broken tower
diagram consisting of $V_{1}\succ V_{2}\succ V_{3}\succ V_{4}$ has $|B_{l}|=k$
and has gluing data $t$.

###### Proof.

This comes directly from 4.13 and the fact that
$BT_{t}=\bigcup_{k=1}^{\infty}BT_{t,k}$. ∎

######  Lemma 4.15.

What follows is a table of values detailing the set of possible gluing data in
the $E_{n}$ case as well parametrizations for the quadruples of $V_{i}$ in
each $BT_{t,k}$ and the number of pairs of pairs of restriction to broken
tower and regular part so that the glue of all the broken towers has a full
sight-set. This is easily brute forceable in the $E_{n}$ case as the number of
nodes outside the main branch is fixed. Given a gluing data one can test all
the possible continuations.

Broken Tower | Sight-Set | Parametrization | Continuations |  Symmetry ($\times$2)
---|---|---|---|---
|  | $u/d$
---
$d$
$d$
$d$
$k>x_{2}>|V_{3}|>|V_{4}|>0$ | 8 | Yes
|  | $u/d$
---
$\emptyset$
$d$
$d$
$k>x_{2}>|V_{4}|>0$ | 1 | Yes
|  | $u/d$
---
$d$
$\emptyset$
$d$
$k>x_{2}>|V_{3}|>0$ | 4 | Yes
|  | $u$
---
$u/d$
$d$
$d$
$k>x_{1},k>x_{2}>|V_{4}|>0$ | 1 | Yes
|  | $u/d$
---
$d$
$d$
$k>x_{1}>|V_{3}|>0$ | 29 | Yes
|  | $u/d$
---
$\emptyset$
$d$
$k>x_{1}>0$ | 13 | Yes
|  | $u$
---
$d$
$d$
$k>x_{1}>0$ | 1 | Yes
Broken Tower | Sight-Set | Parametrization | Continuations | Symmetry (x2)
---|---|---|---|---
|  | $u$
---
$u/d$
$d$
$k>x_{1}>0,k>x_{2}>0$ | 12 |  |  Not if $x_{1}=x_{2}$
---
|  | $u/d$
---
$d$
$k>x_{2}>0$ | 41 | Yes
|  | $u$
---
$d$
$k$ | 33 | No
| $u/d$ | $k$ | 10 | No

Broken Tower | Generating Function
---|---
|
$\sum_{k=1}^{\infty}\left(\sum_{x_{1}=1}^{k-1}\frac{(x_{1}-2)(x_{1}-1)}{2}x^{x_{1}}B_{k-x_{1}}(x)\right)+\frac{(k-3)(k-2)(k-1)}{6}B_{k}(x)+x^{k}$
|
$\sum_{k=1}^{\infty}\left(\sum_{x_{1}=1}^{k-1}(x_{1}-1)x^{x_{1}}B_{k-x_{1}}(x)\right)+\frac{1}{2}(k-2)(k-1)B_{k}(x)+x^{k}$
|
$\sum_{k=1}^{\infty}\left(\sum_{x_{1}=1}^{k-1}(x_{1}-1)x^{x_{1}}B_{k-x_{1}}(x)\right)+\frac{1}{2}(k-2)(k-1)B_{k}(x)+x^{k}$
|
$\sum_{k=1}^{\infty}\left(\sum_{x_{1}=1}^{k-1}\frac{(k-2)(k-1)}{2}x^{x_{1}}B_{k-x_{1}}(x)+(k-1)(x_{1}-1)x^{x_{1}}B_{k-x_{1}}(x)\right)+\frac{(k-2)(k-1)^{2}}{2}x^{k}$
|
$\sum_{k=1}^{\infty}\left(\sum_{x_{1}=1}^{k-1}(x_{1}-1)x^{x_{1}}B_{k-x_{1}}(x)\right)+\frac{1}{2}(k-2)(k-1)B_{k}(x)+x^{k}$
|
$\sum_{k=1}^{\infty}\left(\sum_{x_{1}=1}^{k-1}x^{x_{1}}B_{k-x_{1}}(x)\right)+(k-1)B_{k}(x)+(k-1)x^{k}$
|
$\sum_{k=1}^{\infty}\left(\sum_{x_{1}=1}^{k-1}x^{x_{1}}B_{k-x_{1}}(x)\right)+(k-1)B_{k}(x)+(k-1)x^{k}$
|
$\sum_{k=1}^{\infty}2(k-1)\left(\sum_{x_{1}=1}^{k-1}x^{x_{1}}B_{k-x_{1}}(x)\right)+(k-1)^{2}x^{k}$
|
$\sum_{k=1}^{\infty}\left(\sum_{x_{1}=1}^{k-1}x^{x_{1}}B_{k-x_{1}}(x)\right)+(k-1)B_{k}(x)+(k-1)x^{k}$
| $\sum_{k=1}^{\infty}2B_{k}(x)+x^{k}$
| $\sum_{k=1}^{\infty}2B_{k}(x)+x^{k}$

######  Theorem 4.16.

The generating function for

$E(x)=\sum_{i}e_{n}x^{n}=x^{3}\sum_{t\in\mathcal{T}}Cont(t)\sum_{k}\sum_{(V_{1},V_{2},V_{3},V_{4})\in
BT_{t,k}}x^{x_{1}}B_{k-x_{1}}(x)+x^{x_{2}}B_{k-x_{2}}$

is

$\frac{P(x)+Q(x)\sqrt{1-4x}}{R(x)}.$

Where

$P(x)=-3224x^{16}+6436x^{15}+2494x^{14}-12226x^{13}+7795x^{12}-6091x^{11}+14594x^{10}-18859x^{9}+$
$13325x^{8}-4899x^{7}+569x^{6}+350x^{5}-363x^{4}+217x^{3}-70x^{2}+8x$
$Q(x)=-2034x^{15}+7488x^{14}-9888x^{13}+5071x^{12}+815x^{11}-5070x^{10}+8318x^{9}-$
$7544x^{8}+3537x^{7}-681x^{6}-79x^{5}+146x^{4}-125x^{3}+54x^{2}-8x$

and

$R(x)=(x-1)^{4}\left(x^{2}+4x-1\right).$

###### Proof.

This is obtained from the expression in 4.14 with the data obtained from 4.15
in $3$ parts. We sum those parts containing pure $x^{k}$ terms, those
containing terms with $x_{1}=0$ ($B_{k}(x)$ terms) and those containing
$B_{k-x_{1}}(x)$ terms. For each broken tower this gives a distinct generating
function, adding each case together gives the theorem. ∎

##  5\. $E$-type Diagrams without full support

######  Theorem 5.1.

The generating function $E_{Disc.}(x)$, enumerating the total number of
staircase diagrams over the graph $E_{n}$ (including disconnected) is the
rational $E_{Disc.}$, given below.

###### Proof.

From [5] we are given the generating function for diagrams and fully supported
diagrams of type $A_{n}$ and $\mathcal{D}_{n}$. We denote these by
$A_{Disc.}(x)$, $A(x)$, $\mathcal{D}_{Disc.}(x)$, and $D(x)$, respectively.To
enumerate the total number of staircase diagrams over the graph $E_{n}$ we
consider $4$ key vertices and apply an inclusion-exclusion argument. The
vertices in question are vertices in the $2$ minor branches and the branch
vertex. Any vertex can either have blocks lying over it or not. In the
following diagrams an $X$ denotes no blocks lying over this vertex while an
$O$ denotes blocks do lie over this vertex. The generating function for each
possibility is also given. Note that in the case where he branch vertex is
non-empty ($|D_{v_{b}}\neq\emptyset|$) we must multiply by a factor of
$1+A_{Disc.}(x)$ because the diagram is either fully supported on the main
branch or is not and has another diagram (of type $A_{Disc.}(x)$ within it).

$X$$X$$X$$X$$O$$X$$X$$O$$O$$X$$X$$X$$X$$X$$X$$O$$xA_{Disc.}(x)$$xA_{Disc.}(x)$$xA_{Disc.}(x)$$3xA_{Disc.}(x)$

$X$$X$$O$$X$$O$$X$$O$$O$$O$$X$$O$$X$$X$$X$$O$$O$$xA_{Disc.}(x)$$xA_{Disc.}(x)$$xA_{Disc.}(x)$$3xA_{Disc.}(x)$

$X$$O$$X$$X$$O$$O$$X$$O$$O$$O$$X$$X$$X$$O$$X$$O$$A(x)$$A(x)$$\frac{A(x)-x}{x}$$\frac{A(x)-x-3x^{2}}{x^{2}}$

$X$$O$$O$$X$$O$$O$$O$$O$$O$$O$$O$$X$$X$$O$$O$$O$$\frac{A(x)-x}{x}$$\frac{A(x)-x}{x}$$\frac{D(x)}{x^{2}}$$\frac{E(x)}{x^{3}}$

Summing yields:

$E_{Disc.}(x)=12x^{4}A_{Disc.}(x)+x^{3}\left(1+A_{Disc.}(x)\right)\left(2A(x)+3\frac{A(x)-x}{x}+\frac{A(x)-x-3x^{2}}{x^{2}}+\frac{D(x)}{x^{2}}+\frac{E(x)}{x^{3}}\right)$
$=\frac{P_{Disc.}(x)+Q_{Disc.}(x)\sqrt{1-4x}}{R_{Disc.}(x)}.$

Where

$P_{Disc.}(x)=1536x^{16}-9586x^{15}+20762x^{14}-19750x^{13}+10942x^{12}-15139x^{11}+27760x^{10}-28954x^{9}$
$+16898x^{8}-4690x^{7}-173x^{6}+689x^{5}-454x^{4}+238x^{3}-73x^{2}+8x$
$Q_{Disc.}(x)=3224x^{16}-13230x^{15}+21016x^{14}-15930x^{13}+4800x^{12}+3759x^{11}-10616x^{10}+13958x^{9}$
$-10482x^{8}+4200x^{7}-695x^{6}-95x^{5}+168x^{4}-140x^{3}+57x^{2}-8x$

and

$R_{Disc.}(x)=(-1+x)^{4}(-1+6x-8x^{2}+4x^{3}).$

∎

##  6\. Rationally Smooth Schubert Varieties

In this section we enumerate the number of rationally smooth Schubert
varieties in the classical finite type $E$. For any simple Lie group $G$ and
any Borel subgroup $B$ take the Weyl group of $G$, $W$. We can index the
Schubert varieties $X(w)$ in the flag variety $G/B$ by elements $w\in W$. When
$W=E_{n}$ for some $n$ we can answer how many rationally smooth Schubert
varieties there are.

First we introduce the notion of a Billey-Postnikov (BP) decomposition. Let
$S$ be the set of vertices in the Dynkin diagram of $W$, that is, the
generators $W$ as a Coxeter group. Let $J\subset S$ and $W_{J}$ be the
subgroup generated by $J$.

######  Definition 6.1.

We say a staircase $\mathcal{D}$ is spherical if $W_{B}$ is finite for all
$B\in D$.

We let $W^{J}$ denote the set of minimal length left cosets representatives of
the quotient $W/W_{J}$. Every element $w\in W$ can be uniquely decomposed as
$w=vu$ with $v\in W^{J}$ and $u\in W_{J}$. This is known as the parabolic
decomposition of $w$ with respect to $J$.

######  Definition 6.2.

We say a parabolic decomposition $w=vu$ is a Billey-Postnikov decomposition if
$S(v)\cap J\subseteq\mathcal{D}_{L}(u)$ where $\mathcal{D}_{L}(u)$ is the left
descent set of $u$, that is $\mathcal{D}_{L}(u)=\\{s\in S|l(su)=l(u)-1\\}$.

Furthermore we define a complete Billey-Postnikov decomposition as follows:

######  Definition 6.3.

$w$ has a complete BP decomposition, $w=v_{n}\cdots v_{1}$, if there is a
increasing sequence of sets $\emptyset\subsetneq J_{1}\subsetneq J_{2}\cdots
J_{k}=S(w)$ with $k=|S(w)|$ with $S(v_{j})\subset J_{j}$, $v_{j}\in W^{J_{j}}$
and $v_{j}(v_{j-1}\cdots v_{1})$ is a BP decomposition.

We now use a theorem from [4] showing a bijection between rationally smooth
Schubert varieties and complete BP decompositions.

######  Theorem 6.4.

A Schubert variety $X(w)$ is smooth if and only if $w$ has a complete BP
decomposition $w=v_{k}\cdots v_{1}$ where $X^{J_{j-1}}(v_{j})$ is smooth.

Inducing the following fiber bundle structure on $X(w)$ given the complete BP
decomposition $w=v_{k}\cdots v_{1}$:

$X(v_{1})$$X(v_{2}v_{1})$$X(v_{k-1}...v_{1})$$X(v_{k}\
...v_{1})$$X(v_{1})$$X^{J_{1}}(v_{2})$$X^{J_{k-2}}(v_{k-1})$$X^{J_{k-1}}(v_{k})$$...$

We also have the following theorem.

######  Theorem 6.5.

The subgroup $W_{J}$ is finite if $J$ is a finite type Dynkin diagram.

Which give the following corollary due to 6.1.

######  Corollary 6.6.

A staircase is spherical if each block is a finite type Dynkin diagram.

Now from [5] we have the following bijection between spherical staircase
diagrams and elements $w\in W$ with complete BP decompositions.

######  Theorem 6.7.

There is a bijection between spherical staircase diagrams and elements $w\in
W$ with complete BP decomposition $w=v_{k}\cdots v_{1}$ with $v_{j}$ the
largest element in $W_{S(v_{j})}\cap W^{J}$ for all $j\in\\{1,\cdots,n\\}$.

## References

  * [1] Sara Billey and Alexander Postnikov  “Smoothness of Schubert varieties via patterns in root subsystems”  In _Advances in Applied Mathematics_ 34.3 Elsevier, 2005, pp. 447–466
  * [2] William Feller  “An Introduction to Probability Theory and Its Applications”  Wiley, Hardcover, 1968  URL: http://www.amazon.ca/exec/obidos/redirect?tag=citeulike04-20{&}path=ASIN/0471257087
  * [3] R. Sedgewick P. Flajolet  “Analytic Combinatorics”  In _Choice Reviews Online_ 46.12 American Library Association, 2009, pp. 46–6858–46–6858  DOI: 10.5860/choice.46-6858
  * [4] Edward Richmond and William Slofstra  “Billey–Postnikov decompositions and the fibre bundle structure of Schubert varieties”  In _Mathematische Annalen_ 366.1-2 Springer ScienceBusiness Media LLC, 2015, pp. 31–55  DOI: 10.1007/s00208-015-1299-4
  * [5] Edward Richmond and William Slofstra  “Staircase diagrams and enumeration of smooth Schubert varieties”  In _Journal of Combinatorial Theory, Series A_ 150 Elsevier BV, 2017, pp. 328–376  DOI: 10.1016/j.jcta.2017.03.009