#  $q$-Rationals and Finite Schubert Varieties

Nicholas Ovenhouse

###### Abstract

The classical $q$-analogue of the integers was recently generalized by Morier-
Genoud and Ovsienko to give $q$-analogues of rational numbers. Some
combinatorial interpretations are already known, namely as the rank generating
functions for certain partially ordered sets. We review some of these
interpretations, and additionally give a slightly novel approach in terms of
planar graphs called snake graphs. Using the snake graph approach, we show
that the numerators of $q$-rationals count the sizes of certain varieties over
finite fields, which are unions of open Schubert cells in some Grassmannian.

###### Contents

  1. 1 Background
    1. 1.1 Geometry Over Finite Fields
    2. 1.2 Weight Generating Functions
  2. 2 $q$-Rational Numbers
    1. 2.1 Combinatorial Interpretations
    2. 2.2 Snake Graphs
  3. 3 Schubert Varieties
    1. 3.1 Definitions
    2. 3.2 Numerators of $q$-Rationals

##  1 Background

The classical “ _$q$ -analogue_” of a positive integer $n$ is the following
polynomial in $\mathbb{Z}[q]$

$[n]_{q}:=1+q+q^{2}+\cdots+q^{n-1}$

It satisfies the property that evaluation at $q=1$ yields the number $n$. In
general, any “ _$q$ -analogue_” should satisfy at least this simple property —
that evaluating $q=1$ (or more generally taking a limit $q\to 1$) should
recover the classical case.

Some simple well-known examples are built from the $q$-integers in the obvious
way. First is the $q$-factorial:

$[n]_{q}!:=[n]_{q}[n-1]_{q}\cdots[2]_{q}[1]_{q}$

Also we have the $q$-binomial coefficients:

$\binom{n}{k}_{q}:=\frac{[n]_{q}!}{[k]_{q}![n-k]_{q}!}$

However, the significance of a $q$-analogue would be superficial if the only
connection were this “$q=1$” property. Usually, a $q$-analogue satisfies some
other interesting properties, and has some deeper connection with the
underlying expression. In particular, $q$-analogues often have two other
interesting combinatorial interpretations.

  * (1)

They count the size of some algebraic variety defined over a finite field
$\mathbb{F}_{q}$.

  * (2)

They appear as weight generating functions of some combinatorial set for some
natural statistic.

Recently, Morier-Genoud and Ovsienko defined $q$-analogues of rational numbers
[MGO20]. Given a rational number $\frac{r}{s}$, they define a rational
function $\left[\frac{r}{s}\right]_{q}=\frac{\mathcal{R}(q)}{\mathcal{S}(q)}$,
which is defined in terms of the continued fraction expansion of
$\frac{r}{s}$. The formal definition will be given later, but here are some
examples:

$\left[\frac{5}{2}\right]_{q}=\frac{1+2q+q^{2}+q^{3}}{1+q},\quad\quad\left[\frac{10}{7}\right]_{q}=\frac{1+q+2q^{2}+3q^{3}+2q^{4}+q^{5}}{1+q+2q^{2}+2q^{3}+q^{4}}$

Morier-Genoud and Ovsienko give some combinatorial interpretations and
formulas for the polynomials $\mathcal{R}(q)$, but they pose the open problem
of finding more. They say the following in [MGO20]:

“ _It is a challenging problem to find more different combinatorial and
geometric interpretations of the polynomials $\mathcal{R}$ and $\mathcal{S}$.
A natural question is to connect the polynomials $\mathcal{R}$ and
$\mathcal{S}$ with counting of points in varieties defined over the finite
fields $\mathbb{F}_{q}$. This property would be similar to that of the
Gaussian $q$-binomial coefficients.”_

The aim of this paper is to address this question. We will review some known
interpretations of the polynomials $\mathcal{R}(q)$, as well as present some
new ones. In particular, we answer the question of giving an interpretation in
terms of varieties over finite fields. The organization of the rest of the
paper is as follows.

In the remainder of Section 1, we discuss, for $[n]_{q}!$ and
$\binom{n}{k}_{q}$, the two points mentioned above, how $q$-analogues often
have interpretations in terms of finite algebraic varieties as well as weight
generating functions.

In Section 2, we give the definition of $\left[\frac{r}{s}\right]_{q}$, the
$q$-analogue of the rational number $\frac{r}{s}$, and discuss some
combinatorial interpretations in terms of weight generating functions which
have already appeared in the literature. In this section we define “ _snake
graphs_ ”, and the main combinatorial interpretation of $q$-rationals for our
purposes will be as weight generating functions for certain paths on these
graphs. In this section we also define two partitions $\lambda$ and $\mu$
(with $\mu<\lambda$) associated with a rational number, which depend on the
continued fraction expansion. These partitions play an important part in the
statement of the main theorem later.

In Section 3, we begin by reviewing the decomposition of the Grassmannian
$\mathrm{Gr}_{k}(n)$ into “ _open Schubert cells_ ”. These cells are indexed
by partitions which fit inside a $k\times(n-k)$ rectangle. For each partition
$\lambda$, we write $\Omega_{\lambda}^{\circ}$ for the open Schubert cell. The
main result (see Theorem 3.2.1) is the following.

######  Theorem.

Let $\frac{r}{s}$ be a rational number, and
$\left[\frac{r}{s}\right]_{q}=\frac{\mathcal{R}(q)}{\mathcal{S}(q)}$. Also let
$\mu$ and $\lambda$ be the partitions defined in Section 2 associated with
$\frac{r}{s}$. Then

$q^{|\mu|}\mathcal{R}(q)=\left|\bigcup_{\mu\leq\nu\leq\lambda}\Omega_{\nu}^{\circ}\right|$

###  1.1 Geometry Over Finite Fields

Now let us recall how the above-mentioned examples (the $q$-factorial and
$q$-binomial coefficients) can be interpreted as the sizes of certain
algebraic varieties over finite fields. The results in this section are well-
known (see [Sta11] for example).

Before stating the results, let us establish some notation. Let $V$ be a
vector space (over a field $\mathbb{K}$), say of dimension $n$. The
_(complete) flag variety_ , which we will denote by $\mathrm{Fl}(V)$, is the
set of all complete flags in $V$. By a complete flag we mean a chain of nested
subspaces

$0\subset V_{1}\subset V_{2}\subset\cdots\subset V_{n-1}\subset V$

In the case that $V=\mathbb{K}^{n}$, we will just write $\mathrm{Fl}(n)$.

Recall also that the set of $k$-dimensional subspaces of a vector space is
called a _Grassmannian_ , and denoted $\mathrm{Gr}_{k}(V)$, or if
$V=\mathbb{K}^{n}$, just $\mathrm{Gr}_{k}(n)$.

######  Theorem 1.1.1.

If $\mathbb{K}=\mathbb{F}_{q}$ is a finite field, then

  * $(a)$

$\displaystyle\left|\mathrm{Fl}(n)\right|=[n]_{q}!$

  * $(b)$

$\displaystyle\left|\mathrm{Gr}_{k}(n)\right|=\binom{n}{k}_{q}$

######  Example 1.1.1.

The projective space $\mathbb{P}^{n-1}$ is the special case
$\mathrm{Gr}_{1}(n)$, of lines in $\mathbb{K}^{n}$. In particular, we have
that the $q$-integer $[n]_{q}=\binom{n}{1}_{q}$ is the size of
$\mathbb{P}^{n-1}$ over $\mathbb{F}_{q}$. This is also easy to see using the
usual definition of projective space:
$\mathbb{P}^{n-1}=(\mathbb{K}^{n}\setminus\\{0\\})/\sim$, where $\sim$ is the
equivalence relation $p\sim\alpha\,p$ for any non-zero $\alpha\in\mathbb{K}$.
In the case $\mathbb{K}=\mathbb{F}_{q}$, we have
$\left|\mathbb{K}\setminus\\{0\\}\right|=q^{n}-1$, and the number of non-zero
$\alpha$’s is $q-1$. And indeed we have $[n]_{q}=\frac{q^{n}-1}{q-1}$.

###  1.2 Weight Generating Functions

We will now recall the interpretations of $q$-factorials and $q$-binomial
coefficients as weight generating functions. As in the previous section, all
results presented here are well-known (see [Sta11] for a standard reference).

First we will discuss the $q$-factorial. The natural set that comes to mind
when one thinks of the number $n!$ is the symmetric group $S_{n}$, and indeed
the $q$-analogue $[n]_{q}!$ is a weight generating function for a statistic on
this set. Given a permutation $\sigma\in S_{n}$, an _inversion_ of $\sigma$ is
a pair $(i,j)$ such that $i<j$ and $\sigma(i)>\sigma(j)$. Define
$\mathrm{inv}(\sigma)$ to be the total number of inversions.

######  Theorem 1.2.1.

$\displaystyle[n]_{q}!=\sum_{\sigma\in S_{n}}q^{\mathrm{inv}(\sigma)}$

Next we turn to the $q$-binomial coefficients. When one thinks of the numbers
$\binom{n}{k}$, one naturally thinks of the set $\binom{[n]}{k}$, the set of
subsets of $[n]=\\{1,2,\dots,n\\}$ of size $k$. However, for the present
purposes it is more natural to consider a couple other sets which are in
bijection with $\binom{[n]}{k}$.

Consider the set $P_{n,k}$ of lattice paths from $(0,0)$ to $(n-k,k)$ which
only take unit steps right or up. All such paths must take a total of $n$
steps, $k$ of which are “up”. So there is a natural bijection
$\binom{[n]}{k}\to P_{n,k}$. For a path $p\in P_{n,k}$, let $|p|$ be the area
above $p$ and below the horizontal line $y=k$.

######  Theorem 1.2.2.

$\displaystyle\sum_{p\in P_{n,k}}q^{|p|}=\binom{n}{k}_{q}$

This can also be described nicely in terms of integer partitions. Recall that
a _partition_ $\lambda$ is a sequence of weakly decreasing integers
$\lambda_{1}\geq\lambda_{2}\geq\cdots\geq\lambda_{k}$. If $\sum\lambda_{i}=n$,
we write $\lambda\vdash n$, and also $|\lambda|=n$. Partitions are visualized
by their Young diagram, which is an array of boxes (left-aligned) with
$\lambda_{i}$ boxes in row $i$. Let $Y_{n,k}$ be the set of all partitions
whose Young diagram fits inside the rectangle with height $k$ and width $n-k$.
There is a clear bijection $Y_{n,k}\to P_{n,k}$. The bottom boundary of any
Young diagram is a lattice path in $P_{n,k}$, and the number of boxes in the
Young diagram is precicely the area above the lattice path.

######  Corollary 1.2.1.

$\displaystyle\sum_{\lambda\in Y_{n,k}}q^{|\lambda|}=\binom{n}{k}_{q}$

Lastly, let us just mention that the previous expressions can also be thought
of in terms of posets. In particular, the set of all Young diagrams (or
partitions) has a natural partial order where $\mu\leq\lambda$ means that
$\mu_{i}\leq\lambda_{i}$ for all $i$, or equivalently the Young diagram of
$\mu$ fits inside $\lambda$. This poset is called _Young’s lattice_. The poset
has a natural _rank_ function, given by the size of the partition (the number
of boxes). If we let $\lambda=(n-k)^{k}$ denote the Young diagram given by the
$k\times(n-k)$ rectangle, then the set $Y_{n,k}$ above is simply the interval
of Young’s lattice $[\varnothing,\lambda]=\\{\mu~{}|~{}\mu\leq\lambda\\}$, and
the generating function given above is the rank generating function for this
poset. This simply means the coefficient of $q^{k}$ is the number of elements
of rank $k$.

######  Example 1.2.1.

The poset structure for $Y_{4,2}$ and $P_{4,2}$ is pictured in Figure 1. The
sizes of the ranks are the coefficients of
$\binom{4}{2}_{q}=1+q+2q^{2}+q^{3}+q^{4}$.

Figure 1: Poset structure for $Y_{4,2}$

##  2 $q$-Rational Numbers

In [MGO20], Morier-Genoud and Ovsienko extend the definition of $[n]_{q}$ to
include the case when $n=\frac{r}{s}\in\mathbb{Q}$ is a rational number. Their
definition uses the continued fraction expansion of $\frac{r}{s}$.
Specifically, if $\frac{r}{s}=[a_{1},a_{2},\dots,a_{2m}]$ is the finite
continued fraction expansion (and $a_{1}>1$), then they define

$\left[\frac{r}{s}\right]_{q}:=[a_{1}]_{q}+\cfrac{q^{a_{1}}}{[a_{2}]_{q^{-1}}+\cfrac{q^{-a_{2}}}{[a_{3}]_{q}+\cfrac{q^{a_{3}}}{[a_{4}]_{q^{-1}}+\cfrac{q^{-a_{4}}}{\ddots\,+\,\cfrac{q^{a_{2m-1}}}{[a_{2m}]_{q^{-1}}}}}}}$

Morier-Genoud and Ovsienko also give several other ways to compute these
expressions, including some determinantal formulas, a recursive procedure
involving triangulated polygons, and using products of $q$-deformed matrices
in $\mathrm{PSL}_{2}(\mathbb{Z})$. We will briefly describe this last
interpretation in terms of matrices (for more details, see [MGO20] and
[LMG21]).

Define the following two matrices:

$A=\begin{pmatrix}1&1\\\ 0&1\end{pmatrix}\quad\text{ and }\quad
B=\begin{pmatrix}1&0\\\ 1&1\end{pmatrix}$

They generate $\mathrm{PSL}_{2}(\mathbb{Z})$, and the group acts transitively
on $\mathbb{Q}\cup\\{\infty\\}$ by the rule:

$\begin{pmatrix}a&b\\\ c&d\end{pmatrix}\cdot x:=\frac{ax+b}{cx+d}$

Define $q$-deformed versions of these matrices:

$A_{q}=\begin{pmatrix}q&1\\\ 0&1\end{pmatrix}\quad\text{ and }\quad
B_{q}=\begin{pmatrix}q&0\\\ q&1\end{pmatrix}$

Let $\mathrm{PSL}_{2}^{q}(\mathbb{Z})$ be the group generated by $A_{q}$ and
$B_{q}$, modulo scaling by monomials $q^{\pm n}$. In general, for any
$M\in\mathrm{PSL}_{2}(\mathbb{Z})$, we get a corresponding element
$[M]_{q}\in\mathrm{PSL}_{2}^{q}(\mathbb{Z})$. This group acts on the set of
power series $\mathbb{Z}[[q]]\cup\\{\infty\\}$ by the same formula as above.
The following is another, more conceptual, definition of $q$-rational numbers.

######  Theorem 2.0.1.

[MGO20] [LMG21] The $q$-deformation $x\mapsto[x]_{q}$ commutes with the
$\mathrm{PSL}_{2}(\mathbb{Z})$ action. In particular, this means if
$x\in\mathbb{Q}$ and $M\in\mathrm{PSL}_{2}(\mathbb{Z})$, then $[M\cdot
x]_{q}=[M]_{q}\cdot[x]_{q}$.

######  Example 2.0.1.

The continued fraction expansion of $\frac{7}{3}$ is $[2,3]$. Using the
definition, we get

$\left[\frac{7}{3}\right]_{q}=[2]_{q}+\cfrac{q^{2}}{[3]_{q^{-1}}}=(1+q)+\frac{q^{2}}{1+q^{-1}+q^{-2}}=\frac{1+2q+2q^{2}+q^{3}+q^{4}}{1+q+q^{2}}$

Alternatively, we can use Theorem 2.0.1 to compute. Notice that the generators
$A$ and $B$ act by $A\cdot x=x+1$ and $B\cdot x=\frac{x}{x+1}$. Notice that
$\frac{7}{3}=A^{2}B^{2}\cdot 1$:

$1\xrightarrow{~{}B~{}}\frac{1}{2}\xrightarrow{~{}B~{}}\frac{1}{3}\xrightarrow{~{}A~{}}\frac{4}{3}\xrightarrow{~{}A~{}}\frac{7}{3}$

According to Theorem 2.0.1, we should be able to start with $1$ and apply
$A_{q}^{2}B_{q}^{2}$ to compute $\left[\frac{7}{3}\right]_{q}$. Notice that
the action of $A_{q}$ and $B_{q}$ are given by $A_{q}\cdot x=1+qx$ and
$B_{q}\cdot x=\frac{qx}{1+qx}$.

$1\xrightarrow{~{}B_{q}~{}}\frac{q}{1+q}\xrightarrow{~{}B_{q}~{}}\frac{q^{2}}{1+q+q^{2}}\xrightarrow{~{}A_{q}~{}}\frac{1+q+q^{2}+q^{3}}{1+q+q^{2}}\xrightarrow{~{}A_{q}~{}}\frac{1+2q+2q^{2}+q^{3}+q^{4}}{1+q+q^{2}}$

###  2.1 Combinatorial Interpretations

Let $\left[\frac{r}{s}\right]_{q}=\frac{\mathcal{R}(q)}{\mathcal{S}(q)}$. One
of the interpretations given in [MGO20] says that the coefficient of $q^{k}$
in $\mathcal{R}(q)$ is the number of “ _$k$ -vertex closures_” in a certain
directed graph. Although worded differently, this is equivalent to the
following interpretation in terms of posets (see [MSS21]).

######  Definition 2.1.1.

Define a poset $F\left(\frac{r}{s}\right)$ on the set
$\\{x_{1},x_{2},\dots,x_{N-1}\\}$, where $N=\sum_{i=1}^{2m}a_{i}$, with cover
relations:

$x_{1}<x_{2}<\cdots<x_{a_{1}}>x_{a_{1}+1}>\cdots>x_{a_{1}+a_{2}}<\cdots$

In other words, the Hasse diagram is a “ _fence_ ” which goes up $(a_{1}-1)$
times, then down $a_{2}$ times, then up $a_{3}$ times, $\dots$, and finally up
$a_{2m-1}$ times, and down $(a_{2m}-1)$ times.

Let $L\left(\frac{r}{s}\right)$ be the lattice of order ideals in
$F\left(\frac{r}{s}\right)$. In this language, what Morier-Genoud and Ovsienko
call a “ _$k$ -vertex closure_” is the same as a lower order ideal of
$F\left(\frac{r}{s}\right)$. In particular we have the following.

######  Theorem 2.1.1.

[MGO20] $\mathcal{R}(q)$ is the rank generating function of
$L\left(\frac{r}{s}\right)$.

######  Remark 2.1.1.

The connection between these posets and $F$-polynomials in cluster algebras
has been noted in [MSW11], [Rab18], [BG21], and [Cla20]. These posets were
called “ _fence posets_ ” in [MSS21] and “ _piece-wise linear posets_ ” in
[BG21]. It was shown in [MGO20] that $\mathcal{R}(q)$ can be obtained from an
$F$-polynomial by specialization of the variables.

######  Example 2.1.1.

We continue with the running example of $\frac{r}{s}=\frac{7}{3}$ from Example
2.0.1. Recall that the continued fraction for $\frac{7}{3}$ is $[2,3]$. The
corresponding fence $F(7/3)$ is

The Lattice $L(7/3)$ of order ideals in $F(7/3)$ is pictured in Figure 2.
Notice the number of elements of rank $k$ is the coefficient of $q^{k}$ in
$\mathcal{R}(q)=1+2q+2q^{2}+q^{3}+q^{4}$.

Figure 2: Poset structure of $L(7/3)$

###  2.2 Snake Graphs

Yet another equivalent description can be given in terms of “ _snake graphs_
”. These are certain types of planar graphs built out of square tiles, such
that each tile is either above or to the right of the previous one. To each
snake graph $G$ we can naturally associate a _word_ $W(G)$ in the alphabet
$\\{R,U\\}$, indicating whether going from one tile to the next is “ _right_ ”
or “ _up_ ”. Some examples are shown in Figure 3.

$W(G)=RR$$G=G(4/1)$$4/1=[3,1]$$W(G)=URUR$$G=G(13/8)$$13/8=[1,1,1,1,1,1]$$W(G)=RURR$$G=G(11/4)$$11/4=[2,1,2,1]$
Figure 3: Examples of snake graphs

######  Definition 2.2.1.

Let $\overline{R}=U$ and $\overline{U}=R$. If $w=w_{1}w_{2}\cdots w_{k}$ is a
word in the alphabet $\\{R,U\\}$, define the _dual word_ to be
$w^{*}=\overline{w}_{1}w_{2}\overline{w}_{3}w_{4}\cdots\overline{w}_{2n-1}w_{2n}$
or $w^{*}=\overline{w}_{1}w_{2}\overline{w}_{3}w_{4}\cdots
w_{2n}\overline{w}_{2n+1}$ (depending on parity). In other words, $w^{*}$ is
obtained by toggling the odd-indexed letters. Extend this definition to snake
graphs by defining $G^{*}$ to be the snake graph so that $W(G^{*})=W(G)^{*}$.

######  Remark 2.2.1.

Snake graphs have appeared numerous times in the cluster algebra literature
(see for example [ÇS18] [Rab18] [MSW11], [Pro05]). The dual construction
($G^{*}$ rather than $G$) seems to be more popular in the literature (in
[Pro05], Propp first constructs $G^{*}$ and calls $G$ the “dual” snake). In
what follows, results from the literature are paraphrased in terms of $G$
rather than $G^{*}$.

######  Definition 2.2.2.

[ÇS18] Given a continued fraction $\frac{r}{s}=[a_{1},\dots,a_{2m}]$, define a
snake graph $G\left(\frac{r}{s}\right)$ such that its word is
$W(G)=R^{a_{1}-1}U^{a_{2}}R^{a_{3}}U^{a_{4}}\cdots R^{a_{2m-1}}U^{a_{2m}-1}$.
Figure 3 also indicates which rational number $\frac{r}{s}$ is associated to
each snake graph.

######  Definition 2.2.3.

Given a snake graph $G$, let $P(G)$ be the set of lattice paths on $G$, going
from the bottom-left corner to the top-right corner, which only take steps
right or up. We also use the notation $P(r/s)=P(G(r/s))$.

######  Theorem 2.2.1.

[Pro05] [ÇS18] The number of lattice paths on $G(r/s)$ is $r$. In other words,
$|P(r/s)|=r$.

We now give a $q$-analogue of Theorem 2.2.1, where $r$ is replaced by
$\mathcal{R}(q)$, the numerator of the corresponding $q$-rational. Note that
there is a natural partial order on $P(G)$ so that $a\leq b$ if the set of
boxes above $a$ is a subset of the boxes above $b$. Let $|a|$ denote the
number of boxes above the path $a$.

######  Proposition 2.2.1.

$\mathcal{R}(q)$ is the rank-generating function of the poset $P(r/s)$. In
other words,

$\sum_{p\in P(r/s)}q^{|p|}=\mathcal{R}(q)$

###### Proof.

It is easy to see that $P(r/s)$ is isomorphic to the poset $L(r/s)$ described
earlier. The points of $F(r/s)$ correspond to the boxes of the snake graph,
and the order ideals of $F(r/s)$ are the sets of boxes above the lattice
paths.

More specifically, we can construct $G(r/s)$ from $F(r/s)$ as follows (see
Figure 4 for an illustration). Start with the Hasse diagram of $F(r/s)$, and
reflect it over a horizontal line. Next, rotate it $45^{\circ}$ counter-
clockwise. Then draw a square around each vertex. This will be $G(r/s)$. ∎

$F(r/s)$reflectrotate$G(r/s)$ Figure 4: Proof of Proposition 2.2.1

######  Example 2.2.1.

The poset $P(7/3)$ is shown in Figure 5. Compare with $L(7/3)$ from Example
2.1.1 and Figure 2.

Figure 5: Poset structure of $P(7/3)$

The snake graph interpretation allows to make a nice analogy with the
situation for $q$-binomial coefficients. In particular, $P(r/s)$ is also
isomorphic to an interval in Young’s lattice. This follows from the
observation that any snake graph is a skew Young diagram of shape
$\lambda/\mu$ where $\lambda$ is determined by the bottom boundary of $G$, and
$\mu$ is determined by the top boundary of $G$.

######  Definition 2.2.4.

If $G$ is a snake graph, let $\lambda(G)$ be the partition such that $G$ is
the subset of boxes adjacent to the bottom boundary of $\lambda$. Also, let
$\mu(G)<\lambda(G)$ be the partition such that $G$ is the skew Young diagram
of shape $\lambda(G)/\mu(G)$. If $G=G(r/s)$, then we use the notations
$\lambda(r/s)$ and $\mu(r/s)$.

######  Example 2.2.2.

Some examples of $\lambda$ and $\mu$, along with their corresponding fractions
$\frac{r}{s}$, are shown in Figure 6.

$\frac{r}{s}=\frac{4}{1}=[3,1]$$W(4/1)=RR$$\lambda=(3)$,
$\mu=\varnothing$$\frac{r}{s}=\frac{12}{5}=[2,2,1,1]$$W(12/5)=RUUR$$\lambda=(3,2^{2})$,
$\mu=(1^{2})$$\frac{r}{s}=\frac{31}{18}=[1,1,2,1,1,2]$$W(31/18)=URRURU$$\lambda=(4^{2},3,1)$,
$\mu=(3,2)$ Figure 6: Examples of $\lambda(r/s)$ and $\mu(r/s)$. The partition
$\mu$ is shaded in gray.

In order to state the next result, we introduce the following notation. We
will write partitions as
$\lambda=(\lambda_{1}^{b_{1}},\lambda_{2}^{b_{2}},\dots)$, where
$\lambda_{i}^{b_{i}}$ means that $\lambda_{i}$ is repeated $b_{i}$ times. For
example, $(3^{4},2,1^{2})=(3,3,3,3,2,1,1)$.

######  Proposition 2.2.2.

Suppose $\frac{r}{s}=[a_{1},b_{1},\dots,a_{m},b_{m}]$. The partitions
$\lambda(r/s)$ and $\mu(r/s)$ are given explicitly as follows.

  * $(a)$

$\lambda=(\lambda_{1}^{b_{m}},\lambda_{2}^{b_{m-1}},\dots,\lambda_{m}^{b_{1}})$,
where $\lambda_{k}=\sum\limits_{i=1}^{m+1-k}a_{i}$.

  * $(b)$

$\mu=(\mu_{1}^{c_{1}},\dots,\mu_{m}^{c_{m}})$, where $\mu_{i}=\lambda_{i}-1$
and $c_{i}$ are given by $c_{1}=b_{m}-1$ and $c_{i}=b_{m+1-i}$ otherwise.

###### Proof.

$(a)$ We will induct on $m$. Recall the definition of the snake graph
$G(r/s)$. It is defined by the word
$W(r/s)=R^{a_{1}-1}U^{b_{1}}R^{a_{2}}U^{b_{2}}\cdots R^{a_{m}}U^{b_{m}-1}$. By
definition of $\lambda(r/s)$, the partition $\lambda$ will have the same
bottom boundary as $G(r/s)$. Suppose $m=1$. Then
$W(r/s)=R^{a_{1}-1}U^{b_{1}-1}$. The young diagram which has this as its
bottom boundary is the rectangle with width $a_{1}$ and height $b_{1}$ (i.e.
$\lambda=(a_{1}^{b_{1}})$).

Now suppose the result is true for all continued fractions
$\frac{r^{\prime}}{s^{\prime}}=[a_{1},b_{1},\dots,a_{m-1},b_{m-1}]$. Then by
induction
$\lambda(r^{\prime}/s^{\prime})=\lambda^{\prime}=(\lambda_{1}^{b_{m-1}},\lambda_{2}^{b_{m-2}},\dots,\lambda_{m-1}^{b_{1}})$,
where $\lambda_{k}=\sum_{i=1}^{m-k}a_{i}$. The snake graph for
$\frac{r}{s}=[a_{1},b_{1},\dots,a_{m},b_{m}]$ is build from the previous one
by adding some more boxes to the end. This means the parition
$\lambda=\lambda(r/s)$ will be obtained from
$\lambda^{\prime}=\lambda(r^{\prime}/s^{\prime})$ by adding some more rows to
the top.

First let us point out a subtle point. The word $W(r^{\prime}/s^{\prime})$
ends with $U^{b_{m-1}-1}$, but $W(r/s)$ contains $U^{b_{m-1}}$. This means we
must add one more box going up at the end of $G(r^{\prime}/s^{\prime})$ before
continuing to build $G(r/s)$. This simply means the rows of
$\lambda(r^{\prime}/s^{\prime})$ will not be changed. Now that we have added
an extra $U$, the next part of $W(r/s)$ is $R^{a_{m}}$. This means we add
another row on top of $\lambda(r^{\prime}/s^{\prime})$ whish has $a_{m}$ more
boxes than the previous row. By induction, the previous row has
$\sum_{i=1}^{m-1}a_{i}$ boxes, and so this new row has $\sum_{i=1}^{m}a_{i}$
boxes. Finally, $W(r/s)$ ends with $U^{b_{m}-1}$, and so there are a total of
$b_{m}$ rows at the top, all of this length.

$(b)$ The statement says $\mu$ is obtained from $\lambda$ by subtracting one
from all parts (i.e. $\mu_{i}=\lambda_{i}-1$), and then the multiplicities are
the same for all but the first (i.e. the greatest) part, which is one less. To
see this, consider a part of the word $W(r/s)$ of the form $R^{a}U^{b}$. When
the snake graph goes right $a$ times, this corresponds to the bottom-most row
of of the Young diagram of some length $\lambda_{i}$. After this, every time
it goes up (except the last), we get another row of length $\lambda_{i}$. The
exception is the last time it goes up, since if the snake graph continues to
go right again after this, the final “up” will be part of a longer row.

At each of these “up” steps, the corresponding part of $\mu$ will have length
$(\lambda_{i}-1)$ (including the last “up”). If this part $R^{a}U^{b}$ is not
the end of the snake graph, then by part $(a)$, $b$ is the number of parts of
$\lambda$ of length $\lambda_{i}$. However, in the special case that
$U^{b_{m}-1}$ is the end of the snake graph, we only get $b_{m}-1$ parts of
length $\mu_{m}$ (but there are $b_{m}$ parts of length $\lambda_{m}$). ∎

######  Proposition 2.2.3.

Let $\lambda=\lambda(r/s)$ and $\mu=\mu(r/s)$. The poset $P(r/s)$ is
isomorphic to the interval $[\mu,\lambda]$ in Young’s lattice, and

$\mathcal{R}(q)=\frac{1}{q^{|\mu|}}\sum_{\mu\leq\nu\leq\lambda}q^{|\nu|}$

###### Proof.

There is the natural bijection between $P(r/s)$ and the interval
$[\mu,\lambda]$ in Young’s lattice, where a lattice path corresponds to the
Young diagram whose boxes lie above the path. Under this bijection, the rank
of a path corresponding to a partition $\nu$ is $|\nu|-|\mu|$, which is the
number of boxes in the snake graph above the path. The formula for
$\mathcal{R}(q)$ follows from this bijection together with Proposition 2.2.1.
∎

##  3 Schubert Varieties

###  3.1 Definitions

Let $\mathbb{K}$ be a field, and consider the Grassmannian
$\mathrm{Gr}_{k}(n)$ of $k$-planes in $\mathbb{K}^{n}$. We will identify
$\mathrm{Gr}_{k}(n)$ with the set of $k\times n$ matrices of rank $k$, modulo
left multiplication by elements of $\mathrm{GL}_{k}$. The following
definitions and results are all well-known (see [Ful97] for a standard
reference, although here we use notational conventions as in [Pos06]).

There is a simple bijection between the interval $[\varnothing,(n-k)^{k}]$ in
Young’s lattice (i.e. partitions which fit inside a $k\times(n-k)$ rectangle)
and the set $\binom{[n]}{k}$ of $k$-element subsets of
$[n]=\\{1,2,\dots,n\\}$, given as follows. Recall from Section 1.2 the
bijection $Y_{n,k}\to P_{n,k}$ that associates to a Young diagram $\lambda$
the lattice path from $(0,0)$ to $(n-k,k)$ which is formed by the bottom
boundary of $\lambda$. If the path is traversed backwards, and the steps are
labelled $1,2,\dots,n$, then taking the subset of vertical steps gives an
element $I_{\lambda}\in\binom{[n]}{k}$.

######  Definition 3.1.1.

The _open Schubert cell_ $\Omega^{\circ}_{\lambda}\subseteq\mathrm{Gr}_{k}(n)$
is the subset whose representatives, when written in echelon form, have the
identity matrix in the columns indexed by $I_{\lambda}$. The remaining non-
zero entries (to the right of the pivots) form the shape of $\lambda$ (but
backwards).

From this definition, it is easy to see that
$\dim(\Omega^{\circ}_{\lambda})=|\lambda|$. Once the matrices are in echelon
form, there is a clear parameterization by $\mathbb{K}^{|\lambda|}$.

######  Example 3.1.1.

The partition $\lambda=(3,2,2,1)$ fits inside a $3\times 4$ rectangle
(corresponding to $\mathrm{Gr}_{4}(7)$). The corresponding column set is
$I_{\lambda}=\\{1,3,4,6\\}$. The open Schubert cell $\Omega^{\circ}_{\lambda}$
contains all matrices whose echelon form has the shape given in Figure 7.

$1$$3$$4$$6$$I_{\lambda}=\\{1,3,4,6\\}$$\begin{pmatrix}1&\ast&0&0&\ast&0&\ast\\\\[4.30554pt]
0&0&1&0&\ast&0&\ast\\\\[4.30554pt] 0&0&0&1&\ast&0&\ast\\\\[4.30554pt]
0&0&0&0&0&1&\ast\end{pmatrix}$ Figure 7: (left) The subset
$I_{\lambda}\in\binom{[n]}{k}$ corresponding to a partition $\lambda$, and
(right) the echelon form of a matrix representative in
$\Omega^{\circ}_{\lambda}$

######  Definition 3.1.2.

The _Schubert variety_ (or _closed Schubert cell_) is the closure
$\Omega_{\lambda}:=\overline{\Omega^{\circ}_{\lambda}}$.

It is a well-known fact that $\Omega_{\lambda}$ is the disjoint union of all
open Schubert cells corresponding to partitions that fit inside $\lambda$:

$\Omega_{\lambda}=\bigcup_{\mu\leq\lambda}\Omega^{\circ}_{\mu}$

In particular, the Grassmannian is the closure of the “biggest” cell
$\Omega^{\circ}_{\lambda}$, where $\lambda=(n-k)^{k}$ is the entire
$k\times(n-k)$ rectangle, and thus $\mathrm{Gr}_{k}(n)$ is the disjoint union
of all the open Schubert cells.

Now, putting this all together, we are in a position to see the connection
between the geometric and combinatorial interpretations of the $q$-binomial
coefficients given in Theorem 1.1.1(b) and Theorem 1.2.2. If we take
$\mathbb{K}=\mathbb{F}_{q}$, then clearly
$\left|\Omega^{\circ}_{\lambda}\right|=q^{|\lambda|}$. The fact that the
Grassmannian is the disjoint union of all open Schubert cells implies that

$\left|\mathrm{Gr}_{k}(n)\right|=\sum_{\lambda\leq(n-k)^{k}}\left|\Omega^{\circ}_{\lambda}\right|=\sum_{\lambda\leq(n-k)^{k}}q^{|\lambda|}=\binom{n}{k}_{q}$

###  3.2 Numerators of $q$-Rationals

Finally, we return to the discussion of $\mathcal{R}(q)$, then numerator of
the $q$-rational $\left[\frac{r}{s}\right]_{q}$, and we will show that it is
the size of a certain subvariety of $\mathrm{Gr}_{k}(n)$ (over
$\mathbb{F}_{q}$), where $k$ and $n$ can be determined by the continued
fraction $\frac{r}{s}=[a_{1},a_{2},\dots,a_{2m}]$. Let
$n=\sum_{i=1}^{2m}a_{i}$ and $k=\sum_{i=1}^{m}a_{2i}$.

######  Theorem 3.2.1.

Let $\frac{r}{s}=[a_{1},\dots,a_{2m}]\in\mathbb{Q}$, and let
$\left[\frac{r}{s}\right]_{q}=\frac{\mathcal{R}(q)}{\mathcal{S}(q)}$ be the
corresponding $q$-rational. Let $\mu=\mu(r/s)$ and $\lambda=\lambda(r/s)$.
Then up to a factor of $q^{|\mu|}$, the polynomial $\mathcal{R}(q)$ is the
number of $\mathbb{F}_{q}$ points in the union of open Schubert cells in the
interval $[\mu,\lambda]$ in the Grassmannian $\mathrm{Gr}_{k}(n)$:

$q^{|\mu|}\cdot\mathcal{R}(q)=\left|\bigcup_{\mu\leq\nu\leq\lambda}\Omega^{\circ}_{\nu}\right|$

###### Proof.

Recall from Proposition 2.2.3 that the numerator $\mathcal{R}(q)$ is the rank
generating function of the interval $[\mu,\lambda]$ in Young’s lattice (scaled
by $q^{|\mu|}$). The width of the snake graph (also the width of $\lambda$) is
$\sum_{i=1}^{m}a_{2i-1}$, and the height is $\sum_{i=1}^{m}a_{2i}$. Since the
parameters $n$ and $k$ of the Grassmannian correspond to Young diagrams in a
$k\times(n-k)$ rectangle, this implies that $k=\sum_{i=1}^{m}a_{2i}$ and
$n-k=\sum_{i=1}^{m}a_{2i-1}$.

Since $q^{|\nu|}$ is the size of the open Schubert cell
$\Omega^{\circ}_{\nu}$, we see that

$\mathcal{R}(q)=\frac{1}{q^{|\mu|}}\sum_{\mu\leq\nu\leq\lambda}\left|\Omega^{\circ}_{\nu}\right|$

∎

######  Example 3.2.1.

If $\frac{r}{s}=\frac{n}{1}$ or $\frac{r}{s}=\frac{n+1}{n}$, then the snake
graph $G(r/s)$ is a straight row or column of boxes. In this case, the
partition $\mu$ is empty, and $\lambda$ is the full $n\times 1$ or $1\times n$
rectangle. In this case Theorem 3.2.1 reduces to a special case of Theorem
1.1.1(b), which, as described in Example 1.1.1, simply says that
$|\mathbb{P}^{n-1}|=[n]_{q}$. The interpretation given in Theorem 3.2.1 in
terms of Schubert cells just states that
$\mathbb{P}^{n-1}=\bigcup_{k=1}^{n-1}X_{k}$, where $X_{k}$ consists of those
points whose homogeneous coordinates have the form

$[x_{0}:x_{1}:\cdots:x_{n-1}]=[0:0:\cdots:0:1:\ast:\cdots:\ast]$

where $x_{k}=1$ and $x_{i}=0$ for $i<k$.

######  Example 3.2.2.

Continuing with Example 2.1.1 and Example 2.2.1, let
$\frac{r}{s}=\frac{7}{3}=[2,3]$. The corresponding $q$-rational is

$\left[\frac{7}{3}\right]_{q}=\frac{\mathcal{R}(q)}{\mathcal{S}(q)}=\frac{1+2q+2q^{2}+q^{3}+q^{4}}{1+q+q^{2}}$

The snake graph $G(7/3)$ has word $W(G)=RUU$, and is pictured (along with the
poset $P(7/3)$) in Figure 5. The snake graph has width 2 and height 3, which
means the corresponding union of Schubert cells lives in $\mathrm{Gr}_{3}(5)$
(since $k=3$ and $n-k=2$).

In the notation of Theorem 3.2.1, we have $\lambda=(2,2,2)$ and $\mu=(1,1)$.
In this case Theorem 3.2.1 says that $q^{2}\mathcal{R}(q)$ counts the number
of points in $\bigcup_{\nu\in[\mu,\lambda]}\Omega^{\circ}_{\nu}$, which
consists of all 3-dimensional subspaces of $\mathbb{F}_{q}^{5}$ which have a
matrix representative of one of the following seven forms:

$\begin{pmatrix}1&0&0&\ast&\ast\\\\[4.30554pt] 0&1&0&\ast&\ast\\\\[4.30554pt]
0&0&1&\ast&\ast\end{pmatrix},\quad\begin{pmatrix}1&0&\ast&0&\ast\\\\[4.30554pt]
0&1&\ast&0&\ast\\\\[4.30554pt]
0&0&0&1&\ast\end{pmatrix},\quad\begin{pmatrix}1&0&\ast&\ast&0\\\\[4.30554pt]
0&1&\ast&\ast&0\\\\[4.30554pt]
0&0&0&0&1\end{pmatrix},\quad\begin{pmatrix}1&\ast&0&0&\ast\\\\[4.30554pt]
0&0&1&0&\ast\\\\[4.30554pt] 0&0&0&1&\ast\end{pmatrix},$
$\begin{pmatrix}0&1&0&0&\ast\\\\[4.30554pt] 0&0&1&0&\ast\\\\[4.30554pt]
0&0&0&1&\ast\end{pmatrix},\quad\begin{pmatrix}1&\ast&0&\ast&0\\\\[4.30554pt]
0&0&1&\ast&0\\\\[4.30554pt]
0&0&0&0&1\end{pmatrix},\quad\begin{pmatrix}0&1&0&\ast&0\\\\[4.30554pt]
0&0&1&\ast&0\\\\[4.30554pt] 0&0&0&0&1\end{pmatrix}$

## Acknowledgments

I would like to acknowledge the support of the NSF grant DMS-1745638. I would
also like to thank Valentin Ovsienko and Vic Reiner for helpful comments and
conversations.

## References

  * [BG21] Rachel Bailey and Emily Gunawan.  Cluster algebras and binary subwords.  Order, pages 1–15, 2021.
  * [Cla20] Andrew Claussen.  Expansion posets for polygon cluster algebras.  arXiv preprint arXiv:2005.02083, 2020.
  * [ÇS18] İlke Çanakçı and Ralf Schiffler.  Cluster algebras and continued fractions.  Compositio mathematica, 154(3):565–593, 2018.
  * [Ful97] William Fulton.  Young tableaux: with applications to representation theory and geometry.  Number 35. Cambridge University Press, 1997.
  * [LMG21] Ludivine Leclere and Sophie Morier-Genoud.  q-deformations in the modular group and of the real quadratic irrational numbers.  Advances in Applied Mathematics, 130:102223, 2021.
  * [MGO20] Sophie Morier-Genoud and Valentin Ovsienko.  $q$-continued fractions.  In Forum of Mathematics, Sigma, volume 8. Cambridge University Press, 2020.
  * [MSS21] Thomas McConville, Bruce E Sagan, and Clifford Smyth.  On a rank-unimodality conjecture of morier-genoud and ovsienko.  Discrete Mathematics, 344(8):112483, 2021.
  * [MSW11] Gregg Musiker, Ralf Schiffler, and Lauren Williams.  Positivity for cluster algebras from surfaces.  Advances in Mathematics, 227(6):2241–2308, 2011.
  * [Pos06] Alexander Postnikov.  Total positivity, grassmannians, and networks.  arXiv preprint math/0609764, 2006.
  * [Pro05] James Propp.  The combinatorics of frieze patterns and markoff numbers.  arXiv preprint math/0511633, 2005.
  * [Rab18] Michelle Rabideau.  F-polynomial formula from continued fractions.  Journal of Algebra, 509:467–475, 2018.
  * [Sta11] Richard Stanley.  Enumerative combinatorics volume 1 second edition.  Cambridge studies in advanced mathematics, 2011.


titleA non uniqueness result for operators of principal type

IOP Publishing

journaltitleClassical and Quantum Gravity
titleMaking anti-de Sitter black holes

journaltitlePhys. Rev. D
titleConstant curvature black holes

American Physical Society

journaltitlePhys. Rev. Lett.
titleBlack hole in three-dimensional spacetime

American Physical Society (APS)

journaltitlePhysical Review D
titleGeometry of the 2+1 black hole

journaltitleMathematical Research Letters
titleContinuation unique a partir de l'infini conforme pour les metriques d'Einstein

Cambridge University Press

titleSpacetime and Geometry

titleA gauge-invariant unique continuation criterion for waves in asymptotically anti-de Sitter spacetimes
yearin preparation

journaltitleJournal of Geometry and Physics
titleUnique continuation and extensions of Killing vectors at boundaries for stationary vacuum space-times

journaltitleCommunications in Mathematical Physics
titleHolographic Reconstruction of Spacetime and Renormalization in the AdS/CFT Correspondence

Société mathématique de France

journaltitleÉlie Cartan et les mathématiques d'aujourd'hui - Lyon, 25-29 juin 1984
titleConformal invariants

titleVolume and Area Renormalizations for Conformally Compact Einstein Metrics

journaltitleNucl. Phys. B
titleConformal Anomaly Of Submanifold Observables In AdS/CFT Correspondence

journaltitleAdvances in Mathematics
titleEinstein metrics with prescribed conformal infinity on the ball

titleConstruction of counter-examples to the wave equation unique continuation problem with a critically singular potential
yearin preparation

Springer Science
Business Media LLC

journaltitleCommunications in Mathematical Physics
titleUnique Continuation from Infinity in Asymptotically Anti-de Sitter Spacetimes

Informa UK Limited

journaltitleCommunications in Partial Differential Equations
titleUnique continuation from infinity in asymptotically anti-de Sitter spacetimes II: Non-static boundaries

titleUnique Continuation for the Einstein Equations in Asymptotically Anti-de Sitter Spacetimes
yearin preparation

IOP Publishing

journaltitleClassical and Quantum Gravity
titleDiffeomorphisms and holographic anomalies

Springer Science
Business Media LLC

journaltitleInternational Journal of Theoretical Physics

American Physical Society (APS)

journaltitlePhysical Review D
titleGravitationally collapsing dust in 2 + 1 dimensions

titleNull Geodesics and Improved Unique Continuation for Waves in Asymptotically Anti-de Sitter Spacetimes

Academic Press

titleSemi-Riemannian geometry with applications to relativity

titleThe Near-Boundary Geometry of Einstein-Vacuum Asymptotically Anti-de Sitter Spacetimes

World Scientific Pub Co Pte Lt

journaltitleInternational Journal of Modern Physics A
titleAsymptotically Anti-de Sitter spacetimes and their stress energy tensor

Elsevier BV

journaltitlePhysics Letters B
titleQuantum effective action from the AdS/CFT correspondence

The University of Chicago Press Press

titleGeneral relativity


# Machine learning for structure-guided materials and process design

Lukas Morand  Fraunhofer Institute for Mechanics of Materials IWM, Freiburg,
Germany  Tarek Iraki  Institute for Advanced Simulations - Materials Data
Science and Informatics (IAS-9), Forschungszentrum Jülich GmbH, 52425, Jülich,
Germany  Intelligent Systems Research Group ISRG, University of Applied
Sciences, Karlsruhe, Germany  Johannes Dornheim  Intelligent Systems Research
Group ISRG, University of Applied Sciences, Karlsruhe, Germany  Institute for
Applied Materials - Computational Materials Sciences IAM-CMS, Karlsruhe
Institute of Technology, Karlsruhe, Germany  Stefan Sandfeld  Institute for
Advanced Simulations - Materials Data Science and Informatics (IAS-9),
Forschungszentrum Jülich GmbH, 52425, Jülich, Germany  Norbert Link
Intelligent Systems Research Group ISRG, University of Applied Sciences,
Karlsruhe, Germany  Dirk Helm  Fraunhofer Institute for Mechanics of Materials
IWM, Freiburg, Germany

###### Abstract

In recent years, there has been a growing interest in accelerated materials
innovation in both, research and industry. However, to truly add value to the
development of new advanced materials, it is inevitable to take into account
manufacturing processes and thereby tailor materials design approaches to
support downstream process design approaches. As a major step into this
direction, we present a holistic optimization approach that covers the entire
materials process-structure-property chain. Our approach specifically employs
machine learning techniques to address two critical identification problems.
The first is to solve a materials design problem, which involves identifying
near-optimal material structures that exhibit desired macroscopic properties.
The second is to solve a process design problem that is to find an optimal
processing path to manufacture these material structures. Both identification
problems are typically ill-posed, which presents a significant challenge for
solution approaches. However, the non-unique nature of these problems also
offers an important advantage for processing: By having several target
structures that perform similarly well, the corresponding processes can be
efficiently guided towards manufacturing the best reachable structure. In
particular, we apply deep reinforcement learning for process design in
combination with a multi-task learning-based optimization approach for
materials design. The functionality of the approach will be demonstrated by
using it to manufacture crystallographic textures with desired properties in a
metal forming process.

##  1 Introduction

Accelerated materials innovation has become a core research field in
integrated computational materials engineerng (ICME) and is pushed forward
strongly in materials science (cf. materials genome initiative [1], European
advanced materials initiative [2]). Essentially, the properties of such new
sustainable, resilient, and high-performance materials depend on the structure
of the material and the material structure, in turn, is depending on the
manufacturing process. Therefore, designing new materials without taking into
account the manufacturing process does not add significant value [3]. For an
application in industry, groundbreaking technologies for modeling and
optimization that cover the entire process-structure-properties chain [4] are
required.

One fundamental characteristic of the process-structure-properties chain that
can be leveraged is its modularity: Each individual link represents a specific
identification problem, which includes identifying microstructures for given
desired properties (materials design) and finding optimal processing paths for
targeted microstructures (process design). These identification problems are
typically not well-posed in the sense of Hadamard [5], which presents a
significant challenge for solution approaches [6]. However, the non-unique
nature of these problems offers an important advantage for processing: It
enables a more flexible production as processes can be efficiently guided to
manufacture the best reachable microstructure from a set of equivalent
microstructures with respect to their properties.

The identification problems mentioned above remain highly complex and are
often high-dimensional. To handle this complexity, the usage of machine
learning has shown to be suitable for materials and process design
applications[7, 8]. In this work, a novel machine learning framework is
introduced that combines a reinforcement learning approach for multi-
equivalent goal structure guided processing path optimization (MEG-SGGPO) [9]
with a Siamese multi-task learning-based optimization (SMTLO) approach [10].
The framework is tailored to the specifics of process-structure-property
optimization problems and therefore constitutes a significant advancement
towards accelerated process and materials design.

Although each of the approaches have been shown to work well for their
individual design problems, the combined usage of both is not investigated
yet. This is particularly the aim of the present work, while the approach is
enhanced by using a recently developed novel distance measure for one-point
statistics microstructure representations: the Sinkhorn distance
$\mathscr{D}_{\mathrm{sh}}$ [11]. Specifically, the Sinkhorn distance is
supposed to be suited well as it takes into account neighborhood information
encoded in histogram-based microstructure representations. The importance of
the distance measure can be seen in Figure 1, which depicts the general
concept of the approach. In this work, we demonstrate the approach at
manufacturing metallic materials with desired elastic and anisotropy
properties, which are affected by the crystallographic texture that evolves
during forming.

Figure 1: General concept for structure-guided materials and process design.
The first (materials design) step is addressed by the SMTLO approach, while
the second (process design) step is addressed by the MEG-SGGPO approach.

Pioneering work has already been done in this field, however, typically either
focusing on solving materials design problems (without taking into account
processing) or focusing on solving process design problems for given desired
properties (not taking into account the structure of materials). In the
following, we discuss works that describe approaches for solving materials and
process design problems with an emphasize on the application of machine
learning.

A widely-used approach for designing materials is the microstructure sensitive
design (MSD) approach [12]. This approach primarily focuses on identifying
microstructures that exhibit desired properties, with six out of the seven MSD
steps being dedicated to this task. The processing of structures is only
briefly addressed by the final MSD step. To the authors knowledge, there are
only few works that show how to set up process-structure-property linkages for
crystallographic texture optimization within the framework of MSD. One
approach involves modeling texture evolution as fluid flow in the orientation
space. On this basis, so-called processing streamlines are used to guide from
one point to another [12, 13].

For example, in the case of optimizing the crystallographic texture of an
orthotropic plate, these streamlines were calculated for the processing
operations of tension and compression, and used to guide from a random texture
to required textures [14]. These required textures have been identified in
beforehand based on the MSD approach [15]. Alternatively, so-called texture
evolution networks have been developed within the context of MSD, using a
priory sampled processing paths to create a directed tree graph where
microstructures are represented by the nodes on the graph [16]. Graph search
algorithms are then used to find optimal processing paths from an initial to a
targeted microstructures.

Besides the MSD approach, other approaches exist that solve crystallographic
texture optimization problems in terms of materials design, however, without
solving a corresponding optimal processing problem. For instance, Kuroda and
Ikawa [17] used a genetic algorithm to identify optimal combinations of
typical fcc rolling texture components for given desired properties. Also, Liu
et al. [18] used optimization algorithms, but incorporated machine learning
techniques to efficiently identify significant features and regions of the
orientation space. Surrogate-based optimization has also been explored in
several studies, such as for handling uncertainties in materials design [19].
Alternatively, probabilistic modeling approaches can be used to directly solve
the inverse identification problem [20].

Regarding process design, machine learning-based and data mining-based
approaches have been proposed by several works, however, without solving the
corresponding materials design problem in beforehand. One approach involves
the use of principle component analysis to represent one-step and two- to
three-step deformation processes and to identify the deformation sequences
required for reaching target crystallographic textures [21, 22].
Alternatively, a database approach can be used that stores microstructure
representations and corresponding processing paths [23]. The database can be
searched for desired textures yielding optimal process paths. These process
paths can then be fine-tuned using gradient-based optimization. Another option
is to store microstructure representations in a lower dimensional feature
space generated by a variational autoencoder [24]. In this lower dimensional
feature space, optimal processing paths can be identified using a suitable
distance measure.

##  2 Results

In the following, we demonstrate how the machine learning approach can be used
to identify process paths for manufacturing an optimal crystallographic
texture for given desired properties in a metal forming process. The metal
forming process used for this purpose consists of seven subsequent loading
steps, each of 10% uniaxial strain. In each step, the uniaxial strain
operation can be applied in one out of 25 different loading directions,
leading to a constant change in crystallographic texture. Consequently, the
there are $25^{7}$ different processing paths.

On this basis, in Section 2.1, it is shown how near-optimal crystallographic
textures are identified for given elastic and anisotropy properties using the
SMTLO approach. The properties are in particular the Young’s moduli $E_{i}$
and anisotropy measures $\widetilde{R}_{i}$, both in three orthogonal
directions. Using the set of near-optimal crystallographic textures, in
Section 2.2, it is shown how the MEG-SGGPO approach is used to guide the
forming process to identify and manufacture the best reachable texture.
Further details on the implementation of the machine learning models, the
numerical simulation as well as the generation and representation of texture
data can be found in Section 4.

###  2.1 Designing materials using SMTLO

The basis for applying the SMTLO approach is a data set of 76980 samples,
composed of textures and corresponding properties. 2D projections of the
training data distribution, generated by the metal forming process simulation,
are shown in Figure 2 including the region delineating the desired properties
(target region). In this study, the target region is centered at
$E_{1},E_{2},E_{3}=214,214,221$ GPa and
$\widetilde{R}_{1},\widetilde{R}_{2},\widetilde{R}_{3}=0.65,0.685,0.885$. Its
width equals $2$ GPa for $E_{i}$ and $0.2$ for $\widetilde{R}_{i}$. Both
$E_{i}$ and $\widetilde{R}_{i}$ are determined in three orthogonal directions
at the material point at the end of the process. $E_{i}$ is given by the slope
in the elastic regime, while $\widetilde{R}_{i}$ is calculated by the ratio
between transverse and tensile strain in the plastic regime when applying
uniaxial tension. In Figure 2, we highlighted training data points that are
already located inside the target region. We use this set of $80$ textures as
benchmark set.

Figure 2: Projections of the properties space showing the data distribution
and the target region. The distribution of the underlying point cloud is
displayed in blue, while the gray dots mark data points that are already
located inside the target region. The black isolines indicate regions with the
same point cloud density.

Using the SMTLO approach, we were able to identify a set of 175 near-optimal
textures. As we aim to identify a preferably diverse set of textures, we
depict their pairwise distances in Figure 3 and compare them to the benchmark
set. As one can easily see, the SMTLO approach is able to find a set of
textures that differ more to each other than the benchmark set.

Figure 3: Histogram of pairwise distances between textures belonging to the
set of identified textures (red) and between textures belonging to the
benchmark set (green), as evaluated in the latent space of the SMTLO approach

For the subsequent process design step, we chose ten goal textures from the
set of identified textures that differ strongly from each other [25]. The
distribution of the chosen goal textures in properties space is shown in
Figure 4. The properties of the goal textures, however, are not located
completely inside the target region. To show the differences between the goal
textures, in Figure 5, four textures are depicted exemplary as pole figure
plots. While the pole figure intensities are similar for all textures, the
shape of the represented orientation distribution differs strongly.

Figure 4: Projections of the properties space showing the target region and
the calculated properties of the ten chosen goal textures $g_{0}$ to $g_{9}$
Figure 5: Four exemplary textures of the ten chosen goal textures depicted as
pole figure plots: (001), (110), (111)

###  2.2 Structure-guided process design using MEG-SGGPO

Once a diverse set of crystallographic textures has been identified, MEG-SGGPO
is used to guide the metal forming process to the best reachable texture. For
solving the identification problem, the MEG-SGGPO approach is allowed to
conduct $400$ process runs, so-called episodes. The evolution of the distance
between the produced and the chosen goal texture is depicted over the episodes
in Fig. 6. Initially, the reinforcement learning agent attempts to identify
processing paths that target randomly selected goal textures. As the learning
process progresses, the agent focuses increasingly on producing goal texture
$g_{2}$.

Figure 6: Sinkhorn texture distance $\mathscr{D}_{\mathrm{sh}}$ between the
produced and the goal texture over the episodes of the MEG-SGGPO approach. The
color indicates the targeted texture.

For comparison, Figure 7 presents pole figure plots of the goal texture
$g_{2}$ and the produced texture $g_{2}^{*}$ after 400 episodes. Visually, the
produced texture is highly similar to the goal texture with intensity peaks at
the same positions and of a similar magnitude. This also holds for areas not
covered by orientations at all. To quantify the effectiveness of the approach,
we compute the properties resulting from the produced crystallographic texture
and compare them with the desired ones. The results, listed in Table 1, show
that five of the properties ($E_{1}$, $E_{2}$, $E_{3}$, $R_{2}$ and $R_{3}$)
are very close to the target region, while $R_{1}$ lies inside.

Figure 7: Pole figure plots (001), (110), (111) of the targeted goal texture $g_{2}$ (top) and the produced texture $g_{2}^{*}$ (bottom) Table 1: Properties of the produced crystallographic texture and the distance to the target region Target property | Value | Distance to target region | Unit
---|---|---|---
$E_{1}$ | 217.5 | 2.5 | GPa
$E_{2}$ | 216.6 | 1.6 | GPa
$E_{3}$ | 222.2 | 0.3 | GPa
$\widetilde{R}_{1}$ | 0.744 | 0 | -
$\widetilde{R}_{2}$ | 0.831 | 0.046 | -
$\widetilde{R}_{3}$ | 1.058 | 0.073 | -

In summary, by sequentially applying the SMTLO and the MEG-SGGPO approach, we
were able to successfully produce a crystallographic texture with desired
properties in a simulated metal forming process. The resulting properties fall
within the acceptable range from an engineering point of view, albeit at the
border of the defined properties region.

##  3 Discussion

The results presented in Section 2.1 demonstrate the effectiveness of the
SMTLO approach in identifying sets of near-optimal crystallographic textures
that exhibit desired properties in a given target region. The set of textures
identified by the approach is shown to be more diverse than the benchmark set
obtained from the training data set. However, the properties of the identified
textures are not completely inside the target region, as can be seen in Fig.
4.

There are two reasons for this: First, the formulation of the optimization
objective in the SMTLO approach allows for a trade-off between (i) finding
crystallographic textures with properties inside the target region, (ii)
identifying possibly diverse textures and, (iii) guaranteeing that texture can
be produced by the process under consideration. Second, due to prediction
errors of the underlying machine learning model, the SMTLO approach identifies
textures whose predicted properties lie inside the target region, while the
true properties (calculated by the numerical simulation) slightly lie outside.
Both of these issues can be addressed by modifying the objective function. For
instance, targeting the center of the properties region, instead of its bounds
can mitigate both issues. Additionally, the second issue can be addressed by
enhancing the machine learning model, for example, by increasing the amount of
training data.

Taking a diverse subset of the identified textures as goal textures, the MEG-
SGGPO approach guides the metal forming process closely to one of the chosen
goal textures. Although the reinforcement learning agent was not able to
reproduce one of the goal textures exactly (which is challenging due to the
many possible processing paths), the measured distance between the targeted
texture $g_{2}$ and the produced one $g_{2}^{*}$ is sufficiently low
($\mathscr{D}_{\mathrm{sh}}=0.031$) when relating it to the distance to the
nearest neighbor in the training data set ($\mathscr{D}_{\mathrm{sh}}=0.033$)
and to the furthest data point ($\mathscr{D}_{\mathrm{sh}}=0.141$). We want to
remark here, that it took the reinforcement learning agent only 400 episodes
to achieve this result compared to the baseline set that is grounded on the
generation of 76980 random samples. The reinforcement learning algorithm can
therefore be seen as being data efficient.

Nevertheless, in this study, it seems that a lower bound is existing that is
difficult to overcome by the MEG-SSGPO approach. In general, this can have two
reasons: First, the forming process is unable to produce textures that are
closer to the goal texture identified by the SMTLO approach. Yet, the SMTLO
approach already addresses this issue by enforcing microstructures to remain
within the region delineated by the known microstructures from the training
data set. We expected that a stronger enforcement leads to textures that are
better reachable by the process. This, however, can lead to a less accurate
properties prediction. Second, the MEG-SGGPO approach identified a local
optimum and got stuck. This can be mitigated by longer MEG-SGGPO runs with
optimized hyperparameters. Longer runs, however, have not shown to yield
significant improvements in the presented study. As future work, it is
desirable to incorporate the knowledge contained in the generated training
samples for SMTLO into MEG-SGGPO a priori to enhance its performance.

On the whole, the machine learning-based approach presented in this study
enables accelerated materials development while taking into account the
manufacturing process. Particularly, we want to highlight that the considered
metal forming process offers a total of $25^{7}$ different processing paths,
and despite this complexity,

  * •

$76980$ samples were sufficient for the SMTLO approach to successfully solve
the materials design problem. It allowed us to identify a diverse set of near-
optimal textures with desired properties.

  * •

a total of $400$ episodes was sufficient for the MEG-SGGPO approach to
successfully guide the forming process successfully and solve the process
design problem.

This shows that the applied approach is highly data efficient and capable of
effectively optimizing process-structure-property relations end-to-end in
manufacturing processes. Moreover, we want to emphasize that the approach
leverages the non-uniqueness of the materials and process design problems,
meaning that a diverse set of microstructures is identified for given desired
properties and subsequently, the best reachable microstructure is produced.
This is a significant advantage when transferring the approach to real
manufacturing systems, where often constraints exist, which may exclude
microstructures from being producible. Nonetheless, the application still
needs to be validated in real manufacturing systems. In our specific case, the
amount of used training data is comparably high to what is typically available
in production. Thus, for application in manufacturing, a first step would be
to pre-train the machine learning models with a numerical simulation that
serves as a digital twin of the process and the material.

##  4 Methods

In this section, we present details about the approaches for materials and
process design, SMTLO [10] and MEG-SGGPO [9], as well as the underlying metal
forming simulation and the representation of crystallographic texture.

###  4.1 Metal forming process simulation

The metal forming process simulation [9] used in this study applies a
deformation $\hat{\boldsymbol{F}}$

$\hat{\boldsymbol{F}}=\boldsymbol{R}\widetilde{\boldsymbol{F}}\boldsymbol{R}^{\top},$
(1)

with $\boldsymbol{R}$ being a rotation matrix that describes one out of 25
possible loading directions. The deformation $\widetilde{\boldsymbol{F}}$ is
defined using orthogonal basis vectors $\boldsymbol{e}_{i}$ and the operator
$\otimes$ for the outer product

$\widetilde{\boldsymbol{F}}=\widetilde{F}_{11}\boldsymbol{e}_{1}\otimes\boldsymbol{e}_{1}+\widetilde{F}_{22}\boldsymbol{e}_{2}\otimes\boldsymbol{e}_{2}+\widetilde{F}_{33}\boldsymbol{e}_{3}\otimes\boldsymbol{e}_{3}$
(2)

with $\widetilde{F}_{11}$ corresponding to $10\%$ strain increments.
$\widetilde{F}_{22},\widetilde{F}_{33}$ are adjusted such that the stresses
are in balance. The metal forming process consists of seven subsequent loading
steps, each in a separate loading direction. For this study, we conducted
76980 random process paths to generate the training data for the SMTLO
approach.

The underlying material model is a crystal plasticity model of Taylor-type
[26]. The volume averaged stress for $n_{\mathrm{oris}}$ crystals with
different orientations is calculated by

$\overline{\boldsymbol{T}}=\frac{1}{V}\sum_{i}^{n_{\mathrm{oris}}}\boldsymbol{T}^{(i)}V^{(i)},$
(3)

with the total volume $V$, the individual volume of each crystal $V^{(i)}$,
and the Cauchy stress tensor $\boldsymbol{T}^{(i)}$.

With the multiplicative decomposition of the deformation gradient in its
elastic and plastic part

$\boldsymbol{F}=\boldsymbol{F}_{\mathrm{e}}\cdot\boldsymbol{F}_{\mathrm{p}},$
(4)

and the conversion formula for the stress tensor in the intermediate
configuration $\boldsymbol{T}^{*}$

$\boldsymbol{T}^{*}=\boldsymbol{F}_{\mathrm{e}}^{-1}\cdot(\mathrm{det}(\boldsymbol{F}_{\mathrm{e}})\boldsymbol{T})\cdot\boldsymbol{F}_{\mathrm{e}}^{-\top},$
(5)

the Cauchy stress tensor is derived using

$\boldsymbol{T}^{*}=\frac{1}{2}\mathbb{C}:(\boldsymbol{F}_{\mathrm{e}}^{\top}\cdot\boldsymbol{F}_{\mathrm{e}}-\boldsymbol{I}),$
(6)

where $\boldsymbol{I}$ denotes the second order identity tensor and
$\mathbb{C}$ the fourth order elastic stiffness tensor.

The evolution of the plastic deformation is described using the plastic part
of the velocity gradient

$\boldsymbol{L}_{\mathrm{p}}=\dot{\boldsymbol{F}_{\mathrm{p}}}\cdot\boldsymbol{F}_{\mathrm{p}}^{-1}=\sum_{\alpha}\dot{\gamma}^{(\alpha)}\boldsymbol{m}^{(\alpha)}\otimes\boldsymbol{n}^{(\alpha)},$
(7)

with the slip rates $\dot{\gamma}^{(\alpha)}$ on slip system $\alpha$ that is
defined by the slip plane normal $\boldsymbol{n}^{(\alpha)}$ and the slip
direction $\boldsymbol{m}^{(\alpha)}$. The slip rates are calculated by a
phenomenological power law. The crystal reorientation is calculated by
applying a rigid body rotation derived from the polar decomposition of
$\boldsymbol{F}_{\mathrm{e}}$ to the original orientation [27, 11].

###  4.2 Texture representation and distance measure

Texture is represented by using the histogram-based description introduced by
Dornheim et al. [9]. The orientation histogram (soft-assignment factor of
three) consists of 512 orientation bins that are nearly uniformly distributed
in the cubic-triclinic fundamental zone, which were created using the software
neper [28, 29]. In contrast to the original MEG-SGGPO approach,
crystallographic texture distances are measured using the Sinkhorn distance
applied to the histogramm representation [11]. Specifically, the Sinkhorn
distance is an efficient implementation of the Earth Movers distance that
measures the least amount of work necessary to transform one histogram into
the other [30, 31]. Therefore, local orientation distances [32] encoded in the
histograms are taken into account, in contrast to the originally proposed Chi-
Squared distance, which is basically a bin-wise comparison of two histograms.

###  4.3 The SMTLO approach

The SMTLO approach described in [10] aims to identify diverse microstructures
for given desired properties (defined by a target region). It consists of a
multi-task learning model, which is grounded on an encoder
$f_{\mathrm{x\text{-}enc}}$ that transforms a microstructure representation
$\boldsymbol{x}$ into a lower dimensional latent feature space representation
$\boldsymbol{z}$

$\boldsymbol{z}=f_{\mathrm{x\text{-}enc}}(\boldsymbol{x},\boldsymbol{\theta}_{\mathrm{x\text{-}enc}}),$
(8)

with the trainable parameters $\boldsymbol{\theta}_{\mathrm{x\text{-}enc}}$.
The encoder is equipped with three heads, each of which is designed to solve a
specific learning task. These are

  1. 1.

a decoder head that reconstructs the microstructure representations
$\boldsymbol{x}$ from $\boldsymbol{z}$

$\boldsymbol{x}^{\prime}=f_{\mathrm{x\text{-}dec}}(\boldsymbol{z},\boldsymbol{\theta}_{\mathrm{x\text{-}dec}}),$
(9)

with the trainable parameters $\boldsymbol{\theta}_{\mathrm{x\text{-}dec}}$.
The encoder-decoder part is trained using a loss term that minimizes the
Sinkhorn distance between the original microstructure $\boldsymbol{x}$ and its
reconstruction $\boldsymbol{x}^{\prime}$ [11]

$\mathscr{L}_{1}=\mathscr{D}_{\mathrm{sh}}(\boldsymbol{x},\boldsymbol{x}^{\prime}).$
(10)

  2. 2.

a prediction head $f_{\mathrm{p\text{-}pred}}$ to infer material properties
$\hat{\boldsymbol{p}}$

$\hat{\boldsymbol{p}}=f_{\mathrm{p\text{-}pred}}(\boldsymbol{z},\boldsymbol{\theta}_{\mathrm{p\text{-}pred}}),$
(11)

with the trainable parameters $\boldsymbol{\theta}_{\mathrm{p\text{-}pred}}$.
The sample-wise loss term is defined by the mean squared error between
predicted and true properties

$\mathscr{L}_{2}=\frac{1}{n_{\mathrm{p}}}\sum_{i}^{n_{\mathrm{p}}}(p_{i}-\hat{p}_{i})^{2},$
(12)

with the number of properties $n_{\mathrm{p}}$.

  3. 3.

an auto-encoder head that serves as an anomaly detector to estimate whether a
microstructure in its latent representation belongs to the set of known
microstructures (defined by the training data) or not:

$\boldsymbol{z}^{\prime}=f_{\mathrm{z\text{-}dec}}(f_{\mathrm{z\text{-}enc}}(\boldsymbol{z},\boldsymbol{\theta}_{\mathrm{z\text{-}enc}}),\boldsymbol{\theta}_{\mathrm{z\text{-}dec}}),$
(13)

with the encoder and decoder parts $f_{\mathrm{z\text{-}enc}}$ and
$f_{\mathrm{z\text{-}dec}}$ and the trainable parameters
$\boldsymbol{\theta}_{\mathrm{z\text{-}enc}}$ and
$\boldsymbol{\theta}_{\mathrm{z\text{-}dec}}$, respectively. The auto-encoder
is trained using a mean squared error loss function

$\mathscr{L}_{3}=\frac{1}{n_{\mathrm{z}}}\sum_{i}^{n_{\mathrm{z}}}(z_{i}-z^{\prime}_{i})^{2},$
(14)

with $n_{\mathrm{z}}$ dimensions of the latent feature space.

A neural network is used to solve the above mentioned learning tasks, which is
trained using a combined loss function

$\mathscr{L}_{\mathrm{MTL}}=W_{1}\mathscr{L}_{1}+W_{2}\mathscr{L}_{2}+W_{3}\mathscr{L}_{3},$
(15)

with individually weighted loss terms using the parameters $W_{1}=0.2$,
$W_{2}=0.06$, and $W_{3}=0.04$.

To ensure that the original microstructure distance is preserved in the latent
feature space, a Siamese training method is applied [33]. This involves
training two twin models simultaneously, with shared weights, and adding a
further loss term that applies to the latent feature space. This preservation
loss writes

$\mathscr{L}_{4}=(\mathscr{D}_{\mathrm{sh}}(\boldsymbol{x}^{(1)},\boldsymbol{x}^{(2)})-\mathrm{dist}(\boldsymbol{z}^{(1)},\boldsymbol{z}^{(2)}))^{2},$
(16)

with the absolute distance in the latent feature space

$\mathrm{dist}(z_{i}^{(1)},z_{i}^{(2)})=\frac{1}{n_{\mathrm{z}}}\sum_{i}^{n_{\mathrm{z}}}|z_{i}^{(1)}-z_{i}^{(2)}|.$
(17)

The superscripts (1) and (2) are used to distinguish between the two
simultaneously trained twins parts. The overall loss function writes

$\mathscr{L}=\mathscr{L}_{\mathrm{MTL}}+W_{4}\mathscr{L}_{4}+\lambda\Omega(\boldsymbol{\theta}),$
(18)

with the regularization term $\Omega(\boldsymbol{\theta})$, $\lambda=10^{-6}$,
and $W_{4}=0.7$.

In order to identify microstructures with desired properties, we use a genetic
optimizer [34] to search for candidate microstructures in the latent feature
space generated by the Siamese multi-task learning model. The optimizer
minimizes three cost terms: $\mathscr{C}_{1},\mathscr{C}_{2},\mathscr{C}_{3}$
[10]. $\mathscr{C}_{1}$ minimizes the distance of the predicted property
$\hat{\boldsymbol{p}}$ for a candidate microstructure $\boldsymbol{z}^{*}$ to
the target region. $\mathscr{C}_{2}$ penalizes candidate microstructures that
are not inside the region of known microstructures. $\mathscr{C}_{3}$ enforces
candidate microstructures being as far away as possible from each other in the
actual population. The overall objective function writes

$\mathscr{C}=V_{1}\mathscr{C}_{1}+V_{2}\mathscr{C}_{2}+V_{3}(1+\mathscr{C}_{3}),$
(19)

with individual weights $V_{1}=0.75$, $V_{2}=0.05$, and $V_{3}=0.2$.

###  4.4 The MEG-SGGPO approach

In reinforcement learning, an agent learns to take optimal decisions by
interacting with its environment in a sequence of discrete time steps $t$. The
MEG-SGGPO approach [9] is tailored for efficient learning in the case of
multiple equivalent target microstructures. Given a set $\mathcal{G}$ of near-
optimal microstructures $s^{(\text{g})}\in\mathcal{G}$, an initial
microstructure $s_{0}$, and a microstructure generating process
$p(s_{t},a_{t})=s_{t+1}$, the goal of the approach is to find a sequence of
processing parameters $a_{t}\in A$ that guide the process towards
manufacturing the best reachable microstructure $s^{*}\in\mathcal{G}$.

MEG-SGGPO is based on deep Q-networks[35] and its extensions [36, 37, 38].
Deep Q-networks are derived from Q-learning, where the so-called Q-function is
approximated from data gathered by the reinforcement learning agent while
following a policy $a=\pi(s)$. The Q-function models the expected sum of
future reward signals for state action pairs $(s_{t},a_{t})$. In MEG-SGGPO the
Q-function is generalized to also take the goal microstructure into account
and is, in its recursive form, defined as

$\mathcal{Q}(s_{t},a_{t},s^{(\text{g})})=\mathbb{E}_{P,\pi}\Big{[}R(s_{t},s_{t+1},s^{(\text{g})})+\max_{a_{t+1}\in
A}\mathcal{Q}(s_{t+1},a_{t+1},s^{(\text{g})})\Big{]},$ (20)

where $R(s_{t},s_{t+1},s^{(\text{g})})$ is a per goal microstructure pseudo
reward function

$R(s_{t},s_{t+1},s^{(\text{g})})=\dfrac{1}{\mathscr{D}(s_{t+1},s^{(\text{g})})}-\dfrac{1}{\mathscr{D}(s_{t},s^{(\text{g})})},$
(21)

which is based on a microstructure distance function $\mathscr{D}$.

The process design task is a two-fold optimization problem to(i) identify the
best reachable goal microstructure $s^{*}$ and (ii) learn the optimal policy
$\pi^{*}$. MEG-SGGPO addresses this by using the generalized functions defined
above in a nested reinforcement learning procedure. At the beginning of each
episode (i.e. an execution of the process during learning), the agent uses the
generalized Q-function to identify the targeted microstructure $s^{\prime}$
from $\mathcal{G}$ and determines the processing parameters during the
episode. In a first step, an estimation of the best reachable goal
microstructure $\bar{s^{*}}$ is extracted from the current $\mathcal{Q}$
estimation by

$\bar{s^{*}}=\operatorname*{arg\,max}_{s^{(\text{g})}\in\mathcal{G}}\Big{[}\mathcal{V}(s_{0},s^{(\text{g})})+\dfrac{1}{\mathscr{D}(s_{0},s^{(\text{g})})}\Big{]},$
(22)

where the generalized state-value function $\mathcal{V}$ is defined as

$\mathcal{V}(s,s^{(\text{g})})=\max_{a\in A}\mathcal{Q}(s,a,s^{(\text{g})}).$
(23)

The targeted microstructure $s^{\prime}$ is chosen in an $\epsilon$-greedy
approach [39], where the agent chooses random targets and processing
parameters in an $\epsilon$ fraction of the cases ($0\leq\epsilon\leq 1$) and
optimizes targets and processing parameters in the remaining cases. The
optimal processing path is identified simultaneously in an inner loop by using
the Q-learning approach.

## Data availability statement

Data and code has been published in the following repositories:

  * •

Training data for the SMTLO approach is made available via Fraunhofer Fordatis
repository at https://fordatis.fraunhofer.de/handle/fordatis/319 [40].

  * •

The process simulation is made available via Fraunhofer Fordatis repository at
https://fordatis.fraunhofer.de/handle/fordatis/201.2[41].

  * •

The source code for the Siamese multi-task learning framework is made
available via gitlab at https://gitlab.com/tarekiraki/the_smtl_framework.

  * •

The reinforcement learning environment is made available via github at
https://github.com/Intelligent-Systems-Research-
Group/RL4MicrostructureEvolution.

## Acknowledgments

The authors would like to thank the German Research Foundation (DFG) for
funding this work, which was carried out within the research project number
415804944: ’Taylored Material Properties via Microstructure Optimization:
Machine Learning for Modelling and Inversion of Structure-Property-
Relationships and the Application to Sheet Metals’.

## Author contribution

LM set up the numerical simulation, generated the training data for solving
the materials design problem and wrote the manuscript together with all other
authors. TI developed the SMTLO approach and applied both, the SMTLO and the
MEG-SGGPO approach to the problem analyzed in this study. JD developed the
MEG-SGGPO approach. The approach was extended using the Sinkhorn distance
measure by TI. SS supported in terms of machine learning and materials
science. NL supported in terms of machine learning. DH supported in terms of
materials science. All authors contributed equally to the discussion of the
results and to reviewing the manuscript.

## Competing interests

The authors have no competing interests to declare that are relevant to the
content of this article.

## References

  * [1] J. J. de Pablo, N. E. Jackson, M. A. Webb, L.-Q. Chen, J. E. Moore, D. Morgan, R. Jacobs, T. Pollock, D. G. Schlom, E. S. Toberer, et al., “New frontiers for the materials genome initiative,” npj Computational Materials, vol. 5, no. 1, p. 41, 2019.
  * [2] Advanced materials initiative 2030, “The materials roadmap 2030,” tech. rep., 2022.
  * [3] P. Grant, “Foresight – new and advanced materials,” tech. rep., 2013.
  * [4] G. B. Olson, “Computational design of hierarchically structured materials,” Science, vol. 277, no. 5330, pp. 1237–1242, 1997.
  * [5] J. Hadamard, “Sur les problèmes aux dérivées partielles et leur signification physique,” Princeton university bulletin, pp. 49–52, 1902\.
  * [6] A. Agrawal and A. Choudhary, “Perspective: Materials informatics and big data: Realization of the “fourth paradigm” of science in materials science,” Apl Materials, vol. 4, no. 5, p. 053208, 2016.
  * [7] Y. Liu, T. Zhao, W. Ju, and S. Shi, “Materials discovery and design using machine learning,” Journal of Materiomics, vol. 3, no. 3, pp. 159–177, 2017.
  * [8] M. Mozaffar, S. Liao, X. Xie, S. Saha, C. Park, J. Cao, W. K. Liu, and Z. Gan, “Mechanistic artificial intelligence (mechanistic-ai) for modeling, design, and control of advanced manufacturing processes: Current state and perspectives,” Journal of Materials Processing Technology, vol. 302, p. 117485, 2022.
  * [9] J. Dornheim, L. Morand, S. Zeitvogel, T. Iraki, N. Link, and D. Helm, “Deep reinforcement learning methods for structure-guided processing path optimization,” Journal of Intelligent Manufacturing, vol. 33, pp. 333–352, 2021.
  * [10] T. Iraki, L. Morand, J. Dornheim, N. Link, and D. Helm, “A multi-task learning-based optimization approach for finding diverse sets of material microstructures with desired properties and its application to texture optimization,” Journal of Intelligent Manufacturing, 2023.
  * [11] T. Iraki, L. Morand, N. Link, S. Sandfeld, and H. Dirk, “Accurate distances measures and machine learning of the texture-property relation for crystallographic textures represented by one-point statistics,” arXiv:2312.04214, 2023.
  * [12] D. T. Fullwood, S. R. Niezgoda, B. L. Adams, and S. R. Kalidindi, “Microstructure sensitive design for performance optimization,” Progress in Materials Science, vol. 55, no. 6, pp. 477–562, 2010.
  * [13] D. Li, H. Garmestani, and S. Ahzi, “Processing path optimization to achieve desired texture in polycrystalline materials,” Acta materialia, vol. 55, no. 2, pp. 647–654, 2007.
  * [14] B. L. Adams, S. Kalidindi, D. T. Fullwood, and D. Fullwood, Microstructure sensitive design for performance optimization.  Butterworth-Heinemann, 2012.
  * [15] S. R. Kalidindi, J. R. Houskamp, M. Lyons, and B. L. Adams, “Microstructure sensitive design of an orthotropic plate subjected to tensile load,” International Journal of Plasticity, vol. 20, no. 8-9, pp. 1561–1575, 2004.
  * [16] J. B. Shaffer, M. Knezevic, and S. R. Kalidindi, “Building texture evolution networks for deformation processing of polycrystalline fcc metals using spectral approaches: Applications to design for targeted performance,” International Journal of Plasticity, vol. 26, no. 8, pp. 1183–1194, 2010.
  * [17] M. Kuroda and S. Ikawa, “Texture optimization of rolled aluminum alloy sheets using a genetic algorithm,” Materials Science and Engineering: A, vol. 385, no. 1-2, pp. 235–244, 2004.
  * [18] R. Liu, A. Kumar, Z. Chen, A. Agrawal, V. Sundararaghavan, and A. Choudhary, “A predictive machine learning approach for microstructure optimization and materials design,” Scientific Reports, vol. 5, no. 1, pp. 1–12, 2015.
  * [19] P. V. Balachandran, D. Xue, J. Theiler, J. Hogden, and T. Lookman, “Adaptive strategies for materials design using uncertainties,” Scientific reports, vol. 6, no. 1, pp. 1–9, 2016.
  * [20] A. Tran and T. Wildey, “Solving stochastic inverse problems for property–structure linkages using data-consistent inversion and machine learning,” JOM, vol. 73, no. 1, pp. 72–89, 2021.
  * [21] P. Acar and V. Sundararaghavan, “Linear solution scheme for microstructure design with process constraints,” AIAA Journal, vol. 54, no. 12, pp. 4022–4031, 2016.
  * [22] P. Acar and V. Sundararaghavan, “Reduced-order modeling approach for materials design with a sequence of processes,” AIAA Journal, vol. 56, no. 12, pp. 5041–5044, 2018.
  * [23] V. Sundararaghavan and N. Zabaras, “On the synergy between texture classification and deformation process sequence selection for the control of texture-dependent properties,” Acta materialia, vol. 53, no. 4, pp. 1015–1027, 2005.
  * [24] S. Sundar and V. Sundararaghavan, “Database development and exploration of process–microstructure relationships using variational autoencoders,” Materials Today Communications, vol. 25, p. 101201, 2020.
  * [25] C. Bauckhage, “Ml2r coding nuggets: Hopfield nets for max-sum diversification,” 2021.
  * [26] S. R. Kalidindi, C. A. Bronkhorst, and L. Anand, “Crystallographic texture evolution in bulk deformation processing of fcc metals,” Journal of the Mechanics and Physics of Solids, vol. 40, no. 3, pp. 537–569, 1992.
  * [27] X. Ling, M. Horstemeyer, and G. Potirniche, “On the numerical implementation of 3d rate-dependent single crystal plasticity formulations,” International Journal for Numerical Methods in Engineering, vol. 63, no. 4, pp. 548–568, 2005.
  * [28] R. Quey, P. Dawson, and F. Barbe, “Large-scale 3d random polycrystals for the finite element method: Generation, meshing and remeshing,” Computer Methods in Applied Mechanics and Engineering, vol. 200, no. 17-20, pp. 1729–1745, 2011.
  * [29] R. Quey, A. Villani, and C. Maurice, “Nearly uniform sampling of crystal orientations,” Journal of Applied Crystallography, vol. 51, no. 4, pp. 1162–1173, 2018.
  * [30] Y. Rubner, C. Tomasi, and L. J. Guibas, “The earth mover’s distance as a metric for image retrieval,” International Journal of Computer Vision, vol. 40, pp. 99–121, 2000.
  * [31] M. Cuturi, “Sinkhorn distances: Lightspeed computation of optimal transport,” in Advances in Neural Information Processing Systems (C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger, eds.), vol. 26, Curran Associates, Inc., 2013.
  * [32] D. Q. Huynh, “Metrics for 3D rotations: Comparison and analysis,” Journal of Mathematical Imaging and Vision, vol. 35, no. 2, pp. 155–164, 2009\.
  * [33] J. Bromley, I. Guyon, Y. LeCun, E. Säckinger, and R. Shah, “Signature verification using a” siamese” time delay neural network,” Advances in neural information processing systems, vol. 6, 1993.
  * [34] J. Zhang and A. C. Sanderson, “Jade: Adaptive differential evolution with optional external archive,” IEEE Transactions on Evolutionary Computation, vol. 13, no. 5, pp. 945–958, 2009.
  * [35] V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, et al., “Human-level control through deep reinforcement learning,” nature, vol. 518, no. 7540, pp. 529–533, 2015.
  * [36] H. Van Hasselt, A. Guez, and D. Silver, “Deep reinforcement learning with double q-learning,” in Thirtieth AAAI conference on artificial intelligence, 2016.
  * [37] T. Schaul, J. Quan, I. Antonoglou, and D. Silver, “Prioritized experience replay,” arXiv:1511.05952, 2015.
  * [38] Z. Wang, T. Schaul, M. Hessel, H. Van Hasselt, M. Lanctot, and N. De Freitas, “Dueling network architectures for deep reinforcement learning,” arXiv:1511.06581, 2015.
  * [39] R. S. Sutton, A. G. Barto, and R. J. Williams, “Reinforcement learning is direct adaptive optimal control,” IEEE control systems magazine, vol. 12, no. 2, pp. 19–22, 1992.
  * [40] L. Morand, T. Iraki, and D. Helm, “Crystallographic texture-property data set originating from a simulated multi-step metal forming process,” 2023.  \- Fraunhofer Fordatis repository https://fordatis.fraunhofer.de/handle/fordatis/319.
  * [41] L. Morand, J. Dornheim, J. Pagenkopf, and D. Helm, “Simulation of texture evolution for a multi-step metal forming process,” 2021.  \- Fraunhofer Fordatis repository https://fordatis.fraunhofer.de/handle/fordatis/201.2.