# Integer Division by Constants: Optimal Bounds

Daniel Lemire<EMAIL_ADDRESS>Colin Bartlett<EMAIL_ADDRESS>Owen
Kaser<EMAIL_ADDRESS>Université du Québec (TELUQ), 5800 Saint-Denis,
Montreal, Quebec, Canada  Computer Science Department, UNB Saint John, New
Brunswick, Canada

###### Abstract

The integer division of a numerator $n$ by a divisor $d$ gives a quotient $q$
and a remainder $r$. Optimizing compilers accelerate software by replacing the
division of $n$ by $d$ with the division of $c*n$ (or $c*n+c$) by $m$ for
convenient integers $c$ and $m$ chosen so that they approximate the
reciprocal: $c/m\approx 1/d$. Such techniques are especially advantageous when
$m$ is chosen to be a power of two and when $d$ is a constant so that $c$ and
$m$ can be precomputed. The literature contains many bounds on the distance
between $c/m$ and the divisor $d$. Some of these bounds are optimally tight,
while others are not. Using accessible mathematics, we present optimally tight
bounds for quotient and remainder computations.

###### keywords:

Integer Division , Compiler Optimization , Tight Bounds

††journal: Information Processing Letters

##  1 Introduction

The problem of computing the integer division given constant divisors has a
long history in computer science [1, 2, 3, 4]. Granlund and Montgomery [5]
present the first general-purpose algorithms to divide integers by constants
using a multiplication and a division by a power of two: their work was
adopted by the GNU GCC compiler. Given any non-zero 32-bit divisor known at
compile time, the optimizing compiler can replace the division by a
multiplication followed by a shift. Warren [6] improved on the Granlund and
Montgomery technique by deriving a better bound that gives a wider range of
choices. Warren’s better approach is found in LLVM’s Clang compiler. Many
optimizing compilers rely on equivalent techniques, either based on the
original Granlund-Montgomery article or on Warren’s technique.

Robison [7] describes a slightly superior alternative for some divisors in
that we multiply and add the multiplier before dividing by a power of two
(henceforth the multiply-add technique). Though it comes at the cost of an
addition, it allows one to choose a smaller multiplier, which can be
advantageous. Robison’s approach is implemented in the popular libdivide
library [8].

Most of the literature is focused on the computation of the quotient $q$ of
the division of $n$ by $d$. From the quotient $q$, we can compute the
remainder as $n-q*d$. We can also compute the remainder directly [9] without
first computing the quotient: it is given by taking remainder of $c*n$ divided
by $m$, and then multiplying it by $m$. However, for the remainder and the
quotient to be exact, it is necessary that $c/m$ approximates $1/d$ more
closely than if we merely need the quotient.

From the computation of remainders, we can derive a divisibility check, that
is check whether $d$ divides $n$, or, equivalently, check that $n$ is a
multiple of $d$. Though it may seem that computing the remainder and checking
whether it is zero is efficient, we can simplify and accelerate the algorithm
by avoiding the computation of the remainder.

The literature commonly assumes that $m$ is a power of two. We approach the
problem more generally, letting $m$ and $c$ be any integer, and restricting
the numerator to an interval $[0,N]$ where $N$ can be any integer. It makes
our exposition more general, while simplifying the notation.

Some our novel contributions are as follows:

  * 1.

We improve Robison’s bound [7], in a manner similar to how Warren improved
Granlund and Montgomery’s bound. That is, we provide an optimal bound for the
multiply-add technique.111Drane et al. [10] have a related bound, but they
also have additional constraints on the divisor $d$.

  * 2.

We derive a new tighter bounds for computing the quotient directly and
checking the divisibility, thus improving on our earlier work [9].

  * 3.

We show that we can adapt Robison’s technique to compute remainders directly
and derive a novel bound. We adapt the multiply-add technique for the purpose
of a divisibility check. To our knowledge, these results are novel.

All our bounds on how close $c/m$ must be to $1/d$ are optimal and form
necessary and sufficient conditions. Table 1 presents our core results in
concise manner.

Table 1:  Summary of main results. Throughout, all values are non-negative
integers, the divisor is non-zero $d>0$, and the numerator is bounded by
$N\geq d$, so that $n\in[0,N]$. We add the constraint that $c\in[0,m)$ so that
$c$ is as small as possible. Theorem 1 and Warren [11]
---
statement: |  $\operatorname{division}(n,d)=\operatorname{division}(c*n,m)$ for all $n\in[0,N]$
condition: |  $1/d\leq c/m<\left(1+\frac{1}{N-\operatorname{remainder}(N+1,d)}\right)1/d$
Theorem 3 (improves Robison [7])
statement: |  $\operatorname{division}(n,d)=\operatorname{division}(c*n+c,m)$ for all $n\in[0,N]$
condition: |  $\left(1-\frac{1}{N-\operatorname{remainder}(N,d)+1}\right)1/d\leq c/m<1/d$
Theorem 2 (novel, improves Lemire et al. [9])
statement: |  $\operatorname{division}(n,d)=\operatorname{division}(c*n,m)$ and $\operatorname{remainder}(n,d)=\operatorname{division}(\operatorname{remainder}(c*n,m)*d,m)$ for all $n\in[0,N]$
condition: |  $1/d\leq c/m<\left(1+\frac{1}{N}\right)1/d$
Proposition 1 (novel, generalizes Lemire et al. [9])
statement: |  $d$ divides $n\in[0,N]$ if and only if $\operatorname{remainder}(c*n,m)<c$
condition: |  $1/d\leq c/m<\left(1+\frac{1}{N}\right)1/d$
Theorem 4 (novel)
statement: |  $\operatorname{division}(n,d)=\operatorname{division}(c*n+c,m)$ and $\operatorname{remainder}(n,d)=\operatorname{division}(\operatorname{remainder}(c*n+c,m)*d,m)$ for all $n\in[0,N]$
condition: |  $\left(1-\frac{1}{N+1}\right)1/d\leq c/m<1/d$
Proposition 2 (novel)
statement: |  $d$ divides $n\in[0,N]$ if and only if $\operatorname{remainder}(c*n+c,m)<c$
condition: |  $\left(1-\frac{1}{N+1}\right)1/d\leq c/m<1/d$

##  2 Technical Preliminaries

For non-negative real numbers $z$, $\operatorname{floor}(z)$ is the greatest
integer no larger than $z$. It is a monotonic function: if $z_{1}\geq z_{2}$
then $\operatorname{floor}(z_{1})\geq\operatorname{floor}(z_{2})$.

We define $\operatorname{division}(x,y)\equiv\operatorname{floor}(x/y)$ and

$\displaystyle\operatorname{remainder}(x,y)$ $\displaystyle\equiv$
$\displaystyle x-\operatorname{division}(x,y)*y$ $\displaystyle=$
$\displaystyle x-\operatorname{floor}(x/y)*y$

for positive real numbers $x,y$ with the constraint that $y\neq 0$. We have
that $\operatorname{remainder}(x,y)\in[0,y)$. If $y$ is an integer and $x$ is
not an integer, then $\operatorname{remainder}(x,y)\neq 0$. By definition, we
always have that
$x=\operatorname{division}(x,y)*y+\operatorname{remainder}(x,y)$.

######  Lemma 1.

Consider a positive integer $d>0$, a non-negative integer $n$ and a non-
negative real number $x$. We have that
$\operatorname{remainder}(n,d)=\operatorname{floor}(\operatorname{remainder}(x,d))$
and $\operatorname{division}(n,d)=\operatorname{division}(x,d)$ if and only if
$n\leq x<n+1.$

###### Proof.

($\Leftarrow$) We can verify that if $n\leq x<n+1$, the previous two
conditions are satisfied.

($\Rightarrow$) Assume that
$\operatorname{remainder}(n,d)=\operatorname{floor}(\operatorname{remainder}(x,d))$
and $\operatorname{division}(n,d)=\operatorname{division}(x,d).$ We have

$\displaystyle\operatorname{remainder}(n,d)$ $\displaystyle\leq$
$\displaystyle\operatorname{remainder}(x,d)$ $\displaystyle<$
$\displaystyle\operatorname{remainder}(n,d)+1.$

By expanding out
$\operatorname{remainder}(n,d)=\operatorname{floor}(\operatorname{remainder}(x,d)),$
we have that

$\displaystyle n-\operatorname{floor}(n/d)*d$ $\displaystyle\leq$
$\displaystyle x-\operatorname{floor}(x/d)*d$ $\displaystyle<$ $\displaystyle
n-\operatorname{floor}(n/d)*d+1.$

Expanding out $\operatorname{division}(n,d)=\operatorname{division}(x,d)$ and
multiplying by $d$, we have
$\operatorname{floor}(n/d)*d=\operatorname{floor}(x/d)*d.$ We establish the
lemma by adding this last equation to the previous inequality. ∎

##  3 Multiply-Divide Results

Given a non-negative numerator $n$ and non-zero divisor $d$, we want to show
that by choosing integer constants $c$ and $m$ carefully, we can compute
$\operatorname{division}(n,d)$ and $\operatorname{remainder}(n,d)$ by starting
from $c*n$ and dividing by $m$.

###  3.1 Quotient

We want to find $c$ and $m$ such that
$\operatorname{division}(n,d)=\operatorname{division}(c*n,m)$. Intuitively,
this equation implies that $n/d\approx c*n/m$ and $n\approx c*n*d/m$. Let us
formalize this intuition.

For any non-negative real number $x$ and non-negative integer $Q$, we have
that $\operatorname{floor}(x/d)=Q$ is equivalent to $x\in[Qd,Qd+d)$. Letting
$Q=\operatorname{floor}(n/d)$ and $x=c*n*d/m$, we get
$d*\operatorname{floor}(n/d)\leq c*n*d/m<d*\operatorname{floor}(n/d)+d$. Since
$d*\operatorname{floor}(n/d)=n-\operatorname{remainder}(n,d)$,
$\operatorname{floor}(x/d)=\operatorname{division}(c*n,m)$ and
$Q=\operatorname{division}(n,d)$, we have that
$\operatorname{division}(n,d)=\operatorname{division}(c*n,m)$ is equivalent to
$n-\operatorname{remainder}(n,d)\leq
c*n*d/m<n-\operatorname{remainder}(n,d)+d.$

Consider a range of integer numerators $n\in[0,N]$ for some maximal integer
numerator $N\geq d$. We want this equation to hold for all $n$. The equation
is satisfied trivially when $n=0$ and $d>0$. Suppose that $n>0$ and rewrite
the inequalities as $(n-\operatorname{remainder}(n,d))/n\leq
c*d/m<(n-\operatorname{remainder}(n,d)+d)/n.$

Given any $n\in[1,N]$ for some integer $N\geq d$, we have that the leftmost
expression
$(n-\operatorname{remainder}(n,d))/n=1-\operatorname{remainder}(n,d)/n$ is
largest and equal to 1 when $\operatorname{remainder}(n,d)=0.$ Meanwhile the
rightmost expression $1+(d-\operatorname{remainder}(n,d))/n$ is smallest when
$n$ is as large as possible with $\operatorname{remainder}(n,d)=d-1$. To prove
this bound, partition the possible numerators into sets
$N_{k}=\\{n\in[0,N]\,|\,\operatorname{remainder}(n,d)=k\\}.$ Fixing $N$ and
$d$, we seek the value $n\in[1,N]$ minimizing
$f(n)=1+(d-\operatorname{remainder}(n,d))/n.$ For $n\in N_{k}$ we have
$f(n)=1+(d-k)/n$ which is minimized for the largest member of $N_{k}$. Let $v$
be the largest member of $N_{d-1}$; we have $f(v)=1+1/v$. We see that the
values of $n$ in $[N-d+1,N]$ are the minimizing values in each $N_{k}$. Among
these, we can show $v$ minimizes $f$.

  * 1.

Consider any $n\in[N-d+1,N]$ with $n>v$. Write it as $n=v+k$ with $k>0$ and
$k\leq d-1$ and so $\operatorname{remainder}(n,d)=k-1$ and
$f(n)=1+(d-k+1)/(v+k)$. As $k$ increases, the numerator decreases and the
denominator increases, we have that the minimum is reached when $k$ is largest
($d-1$), in which case $f(n)=1+2/(v+d-1)\geq 1+2/(v+v)=1+1/v=f(v)$,since
$v\geq d-1$.

  * 2.

Consider any $n\in[N-d+1,N]$ with $n<v$. Write $n$ as $v-d+k$, so
$\operatorname{remainder}(n,d)$ is again $k-1$. We have
$f(n)=1+(d-k+1)/(v-d+k)$. Again, as $k$ increases, the numerator decreases and
the denominator increases, we have that the minimum is reached when $k$ is
largest ($d-1$) in which case $f(n)=1+2/(v-1)<1+1/v=f(v)$.

Thus $n=v$ minimizes $f(n)$. We have shown Lemma 2 because
$v=N-\operatorname{remainder}(N+1,d)$.

######  Lemma 2.

Given an integer $d>0$, the value of $1+(d-\operatorname{remainder}(n,d))/n$
over $n=0,1,\ldots,N$ is minimized when $n$ is
$n=N-\operatorname{remainder}(N+1,d)$.

Hence we have that $1\leq c*d/m<1+\frac{1}{N-\operatorname{remainder}(N+1,d)}$
is equivalent to $\operatorname{division}(n,d)=\operatorname{division}(c*n,m)$
for all $n\in[0,N]$.

######  Theorem 1.

Consider an integer divisor $d>0$ and a range of integer numerators
$n\in[0,N]$ where $N\geq d$ is an integer. We have that
$\operatorname{division}(n,d)=\operatorname{division}(c*n,m)$ for all integer
numerators $n$ in the range if and only if

$\displaystyle 1/d\leq
c/m<\left(1+\frac{1}{N-\operatorname{remainder}(N+1,d)}\right)1/d.$

######  Remark 1.

Granlund and Montgomery [5] have an upper bound of $c/m\leq(1+1/(N+1))/d$ as a
sufficient (but not necessary) condition. A bound equivalent to Theorem 1 is
derived by Warren [6].

Once we have a pair of inequalities as in Theorem 1, we can solve for $c$ and
$m$. It is always possible to do so: we can verify that $c=1$, $m=d$ is always
a solution. However, we may have further constraints on $c$ and $m$: maybe we
require $m$ to be a power of two. We can show that as long as we can choose
$m$ arbitrarily large, there is always a solution. Letting
$K=N-\operatorname{remainder}(N+1,d)$, we can rewrite the inequalities as
$m/d\leq c<\left(1+\frac{1}{K}\right)\frac{m}{d}.$ Thus if $c$ is to be as
small as possible, we must have that $c=\operatorname{ceiling}(m/d)$. It
remains to solve for $m$ such that
$\operatorname{ceiling}\left(\frac{m}{d}\right)<\left(1+\frac{1}{K}\right)\frac{m}{d}.$
Because $\operatorname{ceiling}(m/d)-m/d<1$, we have that the inequality is
always satisfied when $m\geq K*d.$ This bound indicates that it is always
possible to find a solution, by picking $m$ large enough.

###  3.2 Remainder

From the quotient $\operatorname{division}(c*n,m)$, we get the quotient of the
division of $n$ by $d$; it is maybe intuitive that we can derive the remainder
of the division of $n$ by $d$ from $\operatorname{remainder}(c*n,m)$.

Formally, we want to find integer constants $c>0$ and $m>0$ such that for any
integer numerator $n\in[0,N]$ and integer divisor $d>0$, we have that
$\operatorname{remainder}(n,d)=\operatorname{division}(\operatorname{remainder}(c*n,m)*d,m)$.

If we find $c$ and $m$ such that
$\operatorname{remainder}(n,d)=\operatorname{division}(\operatorname{remainder}(c*n,m)*d,m)$
is satisfied, then replacing $c$ with $c+m$ or $c+2m$ would still work: in
fact
$\operatorname{remainder}(c*n,m)=\operatorname{remainder}(c*n+k*m*n,m)=\operatorname{remainder}((c+k*m)*n,m)$
for any integer $k$. Thus we require $c$ to be in $[0,m)$.

With this constraint ($c\in[0,m)$), we are able to show (see Lemma 3) that the
ability to compute remainders via
$\operatorname{remainder}(n,d)=\operatorname{division}(\operatorname{remainder}(c*n,m)*d,m)$
implies that the quotient of $n$ divided by $d$ is given by
$\operatorname{division}(n,d)=\operatorname{division}(c*n,m)$. Intuitively, it
is strictly more difficult to compute the remainder than to compute the
quotient. Hence, if we just need the remainder, and not the quotient, we
cannot relax our conditions when $c\in[0,m)$.

######  Lemma 3.

Consider an integer divisor $d>1$. Suppose that we have integer constants $c$
and $m$ such that $c\in[0,m)$ and
$\operatorname{remainder}(n,d)=\operatorname{division}(\operatorname{remainder}(c*n,m)*d,m)$
for all numerators $n\in[0,N]$ then we must have that
$\operatorname{division}(n,d)=\operatorname{division}(c*n,m)$.

###### Proof.

We $n=0$, we have that
$\operatorname{division}(n,d)=\operatorname{division}(c*n,m)$ holds trivially.
Since $c\in[0,m)$ then $c*(n+1)-c*n<m$ so when
$\operatorname{division}(c*n,m)$ increases following an increment of $n$ by
one, it must increase by at most one. We just have to show that it happens
exactly when $\operatorname{remainder}(n,d)=0$.

We have
$c*n=\operatorname{remainder}(c*n,m)+m*\operatorname{division}(c*n,m)$. The
left side of this equation increases by $c$ exactly when $n$ is incremented by
one. When $\operatorname{division}(c*n,m)$ increases by one, then it
contributes $m$ to the right side. Since $m>c$, we have that an increase of
$\operatorname{division}(c*n,m)$ corresponds to a decrease of
$\operatorname{remainder}(c*n,m)$.

However, we have that
$\operatorname{remainder}(n,d)=\operatorname{division}(\operatorname{remainder}(c*n,m)*d,m)$.
From this equation, we have that whenever $\operatorname{remainder}(n,d)$
increases when we increment $n$ by one, then $\operatorname{remainder}(c*n,m)$
must also increase. We know that when $n$ is incremented, then either
$\operatorname{remainder}(n,d)$ increases by one, or goes back to zero. It is
not possible for $\operatorname{remainder}(n,d)$ to increase if
$\operatorname{remainder}(c*n,m)$ decreases: it must therefore be that a
decrease in $\operatorname{remainder}(c*n,m)$ corresponds to
$\operatorname{remainder}(n,d)=0$. Thus we have that an increase of
$\operatorname{division}(c*n,m)$ following an increment of $n$ corresponds
$\operatorname{remainder}(n,d)=0$. It follows that
$\operatorname{division}(n,d)=\operatorname{division}(c*n,m)$. ∎

We still must derive the conditions on $c$ and $m$. We can expand the
condition that
$\operatorname{remainder}(n,d)=\operatorname{division}(\operatorname{remainder}(c*n,m)*d,m)$
as follows:

$\displaystyle\operatorname{remainder}(n,d)$
$\displaystyle=\operatorname{floor}\left(\frac{\left(c*n-\operatorname{floor}\left(\frac{c*n}{m}\right)*m\right)*d}{m}\right)$
$\displaystyle=\operatorname{floor}\left(c*n*d/m-\operatorname{floor}\left(\frac{c*n*d/m}{d}\right)*d\right)$
$\displaystyle=\operatorname{floor}(\operatorname{remainder}(c*n*d/m,d)).$

Then by Lemma 1, we have that the two constraints
($\operatorname{remainder}(n,d)=\operatorname{floor}(\operatorname{remainder}(c*n*d/m,d))$
and $\operatorname{division}(n,d)=\operatorname{division}(c*n*d/m,d)$) are
equivalent to $n\leq c*n*d/m<n+1$, or $m/d\leq c<(1+1/n)m/d$. This condition
should hold for all applicable values of $n$, and thus we choose to use the
maximal value of $n$ (i.e., $N$) as it provides the tightest bound — so an
equivalent expression is $m/d\leq c<(1+1/N)m/d$.

We have derived the following theorem.

######  Theorem 2.

Consider an integer divisor $d>0$ and a range of integer numerators
$n\in[0,N]$ where $N\geq d$ is an integer. We have that
$\operatorname{division}(n,d)=\operatorname{division}(c*n,m)$ and
$\operatorname{remainder}(n,d)=\operatorname{division}(\operatorname{remainder}(c*n,m)*d,m)$
for all integer numerators $n$ in the range if and only if

$\displaystyle 1/d\leq c/m<\left(1+\frac{1}{N}\right)1/d$

######  Remark 2.

In previous work [9], we reported an upper bound of $c/m\leq(1+1/(N+1))1/d$ as
a sufficient (but not necessary) condition. For the difference to matter, we
need that there is an integer in the interval $((1+1/(N+1))m/d,(1+1/N)m/d)$.
It happens in some instances, for example if $N=10$, $d=5$, and $m=2^{5}$, we
have that $7\in((1+1/(N+1))m/d,(1+1/N)m/d)$. However, in the previous work
[9], we considered only the case where $m=N+1$. We can show that if $m=N+1$
and $m\geq 3$ then the earlier bound is tight. Indeed, if there is an integer
$z$ in $((1+1/(N+1))m/d,(1+1/N)m/d)$ then there must be an integer $z*d$ in
$((1+1/(N+1))m,(1+1/N)m)$. Substituting $N=m-1$, the interval becomes
$(m+1,m+1+1/(m-1))$: because $m+1$ is an integer and $1/(m-1)\leq 1/2$, there
is no integer in this interval. Hence, there cannot be an integer in
$((1+1/(N+1))m/d,(1+1/N)m/d)$ and the earlier bound is tight.

If we only desire the remainder, and not the quotient, we can lift the
restriction that $c\in[0,m)$: we can replace $c$ by $c+k*m$ for any integer
$k$.

###  3.3 Check for Divisibility

We have that $n$ is a multiple of $d$ if and only if
$\operatorname{remainder}(n,d)=0$. Given Theorem 2, we can check whether
$\operatorname{remainder}(n,d)=0$ by checking whether
$\operatorname{division}(\operatorname{remainder}(c*n,m)*d,m)=0$. In turn, we
have that this last equation holds if and only if
$\operatorname{remainder}(c*n,m)*d<m$ or
$\operatorname{remainder}(c*n,m)<m/d$. Thus
$\operatorname{remainder}(c*n,m)<m/d$ is a divisibility test. However, we show
the more elegant result that $\operatorname{remainder}(c*n,m)<c$ is a
divisibility test (see Proposition 1).

By the assumption of Theorem 2, we have that $m/d\leq c$. Thus if $n$ is a
multiple of $d$, then we have that $\operatorname{remainder}(c*n,m)<c$. We
need to prove the counterpart, that $\operatorname{remainder}(c*n,m)<c$
implies that $n$ is a multiple of $d$. By Theorem 2, we have that
$\operatorname{division}(n,d)=\operatorname{division}(c*n,m)$. Hence we have
that
$\operatorname{division}(c*n,m)=\operatorname{division}(c*(n-\operatorname{remainder}(n,d)),m)$
since $n$ and $n-\operatorname{remainder}(n,d)$ have the same quotient with
respect to $d$. When two values $z_{1},z_{2}$ have the same quotient
($\operatorname{division}(z_{1},m)=\operatorname{division}(z_{2},m)$) then
their difference must be captured by their remainders:
$z_{2}-z_{1}=\operatorname{remainder}(z_{2},m)-\operatorname{remainder}(z_{1},m)$.
In this case, taking $z_{1}=c*n$ and
$z_{2}=c*(n-\operatorname{remainder}(n,d))$,we have that their difference is
$c*\operatorname{remainder}(n,d)$. It follows that
$\operatorname{remainder}(c*n,m)-\operatorname{remainder}(c*(n-\operatorname{remainder}(n,d)),m)=c*\operatorname{remainder}(n,d)$
and therefore
$c*\operatorname{remainder}(n,d)\leq\operatorname{remainder}(c*n,m)$. Thus if
$\operatorname{remainder}(c*n,m)<c$, we have
$c*\operatorname{remainder}(n,d)<c$ which implies
$\operatorname{remainder}(n,d)=0$.

######  Proposition 1.

Consider an integer divisor $d>0$. We have that $d$ divides $n\in[0,N]$ if and
only if $\operatorname{remainder}(c*n,m)<c$ subject to the condition that

$\displaystyle 1/d\leq c/m<\left(1+\frac{1}{N}\right)1/d.$

Proposition 1 selects a value of $c$ in $[0,m)$.

##  4 Multiply-Add-Divide Results

Some authors [12, 7] have considered the case where we replace the division by
a formula of the multiply-add form $\operatorname{floor}((c*n+b)/m)$ for some
$b$. The benefit of the multiply-add approach is that it may allow one to pick
a smaller value of $c$, compared to the simpler form
$\operatorname{floor}(c*n/m)$. The derivations are nearly identical as § 3.1,
so we just give our results.

######  Theorem 3.

Consider an integer divisor $d>0$ and a range of integer numerators
$n\in[0,N]$ where $N\geq d$ is an integer. We have that
$\operatorname{division}(n,d)=\operatorname{division}(c*n+c,m)$ for all
integer numerators $n$ in the range if and only if

$\displaystyle\left(1-\frac{1}{N-\operatorname{remainder}(N,d)+1}\right)1/d\leq
c/m<1/d.$

######  Remark 3.

Robison [7] derived the sufficient condition $(1-1/(N+1))1/d\leq c/m<1/d$.
When $\operatorname{remainder}(N,d)\neq 0$, Robison’s bound is suboptimal
unlike Theorem 3. Drane et al. [10] derive a similar result for the case where
$d$ is odd.

######  Theorem 4.

Consider an integer divisor $d>0$ and a range of integer numerators
$n\in[0,N]$ where $N\geq d$ is an integer. We have that
$\operatorname{division}(n,d)=\operatorname{division}(c*n+c,m)$ and
$\operatorname{remainder}(n,d)=\operatorname{division}(\operatorname{remainder}(c*n+c,m)*d,m)$
for all integer numerators $n$ in the range if and only if

$\displaystyle\left(1-\frac{1}{N+1}\right)1/d\leq c/m<1/d.$

######  Proposition 2.

Consider an integer divisor $d>0$. We have that $d$ divides $n\in[0,N]$ if and
only if $\operatorname{remainder}(c*n+c,m)<c$ subject to the condition that

$\displaystyle\left(1-\frac{1}{N+1}\right)1/d\leq c/m<1/d.$

##  5 Rounding

Instead of computing the integer division ($\operatorname{floor}(n/d)$), we
sometimes wish to round the result to the nearest integer. Unsurprisingly and
maybe obviously, it is possible to do with an expression of the form
$\operatorname{floor}(z/d)$ for some integer $z$ that depends on $n$ and $d$.
Hence, our efficient integer quotient computations extend to the computation
of the rounded division. Indeed, we have that
$\operatorname{floor}((n+\operatorname{floor}(d/2))/d)$ is the round-to-
nearest function; when $d$ is odd and $n$ is between two multiple of $d$, it
rounds up. To round down, we can use
$\operatorname{floor}((n+\operatorname{ceiling}(d/2)-1)/d)$ instead.222We
define $\operatorname{ceiling}(x)$ as the smallest integer that is no smaller
than $x$.

It is also possible to handle more complicated scenarios. For example, what if
we wish to round to the nearest integer, rounding to the nearest even integer
when we are in-between two integers? It is only relevant when the division $d$
is even. Let $z=n+\operatorname{floor}(d/2)$. Whenever $z$ is a multiple of
$d$ and $\operatorname{floor}(z/d)$ is odd, we return
$\operatorname{floor}(z/d)-1$, else we return $\operatorname{floor}(z/d)$. We
can check that an integer is a multiple of $d$ efficiently with Proposition 1
for example. The division and the divisibility test can reuse the same
intermediate computations.

## References

  * Artzy et al. [1976] E. Artzy, J. A. Hinds, H. J. Saal,  A fast division technique for constant divisors,  Commun. ACM 19 (1976) 98–101.
  * Jacobsohn [1973] D. H. Jacobsohn,  A combinatoric division algorithm for fixed-integer divisors,  IEEE Trans. on Comput. 100 (1973) 608–610.
  * Li [1985] S.-Y. R. Li,  Fast constant division routines,  IEEE Trans. Comput. 34 (1985) 866–869.
  * Vowels [1992] R. A. Vowels,  Division by 10,  Aust. Comput. J. 24 (1992) 81–85.
  * Granlund and Montgomery [1994] T. Granlund, P. L. Montgomery,  Division by invariant integers using multiplication,  SIGPLAN Not. 29 (1994) 61–72.
  * Warren [2002] H. S. Warren, Jr., Hacker’s Delight, Addison-Wesley, Boston, 1st edition, 2002.
  * Robison [2005] A. D. Robison,  N-bit unsigned division via n-bit multiply-add,  in: Proceedings of the 17th IEEE Symposium on Computer Arithmetic, ARITH ’05, IEEE Computer Society, Washington, DC, USA, 2005, pp. 131–139.
  * Anonymous author [2011] Anonymous author, Labor of division (episode iii): Faster unsigned division by constants, http://ridiculousfish.com/blog/posts/labor-of-division-episode-iii.html (last checked Nov. 2020), 2011.
  * Lemire et al. [2019] D. Lemire, O. Kaser, N. Kurz,  Faster remainder by direct computation: Applications to compilers and software libraries,  Software: Practice and Experience 49 (2019) 953–970.
  * Drane et al. [2012] T. Drane, W.-c. Cheung, G. Constantinides,  Correctly rounded constant integer division via multiply-add,  in: 2012 IEEE International Symposium on Circuits and Systems, IEEE, pp. 1243–1246.
  * Warren [2013] H. S. Warren, Jr., Hacker’s Delight, Addison-Wesley, Boston, 2nd edition, 2013.
  * Magenheimer et al. [1987] D. J. Magenheimer, L. Peters, K. Pettis, D. Zuras,  Integer Multiplication and Division on the HP Precision Architecture,  SIGARCH Comput. Archit. News 15 (1987) 90–99.


# FluxGAN: A Physics-Aware Generative Adversarial Network Model for Generating
Microstructures That Maintain Target Heat Flux

Artem K. Pimachev  Ann and H.J. Smead Aerospace Engineering Sciences,
University of Colorado Boulder, Boulder, CO 80303, USA  Manoj Settipalli  Ann
and H.J. Smead Aerospace Engineering Sciences, University of Colorado Boulder,
Boulder, CO 80303, USA  Sanghamitra Neogi<EMAIL_ADDRESS>Ann
and H.J. Smead Aerospace Engineering Sciences, University of Colorado Boulder,
Boulder, CO 80303, USA

###### Abstract

We propose a physics-aware generative adversarial network model, FluxGAN,
capable of simultaneously generating high-quality images of large
microstructures and description of their thermal properties. During the
training phase, the model learns about the relationship between the local
structural features and the physical processes, such as the heat flux in the
microstructures, due to external temperature gradients. Once trained, the
model generates new structural and associated heat flux environments,
bypassing the computationally expensive modeling. Our model provides a cost
effective and efficient approach over conventional modeling techniques, such
as the finite element method (FEM), for describing the thermal properties of
microstructures. The conventional approach requires computational modeling
that scales with the size of the microstructure model, therefore limiting the
simulation to a given size, resolution, and complexity of the model. In
contrast, the FluxGAN model uses synthesis-by-part approach and generates
arbitrary large size images at low computational cost. We demonstrate that the
model can be utilized to generate designs of thermal sprayed coatings that
satisfies target thermal properties. Furthermore, the model is capable of
generating coating microstructures and physical processes in three-dimensional
(3D) domain after being trained on two-dimensional (2D) examples. Our approach
has the potential to transform the design and optimization of thermal sprayed
coatings for various applications, including high-temperature and long-
duration operation of gas turbines for aircraft or ground-based power
generators.

## Introduction

In recent years, machine learning techniques, such as generative adversarial
networks (GANs), are increasingly being used to explain deterministic
multiphysics phenomena where the underlying physical model is complex or
unknown. Physics-informed generative adversarial networks have been proposed
for solving forward, inverse, and mixed stochastic problems using data from a
limited number of scattered measurements [1]. Instead of relying solely on
data for training, this approach encoded the governing physical laws in the
GAN architecture in the form of stochastic differential equations. FluidGAN
has been proposed for learning and predicting time-dependent convective flow
coupled with energy transport [2]. StressGAN model was developed to generate
mechanical stress distributions for various complex cases of geometry, load,
and boundary conditions [3]. The model was shown to predict more accurate
high-resolution stress distributions than a convolutional neural network
model. In parallel, GANs have shown remarkable progress for creating high-
resolution synthetic images, visually similar to their real counterparts,
supported by revolutionary advents in the graphical data domain. Until
recently, training such model required enormous amount of training images and
was highly unstable. GANs have been shown to be highly effective in generating
realistic microstructural data that can be used to augment limited
experimental data or simulate microstructures that are challenging to obtain
experimentally [4, 5]. The style-based GAN (StyleGAN2) architecture [6],
implemented the style transfer approach [7] with the adaptive discriminator
augmentation (ADA) [8] mechanism, and demonstrated that high-quality images
can be generated with limited training data.

In this study, we propose a new FluxGAN model that simultaneously generates
high-quality microstructural images and high-resolution description of thermal
properties of these microstructures. We adopt the StyleGAN2 architecture to
develop the model and train the model using images that include both
microstructural information and the associated physical (i.e., thermal)
phenomena. We demonstrate the capabilities of our approach for the analysis
and design of microstructure-controlled thermal properties of thermal sprayed
coatings. The thermal sprayed coatings exhibit highly diverse structural
environments, including pores with different shapes and sizes, and varied size
grains aligned at different crystallographic orientations. The microstructural
features strongly affect interface-sensitive material properties, such as
thermal diffusivity. Additionally, the structural features could change and
evolve due to sintering in high temperature environments. The presence of
microstructural features is often quantified by the porosity parameter: (pore
volume)/(total volume)$\times 100$ %. The variation of thermal properties of
coatings has commonly been described in relation with porosity, using
phenomenological models [9]. However, global parameter (e.g. porosity) based
models may not accurately capture the physics of heat transfer across various
microstructural features and feature interfaces. The complexity of structural
features and their evolution makes it highly challenging to characterize the
physical properties of these coatings, especially after aging, using existing
experimental and numerical approaches.

Our proposed FluxGAN model provides highly accurate prediction of thermal
phenomena in coatings with different microstructures that include different
porosities, grain sizes, and crystallographic orientations. We use a small
number of two-dimensional (2D) images of niobium thermal sprayed coatings,
obtained with backscatter electron (BSE) imaging technique, as the ‘true’
image of the coating microstructures. We compute the thermal properties of the
microstructures using the image-based finite element method (FEM). We train
the FluxGAN model using superimposed images that include both microstructural
and thermal property data. The FluxGAN model combines the physical properties
predicted by FEM and the GAN architecture, for generating diverse
microstructures with desired thermal properties. We use separate FEM
simulations to validate the generated thermal data. The model uses synthesis-
by-part approach and generates images of arbitrary size patch by patch, which
requires constant memory usage. In contrast, the FEM memory usage and
computational time scales with the image size. The combination of image-based
FEM and GAN models offers a promising solution to overcome the limitations of
traditional experimental and numerical methods. Our model provides a cost
effective and efficient approach for describing the microstructure-controlled
thermal properties of thermal sprayed coatings. The model establishes a direct
relationship between structural features and thermal properties of coatings,
which could be utilized to better control and optimize their thermal
properties. We demonstrate that our model can be used to design thermal
sprayed coatings that can maintain tailored heat flux that meets application-
specific requirements. Our approach has the potential to transform the design
and optimization of thermal sprayed coatings for various industrial
applications, including high-temperature and long-duration operation of gas
turbines for aircraft or ground-based power generators.

Figure 1: Overview of FluxGAN model and its advantage over conventional
approach: (i) The model includes a training and a generation phase. We train
the model with (b) superimposed RGB images that include (a) experimental
structural images and (c,d) heat flux data, simulated with FEM techniques. The
model classifies the distinct features of the RGB images and assigns them as
different styles. Once trained, the model generates images based on user
input. (f) Spatial map of different styles as input. (g) The model generates
new structural image superimposed with associated heat flux, that satisfies
target thermal properties. (ii) In contrast, conventional design approach
follows an expensive and iterative cycle until microstructures with target
physical properties are obtained.

## Methods

### Model Overview

Figure 1(i) shows the overview of our physics-informed FluxGAN model. The
model includes a training and a generation phase. We train the model with
images that include information about structural as well as thermal properties
of the microstructure. We obtain the 2D structural images, as shown in Fig.
1(a), using backscatter electron (BSE) imaging and simulate the thermal
properties using FEM techniques. Figure 1(e) shows the spatial distribution of
heat flux, $Q(x,y)$, generated in the microstructure, due to the application
of temperature gradient along the top-to-bottom direction. Figure 1(c) and (d)
show the graphical domain representation of the heat flux components, $Q_{x}$
and $Q_{y}$, respectively. We create RGB images by superimposing the
experimental structural images and the heat flux maps, as shown in Fig. 1(b).
We provide the RGB images as input to the FluxGAN model. The model identifies
the different features of the RGB training images, referred to as styles, in
an unsupervised manner. The styles not only classify structural features but
are strongly associated with heat flux patterns of the RGB images. In the
unguided generation phase, the FluxGAN model requires minimal input from the
user and generates a new image corresponding to the input. An example input is
shown in Fig. 1(f). The input specifies the overall height and width of the
image and the spatial map of styles to be included. The spatial map includes
the number of styles that can be identified in the image (styles 1 and 2), the
locations for regions with styles 1 and 2 (anchors), and the merging function
that can be applied to obtain smooth interfaces between the regions. Figure
1(g) shows a representative microstructure, generated by the FluxGAN model.
The generated image includes structural features that correspond to styles 1
and 2 as specified by the user input. The image also includes information
about the heat flux that will be observed if the microstructure is subjected
to temperature gradients along the top-to-bottom image direction. Thus, the
FluxGAN model directly generates designs of representative microstructures and
corresponding physical properties from a minimal user input. The model can
even generate designs of new classes of structures that are not present in
original experimental images. For example, different designs can be created by
combining different styles, varying the positions of the styles in two-
dimensional (2D) or three-dimensional (3D) spatial domains. Additionally, the
model offers guided assignment of styles in the generated microstructures,
based on the specifications of the input image. For example, given a known
heat flux distribution in the heat source or sink region as input, the model
allows for inverse design of microstructures that satisfies the target heat
flux condition. We describe the different components of the FluxGAN model in
the following sections. We demonstrate the effectiveness of the FluxGAN model
by using single phase niobium (Nb) thermal spray coatings as an example.
However, our model can be broadly applied to different materials and
structures to assess how well they satisfy target physical properties.

The conventional approach for designing new coatings, as outlined in Fig.
1(ii), typically iterates through the following steps: (1) fabrication of
coatings, (2) experimental characterization, (3) developing computational
models based on experimental images, (4) physical property calculation, and
repeat (1-4) till the fabricated structures satisfy target physical
properties. Our FluxGAN model provides an expedited approach to inform the
design of new microstructures that satisfy target physical properties,
bypassing the expensive iterative process. In this study, we have chosen the
styles in an arbitrary manner. However, the styles can be associated with
specific physical conditions. For example, the different spray intervals in a
thermal spray process could result in different structural features. The
different features can also be associated with diverse environmental
conditions, such as high temperatures or corrosion. As styles are strongly
associated with the heat flux, physical meaning of styles can be regions of
high or low thermal conductance and temperature hot spots, etc. A FluxGAN
model aware of the relationship between the styles and synthesis or
environmental conditions can offer great practical advantage. For example, the
styles in generated images can inform strategies to synthesize microstructures
with target functionalities and also, provide information about the
reliability of microstructure in diverse environmental conditions.

## Training Phase

### Processing of Experimental Images

Figure 2 shows the 2D experimental BSE images of the Nb thermal spray coatings
that we use to demonstrate the performance of the FluxGAN model. The dimension
of each image is 112$\upmu$m $\times$ 160$\upmu$m (410px $\times$ 587px).
Three distinct regions can be visually identified from the images: the gray
middle region that shows the presence of the coating microstructure, the top
and bottom dark regions where no coating material is present and the two
interface regions between the gray coating and the dark regions. We assume
that the dark regions of the images represent presence of air. We show a
magnified version of a representative coating microstructure in Fig. 2(ii). As
can be noted, the microstructure includes distinct features, such as flat
splats and globular and interfacial porosities.

Figure 2: Experimental images of coating microstructures: (i) Raw BSE images
of niobium thermal spray coatings. (ii) Magnified image of microstructure
shows a wide variety of structural features, such as flat splats and
porosities due to conditions during the synthesis process.

In addition to these features, a wide variety of small features can also be
noted that are representative of materials properties or processing
conditions. However, some features may appear due to difference in lighting
conditions during imaging. Note that the FluxGAN model identifies the
different styles of the training images in an unsupervised manner. To ensure
that the microstructure images only include ‘true’ features, we preprocess the
grayscale BSE images and remove features that appear due to irregular image
brightness. The preprocessing is an essential step since our goal is to
develop a model that can relate ‘true’ structural features with the
corresponding physical properties. First, we obtain the pixel intensity
distribution of the BSE images, by counting the number of pixels that has a
given intensity in the range between 0 and 255. Figure 2(a) shows that the
coating region of the images have intensities $\geq$ 50 in this scale. The
appearance of the distinct peaks can be attributed to different lighting
conditions during imaging. We scale the intensities to values in the range
between 0 and 1. We then shift the mean of the distribution to 0.5 using a
power transformation. Finally, we rescale the pixel intensity distribution of
the coating region to be in the range from 50 to 255. Figure 2(b) shows the
pixel intensity distribution after the transformation. The intensity values
below 50 represent pores or air regions where the coating material is not
present.

### Augmentation of Training Set

Our experimental dataset is limited to nine BSE images, as shown in Fig. 2.
However, the GAN models require enormous amount of training images to generate
high-resolution images as output. Extensive efforts are ongoing to develop
models that can create high quality images with limited training data [10,
11]. Recent approaches, such as StyleGAN2 [6], require a smaller training set,
however, still require substantial number of training images. We implement
different strategies to augment the training dataset. We apply rotation and
reflection operations on the raw images and add the new images to the dataset.
The new images include additional variations of structural features that
impact the thermal properties. We thus obtain a diverse set of images to train
the FluxGAN model. However, it is essential that the new images include
features that are consistent with the physical conditions displayed by the
original images. This constraint imposes restrictions on the type of
operations we could perform to create new images. For example, all
experimental images of Fig. 2 (and also, Fig. 3(i)) show that the images have
flatter bottom surfaces and uneven top surfaces. These features appear because
the thermal spray coatings are fabricated on top of substrates. We only apply
horizontal flip transformations but do not apply any vertical flip operations
to maintain similar layer-like features in the new images. Additionally, when
the droplets of melted material are sprayed in a regular interval, the
material solidifies and forms layers in the top-to-bottom direction. As can be
seen from Fig. 2(ii), the layers are highly uneven. This is because the
sprayed material might solidify at different rates in the sprayed
microstructures during fabrication. Highly anisotropic features may appear
when droplets of melted material are sprayed over already solidified splats.
To ensure that the new images retain such characteristics, we apply only small
angular rotations between -2.5 to +2.5 degrees, with 11 incremental steps. We
set the top and bottom air regions to be 80 pixels wide and add extra space as
needed so that the coating region stays within the image boundary, as shown in
Fig. 3(ii). The augmented dataset contains a total of 198 (9 original $\times$
11 rotated $\times$ 2 flipped) superimposed images.

Figure 3: Preparation of training images: (i) BSE image. Pixel intensity
distribution of BSE images (a) before and (b) after image processing;
Histogram shows the number of pixel counts for different pixel intensity in a
monochrome channel. Heat flux along components, (c) $Q_{x}$, and (d) $Q_{y}$,
for applied temperature gradients. Top horizontal axes labels show heat flux
values and the bottom labels show graphical domain encoding for the monochrome
channels. Three channel representations of (ii) structural and (iii-iv) heat
flux data. The bottom label values of (b-d) are used as color scheme of images
(ii-iv). The three channels are superimposed to create RGB images. (v) Tiles
extracted from RGB images are used as training images.

### Computation of Thermal Properties

We compute the thermal properties of the coating microstructures using image-
based FEM techniques, as implemented in COMSOL software [12]. We assume that
the heat conduction direction is along the top-to-bottom direction of the
coatings. We convert the microstructure images to masks by assigning all
pixels with intensities $<$ 50 to black color, representing air, and the
pixels with intensities $\geq$ 50 to grey color, representing niobium. Figure
1(a) and (e) show an experimental image and the corresponding masked image,
respectively. We use the masked images as input for image-based FEM
simulations. We import the masked images to COMSOL and use the built-in
function, $im(x,y)=\\{0,1\\}$, to mark different regions within each image.
The discrete values correspond to black and gray regions of the images,
respectively. This creates rectangular simulation domains that have same
dimensions as the input images. We use the image function, $im(x,y)$, and the
thermal conductivity values of air and Nb, to define spatially resolved
thermal conductivities, $\kappa_{\text{local}}(x,y)$, in the rectangular
simulation domain:

$\kappa_{\text{local}}(x,y)=\begin{cases}\kappa_{\text{Air}},&\text{if
}im(x,y)=0\\\ \kappa_{\text{Nb}},&\text{if }im(x,y)=1\end{cases}$ (1)

We choose thermal conductivity of niobium ($\kappa_{\text{Nb}}=54$ Wm-1K-1)
for all grey pixels and thermal conductivity of air ($\kappa_{\text{Air}}=0.1$
Wm-1K-1) for all black pixels. We also define the local densities,
$\rho_{\text{local}}(x,y)$, and local specific heat capacities, $C_{\text{p,
local}}(x,y)$, using similar equations as Eq. 1. We choose
$\rho_{\text{Air,Nb}}=\\{1.225,8570\\}$ Kgm${}^{\text{-3}}$ and
$C_{\text{p}}^{\text{Air, Nb}}=\\{1008,260\\}$
JKg${}^{\text{-1}}$K${}^{\text{-1}}$, respectively. However, the heat flux
distribution is primarily determined by the local thermal conductivities, as
expected. The thermal properties of porous materials are commonly calculated
using effective models, such as the Maxwell-Eucken model [9]. These models
only take the porosity of the structure into account and do not consider the
distribution of pores and their shapes to calculate thermal properties.
Instead, we compute the heat flux in the coatings using image-based FEM and
Fourier’s law: $\textbf{q}=-d_{z}\kappa_{\text{local}}\nabla T$. Here, $q$ is
the heat flux along the temperature gradient $\nabla T$ and $d_{z}$=1m is the
thickness along $z$ direction. We only discuss the heat flux in the 2D $x-y$
plane of the coatings and report the values of $Q=q/d_{z}$, in units of Wm-2.
We find that the heat flux and the resulting effective thermal conductivity
can be significantly different for two coatings with the same porosity but
different pore distributions. Such variability cannot be captured by the
effective models.

We assume that both the top and the bottom no-material regions are made of low
thermal conductivity material, i.e., air. We model the top region as heat
source with temperature equal to 700K and the bottom region as heat sink with
temperature 300K. Thus, we simulate an applied temperature bias of 400 K along
the top-to-bottom direction of the coatings, for all BSE masked images. We
model the lateral boundaries of the simulation cells as perfect insulators. We
define a rectangular grid for the FEM simulations of dimension, $p_{y}\times
p_{x}$. Here $p_{x}$ and $p_{y}$ are the number of pixels along the $x$ and
the $y$ directions of the images. We use an adaptive meshing scheme and carry
out five refinements in COMSOL to identify a mesh grid that minimizes the
$L_{2}$ norm of the squared error. The image function $im$ is defined on the
grid and assigns the local material properties accordingly. We compute heat
flux, $q$, using these adaptive nodes and element shape functions. For each
image, we iteratively solve for $q$ that satisfies the boundary conditions and
local thermal conductivities. We assume dz = 1m in our calculations, thus, $Q$
have the same values as $q$, but different units. We refer to $Q(x,y)$ as heat
flux in the subsequent discussion. $Q(x,y)$’s are defined on a rectangular
grid of the same resolution as the input masked image. However, the grid is
different for each image, depending on the original resolution of the
experimental image and the applied transformations as described in the
‘Augmentation of Training Set’ section. The associated heat flux also vary
significantly, as can be expected.

We show the components of heat flux along $x$ and $y$ directions in Fig.
3(c,d), respectively. The labels of the top horizontal axes show the heat flux
values. We transform $Q_{x}$ and $Q_{y}$ to represent them in the graphical
domain. We linearly scale $Q_{x}$ and $Q_{y}$ such that the middle 98% of the
distribution falls in the range from 0 to 255. We assign the left and the
right 1% of heat flux distribution to values of 0 and 255, respectively. We
transform the $Q$ values using the following equations:
$Q_{x}=Q_{x}/10^{6}\times 173+127$; $Q_{y}=Q_{y}/10^{6}\times 153+255$. We
obtain these equations by assigning the bounds of the middle 98% of the
$Q$-distributions to graphical encoding values. The bottom axes labels of Fig.
3 (c,d) correspond to the graphical domain encoding for the monochrome
channels. Note that the FEM simulation domains have horizontal reflection
symmetry. We leverage this aspect and obtain results for 99 additional images
by horizontally reflecting the FEM-generated heat flux $Q(x,y)$ [W/m2]. In the
reflected images, the ($x,y$) values are flipped horizontally and the
magnitudes of $Q_{x}$ change sign, however, the magnitudes of $Q_{y}$ stays
the same. Thus, we obtain data for 198 images from the 99 FEM calculations.

Figure 4: Schematic of the FluxGAN model workflow. Model architecture consists
of (i) a generator, and (ii) a discriminator. (iii) Large-size images are
generated by learning spatially extensible representations living in a latent
space. Images in the first and the second columns are produced by varying
global latent and styles respectively. Top, middle, and bottom images
correspond to the green, red, and blue boxed regions in the implicit image
(i), respectively.

### Preparation of Training Images

We superimpose structural images and the corresponding heat flux maps into
three-channel RGB images. The R, G, and B channels are the graphical encoding
of $Q_{y}$, $Q_{x}$, and the structural images, as shown in Fig. 3(d), (c) and
(b), respectively. We train the FluxGAN model on the relationship between the
structural features of the coatings and the associated thermal properties. To
this end, we create a training set by uniformly sampling overlapping square
tiles of size 17.4$\upmu$m $\times$ 17.4$\upmu$m (64px $\times$ 64px) from the
RGB images. We split each RGB image into $19\times 13$ overlapping tiles and
create a training set that includes 48,906 ($=198\times 19\times 13$) RGB
images. Figure 3(v) shows representative square tiles included in the training
set. Each image is represented by an array of numbers. The dimension of the
array is the height times the width of the tile, times the number of channels
in the image: $64\text{px}\times 64\text{px}\times 3$. The array elements are
pixel intensities in the range from 0 to 255, that correspond to the graphical
encoding of the heat flux maps and the structural images.

### Unsupervised Learning of Styles

We train the FluxGAN model using square 64px $\times$ 64px RGB patches, as
discussed in the previous section. GAN models can learn from small patches and
successfully generate new large-size images. However, the reconstruction of
large scale holistic images can be challenging. For example, techniques such
as consecutive image outpainting can result in discontinuous and repetitive
structures [13, 14]. Several models have been proposed recently that can
generate large images but they fail to maintain a global coherence in the
generated images [15, 16, 17]. The recently proposed InfinityGAN approach has
shown a superior ability to generate holistic large-size images after being
trained on small patches [18]. The generated images show a great diversity and
continuity along all spatial directions. We implement the InfinityGAN approach
in our FluxGAN model to generate large coating images, learning from small
patches. The approach generates large holistic images, also known as implicit
images, with diverse local features. The InfinityGAN architecture adopts
functionality from StyleGAN2 [6], which the FluxGAN model leverages to learn
the different styles or the textures of the images, in an unsupervised manner.
InfinityGAN synthesizes images of arbitrarily large-size by learning spatially
extensible representations living in a latent space. The latent space is
encoded by a global latent vector that guides the holistic layout of the
infinite-pixel image, and a local latent tensor that expresses structural
variations of small region patches.

We show a schematic of the FluxGAN model workflow in Fig. 4. The model
consists of a generator that learns to generate plausible images (Fig. 4(i)),
and a discriminator that learns to distinguish the generated fake images from
real images (Fig. 4(ii)). The generator consists of two modules, a structure
and a texture synthesizer. The structure synthesizer is a neural implicit
function [19, 20, 21, 22], that takes three sets of inputs: a global latent
variable, a local latent variable and a continuous coordinate that samples the
sub-regions of the implicit image. The texture synthesizer is a fully
convolutional StyleGAN2 model [6], with all positional information removed. In
addition to the same three inputs, a set of randomized noises are provided as
input to the texture synthesizer, to model variations of fine-grained texture
within the local structural representations. A mapping layer projects global
latent onto a texture style vector, and the texture style is injected into all
pixels in each layer of the global image via feature modulation.

Figure 5: Representative input maps and output images of the FluxGAN model.
The first column shows input maps and the second and the third columns show
two representative images generated by the model for each input map,
respectively. Increasing the number of styles and varying their spatial
arrangement affects both the overall appearance and the local diversity of the
generated images.

Similar to InfinityGAN, the model uses 256 and 512 dimensions for local and
global latent vectors, respectively, for unsupervised learning of structure
and texture variations. We use coordinate grid to specify the location of the
local structural patches to be sampled. We use sine functions with period $T$
= 56 to sample both $x$ and $y$ directions of the local latents. Our training
dataset includes $64\text{px}\times 64\text{px}$ RGB image tiles, as mentioned
previously. However, the model does not know the positioning of these
$64\text{px}\times 64\text{px}$ tiles in the global image. To address the
issue, the model uses smaller $37\text{px}\times 37\text{px}$ patches and
learns to coordinate the patches relative to each other, during training. We
use the same architecture of the discriminator as implemented in StyleGAN2.
The schematic of the discriminator workflow is shown in Fig. 4(ii). The
discriminator compares generated 37px $\times$ 37px patches with those
included in the training set images. We assess the quality of images created
by the model during training and find that the model achieved the smallest
value of the Fréchet inception distance (FID) metric [23] of 2.64 after
660,000 training steps.

Figure 4(iii) shows representative output images that are generated based on
single-style input maps. The input maps determine the different latent
variables and drive the generation of the environments. Since the inputs are
single-style maps, the model uses a single latent vector and generates all the
regions of the microstructure. We vary the latent vector and compare the
generated images, to examine the information encoded in the latent space. We
obtain three sets of global latent vectors by using different random sampling
from a unit Gaussian distribution. We use the three different latent vectors
and keep all other variables (e.g., coordinates and styles) fixed, to obtain
the three images shown in the first column of Fig. 4(iii). The structural
environments of the images vary significantly when we change the global latent
variables. The images show different regions of the coatings, such as the
microstructure or the surface and air regions. Together the top surface,
material, and bottom surface images constitute the large coating images. The
images are representative of the regions marked with green, red, and blue
boxes in Fig. 4(i). It is interesting to note that the the heat flux patterns
across the images appear similar in magnitude, especially in regions away from
the pores. For each choice of the global latent, we vary the styles and obtain
the images shown in the second column of Fig. 4(iii). The structural
environments remain very similar between the set of images shown in each row,
however, the magnitudes of the heat flux are varied. This result implies that
the global latent variables encode the environment (e.g., structure) and the
styles are highly related to the physical phenomenon of interest (e.g., heat
flux maps). This insight can offer great practical advantage. For example,
styles can inform changes of heating conditions or reveal strategies to design
structural features such that heat flux can be guided away from sensitive
regions of the microstructures. Thus, FluxGAN model trained on style-based
descriptions offers great practical advantage for designing microstructures
that satisfy target thermal properties.

Figure 6: Comparison of structural features of generated (blue) and
experimental images (red). We show the mean dependency (solid lines) and the
range (shaded regions) for the chord-length distribution and the two-point
correlation functions, in the second and the third column, respectively.

## Generation Phase

### Unguided Generation

Figure 5 shows representative inputs (first column) and the corresponding
images generated as output of the model (second and third column). The model
input specifies the overall height and width of the image, the types and the
number of styles, the spatial positioning of styles, and merging functions to
obtain smooth interfaces between the regions. We show visual representations
of the four input maps with different spatial arrangements of styles, in the
first column of Fig. 5(i-iv). The input maps include one, two, three, and six
randomly chosen styles, respectively. Note that the number of styles
determines the global latent variables, and consequently, the overall
appearance of the generated coating images, which is consistent with
InfinityGAN spatial style fusion implementation. The images shown in the
second and the third column of Fig. 5(i-iv) highlight the structural diversity
in generated images for same input maps. We select 540 $1024\text{px}\times
1024\text{px}$ images generated with single-style and 50 $512\text{px}\times
512\text{px}$ images generated with each multi-style input map, and validate
against experimental images using the procedure discussed in the following
section. Although we show structural images in Fig. 5, the model outputs are
RGB images that contain both structural and heat flux information. We only
show the structural channel for the ease of discussion.

### Validation of Structural Information

We compare the structural features of the generated images with those of the
experimental BSE images. We analyze three features: pore size distribution,
chord-length distribution, and two-point correlation functions. Figure 6(i-iv)
shows the comparison between the features of the experimental images and the
images generated from one, two, three and six styles input maps, respectively.
The first column shows the density of pores of given area, defined by the
ratio between the number of pores with the specific area and the total number
of pores. We calculate the pore sizes by counting the number of black pixels
inside each pore. The presence of the black pores enclosed within the gray
material region can be particularly noted from the experimental image shown in
Fig. 2(ii). The smallest pore is of size 1px $\times$ 1px $\sim$
0.07$\upmu$m2. This size pores are the highest in number in the coatings as
can be noted from Fig. 6(a,d,g). This could be a systematic artifact of the
images. Overall, the density of different size pores in all four classes of
images match with those in the experimental images. This implies that the
model is able to produce the anisotropic pore distributions characteristic of
the original thermal spray coatings shown in the BSE images.

Figure 7: Comparison of generated heat flux (blue) and FEM results (red). (i)
Full size generated image with single-style input map. Average generated heat
flux (blue) along (a) horizontal and (b) and vertical directions, and compared
to FEM results (red). Selected red boxed region is enlarged for comparison
between (ii) generated and (iii) FEM results. (iv) (a) Structural and (b-c)
heat flux pixel intensity distributions of 540 similarly generated images.

The porous coatings include numerous interfaces between the material phases
and the pore phases. If an imaginary line is drawn in the porous medium, the
line will intersect multiple material-pore interfaces. The lengths between
these intersections are referred to as chord lengths in the respective phases.
The chord length distribution function is then defined by the probability of
finding a chord of given length in one of the phases [24, 25]. These functions
provide a quantitative measure of the structure of the porous media. The
second column of Fig. 6 shows the comparison of the chord-length distributions
in material phases of experimental (red lines) and generated images (blue
lines). The solid and the dashed lines represent the chord-length
distributions in the $x$ and $y$ direction, respectively. The results show
that there is higher probability of finding longer chords in the $x$ direction
compared to the $y$ direction in the images. This corresponds to the presence
of layer-like features that can be noted from the experimental images, as
shown in Fig. 2. The layer-like structures form along the vertical ($y$)
direction when the droplets of melted material are sprayed in a regular
interval during the thermal spray process. We show the comparison between the
spatial two-point correlation functions in the third column of Fig. 6. The
spatial correlation function between two points $\bf{r}_{1}$ and $\bf{r}_{2}$
is defined as $\langle f({\bf x}+{\bf r_{1}})f({\bf x}+{\bf r_{2}})\rangle$
[26, 25], where and $\langle..\rangle$ indicates a volume average over the
spatial coordinate ${\bf x}=(x,y)$. The characteristic function $f(\bf{x})$ is
either zero or one if the point lies in the material or the pore phase. We
note a close match between the two-point correlation functions for all four
cases. The result for the image generated with the six-style-input map is the
closest to the one observed in BSE images. This can be attributed to the
higher diversity of microstructures present in the images generated with six-
style-input maps.

Figure 8: Image outpainting with holistic heat flux distribution as input. (i)
Bottom air region (green box) is supplied as input. (ii) Outpainted image
(left) and corresponding FEM results (right). (iii) Interface region of (ii).
(iv) (a) Horizontal and (b) vertical averages of generated (blue) heat flux in
input region, compared to input (green) and FEM results (red). MAEs for ML-
INPUT (blue) and ML-FEM (red) comparisons reported for microstructure shown in
(ii). MAEs averaged over seven systems are shown in parentheses

### Heat Flux Data Validation

We validate the heat flux of the generated coatings against values computed
with FEM for the same microstructures. It is essential that the structural
environments and the applied heating conditions of the FEM models are similar
to those of the generated coatings. As we illustrate in the above sections, we
generate coatings with input maps that include spatial arrangement of random
choice of styles. The styles are closely associated with the heat flux
patterns in different regions of the coatings. The coatings generated with
different input maps not only show highly variable heat flux in the
microstructures but also different heating conditions in the surface regions.
The heating conditions in coatings generated with arbitrarily chosen input
maps may not be consistent with the fixed heating conditions imposed in the
FEM models. As we discuss in the “Computation of Thermal Properties” section,
we simulate a temperature bias of 400K to compute heat flux in all the BSE
masks using FEM. To maintain similar heating conditions in generated
structures and the FEM models, we provide only single-style-maps as input and
generate coating images. We find that the trained FluxGAN model is able to
generate full coating images that include material and air regions from
sufficiently large ($1024\text{px}\times 1024\text{px}$) single-style-maps. We
show an example generated microstructure and the associated heat flux in Fig.
7(i). The yellow vector arrows represent the resultant of the horizontal
($Q_{x}$) and the vertical ($Q_{y}$) heat flux components. We produce masked
images from the generated structural channel and compute the heat flux through
the masked images using FEM calculations. We compute the average heat flux
along the horizontal and the vertical directions of the coatings from the
$Q_{x}$ and $Q_{y}$ values of the 1024 pixels of the square image. We show the
comparison between the average generated (blue) and FEM-obtained (red) heat
flux along the horizontal and the vertical directions, in Fig. 7(i)(a) and
(b), respectively. The generated heat flux is in good agreement with FEM
results, with mean absolute errors (MAE) of 0.009-0.012 MW/m2 for $Q_{x}$ and
0.013-0.020 MW/m2 for $Q_{y}$ components, respectively. Note that the average
$Q_{x}$ is approximately zero and the average $Q_{y}$ is always negative. The
$Q_{y}$ components have a dominant contribution to the overall heat flux,
since we assume that the temperature gradient is applied along top-to-bottom
(negative $y$) direction. We show an enlarged image of the $100\text{px}\times
100\text{px}$ red boxed section within the large $1024\text{px}\times
1024\text{px}$ in Fig. 7 (ii). The corresponding masked image and the FEM
result in shown in Fig. 7 (iii). The close match between the heat flux
patterns can be particularly noted from the two images. We analyze 540
$1024\text{px}\times 1024\text{px}$ images generated with single-style-input
maps. We show the pixel intensity distributions for the three channels in
these 540 generated images in Fig. 7(iv). The distributions show high
diversity and are consistent with those in the training set, see Fig. 3(b-d).

### Guided Generation

Here, we illustrate how our FluxGAN model provides an expedited approach to
design new coatings with target thermal properties. In the conventional
approach for designing new coatings, one typically fabricates the coatings and
then estimates the physical properties using experiments or computational
modelling. The process is repeated till the fabricated microstructures satisfy
target physical properties, as we outline in Fig. 1(ii). In contrast, the
FluxGAN model directly guides the inverse design of coatings that satisfy
target thermal properties. Given a target thermal property, the model
generates rational designs of new microstructures that satisfy the properties.
We demonstrate the targeted generation approach in Fig. 8. We provide the
model with the description of physical phenomena in a particular region, e.g.,
the heat flux near the heat sink, indicated by the green boxes in the images
of Fig. 8(i). The model input is a single 80px $\times$ 587px RGB image that
we extract from one of the 410px $\times$ 587px images. The input image only
contain information about the heat flux in the boxed air region where no
material is present, referred to as $Q$ INPUT. The model uses outpaintaing
technique to generate the microstructure, the top air region, and the
associated heat flux ($Q$ ML) throughout the large 400px $\times$ 587px size
images. We show the resulting images in Fig. 8(i-ii). The model uses GAN
inversion [27, 28, 29, 30, 31, 32] to find the latent variables that match the
input image and conditional coordination to independently generate spatially
consistent micro-patches [17]. Note that the model is capable of generating
different coating images shown in Fig. 8(i-ii), when supplied with the same
input image. This is possible by selecting different latent variables during
outpainting procedure.

Figure 9: Image outpainting with artificially constructed heat flux
distribution as input. (i) Input image. (ii) Example outpainted images (left)
and corresponding FEM results (right). (iii) Interface region of
representative image shown in (ii). (iv) (a) Horizontal and (b) vertical
averages of generated (blue) heat flux in input region, compared to input
(green) and FEM results (red). MAEs for ML-INPUT (blue) and ML-FEM (red)
comparisons reported for bottom microstructure of (ii). Average MAEs of the
three systems are shown in parentheses.

Finally, we calculate the heat flux associated with the generated
microstructures using FEM, $Q$ FEM, for the same temperature bias of 400 K. We
show an outpainted image and the corresponding FEM results for the masked
image in the left and the right columns of Fig. 8(ii), respectively. We show
the comparison between $Q$ INPUT (green), $Q$ ML (blue) and $Q$ FEM (red) in
Fig. 8(iv). Figure 8 (iv)(a) and (b) show the average horizontal and vertical
heat flux, computed from the $Q_{x}$ and $Q_{y}$ values of the 80 and 587
pixels of the input region, respectively. The shaded regions represent the
range of heat flux values observed in all seven generated images. The MAEs
represent the average absolute distance between the two plotted dependencies.
The FluxGAN generated heat flux match with $Q$ INPUT and FEM results, with
great accuracy. The errors range from 0.003 to 0.01 MW/m2 for all the results,
or around 2% of the average heat flux value in the input region. The $Q_{y}$
components have a dominant contribution to the overall heat flux, compared to
$Q_{x}$. This is expected since we assume that the temperature gradient is
applied along top-to-bottom ($y$) direction. The $Q_{y}$ ML values show larger
deviation at the boundaries since the FluxGAN model is unaware of insulating
boundary conditions while the FEM results are obtained using this restriction.
We marked three points of interest in the interface region with magenta
circles, as shown in Fig. 8(iii). These points represent the location of the
cusps at the bottom material-air interface and are next to dips at the
surfaces. They are closest to the heat sink, as can be noted by following the
thin green line that marks the top boundary of the input region. The heat flux
arrows at this locations (Fig. 8(iii)) indicate that the heat is being guided
sideways from the cusps, with the magnitudes being the highest among all
values in the bottom air region (Fig. 8 (iv)(b)). Such characteristic heat
flux patterns are associated with the different color environments around
these points in all the input green boxed regions of Fig. 8(i). Consequently,
similar cusp points can be noted from all generated microstructures of Fig.
8(i) and (ii). The common presence of these points in all generated
microstructures refer to the fact that they all are rational designs that
satisfy the target input heat flux.

To further challenge the FluxGAN model, we test its ability to generate images
for an artificially constructed input. We construct the input image by
periodically extending the red-boxed 80px $\times$ 80px region marked in the
top panel of Fig. 9(i). The full panel shows the resulting input image of size
80px $\times$ 587px. We show three outpainted images and the corresponding FEM
results in Fig. 9(ii). We show the comparison of the average horizontal and
vertical heat flux, computed from the $Q_{x}$ and $Q_{y}$ values of the 80 and
587 pixels of the input region, in Fig. 9(iv)(a) and (b), respectively. The
shaded regions represent the range of heat flux values observed in three
different images. The periodic nature of heat flux can be particularly visible
from the average vertical heat flux, shown in Fig. 9(iv)(b). The FluxGAN
generated heat flux match with $Q$ INPUT and FEM results, with great accuracy.
The MAEs of the FluxGAN results range from 0.002 to 0.011 MW/m2 when compared
with the INPUT and from 0.007 to 0.012 MW/m2 when compared with the FEM
results, respectively. Average vertical heat flux agrees with the FEM results
in the input region within 3% error margin. These results demonstrate that the
model is able to generate rational microstructures that best fit the criterion
imposed by the input image. Note that the generated microstructures are highly
diverse away from the periodic input region and do not show any visible
repetitions. The diversity of generated images indicates that the model
provides multiple designs of microstructures to satisfy target conditions.

### Extension to Three-Dimensional Domain

Figure 10: Microstructure representation in 3D domain. Two discriminators are
implemented to test perpendicular RGB image slices for (i) surface and (ii)
material regions. Representative volumes generated for (iii) surface and (iv)
material regions with superimposed heat flux. (v) Two representative 3D
microstructures generated by combined FluxGAN-SliceGAN approach.

Although our study primarily focuses on 2D microstructures, here we discuss
how we can leverage the FluxGAN model to generate 3D microstructures with high
structural diversity. GANs can be trained to generate 3D volumes, which are
useful for design optimization and detailed analysis of structural features
and physical processes [33, 4, 34]. These models can be used for rapid
generation of complex 3D structures over conventional approaches. A recent
article used a bounded random walk algorithm to generate an interface,
followed by seeding of random size particles to produce a two-phase material,
resulting in large 128px $\times$ 10000px images. The images were used to
train SliceGAN [33] algorithm and output 3D configurations. However, the lack
of diversity in such approach allows the generator to trick the
discriminators, resulting in unrealistic 3D representations. We demonstrate
that this issue can be resolved by training the GAN models with images that
include both structural and physical phenomena information. We propose a
combined approach that implements both the FluxGAN and the SliceGAN algorithms
to generate complex 3D microstructures and associated heat flux environments
with high diversity. The SliceGAN approach generates a volume by assimilating
2D slices, each of which is refined to match the training set images. However,
a large number of diverse 2D images is required for successful training,
especially to generate complex, anisotropic 3D materials with high density of
interfaces. We leverage the FluxGAN model to generate two large 64px $\times$
320000px images (17.4$\upmu$m $\times$ 8.7cm coatings) for the surface and the
material regions. We make random selections of 64px $\times$ 64px patches from
the generated images and pass them to two perpendicular discriminators of
SliceGAN. Figure 10(i) and (ii) show two perpendicular RGB slices for the
surface and the material regions, respectively. Note that the heat flux
components (red and green channels) are continuous across the two
perpendicular slices and the blue channel produces matching structure in both
the perpendicular planes.

The objective of the generator of the SliceGAN model is to generate realistic
volumes in which the 2D slices along $X$ and $Y$ directions pass the
discriminator test. Note that the outputs from the generator include
information in five channels: [BSE, $Q_{X}$, $Q_{Y}$, $Q_{X}^{\perp}$,
$Q_{Y}^{\perp}$]. The first channel contains structural information as learned
from the experimental BSE images. The other two sets of channels include heat
flux data. ($Q_{X}$, $Q_{Y}$) and ($Q_{X}^{\perp}$, $Q_{Y}^{\perp}$) refer to
the heat flux components of the two perpendicular slices, represented in
graphical domain. The generated information is passed to the two
discriminators. Each of the two perpendicular discriminators receive three-
channel RGB images and share a common structural channel (BSE). However, each
discriminator receives a separate set of heat flux data, ($Q_{X}$, $Q_{Y}$) or
($Q_{X}^{\perp}$, $Q_{Y}^{\perp}$). The model assimilates 64 slices in each of
two perpendicular directions to produce 64 $\times$ 64 $\times$ 64 voxel
volumes, as shown in Fig. 10(iii) and (iv). For the generated 3D
microstructures, we obtain the heat flux along the $Z$-direction, $Q_{Z}$, by
averaging the vertical heat flux components $Q_{Y}$ and $Q_{Y}^{\perp}$
obtained from both discriminators. The resulting volumes have a four-channel
representation: [BSE, $Q_{X}$, $Q_{X}^{\perp}$, $Q_{Z}$]. Finally, the
SliceGAN model outputs two representative 3D configurations of size 64
$\times$ 400 $\times$ 400 voxels using the generated volumes, as shown in Fig.
10(v). The resultant microstructures shows great structural diversity in both
the surface landscape as well as the cross section of inner material region,
which are not accessible in the experimental and training images. The combined
FluxGAN-SliceGAN approach shows remarkable ability to infer 3D representation
of microstructure and associated physical phenomenon from a single 2D image.
We attribute the high quality of the representation to both large size of the
training image and the heat flux data included in the image channels.

## Summary and Outlook

We propose a generative ML model, FluxGAN, capable of generating arbitrary
large images of microstructures and associated heat flux data. The model
generates the information about heat flux that will be observed within the
microstructures and the outside heat source or sink regions, if temperature
gradients are applied externally. We train the model with RGB images in which
the different color channels contain graphical domain representation of
structural information from experimental images and heat flux data obtained
from FEM calculations. In these images, the information about the structural
features are superimposed with heat flux patterns that correspond to the
structural features. During training, the model learns about the different
combinations of structural features and heat flux patterns, referred to as
styles, in an unsupervised manner. Once trained, the model generates large
images for given inputs maps that specify spatial distribution of styles to be
included in the output images. The generated microstructures show high
diversity and include structural features that have characteristics similar to
those present in the experimental images. We compare the generated heat flux
with the results from separate FEM analyses on the corresponding generated
microstructures. The FluxGAN generated heat flux show good agreement with FEM
results. Furthermore, we demonstrate that the image generation can be guided
by providing partial information to the model. We provide the model with the
heat flux near the bottom heat sink region. The model shows remarkable ability
to generate large microstructures, heat source regions and associated heat
flux that are consistent with the input. The generated heat flux values lie
within 3% of those obtained with separate FEM simulations. Thus, our FluxGAN
model offers an expedited approach for inverse design of microstructures with
target thermal properties. The images show high structural diversity as well.
We illustrate that the FluxGAN generated images can be leveraged to produce
complex 3D microstructures, in combination with the SliceGAN architecture. We
generate 0.1 meter long continuous and structurally diverse 2D structural
images and their associated heat flux environment, to construct the 3D
representations.

In conclusion, FluxGAN offers fast, inexpensive, and reliable approach for
design and analysis of 2D and 3D microstructures. The model is capable of
generating arbitrary large environments with constant computation cost. This
aspect is particularly advantageous over the FEM approach where memory usage
scales linearly with the number of degrees of freedom. Our approach is general
and can be extended to any continuous physical phenomena represented in
graphical domain. The ability of the model to render arbitrary large
environments while trained on small size patches offers great practical
advantage. For example, the model will be highly effective in providing
description of physical phenomena at the micro-meso or even macro scale, or
guiding characterization and fabrication of complex devices, such as
microelectronic chips with extreme design points.

## Data availability

The authors declare that the data supporting the findings of this study are
available within the paper and its Supplementary Information files.

## Acknowledgements

We are indebted to Dr. M. R. Kracum for providing us with the two-dimensional
backscatter electron images of the niobium thermal sprayed coatings, sharing
valuable insights regarding the outstanding challenges of thermal spray
coatings, and several helpful discussions. We gratefully acknowledge funding
from the research partnership initiative between the College of Engineering &
Applied Science at the University of Colorado Boulder and the DOE-Sandia
National Laboratories, Albuquerque, NM. This work utilized the Summit
supercomputer, which is supported by the National Science Foundation (awards
ACI-1532235 and ACI-1532236), the University of Colorado Boulder, and Colorado
State University. The Summit supercomputer is a joint effort of the University
of Colorado Boulder and Colorado State University.

## Author contributions

A.K.P and M.S. contributed to the acquisition and the analysis of data and the
creation of new scripts used in the study. S.N. contributed to the conception
and the design of the work, the interpretation of data, drafting and revision
of the article.

## Competing interests

The authors declare no competing interests.

## References

  * Yang  _et al._ [2020] L. Yang, D. Zhang, and G. E. Karniadakis, Physics-informed generative adversarial networks for stochastic differential equations, SIAM Journal on Scientific Computing 42, A292 (2020).
  * Jiang and Farimani [2020] C. Jiang and A. B. Farimani, Deep learning convective flow using conditional generative adversarial networks, arXiv preprint arXiv:2005.06422 (2020).
  * Jiang  _et al._ [2021] H. Jiang, Z. Nie, R. Yeo, A. B. Farimani, and L. B. Kara, StressGAN: A generative deep learning model for two-dimensional stress distribution prediction, Journal of Applied Mechanics 88 (2021).
  * Hsu  _et al._ [2021] T. Hsu, W. K. Epting, H. Kim, H. W. Abernathy, G. A. Hackett, A. D. Rollett, P. A. Salvador, and E. A. Holm, Microstructure generation via generative adversarial network for heterogeneous, topologically complex 3d materials, Jom 73, 90 (2021).
  * Nguyen  _et al._ [2022] P. C. Nguyen, N. N. Vlassis, B. Bahmani, W. Sun, H. Udaykumar, and S. S. Baek, Synthesizing controlled microstructures of porous media using generative adversarial networks and reinforcement learning, Scientific reports 12, 9034 (2022).
  * Karras  _et al._ [2019] T. Karras, S. Laine, and T. Aila, A style-based generator architecture for generative adversarial networks, in  _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ (2019) pp. 4401–4410.
  * Huang and Belongie [2017] X. Huang and S. Belongie, Arbitrary style transfer in real-time with adaptive instance normalization, in  _Proceedings of the IEEE international conference on computer vision_ (2017) pp. 1501–1510.
  * Karras  _et al._ [2020] T. Karras, M. Aittala, J. Hellsten, S. Laine, J. Lehtinen, and T. Aila, Training generative adversarial networks with limited data, Advances in neural information processing systems 33, 12104 (2020).
  * Wang  _et al._ [2006] J. Wang, J. K. Carson, M. F. North, and D. J. Cleland, A new approach to modelling the effective thermal conductivity of heterogeneous materials, International journal of heat and mass transfer 49, 3075 (2006).
  * Gurumurthy  _et al._ [2017] S. Gurumurthy, R. Kiran Sarvadevabhatla, and R. Venkatesh Babu, Deligan: Generative adversarial networks for diverse and limited data, in  _Proceedings of the IEEE conference on computer vision and pattern recognition_ (2017) pp. 166–174.
  * Gulrajani  _et al._ [2017] I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and A. C. Courville, Improved training of wasserstein gans, Advances in neural information processing systems 30 (2017).
  * Multiphysics [1998] C. Multiphysics, Introduction to comsol multiphysics®, COMSOL Multiphysics, Burlington, MA, accessed Feb 9, 32 (1998).
  * Abdal  _et al._ [2020] R. Abdal, Y. Qin, and P. Wonka, Image2StyleGAN++: How to edit the embedded images?, in  _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ (2020) pp. 8296–8305.
  * Liu  _et al._ [2021] A. Liu, R. Tucker, V. Jampani, A. Makadia, N. Snavely, and A. Kanazawa, Infinite Nature: Perpetual view generation of natural scenes from a single image, in  _Proceedings of the IEEE/CVF International Conference on Computer Vision_ (2021) pp. 14458–14467.
  * Shaham  _et al._ [2019] T. R. Shaham, T. Dekel, and T. Michaeli, SinGAN: Learning a generative model from a single natural image, in  _Proceedings of the IEEE/CVF international conference on computer vision_ (2019) pp. 4570–4580.
  * Shocher  _et al._ [2019] A. Shocher, S. Bagon, P. Isola, and M. Irani, InGAN: Capturing and retargeting the” dna” of a natural image, in  _Proceedings of the IEEE/CVF International Conference on Computer Vision_ (2019) pp. 4492–4501.
  * Lin  _et al._ [2019] C. H. Lin, C.-C. Chang, Y.-S. Chen, D.-C. Juan, W. Wei, and H.-T. Chen, COCO-GAN: Generation by parts via conditional coordinating, in  _Proceedings of the IEEE/CVF international conference on computer vision_ (2019) pp. 4512–4521.
  * Lin  _et al._ [2021] C. H. Lin, H.-Y. Lee, Y.-C. Cheng, S. Tulyakov, and M.-H. Yang, InfinityGAN: Towards infinite-pixel image synthesis, arXiv preprint arXiv:2104.03963 (2021).
  * Park  _et al._ [2019] J. J. Park, P. Florence, J. Straub, R. Newcombe, and S. Lovegrove, DeepSDF: Learning continuous signed distance functions for shape representation, in  _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ (2019) pp. 165–174.
  * Mescheder  _et al._ [2019] L. Mescheder, M. Oechsle, M. Niemeyer, S. Nowozin, and A. Geiger, Occupancy Networks: Learning 3d reconstruction in function space, in  _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ (2019) pp. 4460–4470.
  * Mildenhall  _et al._ [2021] B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng, NERF: Representing scenes as neural radiance fields for view synthesis, Communications of the ACM 65, 99 (2021).
  * Chen  _et al._ [2021] Y. Chen, S. Liu, and X. Wang, Learning continuous image representation with local implicit image function, in  _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ (2021) pp. 8628–8638.
  * Heusel  _et al._ [2017] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, GANs trained by a two time-scale update rule converge to a local nash equilibrium, Advances in neural information processing systems 30 (2017).
  * Torquato and Lu [1993] S. Torquato and B. Lu, Chord-length distribution function for two-phase random media, Physical Review E 47, 2950 (1993).
  * Torquato [2013] S. Torquato,  _Random heterogeneous materials_ (Springer., 2013).
  * Berryman and Blair [1986] J. G. Berryman and S. C. Blair, Use of digital image analysis to estimate fluid permeability of porous materials: Application of two-point correlation functions, Journal of applied Physics 60, 1930 (1986).
  * Creswell and Bharath [2018] A. Creswell and A. A. Bharath, Inverting the generator of a generative adversarial network, IEEE transactions on neural networks and learning systems 30, 1967 (2018).
  * Ma  _et al._ [2018] F. Ma, U. Ayaz, and S. Karaman, Invertibility of convolutional generative networks from partial measurements, Advances in Neural Information Processing Systems 31 (2018).
  * Abdal  _et al._ [2019] R. Abdal, Y. Qin, and P. Wonka, Image2StyleGAN: How to embed images into the StyleGAN latent space?, in  _Proceedings of the IEEE/CVF international conference on computer vision_ (2019) pp. 4432–4441.
  * Bau  _et al._ [2020] D. Bau, H. Strobelt, W. Peebles, J. Wulff, B. Zhou, J.-Y. Zhu, and A. Torralba, Semantic photo manipulation with a generative image prior, arXiv preprint arXiv:2005.07727 (2020).
  * Zhu  _et al._ [2020] J. Zhu, Y. Shen, D. Zhao, and B. Zhou, In-domain GAN inversion for real image editing, in  _European conference on computer vision_ (Springer, 2020) pp. 592–608.
  * Cheng  _et al._ [2022] Y.-C. Cheng, C. H. Lin, H.-Y. Lee, J. Ren, S. Tulyakov, and M.-H. Yang, InOut: Diverse image outpainting via gan inversion, in  _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ (2022) pp. 11431–11440.
  * Kench and Cooper [2021] S. Kench and S. J. Cooper, Generating three-dimensional structures from a two-dimensional slice with generative adversarial network-based dimensionality expansion, Nature Machine Intelligence 3, 299 (2021).
  * Henkes and Wessels [2022] A. Henkes and H. Wessels, Three-dimensional microstructure generation using generative adversarial neural networks in the context of continuum micromechanics, Computer Methods in Applied Mechanics and Engineering 400, 115497 (2022).

## Supplementary Materials

Here we discuss and compare computational costs associated with ML model
generation and FEM validation. We generate a set of images using periodically
extended 80$\times$587 pixels input image discussed in section Guided
Generation. We build a progression of input images extending the length by up
to 6 times. The resulting outpainted images have the sizes 400px $\times$
(N$\times$587)px, with N=1,2,3,4,5,6. We demonstrate convergence of FEM
validation mean absolute error (MAE) for the 400px $\times$ 3522px outpainted
image in Fig. 11(i). We show MAE for the total heat flux magnitude, namely,
$Q=\sqrt{Q_{x}^{2}+Q_{y}^{2}}$,
MAE=$\sum_{i=1}^{H\times W}|Q_{i}^{ML}-Q_{i}^{FEM}|/(H\times W)$,
where $Q_{i}$ is the total heat flux magnitude at a given pixel, $H$ and $W$
are the height and width of an image. MAE converges quickly with the number of
degrees of freedom (DoF) used for FEM calculation and saturates bellow
0.05MW/m2 after 1 million DoF. For the further calculations we fix DoF around
that value. We also show calculation time (red solid line) and memory usage
(green solid line) for FEM validation step. We observe their linear scaling
with DoF. For comparison, the generation time (red dashed line) and memory
usage (green dashed line) are also shown for ML model. In Fig. 11(ii) we
compare ML generation and FEM calculation costs. Both FEM (red solid line) and
ML (red dashed line) have similar linear time scaling with image size for the
DoF fixed around 1 million. However, the memory usage is significantly higher
for FEM (green solid line) compared to ML (green dashed line). We also show
that MAE (blue solid line) stays consistently small (bellow 0.05MW/m2) for all
image sizes. The highest time cost is associated with latent inversion (black
dashed line). The low memory costs associated with ML are due to patch-by-path
generation approach, where each patch (37px $\times$ 37px) is generated
independently to produce large scale images. The time costs for ML generation
shown here are for generating one patch at a time. This can be improved
significantly since the model is capable of generating patches in batches,
allowing for the highly parallelized output.

Figure 11: Supplementary Materials Computation and generation costs. (i) MAE
and computational costs as a function of number of degrees of freedom for
400px $\times$ 3522px image. MAEs are calculated for the magnitude of heat
flux. For the time (red) and memory (green) axes solid lines correspond to
FEM, dashed lines correspond to ML. (ii) MAE and computational costs as a
function of image length. Color lines show time (red) and memory (green) costs
of FEM validation (solid) and ML generation (dashed). MAEs for heat flux
magnitude are shown in blue. Latent inversion time cost is shown in black
dashed line.