# Enhancing Mechanical Metamodels with a Generative Model-Based Augmented
Training Dataset

Hiba Kobeissi
Department of Mechanical Engineering
Boston University
Boston, MA 02215
<EMAIL_ADDRESS>
& Saeed Mohammadzadeh∗
Department of Systems Engineering
Boston University
Boston, MA 0215
<EMAIL_ADDRESS>
& Emma Lejeune
Department of Mechanical Engineering
Boston University
Boston, MA 02215
<EMAIL_ADDRESS>
co-first authorscorresponding author

###### Abstract

Modeling biological soft tissue is complex in part due to material
heterogeneity. Microstructural patterns, which play a major role in defining
the mechanical behavior of these tissues, are both challenging to
characterize, and difficult to simulate. Recently, machine learning (ML)-based
methods to predict the mechanical behavior of heterogeneous materials have
made it possible to more thoroughly explore the massive input parameter space
associated with heterogeneous blocks of material. Specifically, we can train
ML models to closely approximate computationally expensive heterogeneous
material simulations where the ML model is trained on datasets of simulations
with relevant spatial heterogeneity. However, when it comes to applying these
techniques to tissue, there is a major limitation: the number of useful
examples available to characterize the input domain under study is often
limited. In this work, we investigate the efficacy of both ML-based generative
models and procedural methods as tools for augmenting limited input pattern
datasets. We find that a Style-based Generative Adversarial Network with an
adaptive discriminator augmentation mechanism is able to successfully leverage
just 1,000 example patterns to create authentic generated patterns. And, we
find that diverse generated patterns with adequate resemblance to real
patterns can be used as inputs to finite element simulations to meaningfully
augment the training dataset. To enable this methodological contribution, we
have created an open access Finite Element Analysis simulation dataset based
on Cahn-Hilliard patterns. We anticipate that future researchers will be able
to leverage this dataset and build on the work presented here.

_Keywords_ machine learning $\cdot$ mechanics $\cdot$ surrogate modeling
$\cdot$ heterogeneous materials

##  1 Introduction

Establishing models that realistically capture the biomechanical behavior of
soft tissue is a challenging yet crucial endeavor [1, 2]. High fidelity
mechanical models are needed for tasks such as surgical simulation [3, 4, 5],
patient-specific procedure planning [6, 7], modeling of in-vivo biological
mechanisms [8, 9], and inverse material characterization [10, 11]. Capturing
the mechanical behavior of soft tissue is challenging because soft tissues are
often highly nonlinear and anisotropic, they can exhibit a nonlinear
stiffening response, they often undergo large deformations, and they have a
complex hierarchical structure [1, 12, 13, 14]. For example, at the
microstructural level, soft tissue may contain components such as fibers with
a preferred direction which give rise to highly anisotropic material behavior
on the macroscale [14]. In addition to complex constitutive behavior,
biological materials are also challenging to model because they tend to be
highly heterogeneous [13, 15]. As such, developing faithful mechanical models
of soft tissues and numerically implementing them (e.g., in the Finite Element
setting [16]) is both challenging and typically quite computationally
expensive [2, 10, 14, 17, 18, 19]. Notably, both the exact values of the
mechanical properties of biological tissue and their heterogeneous
distribution in space are often uncertain [20, 21]. Therefore, in order to get
a true picture of tissue behavior, it is necessary to run multiple simulations
that capture the range of relevant input parameters [10]. In this context,
there has been substantial recent interest in reducing the computational cost
of these numerical simulations at the cost of marginal decrease in the
simulation accuracy [22].

Markedly, there has been recent interest in using machine learning tools to
create computationally inexpensive data-driven models of soft biological
tissue in particular [23], and for various biomedical applications in general
[24, 25, 26, 27]. In previous work by our group and others [28, 29, 30, 31,
32, 33, 34, 35], metamodels, or surrogate models [36], developed with
supervised machine learning algorithms and multi-fidelity mechanical datasets
have been used successfully to predict the mechanical behavior of
heterogeneous materials via single and full-field Quantities of Interest
(QoIs) (e.g. strain energy, displacement/strain fields, damage fields). For
example, Tonutti et al. [22] used the results of Finite Element Analysis (FEA)
simulations in conjunction with artificial neural networks and support vector
regression to develop computationally inexpensive patient-specific deformation
models for brain pathologies. In addition, Salehi et al. [37] trained graph
neural networks with FEA simulation results to speed-up the approximation of
soft tissue deformation with acceptable loss of accuracy for neurosurgical
applications. And, in Tac et al. [23], fully connected neural networks were
trained with high-fidelity biaxial test data and low-fidelity analytical
approximations to derive a data-driven anisotropic constitutive model of
porcine and murine skin. Notably, due to the limited availability of both
experimental data and high fidelity simulation data, methods that rely on
multiple data fidelities (i.e., multi-fidelity models) have been shown to be
more effective than single fidelity schemes given a small number of high
fidelity data [23, 28, 38, 39]. This is particularly true for methods that
rely on deep learning where training datasets must be large for successful
model implementation [40, 41, 42]. Though multi-fidelity methods can address
the scenario where there are limited high-fidelity simulations results, they
are not necessarily equipped to address the scenario where there is limited
information about what the training dataset should contain. For example, it is
unlikely that researchers will have tens of thousands of accurate examples of
the heterogeneous material property distribution of a given soft tissue of
interest. In this work, our goal is to systematically answer the question: is
it possible to create a meaningful training dataset that ultimately improves
the performance of a deep learning-based metamodel of heterogeneous material
given only a small number of representative examples of the relevant material
property distribution input pattern?

To address this question, we first define a benchmark problem to evaluate our
proposed machine learning approach. This is important because, at present,
there are only a small number of existing open access benchmark datasets
related to problems in solid mechanics [43, 44, 45, 46]. Furthermore, of the
available datasets, few contain a good representation of the heterogeneous
material properties most relevant to soft tissue modeling. Our benchmark
dataset, the “Mechanical MNIST Cahn-Hilliard” dataset, is a contribution to
our previously initiated “Mechanical MNIST” project where we provide
simulation results for heterogeneous materials undergoing large deformation.
The full dataset contains $104,813$ Cahn-Hilliard patterns and associated
equibiaxial extension simulations, and it is straightforward to train a deep
learning-based metamodel to predict QoI from these simulations (e.g., change
in strain energy $\Delta\Psi$). However, if we constrain ourselves to only a
small subset of these example input patterns, for example, if we limit our
knowledge to just $1\text{,}000$ example patterns, it becomes much more
challenging to effectively train a deep learning-based metamodel. With this
benchmark dataset and imposed limitation, we are able to test both the
efficacy of ML-based generative models, models that learn the data
distribution and generate plausible examples from the distribution [47], and
procedural methods at augmenting a constrained version of the available
training dataset. By comparing the results of metamodels that rely on
generated patterns to metamodels that are trained on true input patterns, we
are able to systematically evaluate the efficacy of our proposed size-limited
data augmentation approaches. We note that this premise follows from recent
work in the literature where generative models have been used to augment small
materials characterization datasets [48, 49]. Ultimately, we are able to
clearly demonstrate that leveraging the capabilities of our selected data
generation models is an effective tool for augmenting small datasets of
material property distributions in biological tissue for the purpose of
creating training datasets for ML-based metamodels.

The remainder of the paper is organized as follows. In Section 2, we begin by
introducing our “Mechanical MNIST Cahn-Hilliard” dataset. Then, we describe
our approach to training a metamodel to approximate the mechanical behavior of
the simulations, and our approach to generating synthetic input patterns to
augment the training dataset. In Section 3, we show the performance of our
generative models, and the performance of our metamodel with ML-based and
procedural augmented training dataset. We conclude in Section 4. Finally, we
note briefly that links to the code and dataset required to reproduce our work
are given in Section 5.

##  2 Methods

Here, we begin in Section 2.1 with an introduction to our “Mechanical MNIST
Cahn-Hilliard” dataset. Then, in Section 2.2, we describe our metamodeling
approach where a ML-based metamodel is used to predict a single quantity of
interest (in this case change in strain energy $\Delta\Psi$) from an array-
based representation of the input pattern. Then, in Section 2.3, we detail our
three different approaches to ML-based generative modeling of the input
pattern distribution. In Section 2.4, we introduce two additional procedural
methods for generating synthetic input patterns. And in Section 2.5, we
present the evaluation metrics that we considered to compare the performance
of the different methods that we have implemented to generate synthetic
patterns. Finally, in 2.6 we define our procedure for standard rotation-based
augmentation. We briefly note that in order to stay consistent with the
literature, the Greek letters $\lambda$ and $\mu$ refer to different constants
in Sections 2.1 and 2.5. In both cases, we provide a brief definition of each
term when it is introduced.

###  2.1 The Mechanical MNIST Cahn-Hilliard Dataset

In conjunction with our previous publications [28, 29, 30], we introduced the
“Mechanical MNIST” dataset of heterogeneous materials undergoing large
deformation. In previous iterations of the dataset, heterogeneous input domain
patterns were defined by the MNIST [50] and Fashion MNIST [51] bitmap
patterns. For this manuscript, we extend our “Mechanical MNIST” dataset
collection to include additional patterns from a different input domain
distribution that is more relevant to heterogeneous biological materials. The
input patterns for the “Mechanical MNIST Cahn-Hilliard” dataset are generated
based on Alan Turing’s model of morphogenesis [52] – a common motif during
biological development manifested in many different animal and plant patterns
such as the pigmentation of animal skins, the branching of trees and other
skeletal structures, and the distinct patterns on leaves and petals [53, 54].
We obtain these patterns by solving a nonlinear spatio-temporal fourth-order
partial differential equation (PDE) referred to as the Cahn-Hilliard equation,
that was originally proposed to describe the process of phase separation in
isotropic binary alloys [55, 56, 57].

Figure 1: a) Illustration of the spatial patterns obtained from our Cahn-
Hilliard simulations where each row corresponds to the time evolution in a
single simulation for $c_{0}=0.5$ (Case $1$), $c_{0}=0.63$ (Case $2$) and
$c_{0}=0.75$ (Case $3$) shown in the first, second and the third rows
respectively. b) A UMAP visualization [58] of a representative proportion of
our Cahn-Hilliard patterns using random_state=42, n_neighbors=30, min_dist=0.1
as training parameters. c) A schematic illustration of displacement driven
equibiaxial extension applied to a heterogeneous domain dictated by the Cahn-
Hilliard patterns. Here we show an example from Case $1$: $c_{0}=0.5$ and plot
the deformed state at the six magnitudes of applied displacement. From these
Finite Element simulations, we obtain multiple outputs including the total
change in strain energy $\Delta\Psi$ at each load step.

Our new dataset, “Mechanical MNIST Cahn-Hilliard”, contains not only Cahn-
Hilliard based two-dimensional heterogeneous input patterns, but also the
results of Finite Element simulations of these material domains subjected to
equibiaxial extension. Here we will summarize the process of creating this
dataset. Briefly, the Cahn-Hilliard equation, which is a fourth-order partial
differential equation that governs the evolution of a binary mixture, can
first be reduced to a pair of second-order equations [59, 60]. This mixed
formulation can be expressed in the weak form for the two unknown fields, $c$,
the concentration of one of the components of the binary mixture, and $\mu$,
the chemical potential of a uniform solution:

$\displaystyle\int_{\Omega}\frac{c_{n+1}-c_{n}}{t_{n+1}-t_{n}}q\,dx+\int_{\Omega}M\nabla\mu_{n+\theta}\cdot\nabla
q\,dx=0\qquad$ $\displaystyle\forall\,q\in V$ (1)
$\displaystyle\int_{\Omega}\mu_{n+1}v\,dx-\int_{\Omega}\frac{df_{n+1}}{dc}v\,dx-\int_{\Omega}\lambda\nabla
c_{n+1}\cdot\nabla v\,dx=0\qquad$ $\displaystyle\forall\,v\in V$ (2)

where $M$ is the mobility parameter, $\lambda$ is a positive scalar that
describes the thickness of the interfaces between the phases of the mixture,
$f$ is the chemical free-energy function, and $q$ and $v$ are test functions
[59, 60].

We solve the Cahn-Hilliard equations using the open source Finite Element
software FEniCS [61, 62] and run $2\text{,}072$ phase separation simulations
on a unit square domain $\Omega=[0,1]$ where each simulation differs in the
following: (1) the initial concentration $c_{0}$ with uniform random
fluctuations of zero mean and range between $-0.05$ and $0.05$, (2) the grid
size on which the initialized concentration is allowed to spatially vary, (3)
the interface thickness $\lambda$, and (4) the peak-to-valley value of the
free-energy function $f$, a symmetric double-well function. We record the
concentration parameter at multiple time steps in each simulation to obtain
$105\text{,}427$ spatial distribution patterns which broadly fall under two
qualitative types: spotted (for $c_{0}=0.63$ and $c_{0}=0.75$), and striped
(for $c_{0}=0.5$), as is expected for these types of simulations [59, 63, 64],
and store the obtained images as $400\times 400$ binary bitmaps. Example
patterns are illustrated in Fig. 1a. For further details on the underlying
theory of the Cahn-Hilliard equation and our Finite Element implementation, we
refer the reader to the supplementary document provided with the dataset (see
Section 5). As an additional step, we visualize downsampled $64\times 64$
vectors describing each Cahn-Hilliard pattern array in a two-dimensional space
using the dimension reduction technique, Uniform Manifold Approximation and
Projection (UMAP) [58], which provides us with a qualitative tool to visualize
our high-dimensional dataset input parameter space. Notably, the plot in
Fig.1b clearly reveals the two distinct clusters of patterns which is
consistent with our observation that the dataset is split between the striped
and spotted motifs.

From this collection of $105\text{,}427$ heterogeneous input patterns, we
perform a second set of Finite Element simulations where we use the input
patterns to inform the heterogeneous material property distribution of the
domain and subject it to equibiaxial extension. To accomplish this, we first
convert the binary bitmap patterns into meshed domains of two different
materials. Briefly, we detect the contours of the image features and extract
their coordinates using the OpenCV library [65]. We then translate these
coordinates into a mesh with two different subdomains, background and pattern,
using pygmsh $6.1.1$ [66], a Python implementation of Gmsh $4.6.0$ [67]. We
note briefly that from our initial collection of $105\text{,}427$ images,
$614$ images could not be processed because they exhibited either pattern
features that were too small to be detected as area domains, features that
were in very close proximity to each other, or complex hierarchical contours
that our pipeline was not able to detect and process. Thus, our final dataset
contains $104\text{,}813$ simulation results. Based on a mesh refinement
study, we chose quadratic triangular elements with a characteristic length of
$0.01$. This led to approximately $41\text{,}000$ elements in a typical
domain.

Once the material domain was meshed, we performed equibiaxial extension
simulations in FEniCs [61, 62]. Here we chose a compressible Neo-Hookean
material model defined by strain energy $\Psi$ as:

$\Psi=\frac{1}{2}\,\mu\,\big{[}\textbf{F}:\textbf{F}-3-2\ln(\det\textbf{F})\big{]}+\frac{1}{2}\,\lambda\,\big{[}\frac{1}{2}[(\det\textbf{F})^{2}-1]-\ln(\det\textbf{F})\big{]}$
(3)

where F is the deformation gradient, and $\mu$ and $\lambda$ are the Lamé
parameters equivalent to Young’s modulus $E$ and Poisson’s ratio $\nu$ as
$E=\mu\,(3\lambda+2\mu)/(\lambda+\mu)$ and $\nu=\lambda/(2(\lambda+\mu))$. We
define the Poisson’s ratio as a constant $(\nu=0.3)$, and we specify a Young’s
modulus $E$ for the background domain that is 10 times lower than the Young’s
modulus for the “stiffer” spotted and striped patterns $(E=[1,10])$. We set up
each Finite Element simulation for equibiaxial deformation so that every
external edge of the domain is extended by half of the value of given applied
displacement in the direction of the outward normal to the surface (Fig.1c).
The set of fixed displacements d go up to $50\%$ of the initial domain size
as:

$\textbf{d}=[0.0,0.001,0.1,0.2,0.3,0.4,0.5]\,.$ (4)

The output of each of the $104\text{,}813$ large deformation simulations
consisted of data on the total change in strain energy $\Delta\Psi$, total
reaction force in the $x$ and $y$ directions, and full field domain
displacement collected on a downsampled $64\times 64$ grid (Fig.1c). We chose
the size of the grid to be the smallest possible size that could be reached
without the loss of important image features. In this context, we consider the
borders of the white/dark patterns to be important features that should not be
distorted much by any operation to avoid misclassifiying the cells along the
edges into the wrong subdomain. We note that all code to implement these
simulations is shared on GitHub with access details given in Section 5.

###  2.2 Metamodel Design and Implementation

In this Section, we summarize our approach to creating metamodels for
predicting the change in strain energy $\Delta\Psi$ from the input Cahn-
Hilliard patterns. In Sections 2.3, 2.4, and 2.6, we describe the details of
our generative model-based, procedural-based and standard rotation-based
approaches that we implement to augment the training dataset.

####  2.2.1 Feedforward Convolutional Neural Network

In this paper, we are focused on using machine learning techniques for
predicting single quantities of interest ($\Delta\Psi$) from input arrays
(Cahn-Hilliard patterns). This goal is illustrated schematically in Fig.2a. To
accomplish this, we implemented a basic feedforward convolutional neural
network (CNN) consisting of a total of $9$ convolutional layers each followed
by batch normalization and rectified linear unit (ReLu) activation except for
the last $(9^{th})$ layer. For downsampling input images, we used max pooling
after the first three convolutional layers with same padding while valid
padding is used for the rest of the convolutional layers. Our network has a
total of $3\text{,}734\text{,}625$ trainable parameters. We trained the
network using the PyTorch library [68] with a batch size of $64$ for $100$
epochs. We employ an Adam optimizer [69] with learning rate $\alpha=0.01$
reduced to $0.001$ after $50$ epochs and exponential decay rates
$\beta_{1}=0.9$ and $\beta_{2}=0.999$. The output of the CNN is a single
quantity of interest (QoI) for a $64\times 64$ array input describing the
simulation input pattern. We validated our model performance through a 5-fold
cross-validation approach based on Mean Squared Error (MSE). In Section 3.2,
we report the performance of our model on test data.

####  2.2.2 Transfer Learning

Our original “Mechanical MNIST Cahn-Hilliard” dataset took approximately
$5\text{,}240$ CPU hours to generate. Rather than expending a similar level of
resources to run simulations based on generated input patterns, we decided to
employ a transfer learning approach where we leverage low fidelity simulation
data [70]. Specifically, we followed the approach outlined in our recent
publication [28] to create low fidelity simulation versions of our dataset
that are run on a coarse mesh ($64\times 64$ grid, $8\text{,}192$ elements)
with linear elements and only subject to a perturbation displacement ($0.001$)
rather than the full $50\%$ extension. With these parameters, it took
approximately $4.2$ CPU hours to generate a low fidelity dataset of
$72\text{,}000$ patterns and the corresponding strain energy values only for a
perturbation displacement. Notably, this is $0.08\%$ of the time it would take
to generate the equivalent number of high fidelity simulations described in
Section 2.1. Of course, this speed up comes at the price of introducing
numerical error that must be subsequently dealt with through transfer
learning.

Our implementation of transfer learning is a straightforward model pre-
training approach illustrated schematically in Fig.2b and described in detail
in our previous publication [28]. Part of the appeal of this approach is that
it is quite straightforward to implement. First, we train the metamodel (in
our case the CNN defined in Section 2.2.1) on the low fidelity dataset. Then,
we use this pre-trained metamodel as the weight initialization for additional
training with the high fidelity dataset. In our case, the low fidelity dataset
will contain data from up to $16\text{,}000$ simulations while the high
fidelity dataset will contain data from only $1\text{,}000$ simulations. The
ideal outcome from this approach is to end up with a metamodel that is trained
on predominantly low-fidelity data yet performs comparably to a metamodel
trained on the target high fidelity dataset. In Section 3.2, we first report
the metamodel performance on the low fidelity dataset (Fig. 5) and then in
Section 3.3 we report the performance of the low fidelity models transferred
to the high fidelity dataset via additional training with $1\text{,}000$ high
fidelity real samples (Table 1).

Figure 2: a) A schematic of our ML metamodels that are used to predict change
in strain energy $\Delta\Psi$ at a fixed level of applied displacement from
each material property distribution. b) A schematic of transfer learning
whereby a model trained on one dataset (in this case a low fidelity dataset)
is used to make predictions on another dataset (in this case a high fidelity
dataset). c) Architecture of a generative adversarial network including the
WGAN models trained in this paper. d) An illustration of the StyleGAN2 with an
adaptive discriminator augmentation (ADA) mechanism implemented in this work
as adapted from [42]. e) A schematic of combining simulations based on both
generated and real patterns to create a larger training dataset.

###  2.3 Augmenting the Training Dataset with a ML-Based Generative Model

The main focus of this paper is on developing techniques to effectively train
the metamodels described in Section 2.2 even when we have limited examples of
the relevant input patterns needed for creating our training dataset. Here we
will explore methods for leveraging limited examples of input patterns by
creating synthetic input patterns from a generative model. Briefly, we
implement a Style-based Generative Adversarial Network using adaptive
discriminator augmentation (StyleGAN2-ADA) [42], a Wasserstein Generative
Adversarial Network with weight clipping (WGAN-CP)[71], and a WGAN with
gradient penalty (WGAN-GP) [72] to generate patterns that resemble the real
striped and spotted Cahn-Hilliard patterns detailed in Section 2.1. The
architectures of the generative models explored in this work are schematically
shown in Figs. 2c and 2d. We train all three GAN models with a limited set of
$1\text{,}000$ real Cahn-Hilliard patterns (the same set of real images used
for training the metamodels). We then combine equibiaxial extension simulation
results of both generated and real patterns to create a larger training
dataset for our metamodel as shown schematically in Fig. 2e. In the remainder
of this Section, we provide an overview of GANs, describe the specific GANs
implemented in this work, and briefly present our alternative approaches for
augmenting the metamodel training dataset via procedural pattern generation
and standard rotation-based augmentation.

####  2.3.1 Generative Adversarial Networks

In the context of machine learning, generative models are models that learn
data distributions such that they can then be used to output (i.e.,
“generate”) plausible new examples [73]. Building upon earlier deep generative
models, generative stochastic networks [74] in particular, and inspired by the
work in [75, 76, 77], Goodfellow et al. [47] developed a novel framework for
generative models where the generative network is put in competition with a
discriminative network that learns to distinguish between a sample obtained
from the real data distribution and one that is generated from the model
distribution. Known as Generative Adversarial Networks (GANs), these methods
consist of training two models, a generative model $G$ and a discriminative
model $D$ simultaneously competing in a minimax two-player game fashion [47].
In this framework, $G$ is trained to capture the input data distribution by
fooling the discriminative model $D$ and maximizing the probability of the
latter mistakenly labelling a sample synthesized by $G$ as one from the
training data.

In their original form, GANs have been applied to many domains including the
MNIST dataset of handwritten digits [78, 79], the Toronto Face Database (TFD)
of human faces with expressions [80], and the miscellaneous CIFAR-10 dataset
[81] with promising results [47]. However, major drawbacks of the method
include low resolution of the generated images, relatively low variation in
the output distribution, and unstable training [82]. Furthermore, training
GANs to synthesize high-quality, high-resolution output distributions
typically requires at least $10^{5}\sim 10^{6}$ input images. Without a
dataset of this size, the training tends to diverge as the discriminator
network overfits to the small number of training data examples and can no
longer provide meaningful feedback to the generator network [42]. There have
been many approaches to modifying the original architecture and training
formulation of GANs [47] to improve their performance. Alterations to the
network structure such as the implementation of Deep Convolutional GANs
(DCGANs) [83], where the GAN model is scaled using CNN architectures, result
in more stable behavior. Other enhanced methods include Wasserstein Generative
Adversarial Networks (WGANs) and Style-based Generative Adversarial Networks
(StyleGANs) which are briefly described in Sections 2.3.2 and 2.3.3
respectively.

####  2.3.2 Wasserstein Generative Adversarial Networks

In contrast to modifying the GAN network structure as in DCGANs, Wasserstein
Generative Adversarial Networks (WGANs) improve the stability of GANs by
replacing the bin–to–bin distance function (i.e., the Jensen–Shannon
divergence) of the original architecture with a continuous loss function, the
Earth Mover (EM) or the Wasserstein-1 ($W$) distance [71]. The shortcomings of
the bin–to–bin distance functions, which generally assume an alignment between
the domains of the histograms being compared, are addressed by the more robust
cross–bin EM distance function defined as the minimal cost of a “transport
plan” to transform one distribution into the other [84, 85, 86].

As proposed, the original WGAN model [71] requires that the discriminator lie
within a 1-Lipschitz space so that $W$ is continuous everywhere and
differentiable almost everywhere. This Lipschitz constraint is enforced via
weight clipping (WGAN-CP) whereby the weights of the discriminator are
restricted to a compact space [71]. In this setting, the discriminator is no
longer trained to directly label samples as “real” or “fake,” but rather to
learn the Lipschitz function needed to compute $W$. And, as the model training
proceeds to minimize the loss function, the distance $W$ decreases, signifying
that the generated output distribution is becoming closer to the real data
distribution [72]. Although more stable compared to GANs, the performance of
WGAN-CPs was shown to be limited because: 1) small clipping thresholds lead to
vanishing gradients while larger thresholds result in exploding gradients, and
2) the discriminator is biased to converge to simplified approximations of the
Lipschitz function [72]. Improved training of WGANs was proposed by Gulrajani
et al.[72], who implement a gradient penalty method (WGAN-GP) instead of
weight clipping to constrain the discriminator gradient. WGAN-GP enforces the
Lipschitz constraint by imposing a penalty on the gradient norm if it is not
close to the theoretical value of 1.

In this work, we test the performance of WGAN-CP and WGAN-GP trained with
$1\text{,}000$ samples from our Cahn-Hilliard dataset. Using the PyTorch
library [87], we train typical convolutional feedforward neural networks for
both the generator and the discriminator networks of WGAN-CP and WGAN-GP for a
total of $23\text{,}690\text{,}498$ trainable parameters,
$12\text{,}656\text{,}257$ for the generator network and
$11\text{,}034\text{,}241$ for the discriminator network. We accomplish this
using the code published in conjunction with [88] as a starting point. We
perform no additional parameter tuning and keep all hyper-parameters at their
default values.

####  2.3.3 Style-Based Generative Adversarial Networks

A third approach to enhancing GANs involves modifying the latent space
distributions of the generator network via feature mapping, and incorporating
adaptive instance normalization (AdaIN) [89]. The AdaIN operation was first
implemented by Huang and Belongie [90] in style transfer algorithms [91];
transferring the style of one image to the content of another image.
Specifically, AdaIN first normalizes each feature map and then scales its mean
and variance according to a style input.

In these StyleGAN models, the adjustments to the traditional generator are
twofold: 1) the input latent space is mapped to a much less entangled
intermediate latent feature space via an 8-layer multilayer perceptron
network, and 2) the generator output is controlled by AdaIN processes which
are themselves controlled by learned affine transformations that concentrate
the intermediate latent space to specific styles that dictate the dominant
image features at each convolution layer [89]. The StyleGAN2 architecture was
later developed to remedy artifacts observed in StyleGAN generated images
[92]. The StyleGAN2 using adaptive discriminator augmentation (StyleGAN2-ADA)
[42] is an adaptation of StyleGAN2 specifically designed for small training
datasets. For the simplest implementations of training GANs with augmented
datasets, generated distributions are known to exhibit features that are
present in the augmented dataset, but not in the original dataset [42, 93,
94]. Therefore, to avoid this undesirable outcome, Karras et al. [42] proposed
the ADA method.

For the augmentations to be “non-leaking” (i.e., not present in the generated
examples) and for the GAN model to learn the true input distribution given an
augmented dataset, the set of applied distortions for augmentation are
required to be differentiable and belong to an invertible transformation of a
probability distribution function [42, 95]. This can be achieved for a diverse
set of possible augmentations when they are applied to the dataset with a
probability $p$, with $0<p<0.8$ [42]. However, the target value of $p$ is
sensitive to the size of the dataset and as such, setting a fixed value for it
is far from optimal. For this reason, Karras et al. [42] implemented the
discriminator augmentation method in an adaptive manner where $p$ is set to
$0$ initially and its value is automatically adjusted (increased or decreased)
based on a metric that indicates the extent by which the discriminator is
overfitting. This heuristic is obtained from the discriminator outputs for the
training and validation datasets, as well as the generated images and their
mean over a fixed number of consecutive minibatches. ADA can be implemented on
any GAN model without modifying the network architecture or increasing
training cost [42]. Notably, the StyleGAN2-ADA combination performs
exceptionally well on the limited CIFAR-10 dataset [81], thus motivating our
implementation of the approach in this work.

Here, we train the StyleGAN2-ADA model using the PyTorch library [87] with the
code provided in [42] on a small subset ($1\text{,}000$ samples) of our Cahn-
Hilliard patterns. Of the set of transformations tested in [42], we apply the
ones that contextually fit the Cahn-Hilliard dataset – geometric and color
transformations. Geometric distortions include pixel blitting, isotropic and
anisotropic scaling, fractional translation, and less frequently arbitrary
rotation. We briefly note at this point that these distortions are implemented
during the generation of synthetic patterns only and are not related to the
equibiaxial loading conditions of the Finite Element simulations performed
later once the generated data patterns are obtained. For color
transformations, the image brightness, contrast, and saturation were adjusted,
the luma axis was flipped, and the hue axis was rotated arbitrarily. We
perform no parameter tuning and keep all hyper-parameters at their default
values. In total, the generator network has $22\text{,}238\text{,}990$
trainable parameters and the discriminator network has
$23\text{,}406\text{,}849$ trainable parameters.

###  2.4 Augmenting the Training Dataset with “Procedural” and “Bernoulli”
Randomly Generated Patterns

As discussed in Section 2.3 and later depicted in Fig. 3 and Fig. 4, the three
different ML-based generative models, WGAN-CP, WGAN-GP, and StyleGAN2-ADA are
able to generate synthetic patterns relevant to the real Cahn-Hilliard
patterns without being explicitly programmed to do so. However, there is a
rich history of implementing “Procedural” algorithms for material
microstructure pattern generation [96, 97, 98, 99, 100, 101]. For example,
many researchers have created explicitly programmed algorithm that draws from
experimentally obtained probability distributions for creating and placing
microstructural features within a domain [102, 103]. These algorithms range
from quite simple (e.g., Voronoi tessellation [104, 105]) to quite complex
(e.g., feature shape and placement based on energy minimization [106]). In
this manuscript, we implement two additional pattern generation algorithms to
compare to the ML-based generative models. First, we implement a
straightforward “Procedural” algorithm where we create synthetic patterns with
spatial correlations. In Section 3, we refer to these patterns as “Procedural”
patterns. Second, we create random patterns following a “Bernoulli”
distribution without spatial correlation. In Section 3, we refer to these
patterns as “Bernoulli” patterns.

For the “Procedural” patterns, we begin with a low resolution grid, a $4\times
4$, an $8\times 8$, or a $16\times 16$ grid, and assign each of the grid
pixels an independent and identically distributed random value drawn from a
uniform distribution $\mathcal{U}$[0,1]. Using the multidimensional image
processing package in SciPy “scipy.ndimage” [107], we then increase the
resolution of the resulting grayscale random image to the desired size of
$64\times 64$ and convert the upscaled image to a binary pattern by setting a
brightness threshold. For the “Bernoulli” patterns, we obtain binary images by
simply creating a $64\times 64$ grid of zeros, and then replacing the zeros
with ones based on a probability threshold $p=0.6594$. For both types of
patterns, the value of the threshold was chosen so that the light–to–dark
ratio present in the real patterns is preserved. Notably, the “Procedural”
patterns lead to spatially correlated features while the “Bernoulli” patterns
do not.

###  2.5 Evaluation Metrics

For evaluating and comparing the performance of the implemented GANs and the
procedural methods at creating generated examples, we considered three
indicators. First, we compute the Fréchet inception distance (FID) score, a
quantitative metric to compare the resemblance between the distributions of
the generated and real images [108]. The FID, also known as Wasserstein-2
distance, is computed between the $2\text{,}048$ dimensional feature vectors,
taken as the output of the last pooling layer of the pre-trained Inception
network, of real and generated images by [108]:

$\text{FID}=\left\|\bm{\mu_{1}}-\bm{\mu_{2}}\right\|^{2}_{2}+Tr\left[\mathbf{C_{1}}+\mathbf{C_{2}}-2\left(\mathbf{C_{1}}\mathbf{C_{2}}\right)^{1/2}\right]$
(5)

where $\bm{\mu_{1}}$ and $\bm{\mu_{2}}$, and $\mathbf{C_{1}}$ and
$\mathbf{C_{2}}$ are the means and covariance matrices of the real and
generated feature vectors respectively. The lower the FID score, the higher
the similarity between the generated and the real images, with a
$\text{FID}=0$ indicating that the two sets are identical. Second, we perform
visual inspection of the generated patterns to check for the presence of any
artifacts in the generated images and confirm their resemblance to real
patterns. Finally, we perform an assessment of the diversity of the generated
patterns by comparing the change in strain energy ($\Delta\Psi$) obtained from
Finite Element simulations performed on the generated patterns to the same
quantity obtained from simulations performed on real patterns from the Cahn-
Hilliard dataset. The performance of our generative approaches is reported in
Section 3.1.

###  2.6 Note on Standard Rotation-Based Augmentation

In addition to augmenting our training dataset with generated patterns, we
further augment the training dataset of the metamodel by performing direct
transformations on both real and generated input patterns. This type of
straightforward data augmentation occurs after the real and generated input
patterns have been used to run Finite Element simulations. Because we are
considering an equibiaxial extension load case in this work, we can increase
the size of the training dataset by a factor of $4$ by applying a set of
predefined rotations ($0^{\circ}$, $90^{\circ}$, $180^{\circ}$, $270^{\circ}$)
on the input images. For all $4$ rotated scenarios, the FEA simulation output
$\Delta\Psi$ is identical, thus we can gain four data points per pattern. We
report the significance of this standard augmentation on the metamodel
performance in Section 3.3.

##  3 Results and Discussion

In this Section, we report the results of employing the methods described in
Sections 2.2,  2.3, 2.4, and 2.6 to augment a small dataset of input patterns
and train a convolutional neural network to predict the change in strain
energy $\Delta\Psi$ for a given magnitude of applied equibiaxial extension. We
begin in Section 3.1 by describing the performance of the generative models
when trained with just $1\text{,}000$ examples of real Cahn-Hilliard patterns.
Then, in Section 3.2, we demonstrate the performance of a metamodel where the
training set contains simulations based on both real and generated input
patterns. Finally, in Section 3.3, we summarize the results of our transfer
learning approach and the effect of standard rotation-based augmentations on
metamodel performance.

###  3.1 Generative Model Performance

As stated previously, we have tested three different GAN models, WGAN-CP,
WGAN-GP, and SyleGAN2-ADA, with the aim of generating input patterns from a
small training dataset of $1\text{,}000$ real Cahn-Hilliard patterns. In this
Section, we show the performance of these methods and demonstrate that the
StyleGAN2-ADA approach performs best at capturing the Cahn-Hilliard dataset.
In Fig. 3, we illustrate the performance by plotting the Fréchet Inception
Distance (FID) between $1\text{,}000$ real and $1\text{,}000$ generated
patterns with respect to the number of epochs used for training. This plot
shows that the StyleGAN2-ADA approach consistently has the lowest FID and is
thus producing patterns that are a better match to the real dataset. We note
that as expected, the calculated FID on real vs. real patterns converged to
zero as we increased the size of the comparison datasets of patterns from
$1\text{,}000$ (FID $\approx 13.3$) to $10\text{,}000$ (FID $\approx 1.7$). In
addition, we have annotated the plot in Fig. 3 with illustrated examples of
generated patterns from the three generative models. These illustrations not
only confirm the intuition that as the FID decreases the patterns in the
generated images more closely resemble those in the real dataset, but also
show that for a converged model performance, the generated patterns look quite
qualitatively realistic. Based on the higher FID for the WGAN-CP and WGAN-GP
models, and the fact that FID begins to increase as the number of epochs
increases, we conclude that both are inferior approaches when the goal is to
generate realistic patterns that closely match the original dataset. However,
we note that in terms of model training time, the StyleGAN2-ADA network is
significantly more expensive to train with the training process taking
approximately $7.5$ hours on $4$ NVIDIA Tesla V100 GPUs. In comparison, it
took approximately $0.5$ hours to train each of the WGAN-CP and WGAN-GP models
on NVIDIA GeForce RTX 3060 Ti.

In Fig. 4, we plot the percentage frequency distribution of the change in
strain energy $\Delta\Psi$ for $15\text{,}000$ low fidelity real and generated
patterns subjected to small displacement ($d=0.001$) with equibiaxial
extension Finite Element simulations. From comparing the distributions of
$\Delta\Psi$, it appears that the StyleGAN2-ADA output distribution bears the
most similarity to the real dataset. However, even though the WGAN and
“Procedural” patterns are less authentic than StyleGAN2-ADA patterns, they are
more divergent from the original $1\textit{,}000$ example real dataset while
still maintaining overlap with the real distribution of $\Delta\Psi$. Lastly,
the “Bernoulli” patterns appear only weakly relevant to the real dataset. From
performing these simulations, we now have multiple datasets of low fidelity
Finite Element simulations based on both real and generated input patterns
that we can use to augment our ML model training datasets.

Figure 3:  Fréchet Inception Distance (FID) with respect to the number of
epochs for the StyleGAN2-ADA, WGAN-CP, and WGAN-GP ML-based generative models.
In the right panel, we include examples of output patterns as model training
proceeds to visualize the relation between a lower FID value and improved
resemblance to the real input pattern. We note that all ML-based generative
models are trained with just $1\text{,}000$ examples. Figure 4:  Visualization
of the ML-based and procedural generative model results in order of increasing
FID. For each pattern type, we show a comparison of strain energy $\Delta\Psi$
at $d=0.001$ for real and generated patterns with low fidelity data for: a)
StyleGAN2-ADA patterns, b) WGAN-GP patterns, c) WGAN-CP patterns, d)
“Procedural” patterns and, e) “Bernoulli” patterns. Overall, patterns produced
with StyleGAN2-ADA have the highest similarity to the real dataset. We note
that all ML-based generative models were trained with just $1\text{,}000$
examples, whereas the “Procedural” and “Bernoulli” patterns rely on no
training data, only a knowledge of the average number of bright and dark
pixels in the target dataset.

###  3.2 Metamodel Performance with an Augmented Training Dataset

With our trained ML-based generative models and procedural algorithm-based
generative models, we are able to generate synthetic input patterns and use
them as inputs to Finite Element simulations where the results are used to
augment our metamodel training datasets. In Fig. 5, we show the test
performance of the CNN-based metamodel defined in Section 2.2.1 trained on
these data. We report the R2 score computed on held out test data with respect
to dataset size for five different types of training dataset. The first
training dataset type is composed of real patterns only. The rest of the
training dataset types contain a fixed number of real data points
($1\text{,}000$), and the size of these datasets is increased by adding
simulation results obtained from patterns generated using WGAN-CP, WGAN-GP,
StyleGAN2-ADA, “Procedural”, or “Bernoulli” methods, respectively. For all
training dataset types, we consider sample sizes of $1\text{,}000$,
$2\text{,}000$, $4\text{,}000$, $8\text{,}000$, and $16\text{,}000$ patterns.
For reference, training our CNN-based metamodel for $100$ epochs with
$16\text{,}000$ samples took $\approx 2$ minutes on a single Nvidia Tesla V100
GPU. The results presented in Fig. 5 reveal that metamodels trained with WGAN-
GP and “Procedural” patterns perform nearly equivalently with R2 scores of
$0.9975$, and exhibit only a slightly inferior performance to a metamodel
trained entirely on real patterns (R2 $=0.9992$) for a dataset size of
$16\text{,}000$. Notably, in all cases, the addition of the generated input
patterns improves the performance of the metamodel except for $1\text{,}000$
and $3\text{,}000$ random “Bernoulli” patterns which decreased the metamodel
performance. This is anticipated because these patterns are not spatially
correlated the way real Cahn-Hilliard patterns are, as depicted in Fig. 4. In
fact, we find the very slight improvement in the performance of the metamodels
when the dataset is augmented with more than $8\text{,}000$ of this type of
synthetic patterns to be counter intuitive. Comparing the metamodel
performance for dataset augmentations with StyleGAN2-ADA patterns versus WGAN-
GP and “Procedural” patterns, we anticipate that the diversity of the WGAN-GP
and “Procedural” synthetic patterns proves to be more important than the
authenticity of the StyleGAN2-ADA patterns for enhancing metamodel
performance. Namely, even though the StyleGAN2-ADA patterns were closer to
real patterns than the WGAN-GP patterns, they were perhaps less diverse or
even too similar to the real patterns used for training and thus less
beneficial when training the predictive model.

Figure 5:  Metamodel performance with respect to the size of the training
dataset. Note that “Dataset Size” refers to the combined number of unique real
and generated synthetic patterns. For a dataset of $16\text{,}000$ real
patterns, R2 is $0.9992$. For a dataset of $1\text{,}000$ real and
$15\text{,}000$ synthetic patterns, the metamodel performance with
“Procedural” and WGAN-GP patterns is almost identical with R2 values of
$0.9975$. For augmentations with WGAN-CP, StyleGAN2-ADA, and “Bernoulli”
patterns, the corresponding R2 values are $0.9969$, $0.9911$, and $0.9822$
respectively.

###  3.3 Metamodel Performance with Transfer Learning

After training the metamodels on datasets based on low fidelity simulation
data, we evaluate the efficacy of our straightforward transfer learning
approach described in Section 2.2.2 to make predictions on the corresponding
high fidelity simulation dataset. We begin with our metamodel pre-trained
using the weights obtained from our low fidelity dataset metamodel trained
with $1\text{,}000$ real and $15\text{,}000$ generated patterns with rotation-
based augmentation as described in Section 2.6. With this additional rotation-
based augmentation, a dataset size of $N$ corresponds to $4N$ training points.
Then, we perform additional training with $1\text{,}000$ real pattern-based
high fidelity simulations. As shown in Table 1, this transfer learning-based
training process predicts the change in strain energy $\Delta\Psi$ at the
final displacement for test data with R2 score of $0.9977$ and corresponding
MAE of $0.0074$ when the weights are initialized with the best performing
metamodel trained with a dataset augmented with StyleGAN2-ADA patterns in
addition to the rotation-based methods. We note that although the performance
of metamodels trained with datasets augmented with WGAN-CP, WGAN-GP and
“Procedural” patterns (with or without additional rotations), is better than
equivalent metamodels trained based on StyleGAN2-ADA augmented datasets (see
Fig. 5 and Table 1 “Pre-Training” column), the StyleGAN2-ADA augmented model
performs best after transfer learning. Alternatively, training a metamodel
initialized with random weights predicts $\Delta\Psi$ for the same high
fidelity dataset with an R2 of $0.9783$ and corresponding MAE of $0.0198$. We
note that the real patterns used in the training and test sets in the low
fidelity metamodel training match the patterns used in the high fidelity
metamodel training, and that the same $1\text{,}000$ real patterns are used as
training data for our metamodels and the generative models. Overall, this
demonstrates that synthetic pattern-based and rotation-based data augmentation
strategies can be combined with our previously explored transfer learning
approach [28] to create meaningful training datasets that rely on only a small
number of representative input pattern images and are computationally cheap to
generate. Based on our investigations, we find that “Procedural” patterns,
when possible to generate, can not only be an effective choice, but also may
be a better choice than ML-based generative models in some circumstances. When
it is not possible to generate procedural patterns, our results indicate that
both WGAN-GP and StyleGAN2-ADA are good choices for ML-based generative
models.

Table 1: Results of transfer learning. Pre-training is performed with low
fidelity data of $1\text{,}000$ real Cahn-Hilliard patterns and
$15\text{,}000$ either real or generated patterns subjected to $3$ additional
rotations of the unique domains. Fine-tuning refers to training a metamodel
with $1\text{,}000$ high fidelity real Cahn-Hilliard patterns with the model
initial weights “transferred” from the pre-trained model on the corresponding
row. For a metamodel that is trained on $1\text{,}000$ entirely real Cahn-
Hilliard patterns without transfer learning, the model weights are randomly
initialized. Overall, it is evident that our transfer learning approach
improves the MAE by at least $55\%$ when predicting change in strain energy.
Representative plots of true strain energy vs. predicted strain energy are
shown in Appendix A, Fig. 6 to add additional context to these values.

Transfer Learning
---
Pre-Training (Low Fidelity) |  | Fine-Tuning (High Fidelity)
Pattern Generation Method | R2 Score (Test Set) |  | R2 Score (Test Set) | MAE (Test Set)
Real | 0.9991 |  | 0.9991 | 0.0046
StyleGAN2-ADA | 0.9967 |  | 0.9977 | 0.0074
WGAN-CP | 0.9973 |  | 0.9974 | 0.0088
WGAN-GP | 0.9981 |  | 0.9974 | 0.0077
“Procedural” | 0.9979 |  | 0.9973 | 0.0084
No Transfer Learning |  | 0.9783 | 0.0198

##  4 Conclusion

In this paper, we extend our previous work on using machine learning-based
metamodels to predict mechanical quantities of interest in heterogeneous
materials [28, 29, 30] to include a method for working with size-limited
datasets. Specifically, we are interested in developing tools for making
smaller datasets (with as few as $1\text{,}000$ example input patterns)
amenable to deep learning approaches. To accomplish this, we first create a
new dataset of spatially heterogeneous domains undergoing large deformation
with material property patterns based on the Cahn-Hilliard equation, the
“Mechanical MNIST Cahn-Hilliard” dataset. In contrast to our previous work
[43, 44], these input patterns are more relevant to heterogeneous biological
materials. In this paper, we present a brief overview of the underlying theory
behind the Cahn-Hilliard equations, and describe the procedure for generating
the dataset. Then, with this dataset, we test the efficacy of different
Generative Adversarial Network (GAN) models at generating new Cahn-Hilliard
patterns from a limited training dataset of $1\text{,}000$ example patterns.
Of the approaches that we explored, we found that the StyleGAN2-ADA model
performed best at generating synthetic Cahn-Hilliard patterns (FID $=39.2$).
In addition to GAN-based synthetic patterns, we explored two procedural
approaches and created two additional types of synthetic Cahn-Hilliard
patterns, “Procedural” patterns and spatially uncorrelated “Bernoulli”
patterns. With ML-based and procedural-based generated patterns, we then
created low fidelity (i.e., computationally cheap through coarse mesh and
perturbation displacements) Finite Element simulation datasets comprised of
$1\text{,}000$ simulations based on real input patterns and $15\text{,}000$
simulations based on generated patterns. We then compared the performance of
metamodels trained on these hybrid real and generated input pattern datasets
to a metamodel trained entirely on real patterns and found that our data
augmentation approaches were highly effective (R2 of $0.9975$ for “Procedural”
and WGAN-GP augmentation-based datasets and R2 of $0.9992$ for the dataset
based entirely on real patterns). In addition, we built on our previous work
in using transfer learning to leverage low fidelity simulation datasets [28],
and demonstrated that with just $1\text{,}000$ high fidelity (i.e., refined
mesh, full applied displacement) Finite Element simulations, we could transfer
a low fidelity metamodel to the high fidelity dataset and obtain an R2 score
of $0.9976$ and corresponding MAE of $0.0074$ for predicting change in strain
energy. This final result was obtained with $1\text{,}000$ unique real input
patterns, $1\text{,}000$ real pattern low fidelity simulations, $1\text{,}000$
real pattern high fidelity simulations, and $15\text{,}000$ low fidelity
simulations with StyleGAN2-ADA generated input patterns.

Broadly speaking, we anticipate that the work presented in this paper will
motivate multiple future research directions. To this end, we have made both
our “Mechanical MNIST Cahn-Hilliard” dataset and our metamodel implementation
readily available for other research groups to build on under open-source
licenses (see Section 5). In the future, we anticipate that others may
implement alternative approaches to this problem that exceed the baseline
performance established in this paper. Here, we established baseline
performance for three problems: (1) training generative models with just
$1\text{,}000$ example patterns, (2) demonstrating the effectiveness of simple
procedural data generation and augmentation approaches, and (3) training a
metamodel based on a Finite Element simulation dataset where the relevant
material property distribution is defined by just $1\text{,}000$ example
patterns. However, because our dataset is published under an open source
license, others are free to formulate different challenges and attempt the
same problem with an entirely different metamodeling approach. In particular,
we anticipate future work in developing more sophisticated approaches for
representing the input pattern space, future work in predicting full field
quantities of interest in addition to the single quantity of interest
predicted here, and future work in accounting for more aspects that render
modeling soft tissue very challenging, such as material anisotropy and the
broad uncertainty in material properties. In addition, we plan to extend the
“Mechanical MNIST Cahn-Hilliard” dataset to include additional constitutive
parameters, a more diverse set of constitutive models, and additional loading
scenarios in the future. In addition, we note that there should be further
future investigation into the minimum number of data points required to train
a GAN model for this type of problem. In this work, we relied entirely on a
pragmatic selection of $1\text{,}000$ data points simply because $100$ would
likely be insufficient for training a GAN, and $10\text{,}000$ would no longer
be resolutely in the size-limited datasets regime. Looking forward, we hope
that the findings in this work will make deep learning-based metamodels much
more accessible for researchers working with limited examples of their input
pattern spaces of interest.

##  5 Additional Information

The “Mechanical MNIST Cahn-Hilliard” dataset is available through the OpenBU
Institutional Repository at https://open.bu.edu/handle/2144/43971 [109]. We
provide with this dataset a supplementary document that includes more details
on the theory of the Cahn-Hilliard equation and our Finite Element
implementation. The codes for generating the Cahn-Hilliard patterns and for
performing the Finite Element equibiaxial extension simulations using FEniCS
computing platform (https://fenicsproject.org) are available on github at
https://github.com/elejeune11/Mechanical-MNIST-Cahn-Hilliard. The codes for
implementing the metamodel pipeline including the convolutional neural network
model for a single quantity of interest prediction and the GAN model for data
synthesis are also made available at https://github.com/saeedmhz/cahn-
hilliard.

##  6 Acknowledgements

We would like to thank the staff of the Boston University Research Computing
Services and the OpenBU Institutional Repository (in particular Eleni Castro)
for their invaluable assistance with generating and disseminating the
“Mechanical MNIST Cahn-Hilliard Dataset”. This work was made possible through
start up funds from the Boston University Department of Mechanical
Engineering, the David R. Dalton Career Development Professorship, the Hariri
Institute Junior Faculty Fellowship, the National Science Foundation
Engineering Research Center CELL-MET NSF EEC-1647837, and the Office of Naval
Research Award N00014-22-1-2066.

##  Appendix A Qualitative Visualization of Metamodel Performance

Figure 6: Qualitative interpretation of R2 scores for transfer learning
evaluation. True versus predicted strain energy values of high fidelity test
data are plotted for three different metamodels trained with $1\text{,}000$
high fidelity real data points. (a) Metamodel weights are initialized randomly
(i.e., no transfer learning is performed). (b) Metamodel weights are
transferred from a model trained on $1\text{,}000$ low fidelity real samples
and $15\text{,}000$ generated samples from StyleGAN2-ADA with extra rotation-
based augmentations. (c) Metamodel weights are initialized by transferring
weights of a model trained on $16\text{,}000$ low fidelity real data with
extra rotation-based augmentations.

In this Appendix, we provide additional information to support how synthetic
data augmentation combined with a simple transfer learning approach can help
improve the performance of our metamodel. As shown in Section 3.3,
initializing the weights of our metamodel with the weights of a model trained
on low fidelity real data augmented with the proper set of generated data can
increase the R2 score of predicted high fidelity strain energy values from
$0.9783$ to $0.9977$. In order to qualitatively interpret the benefits of this
improvement, we plotted true versus predicted strain energy values for all
samples in the test set of the high fidelity dataset for three different
models (Fig. 6). Figure 6a shows the results where no transfer learning is
performed, whereas Fig. 6b shows the case where the weights are transferred
from a model trained on $1\text{,}000$ low fidelity real samples and
$15\text{,}000$ synthetic samples generated from StyleGAN2-ADA with extra
rotation-based augmentations. In Fig. 6c the initial weights are transferred
from a low fidelity model trained on $16\text{,}000$ real data with rotation-
based augmentations. Overall, this figure further supports our findings from
Table 1 and Section 3.3 where we state the performance of our metamodels in
terms of R2 score.

## References

  * [1] Maryam Nikpasand, Ryan R Mahutga, Lauren M Bersie-Larson, Elizabeth Gacek, and Victor H Barocas.  A hybrid microstructural-continuum multiscale approach for modeling hyperelastic fibrous soft tissue.  Journal of Elasticity, 145(1):295–319, 2021.
  * [2] Rong Fan and Michael S. Sacks.  Simulation of planar soft tissues using a structural constitutive model: Finite element implementation and validation.  Journal of Biomechanics, 47(9):2043–2054, 2014.  Functional Tissue Engineering.
  * [3] J. Berkley, G. Turkiyyah, D. Berg, M. Ganter, and S. Weghorst.  Real-time finite element modeling for surgery simulation: an application to virtual suturing.  IEEE Transactions on Visualization and Computer Graphics, 10(3):314–325, 2004.
  * [4] Grand Roman Joldes, Adam Wittek, and Karol Miller.  Suite of finite element algorithms for accurate computation of soft tissue deformation for surgical simulation.  Medical Image Analysis, 13(6):912–919, 2009.
  * [5] Jinao Zhang, Yongmin Zhong, and Chengfan Gu.  Deformable models for surgical simulation: A survey.  IEEE Reviews in Biomedical Engineering, 11:143–164, 2018.
  * [6] Grand Joldes, George Bourantas, Benjamin Zwick, Habib Chowdhury, Adam Wittek, Sudip Agrawal, Konstantinos Mountris, Damon Hyde, Simon K. Warfield, and Karol Miller.  Suite of meshless algorithms for accurate computation of soft tissue deformation for surgical simulation.  Medical Image Analysis, 56:152–171, 2019.
  * [7] Francisco Sahli-Costabal, Kinya Seo, Euan Ashley, and Ellen Kuhl.  Classifying drugs by their arrhythmogenic risk using machine learning.  Biophysical Journal, 118(5):1165–1176, 2020.
  * [8] Vivek D Sree, Manuel K Rausch, and Adrian B Tepole.  Linking microvascular collapse to tissue hypoxia in a multiscale model of pressure ulcer initiation.  Biomechanics and modeling in mechanobiology, 18(6):1947–1964, 2019\.
  * [9] Fanwei Kong, Thuy Pham, Caitlin Martin, Raymond McKay, Charles Primiano, Sabet Hashim, Susheel Kodali, and Wei Sun.  Finite element analysis of tricuspid valve deformation from multi-slice computed tomography images.  Annals of biomedical engineering, 46(8):1112–1127, 2018.
  * [10] Sotirios Kakaletsis, William D Meador, Mrudang Mathur, Gabriella P Sugerman, Tomasz Jazwiec, Marcin Malinowski, Emma Lejeune, Tomasz A Timek, and Manuel K Rausch.  Right ventricular myocardial mechanics: Multi-modal deformation, microstructure, modeling, and comparison to the left ventricle.  Acta Biomaterialia, 123:154–166, 2021.
  * [11] Reza Avazmohammadi, David S Li, Thomas Leahy, Elizabeth Shih, João S Soares, Joseph H Gorman, Robert C Gorman, and Michael S Sacks.  An integrated inverse model-experimental approach to determine soft tissue three-dimensional constitutive parameters: application to post-infarcted myocardium.  Biomechanics and modeling in mechanobiology, 17(1):31–53, 2018\.
  * [12] Ray W. Ogden.  Nonlinear Continuum Mechanics and Modeling the Elasticity of Soft Biological Tissues with a Focus on Artery Walls, pages 83–156.  Springer International Publishing, Cham, 2017.
  * [13] Gerard A. Ateshian.  Mixture Theory for Modeling Biological Tissues: Illustrations from Articular Cartilage, pages 1–51.  Springer International Publishing, Cham, 2017.
  * [14] Gerhard Holzapfel.  Biomechanics of soft tissue, pages 1049–1063.  Academic Press, San Diego, CA, 2001.
  * [15] Jay D. Humphrey.  Multiscale modeling of arterial adaptations: Incorporating molecular mechanisms within continuum biomechanical models.  In Gerhard A. Holzapfel and Ellen Kuhl, editors, Computer Models in Biomechanics", year=2013, pages 119–127, Dordrecht, 2013. Springer Netherlands.
  * [16] Thomas JR Hughes.  The finite element method: linear static and dynamic finite element analysis.  Courier Corporation, Mineola, NY, 2012.
  * [17] Christopher E. Korenczuk, Rohit Y. Dhume, Kenneth K. Liao, and Victor H. Barocas.  Ex Vivo Mechanical Tests and Multiscale Computational Modeling Highlight the Importance of Intramural Shear Stress in Ascending Thoracic Aortic Aneurysms.  Journal of Biomechanical Engineering, 141(12), 11 2019.
  * [18] Yu Leng, Mario de Lucio, and Hector Gomez.  Using poro-elasticity to model the large deformation of tissue during subcutaneous injection.  Computer Methods in Applied Mechanics and Engineering, 384:113919, 2021.
  * [19] Yue Mei, Jiahao Liu, Xu Guo, Brandon Zimmerman, Thao D Nguyen, and Stéphane Avril.  General finite-element framework of the virtual fields method in nonlinear elasticity.  Journal of Elasticity, 145(1):265–294, 2021.
  * [20] Sandeep Madireddy, Bhargava Sista, and Kumar Vemaganti.  A bayesian approach to selecting hyperelastic constitutive models of soft tissue.  Computer Methods in Applied Mechanics and Engineering, 291:102–122, 2015.
  * [21] Majid Jadidi, Selda Sherifova, Gerhard Sommer, Alexey Kamenskiy, and Gerhard A. Holzapfel.  Constitutive modeling using structural information on collagen fiber direction and dispersion in human superficial femoral artery specimens of different ages.  Acta Biomaterialia, 121:461–474, 2021.
  * [22] Michele Tonutti, Gauthier Gras, and Guang-Zhong Yang.  A machine learning approach for real-time modelling of tissue deformation in image-guided neurosurgery.  Artificial intelligence in medicine, 80:39–47, 2017.
  * [23] Vahidullah Tac, Vivek D. Sree, Manuel K. Rausch, and Adrian B. Tepole.  Data-driven modeling of the mechanical behavior of anisotropic soft biological tissue.  arXiv preprint, 2021.
  * [24] Mahsa Tajdari, Aishwarya Pawar, Hengyang Li, Farzam Tajdari, Ayesha Maqsood, Emmett Cleary, Sourav Saha, Yongjie Jessica Zhang, John F. Sarwark, and Wing Kam Liu.  Image-based modelling for adolescent idiopathic scoliosis: Mechanistic machine learning analysis and prediction.  Computer Methods in Applied Mechanics and Engineering, 374:113590, 2021.
  * [25] Angran Li, Amir Barati Farimani, and Yongjie Jessica Zhang.  Deep learning of material transport in complex neurite networks.  Scientific reports, 11(1):1–13, 2021.
  * [26] Grace CY Peng, Mark Alber, Adrian Buganza Tepole, William R Cannon, Suvranu De, Savador Dura-Bernal, Krishna Garikipati, George Karniadakis, William W Lytton, Paris Perdikaris, Linda Petzold, and Ellen Kuhl.  Multiscale modeling meets machine learning: What can we learn?  Archives of Computational Methods in Engineering, 28(3):1017–1037, 2021.
  * [27] Mathias Peirlinck, F Sahli Costabal, KL Sack, JS Choy, GS Kassab, JM Guccione, M De Beule, Patrick Segers, and E Kuhl.  Using machine learning to characterize heart failure across the scales.  Biomechanics and modeling in mechanobiology, 18(6):1987–2001, 2019\.
  * [28] Emma Lejeune and Bill Zhao.  Exploring the potential of transfer learning for metamodels of heterogeneous material deformation.  Journal of the Mechanical Behavior of Biomedical Materials, 117:104276, 2021.
  * [29] Emma Lejeune.  Mechanical MNIST: A benchmark dataset for mechanical metamodels.  Extreme Mechanics Letters, 36:100659, 2020.
  * [30] Saeed Mohammadzadeh and Emma Lejeune.  Predicting mechanically driven full-field quantities of interest with deep learning-based metamodels.  Extreme Mechanics Letters, 50:101566, 2022.
  * [31] Zhenze Yang, Chi-Hua Yu, Kai Guo, and Markus J. Buehler.  End-to-end deep learning method to predict complete strain and stress tensors for complex hierarchical composite microstructures.  Journal of the Mechanics and Physics of Solids, 154:104506, 2021\.
  * [32] Jaber Rezaei Mianroodi, Nima H Siboni, and Dierk Raabe.  Teaching solid mechanics to artificial intelligence—a fast solver for heterogeneous materials.  npj Computational Materials, 7(1):1–10, 2021.
  * [33] Casey Stowers, Taeksang Lee, Ilias Bilionis, Arun K. Gosain, and Adrian Buganza Tepole.  Improving reconstructive surgery design using gaussian process surrogates to capture material behavior uncertainty.  Journal of the Mechanical Behavior of Biomedical Materials, 118:104340, 2021.
  * [34] Hang Yang, Xu Guo, Shan Tang, and Wing Kam Liu.  Derivation of heterogeneous material laws via data-driven principal component expansions.  Computational Mechanics, 64(2):365–379, 2019.
  * [35] Zeliang Liu, C.T. Wu, and M. Koishi.  A deep material network for multiscale topology learning and accelerated nonlinear modeling of heterogeneous materials.  Computer Methods in Applied Mechanics and Engineering, 345:1138–1168, 2019.
  * [36] Alexander IJ Forrester and Andy J Keane.  Recent advances in surrogate-based optimization.  Progress in aerospace sciences, 45(1-3):50–79, 2009.
  * [37] Yasmin Salehi and Dennis Giannacopoulos.  Physgnn: A physics-driven graph neural network based model for predicting soft tissue deformation in image-guided neurosurgery.  arXiv preprint arXiv:2109.04352, 2021.
  * [38] Lu Lu, Ming Dao, Punit Kumar, Upadrasta Ramamurty, George Em Karniadakis, and Subra Suresh.  Extraction of mechanical properties of materials through deep learning from instrumented indentation.  Proceedings of the National Academy of Sciences, 117(13):7052–7062, 2020.
  * [39] Gregory H Teichert and Krishna Garikipati.  Machine learning materials physics: Surrogate optimization and multi-fidelity algorithms predict precipitate morphology in an alternative to phase field dynamics.  Computer Methods in Applied Mechanics and Engineering, 344:666–693, 2019.
  * [40] Ehsan Haghighat, Maziar Raissi, Adrian Moure, Hector Gomez, and Ruben Juanes.  A physics-informed deep learning framework for inversion and surrogate modeling in solid mechanics.  Computer Methods in Applied Mechanics and Engineering, 379:113741, 2021.
  * [41] Andrew Brock, Jeff Donahue, and Karen Simonyan.  Large scale GAN training for high fidelity natural image synthesis.  In International Conference on Learning Representation, New Orleans, LA, May 6—9 2019.
  * [42] Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, and Timo Aila.  Training generative adversarial networks with limited data.  In Advances in Neural Information Processing Systems 33, Virtual, Dec. 6—12 2020. Curran Associates.
  * [43] Emma Lejeune.  Mechanical mnist-fashion, 2020.
  * [44] Emma Lejeune.  Mechanical MNIST-Uniaxial Extension, 2019.
  * [45] Saeed Mohammadzadeh and Emma Lejeune.  Mechanical MNIST Crack Path, 2021.
  * [46] Peerasait Prachaseree and Emma Lejeune.  Asymmetric Buckling Columns (ABC), 2021.
  * [47] Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio.  Generative adversarial nets.  In Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2, NIPS’14, page 2672–2680, Montreal, Canada, 2014. MIT Press.
  * [48] Devendra K Jangid, Neal R Brodnik, Amil Khan, Michael G Goebel, McLean P Echlin, Tresa M Pollock, Samantha H Daly, and BS Manjunath.  3d grain shape generation in polycrystals using generative adversarial networks.  Integrating Materials and Manufacturing Innovation, pages 1–14, 2022.
  * [49] Wufei Ma, Elizabeth J Kautz, Arun Baskaran, Aritra Chowdhury, Vineet Joshi, Bulent Yener, and Daniel J Lewis.  Image-driven discriminative and generative machine learning algorithms for establishing microstructure–processing relationships.  Journal of Applied Physics, 128(13):134901, 2020.
  * [50] Yann LeCun and Corinna Cortes.  MNIST handwritten digit database.  http://yann.lecun.com/exdb/mnist/, 2010.
  * [51] Han Xiao, Kashif Rasul, and Roland Vollgraf.  Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms.  arXiv preprint, 2017.
  * [52] Alan Mathison Turing.  The chemical basis of morphogenesis.  Bulletin of mathematical biology, 52(1):153–197, 1990.
  * [53] Philip K Maini, Thomas E Woolley, Ruth E Baker, Eamonn A Gaffney, and S Seirin Lee.  Turing’s model for biological pattern formation and the robustness problem.  Interface focus, 2(4):487–496, 2012.
  * [54] Krishna Garikipati.  Perspectives on the mathematics of biological patterning and morphogenesis.  Journal of the Mechanics and Physics of Solids, 99:192–210, 2017\.
  * [55] Christopher P Grant.  Spinodal decomposition for the Cahn–Hilliard equation.  Communications in Partial Differential Equations, 18(3-4):453–490, 1993.
  * [56] Zhenlin Wang, Xun Huan, and Krishna Garikipati.  Variational system identification of the partial differential equations governing the physics of pattern-formation: Inference under varying fidelity and noise.  Computer Methods in Applied Mechanics and Engineering, 356:44–74, 2019.
  * [57] Z Wang, X Huan, and K Garikipati.  Variational system identification of the partial differential equations governing microstructure evolution in materials: Inference over sparse and spatially unrelated data.  Computer Methods in Applied Mechanics and Engineering, 377:113706, 2021.
  * [58] Tim Sainburg, Leland McInnes, and Timothy Q Gentner.  Parametric UMAP embeddings for representation and semisupervised learning.  Neural Computation, 33(11):2881–2907, 2021.
  * [59] Garth N Wells, Ellen Kuhl, and Krishna Garikipati.  A discontinuous Galerkin method for the Cahn–Hilliard equation.  Journal of Computational Physics, 218(2):860–877, 2006.
  * [60] FEniCS Project.  Cahn-hilliard equation, 2016.
  * [61] Martin Alnæs, Jan Blechta, Johan Hake, August Johansson, Benjamin Kehlet, Anders Logg, Chris Richardson, Johannes Ring, Marie E Rognes, and Garth N Wells.  The FEniCS project version 1.5.  Archive of Numerical Software, 3(100):9–23, 2015.
  * [62] Anders Logg, Kent-Andre Mardal, and Garth Wells.  Automated solution of differential equations by the finite element method: The FEniCS book, volume 84.  Springer Science & Business Media, Germany, 2012.
  * [63] Olga Wodo and Baskar Ganapathysubramanian.  Computationally efficient solution to the cahn–hilliard equation: Adaptive implicit time schemes, mesh sensitivity analysis and the 3d isoperimetric problem.  Journal of Computational Physics, 230(15):6037–6060, 2011.
  * [64] Héctor Gómez, Victor M Calo, Yuri Bazilevs, and Thomas JR Hughes.  Isogeometric analysis of the cahn–hilliard phase-field model.  Computer methods in applied mechanics and engineering, 197(49-50):4333–4352, 2008.
  * [65] Gary Bradski and Adrian Kaehler.  Learning OpenCV: Computer vision with the OpenCV library.  " O’Reilly Media, Inc.", Sebastopol, CA, 2008.
  * [66] Nico Schlömer, Antonio Cervone, Geordie McBain, tryfon mw, Ruben van Staden, Filip Gokstorp, toothstone, Jørgen Schartum Dokken, anzil, Juan Sanchez, Dominic Kempf, Matthias Bussonnier, Yuan Feng, awa5114, Tomislav Maric, Siwei Chen, nilswagner, Nate, ivanmultiwave, and Fred Fu.  nschloe/pygmsh v6.1.1, April 2020.
  * [67] Christophe Geuzaine and Jean-François Remacle.  Gmsh: A 3-d finite element mesh generator with built-in pre-and post-processing facilities.  International journal for numerical methods in engineering, 79(11):1309–1331, 2009.
  * [68] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala.  Pytorch: An imperative style, high-performance deep learning library.  In Advances in Neural Information Processing Systems 32, pages 8024–8035. Curran Associates, Inc., Red Hook, NY, 2019.
  * [69] Diederik P. Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  arXiv preprint, 2017.
  * [70] Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson.  How transferable are features in deep neural networks?  In Advances in Neural Information Processing Systems, Montreal, Canada, Dec. 8—13 2014. Curran Associates.
  * [71] Martin Arjovsky, Soumith Chintala, and Léon Bottou.  Wasserstein generative adversarial networks.  In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 214–223, Sydney, Australia, 06-11 Aug 2017. PMLR.
  * [72] Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville.  Improved training of wasserstein GANs.  In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30, Red Hook, NY, 2017. Curran Associates, Inc.
  * [73] Christopher M Bishop and Nasser M Nasrabadi.  Pattern recognition and machine learning, volume 4, chapter 1, page 43.  Springer, New York, NY, 2006.
  * [74] Yoshua Bengio, Eric Laufer, Guillaume Alain, and Jason Yosinski.  Deep generative stochastic networks trainable by backprop.  In Eric P. Xing and Tony Jebara, editors, Proceedings of the 31st International Conference on Machine Learning, volume 32(2) of Proceedings of Machine Learning Research, pages 226–234, Bejing, China, 22–24 Jun 2014. PMLR.
  * [75] Urs Michael Gutmann and Aapo Hyvärinen.  Noise-contrastive estimation: A new estimation principle for unnormalized statistical models.  In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics (AISTATS2010), JMLR Workshop and Conference Proceedings, pages 297–304, Sardinia, Italy, May 13–15 2010.  International Conference on Artificial Intelligence and Statistics(AISTATS 2010) ; Conference date: 13-05-2010 Through 15-05-2010.
  * [76] Jürgen Schmidhuber.  Learning Factorial Codes by Predictability Minimization.  Neural Computation, 4(6):863–879, 11 1992.
  * [77] Zhuowen Tu.  Learning generative models via discriminative approaches.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1–8, Minneapolis, MN, Jun. 17—22 2007.
  * [78] Li Deng.  The mnist database of handwritten digit images for machine learning research [best of the web].  IEEE Signal Processing Magazine, 29(6):141–142, 2012.
  * [79] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner.  Gradient-based learning applied to document recognition.  Proceedings of the IEEE, 86(11):2278–2324, 1998.
  * [80] Josh M Susskind, Adam K Anderson, and Geoffrey E Hinton.  The toronto face database.  Department of Computer Science, University of Toronto, Toronto, ON, Canada, Tech. Rep, 2010.
  * [81] Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton.  The Cifar-10 dataset, 2014.
  * [82] Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen.  Progressive growing of GANs for improved quality, stability, and variation.  In International Conference on Learning Representations, Vancouver, Canada, Apr. 30—May 3 2018.
  * [83] Alec Radford, Luke Metz, and Soumith Chintala.  Unsupervised representation learning with deep convolutional generative adversarial networks.  In International Conference on Learning Representations, San Juan, Puerto Rico, May 2—4 2016.
  * [84] Yossi Rubner, Carlo Tomasi, and Leonidas J Guibas.  The earth mover’s distance as a metric for image retrieval.  International journal of computer vision, 40(2):99–121, 2000.
  * [85] Yossi Rubner and Carlo Tomasi.  Perceptual metrics for image database navigation.  Springer Science & Business Media, Norwell, MA, 2001.
  * [86] Haibin Ling and Kazunori Okada.  An efficient earth mover’s distance algorithm for robust histogram comparison.  IEEE transactions on pattern analysis and machine intelligence, 29(5):840–853, 2007.
  * [87] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer.  Automatic differentiation in pytorch.  In Proceeding of the 31st Conference on Neural Information Processing Systems, Long Beach, CA, Dec. 4—9 2017.
  * [88] Zeleni9.  pytorch-wgan.  https://github.com/Zeleni9/pytorch-wgan, 2021.
  * [89] Tero Karras, Samuli Laine, and Timo Aila.  A style-based generator architecture for generative adversarial networks.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4396–4405, Long Beach, CA, Jun. 16—2 2019.
  * [90] Xun Huang and Serge Belongie.  Arbitrary style transfer in real-time with adaptive instance normalization.  In Proceedings of the IEEE International Conference on Computer Vision, page 1501–1510, Venice, Italy, Oct. 22—29 2017.
  * [91] Leon A. Gatys, Alexander S. Ecker, and Matthias Bethge.  Image style transfer using convolutional neural networks.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2414–2423, Las Vegas, NV, Jun.26—Jul. 1 2016.
  * [92] Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila.  Analyzing and improving the image quality of StyleGAN.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), page 8110–8119, Virtual, Jun. 14—19 2020.
  * [93] Han Zhang, Zizhao Zhang, Augustus Odena, and Honglak Lee.  Consistency regularization for generative adversarial networks.  In International Conference on Learning Representations, Virtual, Apr. 26—May1 2020.
  * [94] Zhengli Zhao, Sameer Singh, Honglak Lee, Zizhao Zhang, Augustus Odena, and Han Zhang.  Improved consistency regularization for GANs.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35(12), pages 11033–11041, Virtual, Feb. 2—9 2020.
  * [95] Ashish Bora, Eric Price, and Alexandros G. Dimakis.  AmbientGAN: Generative models from lossy measurements.  In International Conference on Learning Representations, Vancouver, Canada, Apr. 30—May 3 2018.
  * [96] D. Cule and S. Torquato.  Generating random media from limited microstructural information via stochastic optimization.  Journal of Applied Physics, 86(6):3428–3437, 1999.
  * [97] D Fujii, BC Chen, and Noboru Kikuchi.  Composite material design of two-dimensional structures using the homogenization design method.  International Journal for Numerical Methods in Engineering, 50(9):2031–2051, 2001.
  * [98] Y. Jiao, F. H. Stillinger, and S. Torquato.  Modeling heterogeneous materials via two-point correlation functions: Basic principles.  Phys. Rev. E, 76:031110, Sep 2007.
  * [99] Claudia Redenbach.  Microstructure models for cellular materials.  Computational Materials Science, 44(4):1397–1407, 2009.
  * [100] Alexander Pasko, Oleg Fryazinov, Turlif Vilbrandt, Pierre-Alain Fayolle, and Valery Adzhiev.  Procedural function-based modelling of volumetric microstructures.  Graphical Models, 73(5):165–181, 2011.
  * [101] David J. Walters, Darby J. Luscher, and John D. Yeager.  Volumetric analysis and mesh generation of real and artificial microstructural geometries.  MethodsX, 7:100856, 2020.
  * [102] T.J. Vaughan and C.T. McCarthy.  A combined experimental–numerical approach for generating statistically equivalent fibre distributions for high strength laminated composite materials.  Composites Science and Technology, 70(2):291–297, 2010.
  * [103] Valentin Romanov, Stepan V. Lomov, Yentl Swolfs, Svetlana Orlova, Larissa Gorbatikh, and Ignaas Verpoest.  Statistical analysis of real and simulated fibre arrangements in unidirectional composites.  Composites Science and Technology, 87:126–134, 2013.
  * [104] Franz Aurenhammer.  Voronoi diagrams—a survey of a fundamental geometric data structure.  ACM Computing Surveys (CSUR), 23(3):345–405, 1991.
  * [105] Simone Falco, Jiawei Jiang, Francesco De Cola, and Nik Petrinic.  Generation of 3d polycrystalline microstructures with a conditioned laguerre-voronoi tessellation technique.  Computational Materials Science, 136:20–28, 2017.
  * [106] Swantje Bargmann, Benjamin Klusemann, Jürgen Markmann, Jan Eike Schnabel, Konrad Schneider, Celal Soyarslan, and Jana Wilmers.  Generation of 3d representative volume elements for heterogeneous materials: A review.  Progress in Materials Science, 96:322–384, 2018.
  * [107] Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan J. van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nelson, Eric Jones, Robert Kern, Eric Larson, C J Carey, İlhan Polat, Yu Feng, Eric W. Moore, Jake VanderPlas, Denis Laxalde, Josef Perktold, Robert Cimrman, Ian Henriksen, E. A. Quintero, Charles R. Harris, Anne M. Archibald, Antônio H. Ribeiro, Fabian Pedregosa, Paul van Mulbregt, and SciPy 1.0 Contributors.  SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python.  Nature Methods, 17:261–272, 2020.
  * [108] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter.  GANs trained by a two time-scale update rule converge to a local nash equilibrium.  In Proceeding of the 31st Conference on Neural Information Processing Systems, volume 30, Long Beach, CA, Dec. 4—9 2017.
  * [109] Hiba Kobeissi and Emma Lejeune.  Mechanical MNIST - Cahn-Hilliard, 2022.