# Generative Model for Material Experiments
Based on Prior Knowledge and Attention Mechanism

Mincong Luo
China Institute of Atomic Energy
Peking,China
<EMAIL_ADDRESS>
X. He
China Institute of Atomic Energy
Peking,China
<EMAIL_ADDRESS>Li Liu
China Institute of Atomic Energy
Peking,China
<EMAIL_ADDRESS>

###### Abstract

Material irradiation experiment is dangerous and complex, thus it requires
those with a vast advanced expertise to process the images and data manually.
In this paper, we propose a generative adversarial model based on prior
knowledge and attention mechanism to achieve the generation of irradiated
material images (data-to-image model), and a prediction model for
corresponding industrial performance (image-to-data model). With the proposed
models, researchers can skip the dangerous and complex irradiation experiments
and obtain the irradiation images and industrial performance parameters
directly by inputing some experimental parameters only. We also introduce a
new dataset ISMD which contains 22000 irradiated images with 22,143 sets of
corresponding parameters. Our model achieved high quality results by compared
with several baseline models. The evaluation and detailed analysis are also
performed.

##  1 introduction

In recent years, significant progress has been made in the development of deep
learning and generation models Kingma & Welling (2013) Goodfellow et al.
(2014) Mirza & Osindero (2014). However, they are far less used in natural
sciences than in services or the arts, experimental images from natural
sciences have rich scientific connotations that can be analyzed using deep
learning and generating models (e.g. medical images, fluid experiments,
materials experiments etc. Kisilev et al. (2015) Jing et al. (2017) Alom et
al. (2018) Li et al. (2018)).

Most of the researches on analyzing experimental images of natural sciences
are based on feature extraction and end-to-end mapping to obtain valuable
experimental target parameters Li et al. (2018) Cireşan et al. (2013) Jing et
al. (2017), which can be called as image-to-data task. And most of these
researches are based on convolutional neural networks(CNN) and semantic
segmentation Ronneberger et al. (2015) Menze et al. (2010). Although
impressive results have been achieved, none of these researches have built a
data-to-image model.

This research is based on the material irradiation experiment, which mainly
focus on the material structural and performance changes after being
irradiated. Researchers observe the structural changes by irradiated material
images from electron microscope, and then confirm the performance changes by
various mechanical experiments.

We propose a generation model based on prior knowledge and attention mechanism
to generate images of the experimental results. The overall architecture of
the model is shown in Fig.3 .First, we propose an embedding model
$fc(m)=Ph_{m}$ for the molecular composition of materials, so that we can get
the feature vector $C^{i}_{m}$ for each material composition $i$, which is
called molecule2vec.Then we can embed the feature vector $C^{i}_{m}$ of the
material composition as a priori knowledge into the latent variable sampling
of the generation model(see Fig.3 for details).For the irradiated material
images, the swelling cavities distribution $H_{v}$ is the most important
feature Ehrlich (1981), so an attention mechanism is introduced to make the
model focus on the section with swelling cavities in the material images(see
Fig.1).The image generated by the feedforward propagation of the model
$X^{\prime}_{img}=G(D_{d},D_{c},z)$ can be encoded by the encoder network to
be transformed into the corresponding swelling cavities distribution
$H^{\prime}_{v}$, we construct the loss term
$\mathcal{L}_{H_{v}}=||H^{\prime}_{v}-H_{v}||^{2}$ to measure the distance
between the real distribution of the swelling cavities and the feature of the
generated image , and then the attention mechanism is optimized by gradient
descent.In the end, we can obtain high-quality images of irradiated materials.
In addition, we pre-trained the image-to-data network $Pred(X_{img})=D_{r}$ to
predict the performance parameters $D_{r}$ of the material through the
corresponding experimental image $X_{img}$, which makes it possible to predict
the material’s performance parameters $D^{\prime}_{r}$ by generating image
$X^{\prime}_{img}$ by $Pred(X^{\prime}_{img})=D^{\prime}_{r}$.

our main contributions:

  * •

data-to-image material images generation model:generation model
$P(X_{img}|D_{d},D_{c},z)$ is built based on prior knowledge and attention
mechanism.

  * •

image-to-data material performance prediction model: using the images
$X_{img}$ and the context association method of CNN+BiLSTM, the network
$P(D_{r}|X_{img},C_{m})$ is established to predict the performance parameters
$D_{r}$ by the image $X_{img}$.

  * •

The experimental dataset ISMD for irradiated materials: the images with
corresponding data from large number of irradiation experiments and manual
annotations.

(a) Overview

(b) Representation Vector

Figure 1:  (a) :data-to-image model and image-to-data model. In the data-to-
image task, model generate images based on prior knowledge and attention
mechanism. In the image-to-data task, images generated are used to predict the
performance parameters. (b): For different alloy materials, the proportion of
the elements in their molecular compositions is calculated, which is used by
element representation vector $m$.

##  2 Related Work

Our research is related to deep generation models for image generation, image
information mining in the natural sciences, and irradiated material science.

Deep generation model for image generation. The caption-to-image generation
model Gregor et al. (2015) Zhang et al. (2016) Mansimov et al. (2015) is the
majority, most of these models will first introduce a semantic capture module
to extract semantic features, then generate images through RNN Mansimov et al.
(2015), stackGAN Zhang et al. (2016), etc. In addition, enhancing the
connection between semantics and images by attention mechanism is also popular
Zhang et al. (2017).

Image information mining in the natural sciences. Medicine, biology, materials
and other fields have recently introduced deep learning models to mine the
connotation information of experimental images Jung et al. (2017) Ronneberger
et al. (2015) Li et al. (2018). Some researches are inspired by natural
language processing(NLP) using the context information association trick Ssm
et al. (2017), which can be seen as an image-to-data process.

Irradiated material science. In recent years, deep learning methods have been
introduced into material images processing Li et al. (2018) Rovinelli (2018),
which has replaced the traditional artificial technology analysis and achieved
good results.

##  3 Model

Recall that our aim is to generate the corresponding experimental image
$X^{\prime}_{img}$ based on the mechanics and thermodynamic parameters $D_{d}$
of the material under the irradiation condition $D_{c}$ by the prior
knowledge, the molecular composition feature vector $C_{m}$, and the attention
mechanism. In addition, after generating experimental image
$X^{\prime}_{img}$, CNN+BiLSTM network $P(D^{\prime}_{r}|X^{\prime}_{img})$ is
used to predict the performance parameters $D^{\prime}_{r}$ for image
$X^{\prime}_{img}$.

###  3.1 Embedding model for the molecular composition of materials

Our dataset contains 14 kinds of alloy materials:

$C_{m}\in\\{Inconel718,InconeX750,Zr1,Zr2,Zr4,Zr1Nb,Zr2.5Nb,$
$1Cr13,2Cr13,00Cr13Ni5Mo4,Au304,Au317,Cr17Ti,Cr25\\}$

The molecular composition of these alloy materials is an important prior
knowledge for images generation task, so embedding molecular composition
features into the generation model is needed. Let
$E=\\{Al,Mg,Si,Cu,Fe,O...\\}$ be the set of all the elements appeared. Let
$m^{i}\in\mathbb{R}^{|E|}$ be the element percentage representation vector of
the alloy material $i$. Inspired by the word2vec method Mikolov et al. (2013)
for word embedding, a method for molecular composition of alloy materials
feature extracting called moleculars2vec is proposed. $Ph_{m}=fc(m)$ is used
to learn the thermodynamic properties $Ph_{m}$ from the alloy material by the
element representation vector $m$ :

$Ph_{m}=fc(m)=Relu(W_{m}fc^{\prime}(m))$

where $fc(\cdot)$ represents a fully connected network, $fc^{\prime}(\cdot)$
represents the network part of $fc(\cdot)$ without the last layer,
$W_{m}\in\mathbb{R}^{|E|\times d_{Ph}}$ is the weight matrix of the last
layer, $d_{Ph}$ is the dimension of the thermodynamic property $Ph_{m}$. Since
the weight matrix $W_{m}$ has the ability to represent both the molecular
composition features and the properties of the alloy material, we take
$C^{i}_{m}=W_{m}^{i,:}$ as the feature vector of the material $i$.

###  3.2 Generative model based on prior knowledge and attention mechanism

The prior distribution. In the previous section we extracted the prior
knowledge as vectors, that is, the features of the molecular composition of
the alloy materials. To introduce it into the model, we define the
distribution of the latent variable $P(z)$ as follow(prior distribution):

$z\sim\mathcal{N}(\mu(C_{m}),\delta(C_{m}))$ $\mu(C_{m})=tanh(W_{\mu}C_{m})$
$\delta(C_{m})=exp(tanh(W_{\delta}C_{m}))$

where $W_{\mu}\in\mathbb{R}^{D\times
dim(C_{m})}$,$W_{\delta}\in\mathbb{R}^{D\times dim(C_{m})}$ are the learnable
parameters. Other similar methods introduce the distribution as latent
variable dependencies are Mansimov et al. (2015) Bachman & Precup (2015), and
our model is optimized in performance after introducing a prior distribution.

The attention module. The swelling cavities distribution $H_{v}$ of the
irradiated material image is a statistics count vector for the distribution of
different sizes, which is an important feature for material performance
prediction Porollo et al. (2000). Therefore the model needs to allocate more
attention to the section contains cavities, which requires attention mechanism
to achieve. With this design goal in mind, we can set the attention module as
an estimate of the corresponding degree of the cavities distribution $H_{v}$
and the image $X_{img}$, which is the soft attention mechanism $Attn(\cdot)$:

$\hat{X}=Attn(X_{conv},\hat{H}_{v})=Relu(CNN(X_{img}),W_{v}H_{v})$

where $X_{conv}\in\mathbb{R}^{L\times L\times C}$ is a feature calculated by
the forward propagation of convolutional layers, which with $C$ feature maps
and with dimension of $L\times L$. $W_{v}\in\mathbb{R}^{|H_{v}|\times
dim(X_{conv})}$ is a learnable parameter matrix that maps the swelling
cavities distribution $H_{v}$ to the visual space. Finally,
$\hat{X}=Attn(X_{conv},\hat{H}_{v})$ is inputed into the discriminator
$D(\cdot)$ to complete feedforward propagation.

###  3.3 Prediction Model For the performance of the materials

After generating the image of the irradiated material, it is necessary to
evaluate the various properties of the material under the condition $D_{c}$. A
network is constructed to achieve the image-to-data task:
$F_{p}:V\rightarrow\mathbb{R}^{dim(D_{r})}$, where $V$ represents the visual
space.

Before we design the structure of this performance evaluate network, let’s
revisit the two challenges in implementing this map: (1) Irradiated material
image $X_{img}$ and performance parameter $D_{r}$ are both highly abstract
data forms. It is very difficult to establish mapping between two highly
abstract data forms Jung et al. (2017). (2) The variables in the performance
parameters $D_{r}$ are not independent to each other, which have mutual
influence and physical connection.

To address the challenge 1, we introduce CNN to extract the visual features
for the image $X_{img}$, and the molecular composition feature vector $C_{m}$
is introduced to change the model $P(D_{r}|X_{img})$ to
$P(D_{r}|X_{img},C_{m})$; For the challenge 2, BiLSTM Schuster (1996) is used
to learn the dependency of the variables in the performance parameters
$D_{r}$. The prediction model is defined as:

$D^{i}_{r}=Relu(W_{h}[\overrightarrow{h_{i}},\overleftarrow{h_{i}}]+b_{h})$
$\overrightarrow{h_{i}}=\delta(W_{x}f\bar{X}+W_{f}h_{1:i-1}+b_{f})$
$\overleftarrow{h_{i}}=\delta(W_{x}b\bar{X}+W_{f}h_{>i}+b_{b})$
$\bar{X}=Relu(W_{x}\hat{X}_{img}+W_{m}C_{m}+b)$

where $\hat{X}_{img}=CNN(X_{img})$ is the visual feature matrix extracted by
convolutional neural networks(CNN),
$\overrightarrow{h_{i}},\overleftarrow{h_{i}}$ are the forward and backward
propagation hidden state vectors, which are concatenated together to predict
the $i$-th variable in the performance parameters $D^{i}_{r}$.

Figure 2: The data-to-image model based on prior knowledge and attention
mechanism achieves the generation of irradiated material images. Prior
knowledge distribution is employed to embed the molecular composition of
materials into the generative model. The attention mechanism is utilized to
generate images with physical connotations(swelling cavities distribution). In
the image-to-data task, CNN+BiLSTM is used to extract the features and learn
the dependency of the performance variables.

###  3.4 Learning

In order to generate real images with physical connotations, the learning loss
consists of two parts: GAN loss $\mathcal{L}_{GAN}$(prompting the model to
generate real images) and swelling cavities feature loss
$\mathcal{L}_{H_{v}}$(prompting the model to generate images with physical
connotations, adjusting the parameters of attention mechanism by
backpropagation). In order to calculate swelling cavities feature loss
$\mathcal{L}_{H_{v}}$, We need to measure the corresponding degree of the
image $X_{img}$ and the swelling cavities feature $H_{v}$, Which requires
encoding the image.

The image encoder: image encoder maps the image $X_{img}$ to the space of the
swelling cavities feature space by the convolutional neural network(CNN). The
middle layer of the CNN can learn local features of the region in the image
$X_{img}$. Finally, we map the CNN features of $X_{img}$ to the swelling
cavities distribution features through the fully connected
layers:$\bar{H}_{v}=En(X_{img})=Wfc(CNN(X_{img}))$.

The loss and learning steps: the loss function consists of the loss of GAN
$\mathcal{L}_{GAN}$ and the swelling cavities distribution features loss
$\mathcal{L}_{H_{v}}$, and the final loss based on prior knowledge and
attention mechanism is defined as
$\mathcal{L}=\mathcal{L}_{GAN}+\lambda\mathcal{L}_{H_{v}}$.

where $\lambda$ is a hyperparameter that balances two losses. The first loss
represents the unconditional loss of GAN Xu et al. (2017), and the second loss
represents the conditional loss constrainted by the swelling cavities
features. The loss $\mathcal{L}_{H_{v}}$ minimizes the difference between the
cavities feature $H^{\prime}_{v}$ of the generated image $X^{\prime}_{img}$
and the true cavities feature $H_{v}$, defined
as:$\mathcal{L}_{H_{v}}=||H^{\prime}_{v}-H_{v}||^{2}=||En(X^{\prime}_{img})-H_{v}||^{2}$.

The loss of GAN consists of the loss of the generator and the loss of the
discriminator. We assign the swelling cavities feature loss
$\mathcal{L}_{H_{v}}$ to the generator and the discriminator. The generator is
trained to generate real images $X^{\prime}_{img}$ with swelling cavities
features(physical connotations), whose loss is defined as:

$\mathcal{L}_{G}=-\mathbb{E}_{X^{\prime}_{img}\sim
P_{G}}logD(X^{\prime}_{img},H_{v})+\frac{\lambda}{2}\mathcal{L}_{H_{v}}$

The discriminator is trained to distinguish whether the input material image
$X^{\prime}_{img}$ is real or fake, whose loss is defined as:

$\mathcal{L}_{D}=-\mathbb{E}_{X_{img}\sim
P_{data}}logD(X_{img},H_{v})-\mathbb{E}_{X^{\prime}_{img}\sim
P_{G}}log(1-D(X^{\prime}_{img},H_{v}))+\frac{\lambda}{2}\mathcal{L}_{H_{v}}$

Finally, we can train the generator and discriminator alternately.

Algorithm 1 The training algorithm using minibatch SGD with learning rate
$\eta$

Input: minibatch images $X_{img}$; minibatch data $D_{d},D_{c},C_{m},H_{v}$;
minibatch size $S$;

1:for $n=1$ to $S$ do do

2: $z\sim N(\mu(C_{m}),\delta(C_{m}))$;

3: $X^{\prime}_{img}\leftarrow G(z,D_{d},D_{c})$;

4: $\mathcal{L}_{r}\leftarrow D(Attn(X_{img},H_{v}))$;

5: $\mathcal{L}_{f}\leftarrow D(Attn(X^{\prime}_{img},H_{v}))$;

6: $\mathcal{L}_{H_{v}}\leftarrow||En(X^{\prime}_{img})-H_{v}||^{2}$;

7: $\mathcal{L}_{D}\leftarrow-
log(\mathcal{L}_{r})-log(1-\mathcal{L}_{f})+\frac{\lambda}{2}\mathcal{L}_{H_{v}}$;

8:
$\theta_{D}\leftarrow\theta_{D}-\eta\bigtriangledown_{\theta_{D}}\mathcal{L}_{D}$;

9: $\mathcal{L}_{G}\leftarrow-
log(\mathcal{L}_{f})+\frac{\lambda}{2}\mathcal{L}_{H_{v}}$;

10:
$\theta_{G}\leftarrow\theta_{G}-\eta\bigtriangledown_{\theta_{G}}\mathcal{L}_{G}$;

11:end for

###  3.5 Training details

The training algorithm is implemented with the deep learning lib Keras. The
networks are randomly intialized without any pre-training and is trained with
decayed Adagrad and RMSprop. We train for a total of 2000 epochs and use a
batch size of 10, a learning rate of $1\times 10^{-5}$, and a weight decay
rate of $1.6\times 10^{-6}$ for the image generation task. The generated image
size is $32\times 32$. The molecule2vec embedding vectors are intialized as
random vectors.

##  4 Experiments

Extensive experiments is carried out to evaluate the proposed model by
comparing with multiple baseline models. First, we performed an experiment on
image generation task and evaluate the ability of the model to generate
experimental material images based on condition parameters(data-to-image
task). Then the network $P(D^{\prime}_{r}|X^{\prime}_{img})$ predicting the
performance parameters $D^{\prime}_{r}$ by the image $X^{\prime}_{img}$ is
tested(image-to-data task), which is also compared with the theoretical models
from computational materials science Ehrlich (1981) Boltax et al. (1978) Rest
& Hofman (1999).

###  4.1 Dataset

We introduce a dataset called ISMD(Irradiation Swelling Material Dataset) into
this task. This dataset includes more than 22,000 images with size of
$32\times 32$ and with shooting scale of $100nm$ from irradiation experiments,
corresponding to the molecular compositions set $m$ of 14 alloys, material
thermodynamics and mechanical properties set $D_{d}$, experimental condition
parameters set $D_{c}$, swelling cavities distribution set $H_{v}$, and the
performance parameters set $D_{r}$. The dataset is completed by data exported
from the experimental devices, manually labeled, image preprocessed. More
details about dataset ISMD can be found in Appendix.A;

###  4.2 Baselines

Baseline models for image generation.

  * •

Pure GAN: the prior distribution and the attention mechanism removed from the
proposed model.

  * •

The prior distribution dropped: only the prior distribution of the proposed
model removed and is used to evaluate the value of prior distribution.

  * •

The attention mechanism dropped: only the attention mechanism of the proposed
model removed and is used to evaluate the value of the attention mechanism.

  * •

Variational Auto-Encoder: To verify whether GAN is a better generation model
for this task.

Baseline model for materials performance prediction: The theoretical model
from computational material science is used to evaluate the prediction model
in image-to-data task.

(a) data-to-image

(b) image-to-data

Figure 3:  (a):The images generated by the proposed model and the baseline
models. (b):The materials performance prediction model compared to theoretical
models and the experimental results.

###  4.3 Evaluation Metrics

Evaluation metrics for image generation: Inception score Salimans et al.
(2016) Springenberg (2015)is the evaluation method for generating images. The
core idea of the method is: images contain meaningful objects should have a
conditional label distribution $p(y|x)$ with low entropy, moreover, the model
is expected to generate varied images, so the marginal $\int
p(y|x^{\prime}=G(z))dz$ should have high entropy.Combining these two
requirements, the evaluation is defined as
$exp(\mathbb{E}_{m}KL(p(X^{\prime}_{img}|m)||p(X_{img})))$, where $m$ is the
material representation vector.

Evaluation metrics for materials performance prediction: the material
performance parameters from the performance evaluation experiments is
$D^{*}_{r}$, the parameters calculated by the theoretical models Ehrlich
(1981) Boltax et al. (1978) Rest & Hofman (1999) is $\hat{D}_{r}$, and which
predicted by the image $X_{img}$ in the proposed model
$P(D_{r}|X_{img},C_{m})$ is $D^{\prime}_{r}$. Finally, the accuracy evaluation
is defined as the RMSE score
$S_{RMSE}=\sqrt{\frac{1}{m}\sum_{i}(D^{\prime(i)}_{r}-\hat{D^{(i)}_{r}})^{2}}$.

Table 1: The data-to-image task. models | Inception score
---|---
VAE | 1.89 $\pm$ 0.05
GAN | 2.30 $\pm$ 0.07
PR+GAN | 2.40 $\pm$ 0.02
Att+GAN | 2.84 $\pm$ 0.07
Our Model | 3.87 $\pm$ 0.08
Real Data | 7.11 $\pm$ 0.14
Table 2: The accuracy evaluation for materials performance prediction. prediction tasks | img-to-data | theoretical models
---|---|---
$\delta_{s}$ | 1.96 | 1.80
$\delta_{b}$ | 1.92 | 1.63
$\delta_{e}$ | 1.89 | 1.91
$\delta_{L}$ | 1.98 | 1.55
$H_{B}$ | 2.19 | 1.75
$H_{RC}$ | 2.23 | 1.57
$H_{V}$ | 2.21 | 2.24
$K$ | 1.96 | 1.98

###  4.4 Results

For the data-to-image task, the proposed model performs better on the ISMD
dataset than other baseline models, achieving higher scores.For the image-to-
data task, our model performs better on the prediction of some material
properties compared to the theoretical model. In summary, the combination of
the image-to-data model and computational material science models should be
better.

Figure 4: We put the trained models on the server and developed a Web App.

##  5 Conclusion

In this work, we propose the data-to-image model based on prior knowledge and
attention mechanism to achieve the generation of irradiated material images.
In the image-to-data task, CNN and BiLSTM are used to extract the features and
learn the dependency of the variables. In the future work, we plan to generate
more controllable and diverse material images with physical connotations and
predict the performance data to replace the experiments.

## Appendix

Figure 5:  Images generation of the baseline models and proposed model from
different epochs.

###  .1 material thermodynamics and mechanical properties set $D_{d}$

mechanical properties:

$E$: modulus of elasticity (unit: $MPa$); $\nu$: Poisson’s ratio;
$\bar{\delta}$: Equivalent stress (unit: $MPa$); $\bar{\varepsilon}$:
Equivalent plastic strain (unit: $MPa$);
$n$: strain hardening index; $m$: strain rate sensitivity index;
$K$: intensity factor; Lame constant $\lambda$;
Lame constant $G$;

thermodynamics:

$C_{t}$: crystal type; $C_{d}$: lattice parameter (unit: $nm$);
$T_{m}$: melting point (unit: $K$); $\rho$: theoretical density (unit:
$g/cm^{3}$);
$\hat{V}$: Thermal expansion coefficient (unit: $10^{-6}\cdot K^{-1}$);
$C_{h}$: Thermal conductivity (unit: $W\cdot m^{-1}\cdot C^{-1}$);
$H_{c}$: heat capacity (unit: $KJ/mol\cdot K$); $E_{h}$: hot (unit: $KJ/mol$);
$C_{e}$: Seebeck temperature difference electromotive force factor (unit:
$uV/K$); $C_{R}$: resistivity (unit: $10^{3}\cdot\Omega\cdot cm$);

###  .2 experimental condition parameters set $D_{d}$

Fast neutron injection volume $\phi_{f}$(unit: $10^{19}n/cm^{2}$); Thermal
neutron injection amount $\phi_{t}$(unit: $10^{19}n/cm^{2}$);
Irradiation flux $\phi_{i}$(unit: $10^{19}n/cm^{2}$); Irradiation temperature
$T_{i}$(unit: $K$);
Experimental temperature $T_{e}$(unit: $K$);

Figure 6: Left: We visualize the distribution of the molecular composition
feature vector $C_{m}$ in two-dimensional space.Right: The swelling cavities
distribution $H_{v}$.

###  .3 performance parameters set $D_{r}$

Yield limit $\delta_{s}$(unit: $MPa$); Stretch limit $\delta_{b}$(unit:
$MPa$);
Elastic limit $\delta_{e}$(unit: $MPa$); The total extension rate is
$\delta_{L}$(unit: $\%$);
Brinell hardness $H_{B}$(unit: $kg/mm^{2}$); Rockwell hardness $H_{RC}$(unit:
$mm$) (dimensionless: HR*T);
Vickers hardness $H_{V}$(unit: $kg/mm^{2}$); Volume expansion rate
$K_{v}$(unit: $\%$);
Irradiation growth rate $K_{L}$(unit: $\%$); Fracture toughness $K_{Ic}$(unit:
$MPa/m^{1/2}$);
Creep performance $\delta_{t}$(unit: $MPa$); Crisp feature
$C_{He}\in\\{0,1\\}$;

## References

  * Alom et al. (2018) Md Zahangir Alom, Mahmudul Hasan, Chris Yakopcic, Tarek M. Taha, and Vijayan K. Asari.  Recurrent residual convolutional neural network based on u-net (r2u-net) for medical image segmentation.  _IEEE Conf_ , 2018.
  * Bachman & Precup (2015) Philip Bachman and Doina Precup.  Data generation as sequential decision making.  _Computer Science_ , pp. 3249–3257, 2015.
  * Boltax et al. (1978) A. Boltax, J. P. Foster, J. E. Kalinowski, and D. C. Swenson.  Design applications of irradiation creep and swelling data.  _Trans. Am. Nucl. Soc.; (United States)_ , 28, 1978.
  * Cireşan et al. (2013) D. C. Cireşan, A Giusti, L. M. Gambardella, and J Schmidhuber.  Mitosis detection in breast cancer histology images with deep neural networks.  In _International Conference on Medical Image Computing and Computer-Assisted Intervention_ , pp. 411–8, 2013.
  * Ehrlich (1981) Karl Ehrlich.  Irradiation creep and interrelation with swelling in austenitic stainless steels.  _Journal of Nuclear Materials_ , 100(1):149–166, 1981.
  * Goodfellow et al. (2014) Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio.  Generative adversarial nets.  In _International Conference on Neural Information Processing Systems_ , pp. 2672–2680, 2014.
  * Gregor et al. (2015) Karol Gregor, Ivo Danihelka, Alex Graves, Danilo Jimenez Rezende, and Daan Wierstra.  Draw: a recurrent neural network for image generation.  _Computer Science_ , pp. 1462–1471, 2015.
  * Jing et al. (2017) Baoyu Jing, Pengtao Xie, and Eric Xing.  On the automatic generation of medical imaging reports.  _ICML_ , 2017.
  * Jung et al. (2017) Kim Uk Jung, Kim Gu Hak, and Ro Man Yong.  Iterative deep convolutional encoder-decoder network for medical image segmentation.  _IEEE Conf_ , 2017.
  * Kingma & Welling (2013) Diederik P Kingma and Max Welling.  Auto-encoding variational bayes.  _Arxiv_ , 2013.
  * Kisilev et al. (2015) P. Kisilev, E. Walach, E. Barkan, B. Ophir, S. Alpert, and S. Y. Hashoul.  From medical image to automatic medical report generation.  _Ibm Journal of Research and Development_ , 59(2/3):2:1–2:7, 2015.
  * Li et al. (2018) Wei Li, Kevin G. Field, and Dane Morgan.  Automated defect analysis in electron microscopic images.  _Arxiv_ , 2018.
  * Mansimov et al. (2015) Elman Mansimov, Emilio Parisotto, Jimmy Lei Ba, and Ruslan Salakhutdinov.  Generating images from captions with attention.  _Computer Science_ , 2015.
  * Menze et al. (2010) B. H. Menze, Leemput K Van, D Lashkari, M. A. Weber, N Ayache, and P Golland.  A generative model for brain tumor segmentation in multi-modal images.  _Ibm Journal of Research and Development_ , 13(2):151–9, 2010.
  * Mikolov et al. (2013) Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean.  Efficient estimation of word representations in vector space.  _Computer Science_ , 2013.
  * Mirza & Osindero (2014) Mehdi Mirza and Simon Osindero.  Conditional generative adversarial nets.  _Computer Science_ , pp. 2672–2680, 2014.
  * Porollo et al. (2000) S. I. Porollo, A. N. Vorobjev, Yu. V. Konobeev, N. I. Budylkin, E. G. Mironova, F. A. Garner, S. I. Porollo, A. N. Vorobjev, Yu. V. Konobeev, and N. I. Budylkin.  Irradiation creep and stress-affected swelling in austenitic stainless steel 16cr-15ni-3mo-nb-b irradiated in the bn-350 reactor.  _Aaps Pharmscitech_ , 13(4):1386–1395, 2000.
  * Rest & Hofman (1999) J. Rest and G. L. Hofman.  Dart model for irradiation-induced swelling of uranium silicide dispersion fuel elements.  _Nuclear Technology_ , 126(1):88–101, 1999.
  * Ronneberger et al. (2015) Olaf Ronneberger, Philipp Fischer, and Thomas Brox.  _U-Net: Convolutional Networks for Biomedical Image Segmentation_.  Springer International Publishing, 2015.
  * Rovinelli (2018) A Rovinelli.  Using machine learning and a data-driven approach to identify the small fatigue crack driving force in polycrystalline materials.  _Arxiv_ , 2018.
  * Salimans et al. (2016) Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen.  Improved techniques for training gans.  _Computer Science_ , 2016.
  * Schuster (1996) Mike Schuster.  Bi-directional recurrent neural networks for speech recognition.  _Ieice Technical Report Speech_ , 96:7–12, 1996.
  * Springenberg (2015) Jost Tobias Springenberg.  Unsupervised and semi-supervised learning with categorical generative adversarial networks.  _Computer Science_ , 2015.
  * Ssm et al. (2017) Salehi Ssm, D Erdogmus, and A Gholipour.  Auto-context convolutional neural network (auto-net) for brain extraction in magnetic resonance imaging.  _IEEE Transactions on Medical Imaging_ , PP(99):1–1, 2017.
  * Xu et al. (2017) Tao Xu, Pengchuan Zhang, Qiuyuan Huang, Han Zhang, Zhe Gan, Xiaolei Huang, and Xiaodong He.  Attngan: Fine-grained text to image generation with attentional generative adversarial networks.  _Arxiv_ , 2017.
  * Zhang et al. (2016) Han Zhang, Tao Xu, and Hongsheng Li.  Stackgan: Text to photo-realistic image synthesis with stacked generative adversarial networks.  _Computer Science_ , pp. 5908–5916, 2016.
  * Zhang et al. (2017) Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiaogang Wang, Xiaolei Huang, and Dimitris N. Metaxas.  Stackgan++: Realistic image synthesis with stacked generative adversarial networks.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , PP(99), 2017.