# A Lightweight Transformer for Faster and Robust EBSD Data Collection

Harry Dong  Carnegie Mellon University, Department of Electrical and Computer
Engineering, Pittsburgh, 15289, USA<EMAIL_ADDRESS>Sean Donegan  Air
Force Research Laboratory, Materials and Manufacturing Directorate, Wright-
Patterson AFB, 45433, USA  Megna Shah  Air Force Research Laboratory,
Materials and Manufacturing Directorate, Wright-Patterson AFB, 45433, USA
Yuejie Chi  Carnegie Mellon University, Department of Electrical and Computer
Engineering, Pittsburgh, 15289, USA

###### Abstract

Three dimensional electron back-scattered diffraction (EBSD) microscopy is a
critical tool in many applications in materials science, yet its data quality
can fluctuate greatly during the arduous collection process, particularly via
serial-sectioning. Fortunately, 3D EBSD data is inherently sequential, opening
up the opportunity to use transformers, state-of-the-art deep learning
architectures that have made breakthroughs in a plethora of domains, for data
processing and recovery. To be more robust to errors and accelerate this 3D
EBSD data collection, we introduce a two step method that recovers missing
slices in an 3D EBSD volume, using an efficient transformer model and a
projection algorithm to process the transformer’s outputs. Overcoming the
computational and practical hurdles of deep learning with scarce high
dimensional data, we train this model using only synthetic 3D EBSD data with
self-supervision and obtain superior recovery accuracy on real 3D EBSD data,
compared to existing methods.

## Introduction

Experimental methodologies for three dimensional tomography of the internal
microstructure of materials has been refined considerably in the past few
decades [1, 2, 3, 4, 5, 6, 7, 8], growing to include a variety of modalities
such as X-ray computed tomography (CT), optical imaging, electron imaging,
energy dispersive X-ray spectroscopy (EDS), and electron back-scattered
diffraction (EBSD) data, among others. In all of these cases, a volume of
material is interrogated slice by slice, and these slices are then stacked and
reconstructed using a variety of software tools. Three dimensional
microstructure data, while often laborious to generate compared to collecting
a single 2D section, can provide unique insight, including things like the
topology of microstructural features like grains, pores, precipitates and the
true shape and size distributions of such features. Properties such as fatigue
and oxide transport are sensitive to the three dimensional arrangement of
these features [9, 10, 11]. And novel manufacturing processes like additive
manufacturing also demand three dimensional characterization of microstructure
to fully link the processing to the structure [3, 12].

While advancements have been made in robustness and closed loop collection of
data [2], corruptions in data can still happen. This may not be as significant
of a problem in non-destructive techniques where the data can be recollected,
but in destructive techniques like serial sectioning, it may often be the case
that a few slices out of a thousand have no data or are of very poor quality,
lowering the accuracy of the reconstruction. The corruptions can happen for a
number of reasons including removing more material (via mechanical polishing
or laser/ion abalation) than planned, the electron source nearing the end of
its life resulting in low signal images, the magnification on the microscope
being incorrect for a slice, or the brightness/contrast settings result in an
over/under saturated image. While adjustments to the control software can be
made to prevent these and other issues, it is hard to a-priori imagine every
reason a slice or subset of data can be corrupted.

However, if most of the data was collected properly, we hypothesize it should
be possible to infer a substantial amount of the missing data. The current
common practice is to fill in missing slices is to just copy the layer above
or below the missing slice [2]. This is reasonable in most serial sectioning
cases where the slices are being collected at a high enough frequency that
most of the data stays the same from slice to slice. Still, there is some room
for improvement on nearest neighbor replacement type approaches, and the
transformer model, which has been used to fill in text data, is a tantalizing
framework for also filling in missing data in sequential image data.

Since its introduction [13], transformers have been the backbone of many
impressive breakthroughs in natural language processing (NLP). In their
general form, transformers are excellent at processing sequential information
in parallel. Consequently, its use has spread across multiple domains, such as
in computer vision [14], speech processing [15], and bioinformatics [16]. In
our case, transformers are appealing to EBSD data because the readings are
inherently sequential as it represents a real physical structure.

Early large transformer models such as BERT [17] and GPT [18] were pretrained
on copious amounts of text data with the intention of learning features to
model the language through associations between words and phrases with a
collection of self-supervised tasks [19]. Unlike supervised tasks, such as
classification, which requires labeled datasets, self-supervised tasks involve
automatically generating labels from unlabeled data, such as unscrambling a
sequence of shuffled sentences, which force the model to vaguely learn the
structure of the data. This technique remained fairly intact even when scaling
to billion-parameter large language models (LLMs), enormous models that
process much longer sequences over the course of hundreds of billions of
tokens (language units comprising of a sequence of characters) [20, 21, 22].
Taking inspiration from a similar popular self-supervised task in NLP, our
training procedure involves randomly masking a slice in an EBSD volume and
having the model predict the masked slice.

Our slice recovery task is closely related to that of masked image modeling,
masked autoencoders, and inpainting in computer vision [23, 24, 25, 26, 27].
Even though these may be applicable and there exists plenty of transformers in
computer vision [28, 14], we want our model to also be computationally
efficient as we scale our method and leverage the fact that EBSD volumes have
sparse structures [29], the dynamics of which operate differently from typical
images or videos. Regarding efficiency, the fact that EBSD produces high
dimensional data means the final model’s computational footprint cannot be
ignored.

Taking these into consideration, our contributions are the following:

  1. 1.

We propose a novel method to recover missing EBSD data consisting of a
scalable transformer model and straightforward projection algorithm that
produces superior results compared to existing methods. This is accentuated
when zoning in on the recovery accuracy of grain boundaries where other
methods appear to perform poorly.

  2. 2.

We demonstrate that despite being trained solely on synthetic data, our
transformer can generalize to real EBSD data without additional training while
still outperforming all the baselines. This robustness to out-of-distribution
EBSD data overcomes a major limitation of relatively low amount of available
real EBSD data that deep learning requires.

  3. 3.

Our results suggest that serial sectioning experiments using similar
experimental parameters to those in our test datasets, collection time can
effectively be slashed by up to 25% with very little error in predicted voxel
orientation, as the collection of every fourth slice can be bypassed. With the
current lengthy procedure of EBSD, this is a significant improvement in
efficiency.

## Background

### Terminology and Notation

For this paper, we use bold capital letters (e.g. $\bm{A}$) for matrices and
capital calligraphic letters (e.g. $\bm{\mathcal{A}}$) for tensors. Scalars
are represented by plain uppercase and lowercase letters (e.g. $A$ and $a$).
Tuples are used to describe the shape of a structure, so if
$\bm{\mathcal{A}}\in\mathbb{R}^{N_{1}\times N_{2}\times N_{3}}$, then
$\bm{\mathcal{A}}$ is of shape $(N_{1},N_{2},N_{3})$ and vice versa. A
dimension is an index (starting at 1) in the shape tuple, and the size of a
dimension $i$ is the value of the shape tuple at that index $i$. When we refer
to multiple volumes that share the same first $k$ dimensions (e.g.
$\bm{\mathcal{A}}\in\mathbb{R}^{N_{1}\times N_{2}}$,
$\bm{\mathcal{B}}\in\mathbb{R}^{N_{1}\times N_{2}\times N_{3}}$, and
$\bm{\mathcal{C}}\in\mathbb{R}^{N_{1}\times N_{2}\times N_{3}^{\prime}}$), we
use $*$ to capture all remaining and possibly different dimensions (e.g.
$\bm{\mathcal{A}}$, $\bm{\mathcal{B}}$, and $\bm{\mathcal{C}}$ have shape
$(N_{1},N_{2},*)$).

### Transformers

In this section, we outline the technical details of the encoder-only
transformer based on its first introduction [13]. The same source describes
encoder-decoder and decoder-only transformers for interested readers.

#### Input Processing

The first layer of a transformer is the embedding layer. For a single length
$N$ sequence of $D_{in}$-dimensional vectors, $\bm{X}\in\mathbb{R}^{N\times
D_{in}}$, each vector is transformed into a vector of size $D_{out}$. If the
input is a sequence of tokens such as in language tasks, then $D_{in}=1$. The
transformation could be a linear one, or a more complex nonlinear one such as
another neural network. The result is an embedded sequence of shape
$(N,D_{out})$. Next, positional encoding is added to the embedding to inject
positional information to each vector in the sequence. Positional encodings
can either be fixed or learned.

#### Multi-head Self-attention

The idea of self-attention is to find varying degrees of associations between
elements in a sequence. Transformers typically implement multi-headed self-
attention within each encoder layer, which allows the model to learn different
types of associations. Let $\bm{X}\in\mathbb{R}^{N\times D}$ be the input
sequence into the attention mechanism and
$\bm{W}_{h}^{Q},\bm{W}_{h}^{K},\bm{W}_{h}^{V}\in\mathbb{R}^{D\times\frac{D}{H}}$
for head $h=1,2,\dots,H$. Additionally, let $\sigma$ be the row-wise softmax
operator. Then, the attention for head $h$, $\bm{A}_{h}$ is defined as

$\displaystyle\bm{A}_{h}=\sigma\left(\frac{\bm{Q}_{h}\bm{K}_{h}^{\top}}{\sqrt{D/H}}\right)\bm{V}_{h},$
(1)

where
$\bm{Q}_{h}=\bm{X}\bm{W}_{h}^{Q},\bm{K}_{h}=\bm{X}\bm{W}_{h}^{K},\bm{V}_{h}=\bm{X}\bm{W}_{h}^{V}$
are the query, key, and value matrices, respectively. Next, the results are
concatenated and linearly transformed to produce the output
$\bm{Y}=[\bm{A}_{1}\cdots\bm{A}_{H}]\bm{W}_{O}$ for
$\bm{W}_{O}\in\mathbb{R}^{D\times D}$. Note that $\bm{A}_{1},\dots,\bm{A}_{H}$
can be computed in parallel.

The main source of inefficiency is the computation of
$\bm{Q}_{h}\bm{K}_{h}^{\top}$ which levies a computational and memory cost of
$\mathcal{O}(N^{2})$. Consequently, the utility of transformers can be limited
for long sequences without proper hardware. This becomes a dire issue as the
dimensionality of the input grows, such as in images and videos which can be
flattened into a long sequence. Thankfully, there has been substantial work in
reducing the complexity with more efficient attention approximation
mechanisms, such as Linformer [30], Reformer [31], Big Bird [32], and many
others [33].

#### Axial Self-attention

When it comes to high-dimensional data, such as the EBSD volumes, the sequence
length explodes if we flatten (or patchify in the case of many transformers in
computer vision [14]) the input and apply vanilla self-attention, making
attention computation a serious bottleneck. Furthermore, since we observe a
lot of structure with EBSD data, it is reasonable to utilize some simplified
attention mechanism. Our model uses axial attention [34, 35] which runs the
self-attention mechanism along the dimensions of the input tensor. For
instance, in a cube, each voxel only attends to voxels in the same row,
column, or depth. This greatly reduces the amount of computation and memory,
especially for higher order tensors. Intuitively, axial attention is
appropriate because we hypothesize that since local information can be quite
uniform (nearby voxels are likely to be in the same grain), long range
information should also be included. With axial attention, it is highly likely
to also obtain information in other grains and its boundaries.

In terms of implementation, the formula for multi-headed attention (1) can be
reused. Let $\bm{\mathcal{X}}\in\mathbb{R}^{N_{1},\dots,N_{K},D}$ be a single
$K$-dimensional input where $(N_{1},\dots,N_{K})$ defines the shape of the
volume and $D$ is the embedding size. As an example, suppose we are interested
in finding axial attention along the $k$-th dimension for $1<k<K$. With proper
dimension permutation and flattening, we can reshape $\bm{\mathcal{X}}$ to a
tensor of shape $(\prod_{i\neq k}N_{i},N_{k},D)$ and compute multi-headed
attention by treating the first dimension as the batches. In our model, we
repeatedly use the outputs of axial attention to compute axial attention along
the next dimension.

#### Feedforward Blocks

The next major component to create a transformer is the feedforward block, a
multilayer perceptron with a single hidden layer. The activation function is
often either a Gaussian Error Linear Unit (GELU) or Rectified Linear Unit
(ReLU) function, and although the input and output sizes are identical, the
hidden size can be much larger. Consequently, although self-attention is a
computational bottleneck, the feedforward blocks are the dominant source of
parameters in the model.

#### Putting it All Together

With all these components, we are ready to define the general architecture of
a transformer. First, an input is processed by an embedding layer and
positional encoding layers at the start. Then, the input progresses into the
encoder, a sequence of alternating self-attention and feedforward blocks. For
training stability, residual connections and normalization layers are inserted
in between each attention block and feedforward block. Finally, the features
from the encoder are linearly projected to be the desired shape of the output.
A visualization of a vanilla and axial transformer, which mainly differ in
their attention implementation, is depicted in Figure 1.

Figure 1: Transformer architecture (left) with either vanilla attention
(center) or axial attention (right). The axial transformer is produced by
simply substituting full attention with axial attention. Although axial
attention has significantly more layers, it scales much more favorably for
high-dimensional data. Attention layers take three inputs for the query, key,
and value.

## Method

We propose a transformer model to learn missing slices of EBSD data, followed
by a projection step to smooth out the voxel values. Due to limited real EBSD
data, our goal is to train this model on a large and diverse synthetic dataset
and demonstrate that it can generalize to real EBSD data, which we evaluate on
two nickel superalloy EBSD volumes, one for alloy IN625 [36, 37] and one for
alloy IN718 [38, 39]. The following repository contains the code for our
method: https://github.com/hdong920/ebsd_slice_recovery.

### Data Description and Preparation

Each dataset (both synthetic and experimental) includes orientation
information at every voxel in a 3D image. For the experimental data, a
substatial amount of preprocessing was done to handle the alignment of the
data and clean up noise; a complete description of the preprocessing may be
found in Chapman et al.[2] and Stinville et al.[38] In particular, we remove
any grains smaller than 27 voxels ($3^{3}$) and average orienations per grain.
The original volume containing Euler angles is of shape
$(N_{1},N_{2},N_{3},3)$, where $N_{1},N_{2},N_{3}$ are the physical
dimensions, and the final dimension represents the 3 Euler angles needed to
define an orientation. Additional arrays are then computed from the input
data:

  * •

Cubochorics: A volume of shape $(N_{1},N_{2},N_{3},3)$ where the last
dimension contains the cubochoric coordinates [40] converted from the original
Euler angles at each voxel. Cubochoric coordinates are chosesn since the
Euclidean metric is used for regressing the transfomer model. The Euclidean
distance between points in Euler angle space does not necessarily relate to
the angular distance between those points. While this is also true for points
in cubochoric space, as the cubochoric representation is an equal-volume
mapping of _SO_(3) onto a grid as opposed to an equal-angle mapping, in
practice the angular discretization between Euclidean neighbors in cubochoric
space is relatively consistent across the grid. Thus, for all of our
experiments, we operate completely in cubochoric space, under the assumption
that the Euclidean metric is a reasonable approximation for similarity between
points in this space.

  * •

IDs: A volume of shape $(N_{1},N_{2},N_{3})$ which assigns identification
numbers (IDs) to denote which grain each voxel belongs to. Each ID number has
a unique vector of cubochoric coordinates associated with it. While not needed
during training, these ID numbers will be used to smooth our model outputs and
evaluate model accuracy. For experimental data, the IDs are found by
segmenting the grains using a misorientation tolerance [2, 38], and for
synthetic data, the IDs are generated alongside the orientation data.

  * •

Boundaries: A volume of shape $(N_{1},N_{2},N_{3})$ containing Boolean values
indicating if a voxel is on the grain boundary. More specifically, a voxel is
a boundary voxel is at least one of its neighbors is of a different ID number
than itself, where two voxels are neighbors if they share a face.

To illustrate, Figure 2 contains example slices of (scaled and shifted)
Cubochorics and Boundaries of IN625 and IN718.

(a)

(b)

Figure 2: Three consecutive crops of real EBSD slices from IN625 [2] (Figure
2(a)) and IN718 [38] (Figure 2(b)). The top row shows the cubochoric values
that are scaled and shifted for better visualization, and the bottom row shows
the same region in Boundaries. Note that the height and width of each slice
here is 64 voxels and not the original shape.

Synthetic volumes are generated via DREAM.3D [41]. Each synthetic training and
validation volume is of shape $(192,192,192,*)$ and $(64,192,64,*)$,
respectively. Within DREAM.3D, we generate 9 training volumes while
independently varying the mean grain size and mean transformations per grain.
In particular, we generate volumes with mean grain sizes 2, 2.5, and 3 with no
twins; we also generate volumes with mean twins frequencies 0, 1, 2, 3, 4, and
5 while fixing mean grain size to be 2.3. Due to the nature of the software,
these parameters are unitless, as we can always synthetically increase or
decrease the granularity of the volume. For example slices of these synthetic
volumes, see Figure 3. The grain size distributions for each dataset are shown
in Figure 3 as probability plots. The natural logarithm of the normalized
grain sizes are shown; ideal lognormally distributed data would lie on
straight lines in such plots. In general, the grain sizes are primarily
lognormal near their means, with noted deviations from lognormality in their
tails, which is a known phenomenon [42].

(a)

(b)

(c)

(d)

(e)

Figure 3: Example synthetic slices of cubochoric values that are scaled and
shifted for visualization. Keeping mean grain size fixed, Figures 3(a), 3(b),
and 3(c) show generated slices when we specify the mean frequency of twins per
grain to be 0, 2, and 4, respectively. With no twins, Figures 3(d) and 3(e)
show generated slices when we specify the mean grain size to be 2 and 3,
respectively. All slices have a height and width of 64 voxels.

(a)

(b)

Figure 4: Probablity plots showing the grain size distributions for each
dataset. Figure 4(a) compares the real test datasets to the synthetic training
datasets without twins, while Figure 4(b) compares the real test datasets to
the synthetic training datasets with twins. Grain sizes are represented as
sphere equivalent diameters, D, normalized by the distribution mean.

### Training Details

We first describe a few data augmentation steps. The number of unique
cubochoric coordinates is quite limited, so color shift transformations, along
with other augmentations, are critical. Note that use of the word "color" is
loose here because we treat the cubochoric coordinates the same as color
channels in computer vision. In particular, our augmentations include random
linear color shifts, rotations, and flips.

Using the 9 synthetic volumes generated by DREAM.3D, we train our axial
transformer model in a self-supervised fashion similar to masked language
modeling tasks [17]. These volumes are first normalized along each of the
three cubochoric indices. Each training input is sampled from a randomly
chosen volume with physical dimensions randomly permuted. Not only is cropping
necessary due to computational limits, it also acts as another form of
augmentation. For a sample $\bm{\mathcal{X}}_{\star}\in\mathbb{R}^{64\times
7\times 64\times 3}$, one of the central 5 slices (along the second dimension)
is randomly masked. If $m$ is the index of the masked or unobserved slice,
define $\bm{\mathcal{M}}\in{[0,1]}^{64\times 7\times 64\times 3}$ to be a mask
such that $[\bm{\mathcal{M}}]_{\cdot,m}=0$ and $1$ elsewhere. Then, the model
input and output will be $\bm{\mathcal{X}}_{\star}\odot\bm{\mathcal{M}}$ and
$\widehat{\bm{\mathcal{X}}}\in\mathbb{R}^{64\times 7\times 64\times 3}$,
respectively.

Figure 5: Changes between slices are captured in boundary voxels. The top row
shows two consecutive slices. The bottom row shows their respective slices in
Boundaries (red) with the ID difference between the two slices overlaid
(white). Note that differences between these slices lie fully in the boundary
voxels of both slices.

Since EBSD produces very structured data, we can leverage this to design a
more effective loss function. A simple mean-squared-error (MSE) loss function
across all voxels would not be sufficient because most voxels in the input are
observed, so we would risk learning an identity function which would produce a
fairly low loss value. A better approach would be to find the MSE of only the
missing slice, analogous to masked language modeling objective functions [17].
However, based on Figure 2, we see that many of voxels remain unchanged across
multiple slices, and the slice-to-slice changes all lie along the grain
boundaries (see Figure 5). Since voxels within grains are easy to predict the
values of, the source of difficulty is recovering the boundary voxels.
Therefore, we opt for a MSE loss function that only considers boundary voxels
in the missing slice. More formally, letting $k$ be the index of the missing
slice and $\bm{\mathcal{E}}\in{[0,1]}^{64\times 64\times 1}$ be the missing
slice’s boundaries map in Boundaries, the loss function we use is

$\displaystyle\mathcal{L}(\widehat{\bm{\mathcal{X}}},\bm{\mathcal{X}}_{\star})=\frac{\left\|[\widehat{\bm{\mathcal{X}}}-\bm{\mathcal{X}}_{\star}]_{\cdot,m}\odot\bm{\mathcal{E}}\right\|_{\operatorname{\mathsf{F}}}^{2}}{\left\|\bm{\mathcal{E}}\right\|_{0}},$
(2)

following broadcasting rules. By the way we defined $\bm{\mathcal{E}}$,
$\left\|\bm{\mathcal{E}}\right\|_{0}$ is the number of unobserved boundary
voxels. In summary, this loss function essentially averages MSE error across
all unobserved boundary voxels.

Using this scheme, we train our 8-headed 8-layer model using stochastic
gradient descent with a momentum parameter of 0.9 and weight decay of
$1\mathrm{e}{-5}$. Using cosine schedules, we warm-up the learning rate up to
$0.01$ for 8000 steps. While we decay the learning rate until 160000 total
gradient steps are taken with batches of 1 sample, performance plateaus around
halfway. The model uses learnable positional encoding, 4 attention heads, an
embedding size of 128, and a feedforward size of 512. The feedforward block is
consists of two 3D convolutions with a window size of 3 along each dimension
separated by a GELU. See Figure 1 for a visualization of the architecture.
Notably, our model is compact for a transformer, only consisting of slightly
under 30 million parameters. We also apply 10% dropout. Recall that the
transformer returns predicted cubochoric coordinates at each voxel of the same
shape as the input, but only the masked slice contributes to the loss function
(2).

### Nearest Neighbor Projection

The final step is to use outputs of our transformer, which are continuous
values, to assign each voxel to a grain and produce a smoother slice with
fewer intra-grain variations. To do this, we first construct a dictionary
relating observed grain IDs to cubochoric coordinates. Then, we assign voxels
whose neighbors (voxels that share a face) that reside in the previous and
next slice have the same ID to be that ID. We observe these voxels act like
anchors that provide more neighbors for voxels that are more difficult to
classify, which empirically improves the recovery. Next, we begin projecting
voxels with the most observed or previously projected neighbors and
incrementally decrease the requirement on the number of neighbors when further
projections are impossible until all voxels have been assigned an ID.
Projections are determined by the minimum $\ell_{2}$ distance to relevant
cubochoric coordinates pulled from the dictionary based on neighboring IDs.
Summarized in Figure 6, the projection algorithm essentially turns the
transformer outputs of the missing slice into IDs which can then be converted
into cubochoric coordinates.

Figure 6: Projection algorithm to take transformer outputs and assign voxels
to grains. White boxes denote which voxels have not been projected yet. The
input contains the transformer output and observed adjacent slices. First, the
anchoring step assigns voxels whose neighbors in adjacent slices are from the
same grain. Then, voxels are sequentially projected to their neighbors, as
indicated by the small arrows, starting with the voxels with the most observed
and previously projected neighboring voxels.

## Experiments

With our method defined, we now evaluate its performance on synthetic and real
EBSD data with no additional training or fine tuning. Even though our method
overwhelmingly outperforms the baselines in terms of recovery, we also point
out some weaknesses and avenues for improvement.

There are three baselines we compare to. One is k-nearest neighbors (KNN)
where a voxel’s ID is determined by vote based only on observed or previously
assigned IDs among its neighboring voxels. This is the exact process of the
projection step (including the anchoring procedure), except instead of using a
distance metric, voxels are assigned IDs by a vote system. Ties are broken
randomly. Another method, which is currently employed as a simple solution to
missing slices, is to copy an adjacent slice to replace the missing slice.
This usually maintains fairly decent accuracy since the changes from slice to
slice are minuscule compared to the number of voxels. As such, copying the
previous and next slice are our other two baselines.

### Performance

Because the changes from slice to slice are captured by boundary voxels, we
define two different accuracy metrics using the IDs. We denote the accuracy of
each voxel in the recovered slice as the overall accuracy and the accuracy
only considering boundary voxels in the recovered slice as the boundary
accuracy. The latter poses a greater challenge for all models, as it is
consistently lower than the overall accuracy where the influence of non-
boundary voxels often dominates.

While we report the mean accuracies and standard deviation across validation
samples, the performance of one method is heavily correlated with the
performance of others. For instance, if a particular slice is difficult to
recover for one method, it is likely to be difficult for all other methods. To
better compare the performances between our transformer and each baseline, we
obtain differences in accuracy for each sample. Namely, we find
$d(m,\bm{\mathcal{X}}_{\star},\widehat{\bm{\mathcal{X}}},\widehat{\bm{\mathcal{X}}}_{b}):=m(\bm{\mathcal{X}}_{\star},\widehat{\bm{\mathcal{X}}})-m(\bm{\mathcal{X}}_{\star},\widehat{\bm{\mathcal{X}}}_{b})$
for an accuracy metric $m$, ground truth $\bm{\mathcal{X}}_{\star}$, and
outputs, $\widehat{\bm{\mathcal{X}}}$ and $\widehat{\bm{\mathcal{X}}}_{b}$,
from our transformer and baseline $b$, respectively. Computing this across all
samples, the result is a distribution of accuracy improvements.

#### Synthetic Volumes

First, we generate 4 independent synthetic volumes of shape $(64,192,64)$ for
each setting. Each volume is divided into 27 nonoverlapping segments of shape
$(64,7,64)$ each with one of the five interior slices masked. This results in
108 validation samples for each setting. These validation metrics are
displayed in Figure 7 when varying the mean twins frequency and in Figure 8
when varying mean grain size.

(a) Overall Accuracy

(b) Boundary Accuracy

(c) Overall Accuracy Improvements

(d) Boundary Accuracy Improvements

Figure 7: Overall accuracy (Figure 7(a)), boundary accuracy (Figure 7(b)),
overall accuracy improvements (Figure 7(c)), and boundary accuracy
improvements (Figure 7(d)) of synthetic volumes with varying twin frequencies.
Error bars in Figures 7(a) and 7(b) represent one standard deviation.

(a) Overall Accuracy

(b) Boundary Accuracy

(c) Overall Accuracy Improvements

(d) Boundary Accuracy Improvements

Figure 8: Overall accuracy (Figure 8(a)), boundary accuracy (Figure 8(b)),
overall accuracy improvements (Figure 8(c)), and boundary accuracy
improvements (Figure 8(d)) of synthetic volumes with varying mean grain sizes.
Error bars in Figures 8(a) and 8(b) represent one standard deviation.

We observe that our method more accurately recovers missing slices than all
other baselines for every synthetic validation sample since each slice
observed a positive improvement in overall and boundary accuracy. The
performance gain is much more apparent for boundary voxels. Furthermore, our
transformer performance is much lower variance. Among the baselines, KNN
achieves the closest accuracy to our method, but it still underperforms in
comparison. As expected, the difference between our method and the baselines
diminishes as the scenario get simpler (larger grain sizes and fewer twins)
since the baselines are already producing very accurate results.

#### Real EBSD Datasets

Now, we seek to understand how well our model transfers to real data by
running our trained model on IN625 [36, 37] and IN718 [38, 39]. For both
volumes, we subdivide each volume into nonoverlapping subvolumes such that all
slices are of height and width 64 voxels. Then, each subvolume is then further
partitioned into nonoverlapping segments of shape $(64,7,64,*)$, each
representing a single test sample. Again, each sample contains one masked
slice among the five central slices. In the end, there are 800 samples for
IN625 and 3298 samples for IN718 after discarding a small set of samples whose
missing slice did not have any boundary voxels (these samples would trivially
result in exact recovery regardless of the model we use). Average accuracies
and accuracy improvements for both volumes are shown in Table 1 and Figure 9,
respectively. For recovery examples, see Figure 10.

Table 1: Average overall accuracy and boundary accuracy across samples of real world datasets, including their standard deviations. Dataset | Model | Avg. Acc. (%) | Avg. Boundary Acc. (%)
---|---|---|---
IN625 | Transformer | 91.83 $\pm$ 1.98 | 79.49 $\pm$ 3.20
KNN |  89.18 $\pm$ 2.48 |  72.86 $\pm$ 3.77
Previous Slice |  86.44 $\pm$ 2.94 |  65.83 $\pm$ 4.73
Next Slice |  86.45 $\pm$ 3.15 |  65.91 $\pm$ 4.98
IN718 | Transformer | 98.27 $\pm$ 1.03 | 85.65 $\pm$ 3.49
KNN |  97.54 $\pm$ 1.52 |  79.81 $\pm$ 5.49
Previous Slice |  96.40 $\pm$ 2.23 |  70.43 $\pm$ 8.50
Next Slice |  96.39 $\pm$ 2.22 |  70.25 $\pm$ 8.47

(a) IN625

(b) IN718

Figure 9: Overall and boundary accuracy improvements of IN625 and IN718 test
sets. Note that the y-axes are on different scales.

Surprisingly, even though our model is trained exclusively on synthetic data,
we observe it transfers well to real EBSD data, as it still outperforms every
baseline on nearly every sample, and the interquartile range is positive.
Again, the difference is accentuated for boundary voxels. Test results for the
IN718 dataset had much higher variance, likely due to each slice having
proportionally fewer boundary voxels than IN625 slices. Qualitatively, our
model has a stronger capability to recovery thin features than KNN, which
visually tends to ignore more subtle structures.

(a)

(b)

(c)

(d)

(e)

Figure 10: Four random example predictions of missing slices from IN625 (top
two rows) and IN718 (bottom two rows) test sets . Each row is a separate
example. Figure 10(a) contains the target; Figures 10(b), 10(c), 10(d), and
10(e) are predictions made by our transformer, KNN, the previous slice, and
the following slice, respectively.

### Limitations

While our method provides superior results compared to the baselines, we also
seek to understand the circumstance in which it may underperform. One
challenge is rapid changes between slices which make up a small minority of
test inputs. For instance, if the $k$-th slice is missing and the set of
grains present in the $(k-1)$-th is different from the set of grains in
$(k+1)$-th slice, the model has a lot of freedom to decide on which ones are
present and to what degree in the missing slice. A case in which this can
arise is when the faces of grains or twins are perpendicular to the slicing
direction. However, we observe this is an issue for all methods. Using the
second row of Figure 9 as an example, the bright green grain is a large
crescent shape on the slice before the missing slice but is hardly present in
the following slice. Our model and KNN strives to find some middleground, but
ultimately, both misclassifies many of the voxels. A similar scene plays out
in the fourth row of Figure 9. Thus, future work includes designing a better
loss function to mitigate errors for this or emphasizing these scenarios
during training.

Another limitation arises from the locality of our projection method. Recall
our local projection method projects a voxel to have the ID as one of its
observed or previously assigned voxels in order to encourage grain
connectedness. This means long range dependencies may be ignored in favor of
more smooth structures. Furthermore, connectedness is not guaranteed for very
thin features at an angle. Examples of both edge cases for matrices are
illustrated in Figure 11. Though these scenarios are fairly rare in practice,
further work could be done to improve our projection algorithm.

Figure 11: Matrix examples where projection to observed and previously
projected neighbors is suboptimal, even when giving it the ground truth.
Projection involves using the neighboring slices to predict the center slice.
White squares indicate which pixels have not been projected yet, and the black
arrows point to the possible neighbors that the pixel will choose to project
to. The top row shows an example where the projection operation smooths out
very thin features that lie completely in the missing slice. The bottom row
shows an example where the projection operation may disconnect a grain.

## Conclusion

We have presented a novel method using transformers followed by a projection
algorithm to recover missing 3D EBSD data which vastly outperforms all
baselines. Notably, even though our model is trained on synthetic data, it
still recovers more accurate slices for real 3D EBSD data than the baselines
by a wide margin, making it a powerful data processing tool for faulty 3D EBSD
readings. Furthermore, our model opens the possibility to skip every fourth
slice during data collection (by using every skipped slice and the three
collected slices on each side as the input to our model), potentially reducing
the collection time by 25%. Future work involves addressing the limitations,
scaling our method, and considering more general cases, such as altering the
projection algorithm to apply to consecutive missing slices within a sample.
In terms of scaling, we hope to observe emergent behavior similar to scaling
LLMs and their training data, which have seen vast improvements to performance
and versatility [43]. Beyond EBSD, it would be interesting to investigate our
method’s applicability for other high dimensional material science datasets.

## Data Availability Statement

The IN625 and IN718 datasets are available in the repositories,
https://acdc.alcf.anl.gov/mdf/detail/shade_afrl_am_package_v2.1/ and
https://datadryad.org/stash/dataset/doi:10.5061/dryad.83bk3j9sj, respectively.
Synthetic EBSD data are generated using DREAM.3D and are located at
https://drive.google.com/drive/u/1/folders/1S-jKZ7wxIT4ra4q5VJ_vEl4rO3yYQSgQ.

## References

  * [1] Uchic, M. _et al._ An Automated Multi-Modal Serial Sectioning System for Characterization of Grain-Scale Microstructures in Engineering Materials.  In De Graef, M., Poulsen, H. F., Lewis, A., Simmons, J. & Spanos, G. (eds.) _Proceedings of the 1st International Conference on 3D Materials Science_ , 195–202, DOI: 10.1007/978-3-319-48762-5_30 (Springer International Publishing, Cham, 2016).
  * [2] Chapman, M. G. _et al._ Afrl additive manufacturing modeling series: challenge 4, 3d reconstruction of an in625 high-energy diffraction microscopy sample using multi-modal serial sectioning.  _Integrating Materials and Manufacturing Innovation_ 10, 129–141 (2021).
  * [3] Polonsky, A. T. _et al._ Scan strategies in EBM-printed IN718 and the physics of bulk 3D microstructure development.  _Materials Characterization_ 190, 112043, DOI: 10.1016/j.matchar.2022.112043 (2022).
  * [4] Polonsky, A. T. _et al._ Three-dimensional Analysis and Reconstruction of Additively Manufactured Materials in the Cloud-Based BisQue Infrastructure.  _Integrating Materials and Manufacturing Innovation_ 8, 37–51, DOI: 10.1007/s40192-019-00126-7 (2019).
  * [5] Jolley, B. R., Uchic, M. D., Sparkman, D., Chapman, M. & Schwalbach, E. J.  Application of Serial Sectioning to Evaluate the Performance of x-ray Computed Tomography for Quantitative Porosity Measurements in Additively Manufactured Metals.  _JOM_ DOI: 10.1007/s11837-021-04863-z (2021).
  * [6] Nguyen, L. T. & Rowenhorst, D. J.  The Alignment and Fusion of Multimodal 3D Serial Sectioning Datasets.  _JOM_ 73, 3272–3284, DOI: 10.1007/s11837-021-04865-x (2021).
  * [7] Kotula, P. G., Keenan, M. R. & Michael, J. R.  Tomographic spectral imaging with multivariate statistical analysis: comprehensive 3d microanalysis.  _Microscopy and Microanalysis_ 12, 36–48 (2006).
  * [8] Calcagnotto, M., Ponge, D., Demir, E. & Raabe, D.  Orientation gradients and geometrically necessary dislocations in ultrafine grained dual-phase steels studied by 2d and 3d ebsd.  _Materials Science and Engineering: A_ 527, 2738–2746 (2010).
  * [9] Naragani, D. _et al._ Investigation of fatigue crack initiation from a non-metallic inclusion via high energy x-ray diffraction microscopy.  _Acta Materialia_ 137, 71–84 (2017).
  * [10] Sandgren, H. R. _et al._ Characterization of fatigue crack growth behavior in lens fabricated ti-6al-4v using high-energy synchrotron x-ray microtomography.  _Additive Manufacturing_ 12, 132–141 (2016).
  * [11] Wilson, J. R. _et al._ Three-dimensional reconstruction of a solid-oxide fuel-cell anode.  _Nature materials_ 5, 541–544 (2006).
  * [12] Teferra, K. & Rowenhorst, D. J.  Optimizing the cellular automata finite element model for additive manufacturing to simulate large microstructures.  _Acta Materialia_ 116930, DOI: 10.1016/j.actamat.2021.116930 (2021).
  * [13] Vaswani, A. _et al._ Attention is all you need.  _Advances in neural information processing systems_ 30 (2017).
  * [14] Khan, S. _et al._ Transformers in vision: A survey.  _ACM computing surveys (CSUR)_ 54, 1–41 (2022).
  * [15] Latif, S. _et al._ Transformers in speech processing: A survey.  _arXiv preprint arXiv:2303.11607_ (2023).
  * [16] Zhang, S. _et al._ Applications of transformer-based language models in bioinformatics: A survey.  _Bioinformatics Advances_ (2023).
  * [17] Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_ (2018).
  * [18] Radford, A., Narasimhan, K., Salimans, T., Sutskever, I. _et al._ Improving language understanding by generative pre-training (2018).
  * [19] Kalyan, K. S., Rajasekharan, A. & Sangeetha, S.  Ammus: A survey of transformer-based pretrained models in natural language processing.  _arXiv preprint arXiv:2108.05542_ (2021).
  * [20] Zhang, S. _et al._ Opt: Open pre-trained transformer language models.  _arXiv preprint arXiv:2205.01068_ (2022).
  * [21] Brown, T. _et al._ Language models are few-shot learners.  _Advances in neural information processing systems_ 33, 1877–1901 (2020).
  * [22] Touvron, H. _et al._ Llama: Open and efficient foundation language models (2023).  2302.13971.
  * [23] Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T. & Efros, A. A.  Context encoders: Feature learning by inpainting.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2536–2544 (2016).
  * [24] He, K. _et al._ Masked autoencoders are scalable vision learners.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 16000–16009 (2022).
  * [25] Kong, L. _et al._ Understanding masked autoencoders via hierarchical latent variable models.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 7918–7928 (2023).
  * [26] Chang, Y.-L., Liu, Z. Y., Lee, K.-Y. & Hsu, W.  Free-form video inpainting with 3d gated convolution and temporal patchgan.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 9066–9075 (2019).
  * [27] Liu, R. _et al._ Decoupled spatial-temporal transformer for video inpainting.  _arXiv preprint arXiv:2104.06637_ (2021).
  * [28] Dosovitskiy, A. _et al._ An image is worth 16x16 words: Transformers for image recognition at scale.  _arXiv preprint arXiv:2010.11929_ (2020).
  * [29] Dong, H., Shah, M., Donegan, S. & Chi, Y.  Deep unfolded tensor robust pca with self-supervised learning.  In _ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 1–5 (IEEE, 2023).
  * [30] Wang, S., Li, B. Z., Khabsa, M., Fang, H. & Ma, H.  Linformer: Self-attention with linear complexity.  _arXiv preprint arXiv:2006.04768_ (2020).
  * [31] Kitaev, N., Kaiser, Ł. & Levskaya, A.  Reformer: The efficient transformer.  _arXiv preprint arXiv:2001.04451_ (2020).
  * [32] Zaheer, M. _et al._ Big bird: Transformers for longer sequences.  _Advances in Neural Information Processing Systems_ 33, 17283–17297 (2020).
  * [33] Tay, Y., Dehghani, M., Bahri, D. & Metzler, D.  Efficient transformers: A survey.  _ACM Computing Surveys_ 55, 1–28 (2022).
  * [34] Ho, J., Kalchbrenner, N., Weissenborn, D. & Salimans, T.  Axial attention in multidimensional transformers.  _arXiv preprint arXiv:1912.12180_ (2019).
  * [35] Wang, H. _et al._ Axial-deeplab: Stand-alone axial-attention for panoptic segmentation.  In _Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part IV_ , 108–126 (Springer, 2020).
  * [36] Menasche, D. B. _et al._ Afrl additive manufacturing modeling series: challenge 4, in situ mechanical test of an in625 sample with concurrent high-energy diffraction microscopy characterization.  _Integrating Materials and Manufacturing Innovation_ 10, 338–347 (2021).
  * [37] Shade, P. A. _et al._ Afrl am modeling challenge series: Challenge 4 data package, DOI: 10.18126/K5R2-32IU (2019).
  * [38] Stinville, J. _et al._ Multi-modal dataset of a polycrystalline metallic material: 3d microstructure and deformation fields.  _Scientific Data_ 9, 460 (2022).
  * [39] Stinville, J. _et al._ Multi-modal dataset of a polycrystalline metallic material: 3d microstructure and deformation fields, DOI: 10.5061/dryad.83bk3j9sj (2022).
  * [40] Roşca, D., Morawiec, A. & De Graef, M.  A new method of constructing a grid in the space of 3d rotations and its applications to texture analysis.  _Modelling and Simulation in Materials Science and Engineering_ 22, 075013 (2014).
  * [41] Groeber, M. A. & Jackson, M. A.  Dream. 3d: a digital representation environment for the analysis of microstructure in 3d.  _Integrating materials and manufacturing innovation_ 3, 56–72 (2014).
  * [42] Donegan, S., Tucker, J., Rollett, A., Barmak, K. & M, G.  Extreme value analysis of tail departure from log-normality in experimental and simulated grain size distributions.  _Acta Materialia_ 61, 5595–5604 (2013).
  * [43] Wei, J. _et al._ Emergent abilities of large language models.  _arXiv preprint arXiv:2206.07682_ (2022).

## Acknowledgements

This work is supported in part by the Air Force D3OM2S Center of Excellence
under FA8650-19-2-5209, and by the Carnegie Mellon University Manufacturing
Futures Initiative, made possible by the Richard King Mellon Foundation. The
work of H. Dong is also supported by Liang Ji-Dian Graduate Fellowship and
Michel and Kathy Doreau Graduate Fellowship in Electrical and Computer
Engineering at Carnegie Mellon University. The U.S. Government is authorized
to reproduce and distribute reprints for Governmental purposes notwithstanding
any copyright notation thereon. Authors thank Michael Uchic, Marc DeGraef,
Greg Rohrer, and Zachary Varley for engaging discussions throughout the course
of this work. Diagrams were created in draw.io 21.5.1.

## Author contributions statement

H.D. designed and implemented the model, projection algorithm, and
experiments. S.D. and M.S. processed the raw data and helped integrate domain
knowledge to our method. S.D., M.S., and Y.C. advised and guided the project.
All authors wrote and reviewed the manuscript.

## Additional information

Competing interests: The authors declare no competing interests.


# Comment on "Inflation with a graceful exit and entrance driven by Hawking
radiation"

Javad T. Firouzjaee  School of Astronomy and School of Physics, Institute for
Research in Fundamental Sciences (IPM), Tehran, Iran<EMAIL_ADDRESS>
###### Abstract

Modak and Singleton [Phys. Rev. D 86, 123515 (2012)] have presented Hawking-
like radiation for cosmological inflation which has a natural “turn on" and a
natural “turn off" mechanism. This Hawking-like radiation results in an
effective negative pressure “fluid" which leads to a rapid period of expansion
in the very early Universe. We discuss that the turn on mechanism can not
happen for FRW model in early universe because its horizon is apparent horizon
not event horizon. Hence, we cannot apply geometric optic approximation which
is a necessary condition for tunneling method. It was shown that this model
predict a value for $\frac{\rho}{m_{pl}^{4}}$ which is bigger than the COBE
normalization constraint in the Cosmic Microwave Background (CMB) at the
horizon exit.

###### pacs:

98.80.Cq

In the recent paper modak  the Authors claim that they propose a mechanism for
inflation based on the particle creation due to Hawing radiation in a FRW
space-time. Core of their calculation is based on the associating the
temperature for the FRW apparent horizon cai . This method is called Hamilton-
Jacobi or tunneling method tunneling . The Hamilton-Jacobi method to calculate
the Hawking radiation uses the fact that within the WKB approximation the
tunneling probability for the classically forbidden trajectory from inside to
outside the horizon is given by

$\Gamma\propto\exp\left(-\frac{2}{\hbar}\mbox{Im }S\right),$ (1)

where $S$ is the classical action of the (massless) particle to the leading
order in $\hbar$ tunneling . It was shown javad  that only in the case of the
de Sitter space which FRW apparent horizon is event horizon, we can write the
WKB or geometric optics approximation for the horizon and apply the tunneling
method. There is a confusing point that one can attribute a temperature to the
FRW apparent horizon and write the area law for it cai , but this does not
mean that this system has the Hawking radiation.

Furthermore, if we use the canonical invariance tunneling method zhu , we are
not allowed to write the standard ansatz for scalar wave function
$\phi=exp[-\frac{i}{\hbar}S(r,t)+...]$ and taking the limit as $\hbar\sim 0$
for the FRW apparent horizon in the radiation dominated era, and getting the
Hamilton-Jacobi equation. Because this ansatz can be written in the case that
we are near the (event) horizon padmanabhan . In the case near (event)
horizon, the WKB or geometric optics approximation for the wave is satisfied
and we can write this ansatz for the scalar wave function.

Even in the deriving Hawking radiation from the Bogolubov coefficient
radiation , we only able to drive radiation from slowly varying space time or
the space times which have adiabatic vacuum, and no one are able to define
particle in non static space time such as radiation dominated FRW universe.

According to the modak , The first law of thermodynamics can be rewritten as

$\frac{d(\rho V)}{dt}+p\frac{dV}{dt}=+\frac{dQ}{dt}=\sigma A_{H}T^{4},$ (2)

this equation becomes

$\dot{\rho}+3(\rho+p)\frac{\dot{a}}{a}=\frac{3\sigma}{c}\left(\frac{\hbar}{2\pi
k_{B}}\right)^{4}H^{5},$ (3)

after simplification, we gets

$\frac{\dot{\rho}}{\rho}+3(1+\omega-\omega_{c}(t))\frac{\dot{a}}{a}=0,$ (4)

where $p=\omega\rho$ and the particle creation equation of state is

$p_{c}(t)=\omega_{c}(t)\rho\leavevmode\nobreak\ .$ (5)

We can write the equation of the state in the form

$\omega_{c}(t)=\alpha\rho(t)\leavevmode\nobreak\ \leavevmode\nobreak\ ,{\rm
where}\leavevmode\nobreak\ \leavevmode\nobreak\ \alpha=\frac{\hbar
G^{2}}{45c^{7}}=4.8\times 10^{-116}(J/m^{3})^{-1}$ (6)

This negative pressure that occurs due to Hawking radiation in FRW space-time
could give the inflationary era. More precisely, when the energy density of
the universe is near Planck energy density, $10^{-114}(J/m^{3})$, the Hawking
radiation term become important and act as a negative pressure which cause an
accelerated universe. As the universe expand, this term becomes negligible and
we will have a smooth transition to radiation dominated universe.
Since we can not write the WKB approximation for the FRW universe, we don’t
have any negative pressure that create the expanding universe and therefore no
inflationary phase.

Can this model be consistent with CMB observation?

Having inflationary mechanism from Hawking like radiation method lead to

$w_{c}(t)=\frac{\hbar G^{2}}{45c^{7}}\rho\simeq 4/3.$ (7)

or in the God-given natural units $\dfrac{\rho}{m_{pl}^{4}}\simeq 1$. This
require that this mechanism of inflation be near-Planck-scale physics rather
than grand unified scale.

On the other hand, cosmological perturbation theory says that

$P_{t}=rP_{R}$ (8)

in which $r$ is the ratio of the tensor mode to the scalar mode. The upper
bound on tensor perturbations from WMAP and PLANCK implies that r<0.11. Now,
from the COBE normalization for the curvature perturbations at the horizon
exit point we have $P_{R}\simeq 10^{-9}$. As a result
$P_{t}\simeq\frac{H^{2}}{m_{pl}^{2}}\simeq\frac{\rho}{m_{pl}^{4}}<10^{-10}$
bassett . If we had (7), then the tensor perturbation must be seen in the
Cosmic Microwave Background. Therefore, this model model predict a value for
$\dfrac{\rho}{m_{pl}^{4}}\simeq 1$ which is bigger than the predicted value
for the COBE normalization for the curvature perturbations at the horizon exit
point .

Acknowledgment I would like to thank Hassan Firouzjahi and Mohammad Hossein
Namjoo for useful discussions and comments. I wish to thank the Douglas
Singleton for his useful answers for my questions.

## References

  * (1) S. K. Modak and D. Singleton, Phys. Rev. D 86, 123515 (2012).
  * (2) R. Cai and S. P. Kim, J. High Energy Phys. 02, 050 (2005); R. -G. Cai, L. -M. Cao and Y. -P. Hu, Class. Quant. Grav. 26, 155018 (2009).
  * (3) M. K. Parikh, F.Wilczek, Phys. Rev. Lett. 85, 5042 (2000); M. Visser, Int. J. Mod. Phys. D12, 649 (2003). [hep-th/0106111].
  * (4) A. B. Nielsen and J. T. Firouzjaee, Gen. Relativ. Gravit. 45, 1815 (2013) [arXiv:1207.0064 [gr-qc].
  * (5) T. Zhu, J. R. Ren, and D. Singleton, Int. J. Mod. Phys. D 19, 159 (2010).
  * (6) K. Srinivasan and T. Padmanabhan, Phys. Rev. D, 60, 024007 (1999)
  * (7) L. Parker, Phys. Rev. Lett. 21, 562 (1968); N.D. Birrell and P.C.W. Davies,  Quantum Fields in Curved Space, Cambridge University Press (Cambridge, 1982).
  * (8) B. A. Bassett, S. Tsujikawa and D. Wands, Rev. Mod. Phys. 78, 537 (2006); P. A. R. Ade et al. [Planck Collaboration], arXiv:1303.5082 [astro-ph.CO].