# A novel Fourier neural operator framework for classification of multi-sized
images: Application to 3D digital porous media

Ali Kashefi<EMAIL_ADDRESS>Tapan Mukerji<EMAIL_ADDRESS>
###### Abstract

Fourier neural operators (FNOs) are invariant with respect to the size of
input images, and thus images with any size can be fed into FNO-based
frameworks without any modification of network architectures, in contrast to
traditional convolutional neural networks (CNNs). Leveraging the advantage of
FNOs, we propose a novel deep-learning framework for classifying images with
varying sizes. Particularly, we simultaneously train the proposed network on
multi-sized images. As a practical application, we consider the problem of
predicting the label (e.g., permeability) of three-dimensional digital porous
media. To construct the framework, an intuitive approach is to connect FNO
layers to a classifier using adaptive max pooling. First, we show that this
approach is only effective for porous media with fixed sizes, whereas it fails
for porous media of varying sizes. To overcome this limitation, we introduce
our approach: instead of using adaptive max pooling, we use static max pooling
with the size of channel width of FNO layers. Since the channel width of the
FNO layers is independent of input image size, the introduced framework can
handle multi-sized images during training. We show the effectiveness of the
introduced framework and compare its performance with the intuitive approach
through the example of the classification of three-dimensional digital porous
media of varying sizes.

††journal: arXiv

[first]organization=Department of Civil and Environmental Engineering,
Stanford University, city=Stanford, postcode=94305, state=CA, country=USA

[third]organization=Department of Energy Science and Engineering, Stanford
University, city=Stanford, postcode=94305, state=CA, country=USA

##  1 Introduction and motivation

Since 2020, neural operators have gained extensive popularity, specifically
with two versions of graph neural operators (Li et al., 2020b) and Fourier
neural operators (FNOs) (Li et al., 2020a). In this article, our attention is
on FNOs. From a computer science perspective, regular FNOs fall in the
category of supervised deep learning framework, necessitating a large volume
of labeled data for training. FNOs have demonstrated their proficiency in
input-output mapping across various industrial and scientific applications
such as incompressible flows (Li et al., 2022b; Bonev et al., 2023; Peng et
al., 2024; Choubineh et al., 2023; Lyu et al., 2023; Gupta and Brandstetter,
2022; Peng et al., 2023a), wave equations (Zhu et al., 2023a; Zou et al.,
2023; Yang et al., 2023), thermal fields (Zhao et al., 2024; Hao et al.,
2023), carbon storages and sequestration (Wen et al., 2022; Jiang et al.,
2023b), and other areas (Peng et al., 2023b; You et al., 2022; Kontolati et
al., 2023; Zhu et al., 2023b; Hua and Lu, 2023; White et al., 2023a; Li et
al., 2021; Pathak et al., 2022; Rahman et al., 2022b, a; Yang et al., 2022; Li
et al., 2022a; Maust et al., 2022; Zhao et al., 2022; Renn et al., 2023; Xiong
et al., 2023; Chen et al., 2023; Huang et al., 2023; Poels et al., 2023; White
et al., 2023b; Thodi et al., 2023; Zhao et al., 2023; Tran et al., 2023; Lee,
2022; Brandstetter et al., 2023; Li et al., 2023; Majumdar et al., 2023; Jiang
et al., 2023a; Lehmann et al., 2024). From a computer vision perspective,
these are framed as segmentation problems, where an input image, such as the
geometry of an airfoil, is mapped to another image, for instance, the velocity
field around that airfoil. An analogous area in computer vision is
classification, where an input image is mapped, for example, to a name or
number. While FNOs have potential in classification tasks, there exists only a
limited number of research conducted in this application as per our knowledge
(Johnny et al., 2022; Xi et al., 2022; Kabri et al., 2023).

Johnny et al. (2022) used the FNO architecture for classifying images in the
CIFAR-10 dataset, containing ten different classes; however, they trained the
network only on images with a fixed size of 32 by 32 pixels. Additionally,
Kabri et al. (2023) examined the FNO architecture for image classification.
Although they tested images of various sizes (e.g., 28 by 28 pixels, 112 by
112 pixels, etc.), they trained and then tested the network separately for
each size, assessing its performance on the corresponding size. Xi et al.
(2022) utilized the FNO architecture for the hyperspectral remote sensing
image classification. Their dataset comprised images of various sizes,
including 512 by 614 pixels, 610 by 340 pixels, and 512 by 217 pixels.
However, they adjusted all images to a fixed size by adding patches.
Consequently, although they employed the FNO architecture, in practice, they
limited their analysis to images of a uniform size. In the current study, we
narrow our focus on classification problems. More specifically, we consider
the problem of predicting the permeability of three-dimensional digital porous
media, which vary in size, as a benchmark test case.

FNOs are invariant with respect to the size of input images, and this
characteristic ensures that images of varying sizes can be processed by FNO-
based deep learning frameworks without requiring any architectural
alterations. Note that regular convolutional neural networks (CNNs) lack this
feature (Goodfellow et al., 2016). Building on this strength of FNOs, we
introduce a deep-learning framework for training the network simultaneously on
images with varying sizes for a classification problem. To achieve this deep
learning framework, FNO layers must be connected to a classifier, which is
commonly a multilayer perceptron (MLP). An intuitive approach to set this
would be to link FNO layers to a classifier via adaptive max pooling.
Considering the application of permeability prediction of three-dimensional
porous media, our machine-learning experiments show that this intuitive
approach only works well for porous media with fixed sizes. Pivoting from
this, we propose our novel approach. Rather than using adaptive max pooling,
we implement static max pooling with the size of the channel width of the FNO
layers. Given that the size of the channel width of FNO layers is independent
of the size of input images, our proposed framework can be efficiently trained
on various image sizes at once (see Fig. 1 and Fig. 2).

To explain, at a high level, the difference between using adaptive max pooling
(see Fig. 2) and static max pooling (see Fig. 1), let us consider for example
a three-dimensional image being fed as an input of the deep learning
framework. For both pooling methods, at the framework’s outset, FNO layers
lift the input image from its three-dimensional space to a higher dimensional
space, determined by the size of the channel width of the FNO layers. In the
case of adaptive max pooling, after FNO layer operations, the outcome
eventually is dropped into the initial three-dimensional space with the same
size as the input image. This array then serves as the input of adaptive max
pooling. The output of the adaptive pooling is then the input of the
classifier. In the case of static max pooling, before FNO layers drop the
output, we implement static max pooling, which functions within the high
dimensional space and pools with the size of the channel width of FNO layers.
The resulting output from this pooling then becomes the classifier’s input. A
more detailed exploration of these concepts is provided in Sect. 2.

The study of physical and geometric features of porous media is important in
diverse scientific and industrial areas such as digital rock physics (Andra et
al., 2013a, b), membrane systems (Liang and Fletcher, 2023), geological carbon
storages (Blunt et al., 2013), and medicine (Kumeria, 2022; Das et al., 2018).
Deep learning frameworks have been widely used for predicting the permeability
of porous media (Meng et al., 2023; Xie et al., 2023; Kashefi and Mukerji,
2023, 2021; Liu et al., 2023; Hong and Liu, 2020; Wu et al., 2018; Tembely et
al., 2020; Masroor et al., 2023; Sun et al., 2023), but, to the best of our
knowledge, all these frameworks were trained on a fixed-size porous media.
Note that training the proposed network to predict the permeability of porous
media of varying sizes comes with an exclusive challenge when compared to
training the network on conventional images for the purpose of classifying
them by their names (like those of cats and dogs). For conventional images,
one possible solution to handle images with different sizes is to equalize
them by adding mini patches to the smaller images. Nevertheless, this solution
is inapplicable to the porous media problem. Adding mini patches to porous
media can alter their physical properties such as permeability. For instance,
adding mini patches around a porous medium simulates sealing it with wall
boundaries, which prohibits flow within its pore spaces, resulting in a
permeability of zero. Additionally, the inherently three-dimensional nature of
porous media introduces another layer of complexity compared to the two-
dimensional conventional images.

The remainder of this article is organized as follows. We introduce and
discuss the concept of Fourier neural operators for image classification in
Sect. 2, starting with the traditional strategy of adaptive max pooling,
followed by our novel approach of static max pooling in the high dimension of
the Fourier space channel. A brief review of theoretical aspects of FNOs is
given in Sect. 2.3. Data generation and the training methodologies are
respectively presented in Sect. 3 and Sect. 4. In Sect. 5, we provide results
and discussion, including a comparison between traditional strategy and our
novel approach. Moreover, we present a sensitivity analysis, covering the
number of Fourier modes, the channel width of discrete Fourier space, the
number of FNO units, and the effect of activation functions and average
pooling. The deep learning model generalizability is discussed in this section
as well. Finally, we summarize the work and present insight into future
directions in Sect. 6.

Figure 1: Schematic of the proposed FNO-based framework for multi-size image
classification Figure 2: Schematic of the intuitive FNO-based framework for
multi-size image classification

##  2 Fourier neural operators for image classification

###  2.1 Our novel approach: Static max pooling in channel width of FNO layers

In this subsection, we introduce the architecture of our proposed deep
learning framework. Our explanation heavily uses matrix notation to ensure
clarity and provide a deeper understanding. As illustrated in Fig. 1, the
input of the deep learning framework is a cubic binary porous medium
represented as the matrix $\mathbf{A}_{n\times n\times n}$. As a first step,
the matrix $\mathbf{A}_{n\times n\times n}$ is lifted to a higher dimensional
space using a fully connected network. The dimension of this space is termed
the channel width of an FNO layer, shown by “width” in our matrix notation.
This lifting results in a four-dimensional matrix, denoted as
$\mathbf{B}_{\text{width}\times n\times n\times n}^{0}$. The matrix
$\mathbf{B}_{\text{width}\times n\times n\times n}^{0}$ becomes subsequently
the input of an FNO layer. Within the FNO layer, two operations are applied to
$\mathbf{B}_{\text{width}\times n\times n\times n}^{0}$: the kernel
integration operator, denoted by
$\mathbf{K}_{\text{width}\times\text{width}}^{0}$, and the linear
transformation operator, denoted by
$\mathbf{W}_{\text{width}\times\text{width}}^{0}$. The network computes the
matrix-matrix multiplication of
$\mathbf{K}_{\text{width}\times\text{width}}^{0}\mathbf{B}_{\text{width}\times
n\times n\times n}^{0}$ and
$\mathbf{W}_{\text{width}\times\text{width}}^{0}\mathbf{B}_{\text{width}\times
n\times n\times n}^{0}$ and then sums up the resulting matrices, as depicted
in Fig. 1. The output undergoes element-wise operations of the Rectified
Linear Unit (ReLU) activation function (Goodfellow et al., 2016) defined as

$\sigma(\gamma)=\max(0,\gamma),$ (1)

Resulting in a four-dimensional matrix $\mathbf{B}_{\text{width}\times n\times
n\times n}^{1}$. Mathematically, this procedure can be summarized as

$\displaystyle\mathbf{B}_{\text{width}\times n\times n\times n}^{1}$
$\displaystyle=\sigma\left(\mathbf{K}_{\text{width}\times\text{width}}^{0}\mathbf{B}_{\text{width}\times
n\times n\times n}^{0}\right.$
$\displaystyle\quad\left.+\mathbf{W}_{\text{width}\times\text{width}}^{0}\mathbf{B}_{\text{width}\times
n\times n\times n}^{0}\right).$ (2)

In scenarios where multiple FNO layers exist in the framework, the matrix
$\mathbf{B}_{\text{width}\times n\times n\times n}^{1}$ serves as the input
for the succeeding FNO layers, and the same sequence of operations is applied.
If we assume that there are $l$ number of FNO layers, the output from the
final FNO layer is the matrix $\mathbf{B}_{\text{width}\times n\times n\times
n}^{l}$. In the next step, we implement static max pooling on the first
dimension of matrix $\mathbf{B}_{\text{width}\times n\times n\times n}^{l}$.
Because “width” is independent of the input image dimension (i.e., $n$), the
static pooling works for input images with any desired size (e.g., $n=40$,
$n=48$, and $n=56$). Note that “width” is a hyper parameter of FNO layers and
independent of $n$, as all the matrix-matrix multiplication operates on the
dimension with the size “width”, and not “$n$”. The static max pooling
produces a vector of length width, representing the global features of the
input images. The vector is then connected to a classifier. The classifier is
a Multilayer Perceptron (MLP) composed of three layers of sizes 128, 128, and
1. The ReLU activation function is used in the initial two layers along with a
dropout with a rate of 0.3. Following the third layer, a sigmoid activation
function defined as

$\sigma(\gamma)=\frac{1}{1+e^{-\gamma}},$ (3)

is used to ensure output values are bounded between 0 and 1.

###  2.2 Intuitive approach: Adaptive max pooling in 3D spatial space

In this subsection, we explain the intuitive approach (see Fig. 2). Drawing
parallels to our approach elaborated in the previous subsection, we begin by
considering the input porous medium, which is a three-dimensional matrix
represented by $\mathbf{A}_{n\times n\times n}$. All operations outlined in
Sect. 2.1 are applied to $\mathbf{A}_{n\times n\times n}$ until the network
obtains the matrix $\mathbf{B}_{\text{width}\times n\times n\times n}^{l}$ at
an intermediate step, as depicted in Fig. 2. As the next step, we drop (as an
inverse of the lifting operator explained in Sect. 2.1) the matrix
$\mathbf{B}_{\text{width}\times n\times n\times n}^{l}$ from the high
dimensional space to the default space by means of a fully connected network.
This transformation results in the matrix $\mathbf{Z}_{n\times n\times n}$. At
this juncture, we use adaptive three-dimensional max pooling, a functionality
that is available in deep learning platforms such as PyTorch (Paszke et al.,
2019) or TensorFlow (Abadi et al., 2015). To ensure a fair comparison between
the traditional approach and our novel approach, we keep the size of the
vector of the global feature consistent across both approaches. To this end,
the output of the adaptive max pooling is tailored to yield a vector of size
“width”. The resulting vector represents the global features of the input
images.

Note that because the size of matrix $\mathbf{Z}_{n\times n\times n}$ depends
on the size of the input image (i.e., $n$), the pooling must be adaptive as we
plan to train the network simultaneously on input images with varying sizes
(e.g., $\mathbf{A}_{40\times 40\times 40}$, $\mathbf{A}_{48\times 48\times
48}$, and $\mathbf{A}_{56\times 56\times 56}$). Subsequent to the adaptive max
pooling, the global feature vector is connected to a classifier. This
classifier features and architecture is precisely the same as the one
elucidated in Sect. 2.1.

###  2.3 A brief review of theoretical aspects of Fourier neural operators

We focused on the technical aspects and computer implementation of FNO layers
in Sect. 2.1 and Sect. 2.2. Theoretical aspects of FNO layers have already
been vastly explained and discussed in the literature (Li et al., 2020a). In
this subsection, we briefly review the theory behind FNO layers and highlight
some important features.

As discussed in Sect. 2.1, an FNO layer comprises two main operators: the
integral kernel operator and the linear transformation. We overview the
integral kernel operator. We consider the bounded domain $D$ such that
$D\subset\mathbb{R}^{d}$, where $d$ indicates the physical dimensionality of
the problem and is equal to 3 (i.e., $d=3$) for the current problem since we
deal with three-dimensional porous media. We further show the input of the FNO
layer by $b(x)$ with $x\in D$, where $b$ is a function representing all the
operators applied to $x$ when it arrives at the gate of the FNO layer.
Moreover, we define the periodic kernel function
$\tau:\mathbb{R}^{2(d+d_{a})}\rightarrow\mathbb{R}^{\text{width}\times\text{width}}$,
where $d_{a}$ is the number of input features and is equal to 1 (i.e.,
$d_{a}=1$) in this study, because the input of the deep learning framework is
only a cubic binary array (representing a porous medium), and this array only
provides one feature, which is the geometry of the porous medium.
Additionally, recall that “width” is the channel width of the FNO layer, as
illustrated in Sect. 2.1 and Sect. 2.2. Following the formulation proposed by
Li et al. (2020a), the operation of the integral kernel $\mathcal{K}$ on the
function $b(x)$ in the continuous space is defined as

$\mathcal{K}b(x)=\int_{D}\tau(x,y)b(y)\,dy,\quad\forall x\in D.$ (4)

Following the original design of FNO layers by Li et al. (2020a), we introduce
the condition $\tau(x,y)=\tau(x-y)$. By applying the convolution theorem as
detailed in the literature (Li et al., 2020a), the following expression for
the integral kernel operator is obtained:

$\mathcal{K}b(x)=\mathcal{F}^{-1}\left(\mathcal{F}(\tau)\cdot\mathcal{F}(b(x))\right),\quad\forall
x\in D,$ (5)

where the Fourier transform and its inverse are shown by $\mathcal{F}$ and
$\mathcal{F}^{-1}$, respectively. We introduce $\mathcal{R}$ as the learnable
Fourier transform of $\tau$ such that

$\mathcal{R}=\mathcal{F}(\tau).$ (6)

Beyond the theory, we must implement these mathematical concepts in a deep
learning framework. In this way, we work with discrete spaces and
consequently, discrete modes of the Fourier transform. Hence, $\mathcal{R}$ is
implemented as a neural network. Additionally, each porous medium is
represented by $n^{3}$ discrete points such that
$\\{x_{1},x_{2},\cdots,x_{n^{3}}\\}\subset D$. Moreover, Fourier series
expansions are truncated at a maximum number of modes $m_{\text{max}}$
computed as

$m_{\text{max}}=\left|\left\\{m\in\mathbb{Z}^{d}:|m_{j}|\leq
m_{\text{max},j},\text{ for }j=1,\cdots,d\right\\}\right|,$ (7)

where $m_{\text{max},j}$ is the maximum number of modes taken in the dimension
$j$, and is a hyper-parameter of the FNO layer. Note that since we work on
three-dimensional problems in the current study, $d=3$, and thus, there are
only $m_{\text{max},1}$, $m_{\text{max},2}$, and $m_{\text{max},3}$. As a
result, the components of the $\mathcal{R}\cdot\mathcal{F}(b(x))$ operation
can be computed by the following formulation

$\displaystyle[\mathcal{R}\cdot\mathcal{F}(b(x))]_{m,i}$
$\displaystyle=\sum_{j=1}^{\text{width}}[\mathcal{R}]_{m,i,j}[\mathcal{F}(b(x))]_{m,j},$
(8) $\displaystyle\quad m=1,\cdots,m_{\text{max}},\quad
i=1,\cdots,\text{width},$

where
$[\mathcal{R}]\in\mathbb{C}^{m_{\text{max}}\times\text{width}\times\text{width}}$
is the matrix representation of $\mathcal{R}$ in the discrete space.
$[\mathcal{R}\cdot\mathcal{F}(b(x))]\in\mathbb{C}^{m_{\text{max}}\times\text{width}}$
and $[\mathcal{F}(b(x))]\in\mathbb{C}^{m_{\text{max}}\times\text{width}}$ are
similarly defined. To increase computing efficiency and enable parallel
computing, the operator $[\mathcal{R}]$, for the current three-dimensional
problem, is better to be implemented as a five-dimensional matrix expressed as

$\mathbf{R}_{m_{\text{max},1}\times m_{\text{max},2}\times
m_{\text{max},3}\times\text{width}\times\text{width}}.$ (9)

As can be seen from Eq. 9, the size of matrix $\mathbf{R}$, and thus the count
of trainable parameters in the FNO layer, is a function of the number of
maximum Fourier modes at each dimension and the channel width of the FNO
layer. Recall that these parameters (i.e., $m_{\text{max},1}$,
$m_{\text{max},2}$, $m_{\text{max},3}$, and width) are the hyperparameter of
FNO layers and need to be tuned by potential users.

\begin{overpic}[width=433.62pt]{CNN40.png} \put(5.0,90.0){a} \end{overpic}

\begin{overpic}[width=433.62pt]{CNN48.png} \put(5.0,90.0){b} \end{overpic}

\begin{overpic}[width=433.62pt]{CNN56.png} \put(5.0,90.0){c} \end{overpic}

Figure 3: A few examples of synthetically generated three-dimensional digital
porous media for training the proposed neural network; a an image of size
$40^{3}$, b an image of size $48^{3}$, and c an image of size $56^{3}$

##  3 Data generation

To generate synthetic data to examine the deep learning framework under
investigation in this study, we consider cubic porous medium domains with
length L along each side, spatial correlation length of $l_{c}$, and porosity
of $\phi$. We use the truncated Gaussian algorithm (Lantuejoul, 2002; Le
Ravalec-Dupin et al., 2004) to generate synthetic porous media. In practice,
we create three-dimensional cubic arrays of dimension $n\times n\times n$,
populated with random numbers conforming to a normal distribution with the
characteristics of a mean value of 0.0 and a standard deviation of 1.0.
Subsequently, we filter the arrays by a three-dimensional Gaussian smoothing
kernel with a standard deviation of 5.0 and a filter size commensurate with a
spatial correlation length ($l_{c}$) of 17. We then subject the arrays to a
binarization process via a thresholding number such that the porosity ($\phi$)
of the resulting arrays lies within the range of [0.125, 0.200]. We use the
MATLAB software to handle the above-described steps. We set $L$ as
$n\times\delta x$, where $\delta x$ represents the length of each side of a
pixel in porous media. We set $\delta x$ to 0.003 m. We generate porous media
with three different sizes by considering three different values for $n$, such
that $n_{1}=40$, $n_{2}=48$, and $n_{3}=56$. In this way, each cubic porous
medium can be characterized by its size as $n^{3}$ (e.g., $40^{3}$, $48^{3}$,
and $56^{3}$). For each $n$, we generate 1250 data. We randomly split the
generated data corresponding to each size into three categories of training
(80%, i.e., 1000 data), validation (10%, i.e., 125 data), and test (10%, i.e.,
125 data). Hence, there are 3750 data in total, 3000 data for the training
set, 375 data for the validation set, and 375 data for the test set. Figure 3
exhibits a few examples of the generated synthetic data.

To stimulate the incompressible viscous Newtonian flow within the generated
porous media, we apply a constant pressure gradient in the $x$ direction
($\Delta p/L$). Zero velocity boundary condition is applied at the top and
bottom of the porous medium on the $y-z$ planes. Given the geometry and
boundary conditions illustrated above, we use a Lattice Boltzmann solver
(Keehm et al., 2004) to solve the continuity and steady-state Stokes
equations, which are written as follows:

$\nabla\cdot\bm{\mathit{u}}=0,\quad\text{in }V,$ (10)

$-\nabla p+\mu\Delta\bm{\mathit{u}}=\textbf{0},\quad\text{in }V,$ (11)

where $\mu$ is the dynamic viscosity, $\bm{\mathit{u}}$ and $p$ indicate,
respectively, the velocity vector and pressure fields in the pore space of the
porous medium, $V$. In the next step, we compute the permeability in the
$x-$direction ($k$) using Darcy’s law (Darcy, 1856),

$k=-\frac{\mu\bar{U}}{\Delta p/L},$ (12)

where $\bar{U}$ shows the average velocity in the entire porous medium (i.e.,
including solid matrices). The computed permeabilities of our data set fall in
the range [20 mD, 200 mD].

##  4 Training

To accelerate the convergence of the training procedure, the output training
data (i.e., permeability) are scaled in the range of [0, 1] using the maximum
and minimum values of the training set. Note that although we train a single
neural network simultaneously on porous media with three different sizes
(corresponding to $n_{1}$, $n_{2}$, and $n_{3}$), we normalize the
permeability of porous media of each size using the maximum and minimum values
of the specific size. Mathematically, it can be written as

$\\{\hat{k}_{\text{truth}}\\}_{n_{j}}=\frac{\\{k\\}_{n_{j}}-\min\\{k\\}_{n_{j}}}{\max\\{k\\}_{n_{j}}-\min\\{k\\}_{n_{j}}},\quad
j=1,2,\text{and }3,$ (13)

where, $\hat{k}_{\text{truth}}$ shows the ground truth scaled permeability.
Moreover, for instance, $\\{k\\}_{n_{1}}$ indicates the training data
containing porous media with the size of $40^{3}$ (because $n_{1}=40$). Note
that we eventually rescale predicted permeability
($\hat{k}_{\text{prediction}}$) to the physical domain
($k_{\text{prediction}}$) for analyzing the neural network performances.
Concerning the loss function, we use the mean squared error function defined
as

$\text{Loss}=\frac{1}{N}\sum_{i=1}^{N}(\hat{k}_{\text{prediction}}-\hat{k}_{\text{truth}})^{2},$
(14)

where $N$ is the number of data in the training set (i.e., $N=3000$). Note
that using relative mean squared error as the loss function does not lead to a
significant difference in the results, based on our experiments. We set the
number of modes in each dimension to 2 (i.e., set $m_{\text{max},1}=2$,
$m_{\text{max},2}=2$, and $m_{\text{max},3}=2$). The channel width of the
discrete Fourier space is set to 64 (i.e., $\text{width}=64$). It is worth
noting that both the number of modes and the channel width play pivotal roles
in the network performance. Detailed discussions on their significance and
implications are provided in Sect. 5.2 and 5.3, respectively. Additionally, we
implement three units of FNOs in the network. The Adam optimizer (Kingma and
Ba, 2014) is used. A constant learning rate of 0.001 is selected. We use the
stochastic gradient descent (Goodfellow et al., 2016) with a mini-batch size
of 50. As discussed in Sect. 2, the architecture of FNOs is designed to be
independent of the spatial resolution of input images. During the training
process, however, all the input images within a mini-batch must be the same
size. In practice, each epoch of training is characterized by an inner loop
that iterates through mini-batches of differing porous medium sizes (i.e.,
$40^{3}$, $48^{3}$, and $56^{3}$). Within this loop, the training process
starts with a mini-batch of data of size $40^{3}$, followed by one of size
$48^{3}$, and then continues to $56^{3}$, in sequence until all the data in
the training set are covered within the epoch. Note that the trainable
parameters of the network are updated only at the end of each epoch. Our deep
learning experiments show that the order in which these differently sized
porous media are fed within an epoch has no significant influence on the
result accuracy and convergence speed, whether starting with the porous media
of size $40^{3}$, followed by $48^{3}$ and $56^{3}$, or any other permutation.
From a software perspective, we employ the NVIDIA A100 (SXM4) graphic card
with 80 Gigabytes of RAM for training the networks.

In the last paragraph of this subsection, we address the metric used for
assessing the effectiveness of permeability prediction. We use the coefficient
of determination, also known as the $R^{2}$ score, which can be calculated
using the following formula

$R^{2}=1-\frac{\sum_{i=1}^{Q}({k_{\text{truth}}}_{i}-{k_{\text{prediction}}}_{i})^{2}}{\sum_{i=1}^{Q}({k_{\text{truth}}}_{i}-\bar{k})^{2}},$
(15)

where $Q$ represents the number of the data in a set (e.g., training, test,
etc.) and $\bar{k}$ is the average value of the set
$\\{{k_{\text{truth}}}_{i}\\}_{i=1}^{Q}$.

Figure 4: $R^{2}$ plots for the test set (375 data) using the proposed
approach for classification of multi-sized images

##  5 Results and discussion

###  5.1 General analysis

As illustrated in Fig. 4, the success of our approach is evident in the
$R^{2}$ score, 0.96809, obtained for the test set (e.g., 375 data).
Additionally, Fig. 5 specifically showcases the $R^{2}$ scores for the test
set but individualized for each cubic size (i.e., $40^{3}$, $48^{3}$, and
$56^{3}$). As can be seen in Fig. 5, the $R^{2}$ scores obtained are equal to
0.96830, 0.96978, and 0.96607, respectively for the cubic digital porous media
of sizes $40^{3}$, $48^{3}$, and $56^{3}$. The range of $R^{2}$ scores for the
three different sizes remains at an excellent level, demonstrating that our
FNO-based framework is robust and not overfitted to any specific size.

Table 1: $R^{2}$ score of the test set for different mode numbers of the proposed FNO-based framework Number of modes in each dimension | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10
---|---|---|---|---|---|---|---|---|---
$R^{2}$ score | 0.96809 | 0.15416 | 0.26757 | 0.26361 | 0.23325 | 0.38789 | 0.40433 | 0.31773 | 0.28839

\begin{overpic}[width=433.62pt]{C1_40.pdf} \put(5.0,90.0){a} \end{overpic}

\begin{overpic}[width=433.62pt]{C1_48.pdf} \put(5.0,90.0){b} \end{overpic}

\begin{overpic}[width=433.62pt]{C1_56.pdf} \put(5.0,90.0){c} \end{overpic}

Figure 5: $R^{2}$ plots for the test set (375 data) using the proposed
approach for the classification of multi-sized images. The results are
individually shown for a images of size $40^{3}$ (125 data), b images of size
$48^{3}$ (125 data), and c images of size $56^{3}$ (125 data)

\begin{overpic}[width=433.62pt]{Loss_mode2.pdf} \put(5.0,90.0){a}
\end{overpic}

\begin{overpic}[width=433.62pt]{Loss_mode7.pdf} \put(5.0,90.0){b}
\end{overpic}

\begin{overpic}[width=433.62pt]{Loss_mode10.pdf} \put(5.0,90.0){c}
\end{overpic}

Figure 6: Evolution of the loss function for the validation and training sets
for the choice of a $m_{\text{max},1}=m_{\text{max},2}=m_{\text{max},3}=2$, b
$m_{\text{max},1}=m_{\text{max},2}=m_{\text{max},3}=7$, and c
$m_{\text{max},1}=m_{\text{max},2}=m_{\text{max},3}=10$

\begin{overpic}[width=433.62pt]{C9.pdf} \put(5.0,90.0){a} \end{overpic}

\begin{overpic}[width=433.62pt]{C12.pdf} \put(5.0,90.0){b} \end{overpic}

\begin{overpic}[width=433.62pt]{C15.pdf} \put(5.0,90.0){c} \end{overpic}

Figure 7: $R^{2}$ plots for the test set (375 data) using the proposed
approach for the classification of multi-sized images for the choice of a
$m_{\text{max},1}=m_{\text{max},2}=m_{\text{max},3}=4$, b
$m_{\text{max},1}=m_{\text{max},2}=m_{\text{max},3}=7$, and c
$m_{\text{max},1}=m_{\text{max},2}=m_{\text{max},3}=10$

\begin{overpic}[width=433.62pt]{C24.pdf} \put(5.0,90.0){a} \end{overpic}

\begin{overpic}[width=433.62pt]{C25.pdf} \put(5.0,90.0){b} \end{overpic}

Figure 8: $R^{2}$ plots for the test set (375 data) using the proposed
approach for the classification of multi-sized images for the choice of a
$m_{\text{max},1}=m_{\text{max},2}=2$ and $m_{\text{max},3}=10$, and the
choice of b $m_{\text{max},1}=2$, and $m_{\text{max},2}=m_{\text{max},3}=10$

###  5.2 Number of Fourier modes in each dimension

Our deep learning experiments demonstrate that there is a critical interplay
between the number of modes (i.e., $m_{\text{max},1}$, $m_{\text{max},2}$, and
$m_{\text{max},3}$) set in the proposed FNO framework and the tendency for
overfitting during the training procedure. Accordingly, setting the number of
modes beyond 2 leads to a severe divergence between the training and
validation loss. This fact can be observed in Fig. 6 when we set
$m_{\text{max},1}=7$, $m_{\text{max},2}=7$, and $m_{\text{max},3}=7$ or
$m_{\text{max},1}=10$, $m_{\text{max},2}=10$, and $m_{\text{max},3}=10$. The
reported results indicate that the number of modes plays a critical role in
the FNO model generalization. A further survey of the influence of the number
of modes in the FNO configuration is performed by varying the number of modes
in all three principal directions, from 2 to 10, and the obtained $R^{2}$
scores are tabulated in Table 1. Accordingly, the optimal mode configuration
for avoiding overfitting is 2, as the divergence between the validation and
training loss is minimized. Consequently, a careful selection of the number of
modes in the FNO units is necessary to make the deep learning framework robust
and reliable for the image classification application. The consequence of this
scenario is observable in Fig. 7, where we plot the $R^{2}$ score for the test
sets, for example, for the choice of
$m_{\text{max},1}=m_{\text{max},2}=m_{\text{max},3}=4$,
$m_{\text{max},1}=m_{\text{max},2}=m_{\text{max},3}=7$, and
$m_{\text{max},1}=m_{\text{max},2}=m_{\text{max},3}=10$. In all of these
cases, the $R^{2}$ scores obtained for the prediction of the permeability of
the porous media in the test set are less than 0.4.

We perform two other experiments. In the first one, we set only one mode
(e.g., $m_{\text{max},3}$) to 10 ($m_{\text{max},3}=10$) and the other two
modes to 2 (i.e., $m_{\text{max},1}=2$ and $m_{\text{max},2}=2$). In the
second one, we set only two modes (e.g., $m_{\text{max},2}$ and
$m_{\text{max},3}$) to 10 and the reminder mode to 2 (i.e.,
$m_{\text{max},1}=2$). The outputs of these two experiments are illustrated in
Fig. 8. As can be seen in Fig. 8, the resulting $R^{2}$ scores of the test set
are equal to 0.22298 and 0.34728, respectively, for the first and second
experiments. Accordingly, we conclude that even increasing one mode beyond 2
drastically negatively affects the performance of the proposed FNO framework
for the current application.

###  5.3 Channel width of FNOs

We further analyze the impact of different channel widths on the performance
of the introduced deep learning framework. Based on our machine learning
experiments, $R^{2}$ scores obtained for the channel width of 8, 27, 64, and
125 are 0.49904, 0.81618, 0.96815, and 0.94457, respectively. When the channel
width decreases from 64 to 27 or to 8, a significant drop in the $R^{2}$ score
is observed. Notably, increasing the channel width beyond 64 to 125 also leads
to a slight decrease in the precision of permeability predictions.

As discussed in Sect. 2.3, the choice of channel width is directly related to
the number of trainable parameters, which are 30897, 163783, 828673, and
3096531 for each respective channel width. Moreover, the channel width also
determines the size of the max pooling, representing the size of the global
feature vector. Hence, optimizing channel width is critical. Small channel
width leads to poor performance, whereas large channel width imposes high
computational costs and memory allocation without necessarily a significant
performance improvement.

###  5.4 Number of FNO units

We investigate the effect of varying the number of FNO units (see Fig. 1).
Deep learning experiments are conducted using one, two, three, four, and five
units to assess the impact on the introduced FNO performance. By computing the
$R^{2}$ score across the test set, we realize that there is no significant
improvement in the prediction accuracy. $R^{2}$ score for the FNO
configuration with one, two, three, four, and five units are respectively,
0.82767, 0.91703, 0.96813, 0.96759, and 0.97818. Hence, adding more units
(beyond 3) and making the network deeper does not have a remarkable effect on
the prediction accuracy. However, the number of trainable parameters and
consequently, the computational cost and required GPU memory (e.g., RAM)
escalated by adding FNO units. For example, 820353, 824513, 828673, 832833,
and 836993 are, respectively, the number of trainable parameters of the model
with one, two, three, four, and five layers of FNOs.

###  5.5 Activation functions

We give particular attention to the effect of choosing an activation function
on the prediction ability of our FNO model. In the primary setup, we configure
all layers to employ the ReLU activation function, except the last layer of
the classifier, where we utilize a sigmoid function. We implement two
alternative setups. In the first one, we alter the activation function of the
last layer to ReLU, this configuration results in a drastic reduction in the
$R^{2}$ score of the test set, regardless of if the output permeability is
normalized between 0 and 1. In the second setup, we replace the activation
function in all layers with sigmoid. As a consequence of this setup, a slight
decrease in performance is indicated, as $R^{2}$ score of 0.91478 is obtained
for the test set. Note that the training procedure becomes slower in this
setup, as the derivative of the sigmoid function results in a more complicated
computation graph compared to that one output by the derivation of the ReLU
function.

###  5.6 Static max pooling versus static average pooling

Within the context of capturing global features in the proposed FNO-based
framework, we explore the efficacy of implementing static average pooling as
an alternative to static max pooling. Our machine learning experiment yields a
$R^{2}$ score of 0.94478 in this case, demonstrating a marginal diminishment
in the network performance compared to the presence of static max pooling. As
supported by the literature (Qi et al., 2017a, b; Kashefi et al., 2021;
Kashefi and Mukerji, 2022; Kashefi et al., 2023), max pooling is a preferred
technique for classification tasks compared to average pooling. Our finding
shows a similar pattern for the introduced FNO-based framework.

\begin{overpic}[width=433.62pt]{CNN36.png} \put(5.0,90.0){a} \end{overpic}

\begin{overpic}[width=433.62pt]{CNN44.png} \put(5.0,90.0){b} \end{overpic}

\begin{overpic}[width=433.62pt]{CNN52.png} \put(5.0,90.0){c} \end{overpic}

\begin{overpic}[width=433.62pt]{CNN60.png} \put(5.0,90.0){d} \end{overpic}

Figure 9: A few examples of synthetically generated three-dimensional digital
porous media for examining the generalizability of the proposed neural
network; a an image of size $36^{3}$, b an image of size $44^{3}$, c an image
of size $52^{3}$, and d an image of size $60^{3}$

\begin{overpic}[width=433.62pt]{C26.pdf} \put(5.0,90.0){a} \end{overpic}

\begin{overpic}[width=433.62pt]{C27.pdf} \put(5.0,90.0){b} \end{overpic}

\begin{overpic}[width=433.62pt]{C28.pdf} \put(5.0,90.0){c} \end{overpic}

\begin{overpic}[width=433.62pt]{C29.pdf} \put(5.0,90.0){d} \end{overpic}

Figure 10: $R^{2}$ plots demonstrating the generalizability of the proposed
approach in classifying multi-sized images. The network, trained on images of
sizes $40^{3}$, $48^{3}$, and $56^{3}$, is used to predict images of sizes a
$36^{3}$ (375 data), b $44^{3}$ (375 data), c $52^{3}$ (375 data), and d
$60^{3}$ (375 data).

###  5.7 Generalizability

In this subsection, we assess the generalizability ability of the proposed
FNO-based framework. Note that the concept of generalizability in the context
of the present work extends to the network’s performance to predict the
permeability of cubic porous media with unseen sizes. As discussed in Sect. 4,
the network was initially trained using porous media with cubic geometries of
sizes $40^{3}$, $48^{3}$, and $56^{3}$. To examine the network capacity to
generalize, we predict the permeability of porous media with sizes $36^{3}$,
$44^{3}$, $52^{3}$, and $60^{3}$ with 375 cubes for each of these sizes using
our pretrained FNO-based framework. Figure 9 shows a few examples of these
synthetic data, generated for the purpose of examining the network
generalizability. As shown in Fig. 10, a slight decline is observed in the
accuracy of permeability predictions for porous media with unseen sizes.
However, the obtained $R^{2}$ scores remain in an excellent range. These
scores are 0.93185, 0.91124, 0.91500, and 0.90844 for the porous media sizes
of $36^{3}$, $44^{3}$, $52^{3}$, and $60^{3}$, respectively. As another
observation, the performance of our approach is marginally higher in
predicting the permeability of unseen porous media with smaller cubic sizes.
As highlighted in Fig. 10, $R^{2}$ scores of porous media with sizes of
$36^{3}$ are greater than ones with a size of $44^{3}$. A similar scenario
occurs when we compare porous media of sizes $52^{3}$ and $60^{3}$. This can
be attributed to the fact that, for smaller sizes, the fixed-size vector of
the global feature encodes the features of smaller cubes more effectively.
Moreover, note that the vector size is the same as the width channel. As a
last comment in this subsection, to enhance the network’s generalizability, a
potential strategy could involve expanding the training dataset to include
more than the initial three sets of geometry sizes.

\begin{overpic}[width=433.62pt]{Loss_mode2.pdf} \put(5.0,90.0){a}
\end{overpic}

\begin{overpic}[width=433.62pt]{Loss5_V2.pdf} \put(5.0,90.0){b} \end{overpic}

Figure 11: Evolution of the loss function for the validation and training sets
using a the proposed approach (see Fig. 1) and b the intuitive approach (see
Fig. 2)

###  5.8 Comparison with intuitive approach

####  5.8.1 Classification of fixed-sized image

For the comparison between the proposed approach (see Fig. 1) and the
intuitive approach (see Fig. 2), we consider the problem of predicting the
permeability of porous media with fixed cubic sizes. Specifically, we consider
a size of $48^{3}$. Similar outputs are observed for other sizes. To ensure a
fair comparison, both methodologies are investigated under similar conditions.
Specifically, both methods are set to have an approximately equal number of
trainable parameters (i.e., 828738 for the intuitive strategy and 828673 for
our approach). Accordingly, the size of the vector representing the global
feature is 64 in both methods. All other parameters such as the number of
modes in each direction, the number of FNO units, the classifier architecture,
and size, are the same in both methods and are set as those listed in Sect. 4
(i.e., the training section).

Our results demonstrate that both methods perform proficiently, with the
$R^{2}$ score of 0.99348 and 0.97360 over the test set for the intuitive
approach (see Fig. 2) and the proposed approach (see Fig. 1), respectively.
The evolution of the loss function for the training and validation sets
indicates a convergence after approximately 3000 epochs. This deep learning
experiment confirms an approximately equivalent computational cost between the
two approaches. Hence, when the image size of training data is fixed, both
strategies are effective for the defined image classification task and there
is no significant advantage for one method over the other, according to our
analysis. As a last point in this subsection, we note that one may also use
static max pooling in the architecture of the traditional approach since the
size of porous media is fixed in this experiment. Based on our results, the
performance does not change.

####  5.8.2 Classification of multi-sized images

In this subsection, we compare the performance of the proposed approach (see
Fig. 1) with the intuitive approach (see Fig. 2) in predicting the
permeability of porous media with varying sizes. The evolution of both
training and validation losses is depicted in Fig. 11. Figure 11 indicates a
divergence between the training and validation losses for the network used in
the intuitive approach, which suffers from overfitting, whereas this is not
the case for the proposed approach. The superiority of the proposed approach
is also evident by the $R^{2}$ score obtained for the test set. Accordingly,
the $R^{2}$ scores of the proposed approach and the intuitive approach are
respectively 0.96809 and $-0.42632$. The negative value of the $R^{2}$ score
for the intuitive approach demonstrates that its model makes worse predictions
than a model that simply predicts all outputs as the mean value of the
dataset. Note that changing hyper-parameters, such as the number of modes,
channel width, and number of FNO layers, does not improve the model of the
intuitive approach.

This flaw stems from two reasons. First, using the intuitive approach, the
network captures the global feature after lifting cubes into the original
space, while the trainable parameters of the network are mainly defined in the
Fourier space. Second, the adaptive max pooling’s size is altered depending on
the size of the input cubic porous medium. These two together lead to a
misrepresentation of the global feature of cubes with different sizes, when
the network tends to predict the permeability of the validation and test sets.
Note that in Sect. 5.8.1, we showed that the intuitive approach worked well
when it was trained over porous media with fixed sizes. However, the result of
our machine learning experiments illustrates that the global features of cubes
with different sizes are amalgamated. In contrast, our approach uses static
max pooling consistent with the channel width of Fourier neural operators
before transitioning back to the original space. This approach enables the
capture of global features prior to changing spaces.

##  6 Summary and future outlooks

In this research study, we introduced a novel deep learning framework based on
Fourier neural operators for classifying images with different sizes (see Fig.
1). Because Fourier neural operators are resolution invariant, they have the
potential to be used for the task of multi-sized image classification. To
reach this goal, Fourier neural operators must be connected to a classifier,
ideally using a pooling operator. To this end, we proposed the novel idea of
implementing a static max pooling operator, which functions in a high
dimensional space with the size of Fourier channel width. We showed the
efficiency and robustness of this framework by predicting the permeability of
three-dimensional digital porous media with three different sizes of $40^{3}$,
$48^{3}$, and $56^{3}$. We explored the effect of key parameters such as the
number of Fourier modes in each dimension, the channel width of the discrete
Fourier space, activation functions in different layers, and the number of
Fourier units. Additionally, we showed that while the network was only trained
on the porous media with the sizes of $40^{3}$, $48^{3}$, and $56^{3}$, it
could successfully predict the permeability of the porous media with the sizes
of $36^{3}$, $44^{3}$, $52^{3}$, and $60^{3}$, indicating its
generalizability. Moreover, we demonstrated that the idea of implementing an
adaptive max pooling (see Fig. 2), as an intuitive approach for connecting the
FNO layers to the classifier, showed a lack of performance when predicting the
permeability of porous media of varying sizes. Note that the adaptive max
pooling operated in spatial spaces and that pooling had to be adaptive to
handle input images with varying sizes.

As a future research direction, we aim to adapt the current architecture and
extend its capabilities to image classification. In contrast to the problem of
permeability prediction, this approach reduces the problem’s dimensionality to
two. Additionally, given that the standard dataset for image classification is
usually large, we anticipate improved generalizability of the proposed
framework.

## Data availability

The Python code for the three-dimensional problems is available on the
following GitHub repository, https://github.com/Ali-
Stanford/FNOMultiSizedImages.

## Acknowledgements

Financial support by the Shell-Stanford collaborative project on digital rock
physics is acknowledged. Additionally, the first author would like to thank
Prof. Gege Wen at Imperial College London for her helpful guidance and
discussion on the software engineering aspects of this study.

## References

  * Abadi et al. (2015) Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg, J., Mané, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Viégas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., Zheng, X., 2015.  TensorFlow: Large-scale machine learning on heterogeneous systems.  URL: https://www.tensorflow.org/. software available from tensorflow.org.
  * Andra et al. (2013a) Andra, H., Combaret, N., Dvorkin, J., Glatt, E., Han, J., Kabel, M., Keehm, Y., Krzikalla, F., Lee, M., Madonna, C., Marsh, M., Mukerji, T., Saenger, E.H., Sain, R., Saxena, N., Ricker, S., Wiegmann, A., Zhan, X., 2013a.  Digital rock physics benchmarks—part i: Imaging and segmentation.  Computers & Geosciences 50, 25–32.  doi:https://doi.org/10.1016/j.cageo.2012.09.005. benchmark problems, datasets and methodologies for the computational geosciences.
  * Andra et al. (2013b) Andra, H., Combaret, N., Dvorkin, J., Glatt, E., Han, J., Kabel, M., Keehm, Y., Krzikalla, F., Lee, M., Madonna, C., Marsh, M., Mukerji, T., Saenger, E.H., Sain, R., Saxena, N., Ricker, S., Wiegmann, A., Zhan, X., 2013b.  Digital rock physics benchmarks—part ii: Computing effective properties.  Computers & Geosciences 50, 33–43.  doi:https://doi.org/10.1016/j.cageo.2012.09.008. benchmark problems, datasets and methodologies for the computational geosciences.
  * Blunt et al. (2013) Blunt, M.J., Bijeljic, B., Dong, H., Gharbi, O., Iglauer, S., Mostaghimi, P., Paluszny, A., Pentland, C., 2013.  Pore-scale imaging and modelling.  Advances in Water Resources 51, 197–216.  doi:https://doi.org/10.1016/j.advwatres.2012.03.003. 35th Year Anniversary Issue.
  * Bonev et al. (2023) Bonev, B., Kurth, T., Hundt, C., Pathak, J., Baust, M., Kashinath, K., Anandkumar, A., 2023.  Spherical fourier neural operators: Learning stable dynamics on the sphere.  arXiv preprint arXiv:2306.03838 .
  * Brandstetter et al. (2023) Brandstetter, J., van den Berg, R., Welling, M., Gupta, J.K., 2023.  Clifford neural layers for pde modeling.  arXiv:2209.04934.
  * Chen et al. (2023) Chen, G., Liu, X., Li, Y., Meng, Q., Chen, L., 2023.  Laplace neural operator for complex geometries.  arXiv preprint arXiv:2302.08166 .
  * Choubineh et al. (2023) Choubineh, A., Chen, J., Wood, D.A., Coenen, F., Ma, F., 2023.  Fourier neural operator for fluid flow in small-shape 2d simulated porous media dataset.  Algorithms 16.  URL: https://www.mdpi.com/1999-4893/16/1/24, doi:10.3390/a16010024.
  * Darcy (1856) Darcy, H., 1856.  Les fontaines publiques de la ville de Dijon: exposition et application des principes à suivre et des formules à employer dans les questions de distribution d’eau. volume 1.  Victor Dalmont.
  * Das et al. (2018) Das, M.K., Mukherjee, P.P., Muralidhar, K., 2018.  Porous Media Applications: Biological Systems. Springer International Publishing.
  * Goodfellow et al. (2016) Goodfellow, I., Bengio, Y., Courville, A., 2016.  Deep Learning.  MIT Press.  http://www.deeplearningbook.org.
  * Gupta and Brandstetter (2022) Gupta, J.K., Brandstetter, J., 2022.  Towards multi-spatiotemporal-scale generalized pde modeling.  arXiv:2209.15616.
  * Hao et al. (2023) Hao, Z., Wang, Z., Su, H., Ying, C., Dong, Y., Liu, S., Cheng, Z., Song, J., Zhu, J., 2023.  Gnot: A general neural operator transformer for operator learning, in: International Conference on Machine Learning, PMLR. pp. 12556–12569.
  * Hong and Liu (2020) Hong, J., Liu, J., 2020.  Rapid estimation of permeability from digital rock using 3d convolutional neural network.  Computational Geosciences 24, 1523–1539.  doi:10.1007/s10596-020-09941-w.
  * Hua and Lu (2023) Hua, N., Lu, W., 2023.  Basis operator network: A neural network-based model for learning nonlinear operators via neural basis.  Neural Networks 164, 21–37.
  * Huang et al. (2023) Huang, X., Shi, W., Meng, Q., Wang, Y., Gao, X., Zhang, J., Liu, T.Y., 2023.  Neuralstagger: accelerating physics-constrained neural pde solver with spatial-temporal decomposition.  arXiv preprint arXiv:2302.10255 .
  * Jiang et al. (2023a) Jiang, P., Yang, Z., Wang, J., Huang, C., Xue, P., Chakraborty, T.C., Chen, X., Qian, Y., 2023a.  Efficient super-resolution of near-surface climate modeling using the fourier neural operator.  Journal of Advances in Modeling Earth Systems 15, e2023MS003800.  doi:https://doi.org/10.1029/2023MS003800. e2023MS003800 2023MS003800.
  * Jiang et al. (2023b) Jiang, Z., Zhu, M., Li, D., Li, Q., Yuan, Y.O., Lu, L., 2023b.  Fourier-mionet: Fourier-enhanced multiple-input neural operators for multiphase modeling of geological carbon sequestration.  arXiv:2303.04778.
  * Johnny et al. (2022) Johnny, W., Brigido, H., Ladeira, M., Souza, J.C.F., 2022.  Fourier neural operator for image classification, in: 2022 17th Iberian Conference on Information Systems and Technologies (CISTI), pp. 1–6.  doi:10.23919/CISTI54924.2022.9820128.
  * Kabri et al. (2023) Kabri, S., Roith, T., Tenbrinck, D., Burger, M., 2023.  Resolution-invariant image classification based on fourier neural operators, in: Calatroni, L., Donatelli, M., Morigi, S., Prato, M., Santacesaria, M. (Eds.), Scale Space and Variational Methods in Computer Vision, Springer International Publishing, Cham.
  * Kashefi et al. (2023) Kashefi, A., Guibas, L.J., Mukerji, T., 2023.  Physics-informed PointNet: On how many irregular geometries can it solve an inverse problem simultaneously? application to linear elasticity.  Journal of Machine Learning for Modeling and Computing 4, 1–25.
  * Kashefi and Mukerji (2021) Kashefi, A., Mukerji, T., 2021.  Point-cloud deep learning of porous media for permeability prediction.  Physics of Fluids 33, 097109.
  * Kashefi and Mukerji (2022) Kashefi, A., Mukerji, T., 2022.  Physics-informed pointnet: A deep learning solver for steady-state incompressible flows and thermal fields on multiple sets of irregular geometries.  Journal of Computational Physics 468, 111510.  doi:https://doi.org/10.1016/j.jcp.2022.111510.
  * Kashefi and Mukerji (2023) Kashefi, A., Mukerji, T., 2023.  Prediction of fluid flow in porous media by sparse observations and physics-informed pointnet.  Neural Networks 167, 80–91.
  * Kashefi et al. (2021) Kashefi, A., Rempe, D., Guibas, L.J., 2021.  A point-cloud deep learning framework for prediction of fluid flow fields on irregular geometries.  Physics of Fluids 33, 027104.  doi:10.1063/5.0033376.
  * Keehm et al. (2004) Keehm, Y., Mukerji, T., Nur, A., 2004.  Permeability prediction from thin sections: 3d reconstruction and lattice-boltzmann flow simulation.  Geophysical Research Letters 31.  doi:https://doi.org/10.1029/2003GL018761.
  * Kingma and Ba (2014) Kingma, D.P., Ba, J., 2014.  Adam: A method for stochastic optimization.  arXiv preprint arXiv:1412.6980 .
  * Kontolati et al. (2023) Kontolati, K., Goswami, S., Shields, M.D., Karniadakis, G.E., 2023.  On the influence of over-parameterization in manifold based surrogates and deep neural operators.  Journal of Computational Physics 479, 112008.
  * Kumeria (2022) Kumeria, T., 2022.  Advances on porous nanomaterials for biomedical application (drug delivery, sensing, and tissue engineering).  ACS Biomaterials Science & Engineering 8, 4025–4027.  doi:10.1021/acsbiomaterials.2c01103.
  * Lantuejoul (2002) Lantuejoul, C., 2002.  Geostatistical Simulation: Models and Algorithms.  Springer.
  * Le Ravalec-Dupin et al. (2004) Le Ravalec-Dupin, M., Roggero, F., Froidevaux, R., 2004.  Conditioning truncated gaussian realizations to static and dynamic data.  SPE Journal 9, 475–480.  doi:10.2118/84944-PA.
  * Lee (2022) Lee, S., 2022.  Mesh-independent operator learning for partial differential equations, in: ICML 2022 2nd AI for Science Workshop.
  * Lehmann et al. (2024) Lehmann, F., Gatti, F., Bertin, M., Clouteau, D., 2024.  3d elastic wave propagation with a factorized fourier neural operator (f-fno).  Computer Methods in Applied Mechanics and Engineering 420, 116718.
  * Li et al. (2022a) Li, Z., Huang, D.Z., Liu, B., Anandkumar, A., 2022a.  Fourier neural operator with learned deformations for pdes on general geometries.  arXiv preprint arXiv:2207.05209 .
  * Li et al. (2020a) Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A., Anandkumar, A., 2020a.  Fourier neural operator for parametric partial differential equations.  arXiv preprint arXiv:2010.08895 .
  * Li et al. (2020b) Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A., Anandkumar, A., 2020b.  Neural operator: Graph kernel network for partial differential equations.  arXiv preprint arXiv:2003.03485 .
  * Li et al. (2023) Li, Z., Kovachki, N.B., Choy, C., Li, B., Kossaifi, J., Otta, S.P., Nabian, M.A., Stadler, M., Hundt, C., Azizzadenesheli, K., Anandkumar, A., 2023.  Geometry-informed neural operator for large-scale 3d pdes.  arXiv:2309.00583.
  * Li et al. (2022b) Li, Z., Peng, W., Yuan, Z., Wang, J., 2022b.  Fourier neural operator approach to large eddy simulation of three-dimensional turbulence.  Theoretical and Applied Mechanics Letters 12, 100389.
  * Li et al. (2021) Li, Z., Zheng, H., Kovachki, N., Jin, D., Chen, H., Liu, B., Azizzadenesheli, K., Anandkumar, A., 2021.  Physics-informed neural operator for learning partial differential equations.  arXiv preprint arXiv:2111.03794 .
  * Liang and Fletcher (2023) Liang, Y., Fletcher, D., 2023.  Computational fluid dynamics simulation of forward osmosis (fo) membrane systems: Methodology, state of art, challenges and opportunities.  Desalination 549, 116359.  doi:https://doi.org/10.1016/j.desal.2022.116359.
  * Liu et al. (2023) Liu, M., Ahmad, R., Cai, W., Mukerji, T., 2023.  Hierarchical homogenization with deep-learning-based surrogate model for rapid estimation of effective permeability from digital rocks.  Journal of Geophysical Research: Solid Earth 128, e2022JB025378.  doi:https://doi.org/10.1029/2022JB025378.
  * Lyu et al. (2023) Lyu, Y., Zhao, X., Gong, Z., Kang, X., Yao, W., 2023.  Multi-fidelity prediction of fluid flow based on transfer learning using Fourier neural operator.  Physics of Fluids 35, 077118.  doi:10.1063/5.0155555.
  * Majumdar et al. (2023) Majumdar, R., Karande, S., Vig, L., 2023.  How important are specialized transforms in neural operators?, in: 1st Workshop on the Synergy of Scientific and Machine Learning Modeling @ ICML2023.  URL: https://openreview.net/forum?id=DU3Z6ZdqhZ.
  * Masroor et al. (2023) Masroor, M., Emami Niri, M., Sharifinasab, M.H., 2023.  A multiple-input deep residual convolutional neural network for reservoir permeability prediction.  Geoenergy Science and Engineering 222, 211420.  doi:https://doi.org/10.1016/j.geoen.2023.211420.
  * Maust et al. (2022) Maust, H., Li, Z., Wang, Y., Leibovici, D., Bruno, O., Hou, T., Anandkumar, A., 2022.  Fourier continuation for exact derivative computation in physics-informed neural operators.  arXiv preprint arXiv:2211.15960 .
  * Meng et al. (2023) Meng, Y., Jiang, J., Wu, J., Wang, D., 2023.  Transformer-based deep learning models for predicting permeability of porous media.  Authorea Preprints .
  * Paszke et al. (2019) Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al., 2019.  Pytorch: An imperative style, high-performance deep learning library, in: Advances in Neural Information Processing Systems, pp. 8024–8035.
  * Pathak et al. (2022) Pathak, J., Subramanian, S., Harrington, P., Raja, S., Chattopadhyay, A., Mardani, M., Kurth, T., Hall, D., Li, Z., Azizzadenesheli, K., et al., 2022.  Fourcastnet: A global data-driven high-resolution weather model using adaptive fourier neural operators.  arXiv preprint arXiv:2202.11214 .
  * Peng et al. (2024) Peng, W., Qin, S., Yang, S., Wang, J., Liu, X., Wang, L.L., 2024.  Fourier neural operator for real-time simulation of 3d dynamic urban microclimate.  Building and Environment 248, 111063.
  * Peng et al. (2023a) Peng, W., Yuan, Z., Li, Z., Wang, J., 2023a.  Linear attention coupled Fourier neural operator for simulation of three-dimensional turbulence.  Physics of Fluids 35, 015106.  doi:10.1063/5.0130334.
  * Peng et al. (2023b) Peng, Z., Yang, B., Liu, L., Xu, Y., 2023b.  Rapid surrogate modeling of magnetotelluric in the frequency domain using physics-driven deep neural networks.  Computers & Geosciences 176, 105360.
  * Poels et al. (2023) Poels, Y., Derks, G., Westerhof, E., Minartz, K., Wiesen, S., Menkovski, V., 2023.  Fast dynamic 1d simulation of divertor plasmas with neural pde surrogates.  arXiv preprint arXiv:2305.18944 .
  * Qi et al. (2017a) Qi, C.R., Su, H., Mo, K., Guibas, L.J., 2017a.  Pointnet: Deep learning on point sets for 3d classification and segmentation, in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 652–660.
  * Qi et al. (2017b) Qi, C.R., Yi, L., Su, H., Guibas, L.J., 2017b.  Pointnet++: Deep hierarchical feature learning on point sets in a metric space, in: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (Eds.), Advances in Neural Information Processing Systems, Curran Associates, Inc.
  * Rahman et al. (2022a) Rahman, M.A., Florez, M.A., Anandkumar, A., Ross, Z.E., Azizzadenesheli, K., 2022a.  Generative adversarial neural operators.  arXiv preprint arXiv:2205.03017 .
  * Rahman et al. (2022b) Rahman, M.A., Ross, Z.E., Azizzadenesheli, K., 2022b.  U-no: U-shaped neural operators.  arXiv preprint arXiv:2204.11127 .
  * Renn et al. (2023) Renn, P.I., Wang, C., Lale, S., Li, Z., Anandkumar, A., Gharib, M., 2023.  Forecasting subcritical cylinder wakes with fourier neural operators.  arXiv preprint arXiv:2301.08290 .
  * Sun et al. (2023) Sun, H., Zhou, L., Fan, D., Zhang, L., Yang, Y., Zhang, K., Yao, J., 2023.  Permeability prediction of considering organic matter distribution based on deep learning.  Physics of Fluids 35, 032014.  doi:10.1063/5.0142574.
  * Tembely et al. (2020) Tembely, M., AlSumaiti, A.M., Alameri, W., 2020.  A deep learning perspective on predicting permeability in porous media from network modeling to direct simulation.  Computational Geosciences 24.  doi:10.1007/s10596-020-09941-w.
  * Thodi et al. (2023) Thodi, B.T., Ambadipudi, S.V.R., Jabari, S.E., 2023.  Fourier neural operator for learning solutions to macroscopic traffic flow models: Application to the forward and inverse problems.  arXiv:2308.07051.
  * Tran et al. (2023) Tran, A., Mathews, A., Xie, L., Ong, C.S., 2023.  Factorized fourier neural operators.  arXiv:2111.13802.
  * Wen et al. (2022) Wen, G., Li, Z., Azizzadenesheli, K., Anandkumar, A., Benson, S.M., 2022.  U-fno—an enhanced fourier neural operator-based deep-learning model for multiphase flow.  Advances in Water Resources 163, 104180.
  * White et al. (2023a) White, C., Berner, J., Kossaifi, J., Elleithy, M., Pitt, D., Leibovici, D., Li, Z., Azizzadenesheli, K., Anandkumar, A., 2023a.  Physics-informed neural operators with exact differentiation on arbitrary geometries, in: The Symbiosis of Deep Learning and Differential Equations III.
  * White et al. (2023b) White, C., Tu, R., Kossaifi, J., Pekhimenko, G., Azizzadenesheli, K., Anandkumar, A., 2023b.  Speeding up fourier neural operators via mixed precision.  arXiv preprint arXiv:2307.15034 .
  * Wu et al. (2018) Wu, J., Yin, X., Xiao, H., 2018.  Seeing permeability from images: fast prediction with convolutional neural networks.  Science Bulletin 63, 1215–1222.  doi:https://doi.org/10.1016/j.scib.2018.08.006.
  * Xi et al. (2022) Xi, J., Ersoy, O.K., Cong, M., Zhao, C., Qu, W., Wu, T., 2022.  Wide and deep fourier neural network for hyperspectral remote sensing image classification.  Remote Sensing 14.  URL: https://www.mdpi.com/2072-4292/14/12/2931, doi:10.3390/rs14122931.
  * Xie et al. (2023) Xie, C., Zhu, J., Yang, H., Wang, J., Liu, L., Song, H., 2023.  Relative permeability curve prediction from digital rocks with variable sizes using deep learning.  Physics of Fluids 35, 096605.  doi:10.1063/5.0167998.
  * Xiong et al. (2023) Xiong, W., Ma, M., Sun, P., Tian, Y., 2023.  Koopmanlab: machine learning for solving complex physics equations.  arXiv preprint arXiv:2301.01104 .
  * Yang et al. (2022) Yang, H., Li, Z., Sastry, K., Mukhopadhyay, S., Anandkumar, A., Khailany, B., Singh, V., Ren, H., 2022.  Large scale mask optimization via convolutional fourier neural operator and litho-guided self training.  arXiv preprint arXiv:2207.04056 .
  * Yang et al. (2023) Yang, Y., Gao, A.F., Azizzadenesheli, K., Clayton, R.W., Ross, Z.E., 2023.  Rapid seismic waveform modeling and inversion with neural operators.  IEEE Transactions on Geoscience and Remote Sensing 61, 1–12.  doi:10.1109/TGRS.2023.3264210.
  * You et al. (2022) You, H., Yu, Y., D’Elia, M., Gao, T., Silling, S., 2022.  Nonlocal kernel network (nkn): a stable and resolution-independent deep neural network.  Journal of Computational Physics 469, 111536.
  * Zhao et al. (2022) Zhao, J., George, R.J., Zhang, Y., Li, Z., Anandkumar, A., 2022.  Incremental fourier neural operator.  arXiv preprint arXiv:2211.15188 .
  * Zhao et al. (2024) Zhao, X., Chen, X., Gong, Z., Zhou, W., Yao, W., Zhang, Y., 2024.  Recfno: a resolution-invariant flow and heat field reconstruction method from sparse observations via fourier neural operator.  International Journal of Thermal Sciences 195, 108619.
  * Zhao et al. (2023) Zhao, X., Sun, Y., Zhang, T., Xu, B., 2023.  Local convolution enhanced global fourier neural operator for multiscale dynamic spaces prediction.  arXiv:2311.12902.
  * Zhu et al. (2023a) Zhu, M., Feng, S., Lin, Y., Lu, L., 2023a.  Fourier-deeponet: Fourier-enhanced deep operator networks for full waveform inversion with improved accuracy, generalizability, and robustness.  arXiv preprint arXiv:2305.17289 .
  * Zhu et al. (2023b) Zhu, M., Zhang, H., Jiao, A., Karniadakis, G.E., Lu, L., 2023b.  Reliable extrapolation of deep neural operators informed by physics or sparse observations.  Computer Methods in Applied Mechanics and Engineering 412, 116064.
  * Zou et al. (2023) Zou, C., Azizzadenesheli, K., Ross, Z.E., Clayton, R.W., 2023.  Deep neural helmholtz operators for 3d elastic wave propagation and inversion.  arXiv:2311.09608.