# JefiAtten: An Attention-Based Neural Network Model for Solving Maxwell’s
Equations with Charge and Current Sources

Ming-Yan Sun  Peng Xu  Jun-Jie Zhang  Tai-Jiao Du  Jian-Guo Wang  Xi’an
Research Institute of High Tech, Xi’an 710021, China  Division of
Computational Physics and Intelligent Modeling, Northwest Institute of Nuclear
Technology, Xi’an 710024, China  School of Information and Communications
Engineering, Xi’an Jiaotong University, Xi’an 710049, China

###### Abstract

We present JefiAtten, a novel neural network model employing the attention
mechanism to solve Maxwell’s equations efficiently. JefiAtten uses self-
attention and cross-attention modules to understand the interplay between
charge density, current density, and electromagnetic fields. Our results
indicate that JefiAtten can generalize well to a range of scenarios,
maintaining accuracy across various spatial distribution and handling
amplitude variations. The model showcases an improvement in computation speed
after training, compared to traditional integral methods. The adaptability of
the model suggests potential for broader applications in computational
physics, with further refinements to enhance its predictive capabilities and
computational efficiency. Our work is a testament to the efficacy of
integrating attention mechanisms with numerical simulations, marking a step
forward in the quest for data-driven solutions to physical phenomena.

###### keywords:

Maxwell’s equations; AI for science; AI for electromagnetism; transformer.

††journal: Machine Learning: Science and Technology

##  1 Introduction

Plasma, being the fourth state of matter apart from solid, liquid, and gas,
has garnered extensive attention from researchers due to its complex dynamics
beyond various technological applications [1]. Given the intrinsic properties
of plasma, the Maxwell’s equations occupy a pivotal role in plasma simulation.
These electromagnetic equations, when integrated with additional physical
models—such as fluid[2, 3], collisional[4, 5] and kinetic models—to create
sophisticated simulation frameworks that are particle-based (such as particle-
in-cell (PIC) methods[6, 7, 8, 9]) or field-based (such as magneto-
hydrodynamic (MHD) simulations[10, 11].) In the simulation of complex plasma
systems, the speed and accuracy of solving Maxwell’s equations reflect the
capability of the complex plasma system to a certain extent. The classical
Maxwell solvers, such as the Finite Element Method (FEM) [12] and Finite
Difference Time Domain (FDTD)[13, 14, 15], are esteemed for their high
computational accuracy and extensive applicability.

With the advent of the deep learning, it is natural to ask if we can use these
tools to either increase the simulation performance or tackle challenging
tasks? In an effort to surmount this question, deep learning has been applied
by researchers as a tool for solving all sorts of scientific problems [16]. In
terms of equation related scenarios, the networks utilized are principally
classified into three categories: data-driven (e.g., graph neural networks
(GNNs)[17], Convolutional neural network (CNN)[18], 3D Earth-specific
transformer (3DEST)[19], etc.), model-driven (e.g., Physics-Informed Neural
Networks (PINNs)[20], Fourier Neural Operators (FNOs)[21], Deep Operator
Networks (DeepONet)[22], etc.), and the hybrid-approach (e.g., hybrid data-
and physics-augmented convolutional neural network (WaveY-Net)[23], hybrid
modulation network (HMN)[24], etc.).

Inspired by the success of large language models in areas such as natural
language processing and computer vision[25, 26, 27, 28], numerous researchers
have extended their applications to the field of numerical simulation. A
growing body of work has demonstrated the rapid solution of partial
differential equations by employing attention mechanisms to decode inter-data
relationships, thereby achieving precise outcomes[29, 30, 31]. These
researches offer valuable insights for our own endeavors in solving Maxwell’s
equations.

In this paper, we propose "JefiAtten", a network model designed to solve
Maxwell’s equations. The core of the model is an attention mechanism that is
adept at learning the interconnections among data elements. We show that
JefiAtten can effectively handle the Maxwell’s equations in 3D space-time with
various configurations. Meanwhile, we demonstrate that JefiAtten, when
trained, acquires accelerated computational speed over the classical integral
methods while remaining the generalization capabilities. This generalization
capability is manifested in two levels: 1. use the trained JefiAtten to
predict a different scenario; 2. re-train JefiAtten for the first several time
steps and use it to predict the successive time steps. Remarkably, we find
that when the neural network encounters completely new scenarios that exceed
the numerical scope of the training dataset, we only need to re-train the
network with a few similar scenarios so that JefiAtten can adapt to these new
configurations.

The paper is organized in the following structure. In Sec. 2, we present a
short introduction of the Jefimenko’s equations and the self-attention
mechanism. In Sec. 3, we first introduce the data and platform used during the
model training process. Then, we delve into the model details. In Sec. 4, we
address the model’s computational speed, predictive capacity, and
generalization ability, emphasizing its advantages in comparison to
traditional algorithms. Through tests with various training datasets, we
showcase the merits of JefiAtten in solving Maxwell’s equations. The
conclusion and outlook of JefiAtten is in Sec. 5.

##  2 Background

In this section, we commence by presenting the high-dimensional Jefimenko’s
equations, which constitutes the primary problem that we aim to solve. Then,
we explore the concept of the attention mechanism, a component that plays a
pivotal role within our model’s architecture.

###  2.1 Jefimenko’s equations

The Jefimenko’s equations are named after the physicist Oleg D. Jefimenko.
Incorporating the principles of finite light speed and the relativistic
effects on the delayed propagation of fields, the Jefimenko’s equations yield
the distributions of electric and magnetic fields through the specification of
the spatial distribution of charge and current densities. This formulation
provides an essential theoretical tool for the analysis of electromagnetic
phenomena.

$\displaystyle\mathbf{E}(\mathbf{r},t)$ $\displaystyle=$
$\displaystyle\frac{1}{4\pi\epsilon_{0}}\int\left[\frac{\mathbf{r}-\mathbf{r}^{\prime}}{|\mathbf{r}-\mathbf{r}^{\prime}|^{3}}\rho(\mathbf{r}^{\prime},t_{r})\right.$
(1)
$\displaystyle+\frac{\mathbf{r}-\mathbf{r}^{\prime}}{|\mathbf{r}-\mathbf{r}^{\prime}|^{2}}\text{$\frac{1}{c}$}\frac{\partial\rho(\mathbf{r}^{\prime},t_{r})}{\partial
t}$
$\displaystyle\left.-\frac{1}{|\mathbf{r}-\mathbf{r}^{\prime}|}\text{$\frac{1}{c^{2}}$}\frac{\partial\mathbf{J}(\mathbf{r}^{\prime},t_{r})}{\partial
t}\right]d^{3}\mathbf{r}^{\prime}$ $\displaystyle\mathbf{B}(\mathbf{r},t)$
$\displaystyle=$
$\displaystyle-\frac{1}{4\pi\epsilon_{0}c^{2}}\int\left[\frac{\mathbf{r}-\mathbf{r}^{\prime}}{|\mathbf{r}-\mathbf{r}^{\prime}|^{3}}\times\mathbf{J}(\mathbf{r}^{\prime},t_{r})\right.$
(2)
$\displaystyle+\left.\frac{\mathbf{r}-\mathbf{r}^{\prime}}{|\mathbf{r}-\mathbf{r}^{\prime}|^{2}}\times\text{$\frac{1}{c}$}\frac{\partial\mathbf{J}(\mathbf{r}^{\prime},t_{r})}{\partial
t}\right]d^{3}\mathbf{r}^{\prime}$ $\displaystyle t_{r}$ $\displaystyle=$
$\displaystyle t-\frac{|\mathbf{r}-\mathbf{r}^{\prime}|}{c},$ (3)

where $\mathbf{\rho}$ and $\mathbf{J}$ represent the charge density and
current density respectively, $\mathbf{E}$ and $\mathbf{B}$ signify the
electric field and magnetic field at position $\mathbf{r}$ and time $t$.
$t_{r}$ denotes the time delay. As the generalized solutions of Maxwell’s
equations, Jefimenko’s equations[32] obviate the need for the specification of
boundary conditions within the computational process. Instead, these equations
necessitate only the inputs of current density and charge density to resolve
the electric and magnetic fields, which make it more convenient for us to
train the network solely based on the physical quantities $\mathbf{\rho}$,
$\mathbf{J}$, $\mathbf{E}$ and $\mathbf{B}$. The Jefimenko’s equations can be
used with the employment of transport equations, such as the Boltzmann
equation, so that we can arrive at a plasma simulator[33, 34]. The
interconnections of $\mathbf{\rho}$, $\mathbf{J}$, $\mathbf{E}$ and
$\mathbf{B}$ forge a pronounced interdependence between them, thereby
amplifying the intricacies associated with the solution process, suggesting
the usage of the attention mechanism in deep neural networks.

###  2.2 Attention mechanism

The top-down selective attention[35] is characterized by its intentional
nature and is task-specific in its deployment. Such a paradigm of selective
attention can be efficaciously integrated into machine learning algorithms to
mitigate the challenges posed by the deluge of information. A notable
advancement was made in 2017 [36], which adeptly harnessed the attention
mechanism to the realm of machine translation. The quintessence of the
attention mechanism is encapsulated in its ability to compute the relational
significance among data points. It segments the data into three fundamental
constituents: query vectors $q_{j}$, key vectors $k_{j}$, and values $v_{j}$.
The core computational process of this mechanism is depicted in Fig. 1 a.

Figure 1:  (a) Scaled Dot-Product Attention; (b) The multi-head attention
mechanism, composed of $h$ Scaled Dot-Product Attention modules.

In attention mechanism, a linear mapping is firstly performed on the input
vectors $q$, $k$, and $v$, so that these vectors are converted into the
embedding space. Next, the similarity between $Q$ and $K$ ($Q=W^{Q}q$,
$K=W^{K}k$, where $W^{Q}$ and $W^{K}$ are weight matrices) is obtained,
followed by the Softmax normalization. Finally, the obtained weights are used
to perform a weighted summation on all the values in V ($Q=W^{V}v$).

$\begin{array}[]{cc}\text{Attention}(Q,K,V)=\text{softmax}(\frac{QK^{T}}{\sqrt{d_{k}}})V\end{array}$
(4)

Due to the limitations of single attention operation in effectively handling
complex tasks, a multi-head attention mechanism is introduced [36], as
illustrated in Fig. 1 b.

This mechanism involves distributing the parameters into multiple subspaces,
$Q_{i}=QW^{Q}_{i},K_{i}=KW^{K}_{i},V_{i}=VW^{V}_{i},i=1,...,n$ (where $n$ is
the number of subspaces) where each subspace performs Scaled Dot-Product
Attention independently.

$\begin{array}[]{cc}\text{head}_{i}=\text{Attention}(Q_{i},K_{i},V_{i}),i=1,...,n\end{array}$
(5)

Owing to the independence of parameters within each distinct subspace, the
multi-head attention mechanism surpasses a singular attention framework in its
capacity to excavate information interrelations among data. Upon the
completion of informational computations within the individual subspaces, the
multi-head attention mechanism amalgamates the resultant outputs and
subsequently conducts multiplication with a weight matrix.

$\begin{array}[]{cc}\text{MultiHead}(Q,K,V)=\text{Concact}(\text{head}_{1},...,\text{head}_{n})W^{O}\end{array}$
(6)

It is evident that the integration of attention mechanisms within neural
networks substantially enhances their capacity to identify relationships among
data. The depth of the network, established through the superposition of
multi-layer attention mechanisms, is often instrumental in discerning the
subtle and complex interrelations among data. Predicated upon this fundamental
principle, we have engineered a model designed for the resolution of high-
dimensional Maxwell’s equations.

##  3 Structure of the model

In this section, we first provide the requisite data and computational
platform necessary for the training of the model. Subsequently, we present a
detailed description of the model specifics.

###  3.1 Data and Platform

The dataset employed in this training exercise was produced through the
utilization of JefiGPU [33], our custom-developed, GPU-accelerated framework
designed specifically for the computation of Jefimenko’s equations. JefiGPU
integrates discretized data with CUDA cores, harnessing the parallel
processing of GPUs to expedite the calculation of Jefimenko’s equations.
Considering that the purpose of the training is to utilize Jefimenko’s
equation in solving Maxwell’s equations, it is essential for $\mathbf{\rho}$
and $\mathbf{J}$ to meet the $\nabla\cdot\mathbf{J}+\partial\rho/\partial t=0$
continuum relation.

To ensure the precision of the data, we have employed Mathematica [37] for the
computation of $\mathbf{\rho}$ and $\mathbf{J}$. Due to the wide range of
options for selecting $\mathbf{\rho}$ and $\mathbf{J}$, we aim to strike a
balance between the complexity of the data and the computational time. To this
end, we adopted the trigonometric functions as the basis functions and
combined them in a flexible manner, resulting in a database comprising 500
evolution data sequences for model training and testing. The outcomes of these
computations are accessible at the following repository:
https://github.com/sunminmgyan/Data.git.

We utilize the natural unit system for all mentioned physical quantities in
the text,(we set $\hbar=c=k_{B}=\epsilon_{0}=1$) and the corresponding
conversion relations can be found in Ref. [38]. We discretized the spatial
domain into a $20\times 20\times 20$ grid and delineated the region containing
$\rho$ and $\mathbf{J}$ as $[[-2.85,2.85],[-2.85,2.85],[-2.85,2.85]]\text{
GeV}^{-3}$. The designated observation domain for the electric and magnetic
fields was similarly confined to this 3D region. The temporal resolution was
set with an interval, $dt$, of 0.005 $\text{GeV}^{-1}$. Electromagnetic field
calculations were conducted using JefiGPU, with a running of 10,000 time
steps, during which the values of $\rho$ and $\mathbf{J}$ were continuously
updated by analytical equations. The details of the computational procedures
are available for review within an example hosted at
https://codeocean.com/capsule/0744578/tree/v1.

Beyond employing various combinations of trigonometric functions, we have also
experimented with alterations to their amplitude and periodicity in order to
evaluate the model’s generalizability in the face of such changes. The
computational process and the parameters remained consistent throughout; only
the input values for $\mathbf{J}$ and $\rho$ were modified. The visualization
of subsets from the dataset is shown in Fig. 2 and Fig. 3

In current work, we utilized four NVIDIA A100 cards for training the code and
implemented parallel computing using the DataParallel package in PyTorch.

Figure 2:  Visualization of mesh of the sources $\rho$ and $\mathbf{J}$. (a)
depicting the three-dimensional distribution of the charge density $\rho$,
accompanied by the cross-sectional views of $\rho$ in the $x$, $y$, and $z$
directions; (b) illustrates the three-dimensional distribution of the current
density $J_{y}$, with corresponding cross-sectional views of $J_{y}$ in the
$x$, $y$, and $z$ directions.  Figure 3:  Visualization of the electromagnetic
fields $\mathbf{E}$ and $\mathbf{B}$. (a) depicting the three-dimensional
distribution of the electric field $E_{x}$, accompanied by the cross-sectional
views of $E_{x}$ in the $x$, $y$, and $z$ directions; (b) the three-
dimensional distribution of the magnetic field $B_{x}$, with corresponding
cross-sectional views of $B_{x}$ in the $x$, $y$, and $z$ directions.

###  3.2  Overview of Model Architecture

Here, we provide a succinct exposition of the model we have constructed. The
Transformer architecture has gained considerable popularity in recent years,
with numerous studies demonstrating its formidable capacity to process complex
data and to extract information effectively. However, limitations of the
Transformer persist in addressing multidimensional, highly-correlated, and
structurally irregular datasets.

Due to the complexity of high-dimensional, time-dependent nature of the
Maxwell equations, we have considered the potential impact of varying inputs
on the network during the computational process. To address this, we have
designed a fully connected neural network to preprocess the input data. This
process is capable of not only processing the data across its multiple
dimensions but also of preserving, to some extent, the inherent
characteristics of the data. Diverging from the traditional Transformer
architecture, we have removed the use of positional encoding to convey
locational information within the data. Instead, we have instituted an
attention module subsequent to the fully connected input network, which serves
as a surrogate for the conventional positional encoding.

Next, owing to the high degree of complexity inherent problems, a unitary
network structure would be categorically insufficient for successful
resolution. Inspired by the parallel computing approach, we use multiple
attention modules in our network structure. The significance of the existence
of these modules is to disassemble the original problem into different parts,
and learn the correlation relationship in the same data by using independent
parameters respectively, so as to achieve the purpose of fully mining the data
relationship, and the structure of the whole model is shown in Fig. 4.

Figure 4: The JefiAtten’s model structure. (a) Input current density and
charge density at time $t_{1}$ and electromagnetic field at time $t_{1}$. (b)
Grid partitioning of the input data. (c) The main structure of the network.(d)
Output electric field and magnetic field at time $t_{2}=t_{1}+dt$. (e)
Attention Block; (f) Self-attention Block.

###  3.3 Attention module

In our model, there are two types of attention modules, namely self-attention
module and cross-attention module. The self-attention module is primarily used
for learning the relationships within the data itself, such as the physical
information, spatial information, and temporal information inherent in the
data. On the other hand, the cross-attention module learns the relationships
between the inputs $\rho$, $\mathbf{J}$, $\mathbf{E}$, and $\mathbf{B}$. The
structure of each attention module resembles that of the Transformer model
[36],

$\begin{array}[]{cc}\begin{matrix}f_{\text{middle}}=\text{Atten}(f_{\text{input}})+f_{\text{input}}\\\
f_{\text{output}}=\text{LayerNorm}(FFN(f_{\text{middle}})+f_{\text{middle}}),\end{matrix}\end{array}$
(7)

where $Atten()$ represents the attention mechanism introduced in Sec. 2.2, and
here we employ the multi-head attention mechanism. LayerNorm refers to layer
normalization[39], a technique that [40] has demonstrated to confer stability
to the input distribution during forward propagation within the network.
Furthermore, it also contributes to the stabilization of the gradient during
back-propagation. To prevent loss of information in the data, we only use
LayerNorm at the end of cross-attention. $FFN()$ stands for the feed-forward
neural network. If the model solely relies on the attention mechanism, the
network would be limited to a linear expression. However, our purpose in
constructing the network is to enable it to acquire more information from the
data. Thus, the incorporation of non-linear features brought by FFN is
crucial. Furthermore, the activation functions effectively transmit error
signals during the back-propagation process, thus facilitating the
construction of deeper networks. By augmenting the attention mechanism with
FFN, we achieve a better fusion of linear and non-linear components, enabling
more sophisticated data learning. Additionally, Ref. [41] has also discovered
the memory function of FFN in the field of NLP (Natural Language Processing),
further validating its necessity in the network, which is the reason why we
retain FFN in the attention module.

###  3.4  Training techniques

To ensure the effectiveness of model learning, we incorporate a pre-training
process before the formal training of the model. During pre-training, we use
the input data at time $t$ and to predict the electromagnetic field at time
$t+dt$. Additionally, we reduce the number of pre-training epochs. This
approach has the advantage of using less time to obtain a set of initial
weights for the entire network, thereby accelerating the training process of
the entire task.

During the formal training process, to reduce the impact of accumulated errors
and increase the network’s performance, we combine $\rho$, $\mathbf{J}$,
$\mathbf{E}$ and $\mathbf{B}$ at every ten time steps ($[t_{n-9},...,t_{n}]$)
as the input to the model to predict the electromagnetic fields at $t_{n+1}$.
The predicted electromagnetic fields at time step $t_{n+1}$ is included as the
input ($[t_{n-8},...,t_{n+1}]$) to predict the fields at time $t_{n+2}$, etc.
This procedure increased the training dataset into ten times larger. While
this training setup may increase the learning time of the model, it enhances
the model’s acceptance of errors, effectively improving its generalization
ability. Furthermore, the updating process with standard data ensures that the
model does not face the issue of being unable to fit the data.

We have selected the $\text{Smooth}L_{1}\text{Loss}$ function as our loss
function,

$\begin{array}[]{cc}\begin{matrix}\text{Smooth}L_{1}\text{Loss}(f_{\text{output}},f_{\text{label}})\\\
=\frac{1}{n}{\textstyle\sum_{i=1}^{n}}\left\\{\begin{matrix}0.5(f_{\text{output}}-f_{\text{label}})^{2},&\text{if
}\begin{vmatrix}f_{\text{output}}-f_{\text{label}}\end{vmatrix}<1\\\
\begin{vmatrix}f_{\text{output}}-f_{\text{label}}\end{vmatrix}-0.5,&\text{otherwise}\end{matrix}\right.\end{matrix}\end{array}$
(8)

which ensures that the gradient values do not become too large when there is a
large discrepancy between the output values and the standard values. At the
same time, it guarantees that the gradient values are sufficiently small when
the output values are close to the standard values. Throughout the learning
process, we utilize Adaptive Moment Estimation (Adam) to minimize the loss
function and update the parameters of the entire network. Through the design
of the network and optimization of the training process, our model is capable
of adapting well to complex high-dimensional partial differential equations.
Due to the strong versatility of our model design, we can naturally extend the
model to solve different types of partial differential equations by simply
adjusting the parameters of the input and output.

##  4 Experiment

In this section, we train and test the JefiAtten model on the established
dataset. Our investigation focuses on assessing the model’s computational
precision, training duration, generalization capacity, and predictive
efficacy.

###  4.1 Model performance for scenarios with different $\rho$, $\mathbf{J}$
spatial distribution

We first apply the model to the dataset obtained using JefiGPU, as described
in Sec. 3.1. During the training process, we use ten consecutive time steps
($[t_{n-9},...,t_{n}]$) as input, where for each time step the input consists
of $\rho$ and $\mathbf{J}$, as well as $\mathbf{E}$ and $\mathbf{B}$ of the
previous time step, e.g., [$\rho_{n}$, $\mathbf{J}_{n}$, $\mathbf{E}_{n-1}$,
$\mathbf{B}_{n-1}$]. The model is capable of predicting the next moment’s
$\mathbf{E}_{n}$ and $\mathbf{B}_{n}$ (or, if desired, predicting $\mathbf{E}$
and $\mathbf{B}$ for multiple future moments).

Here, we randomly choose 10 out of 500 time series for training, and utilize
another randomly chosen 400 series for testing. In Sec. 3.1, we demonstrate
that the data prepared possess varying spatial distribution, with substantial
differences between them. Our purpose is to see if the network can correctly
predict the electromagnetic fields of the testing samples with a few training
samples, especially when these training and testing samples are very different
in terms of spatial distribution. The result is depicted in Fig. 5.

Figure 5: Relative error ($\text{Mean}(\text{(Predicted-Ground-
Truth)}^{2})/\text{Mean}(\text{(Ground-Truth)}^{2})$) between the predicted
data and the ground-truth. The blue represents a group of 400 data points,
while the red represents their mean.

In Fig. 5, we see that the maximum relative error of the 400 testing samples
is about 0.06, and the average value stands at about 0.0234. With 10 training
series, JefiAtten is able to grasp the intricate relationships between the
input data $\rho$, $\mathbf{J}$, $\mathbf{E}$, $\mathbf{B}$. Meanwhile,
JefiAtten boasts a faster computational speed compared to traditional
algorithms once it has been trained. The training duration for JefiAtten
stands at 3.34 hours (10 time series with each series having 1920 time steps;
the total epoch number is taken to be 100). In the testing phase, for 2000
time steps on the same configuration, the traditional code JefiGPU requires
0.0542 hours on one NVIDIA A100 card, while JefiAtten completes the task in
merely 0.0038 hours on the same device. The result indicates that JefiAtten is
applicable in practical scenarios where, despite fixed boundary conditions and
grid sizes, variations in current density and charge density occur. In
traditional algorithms one would necessitate re-computation to adapt to these
changes. In contrast, JefiAtten utilizes its generalization capability to
solve Jefimenko’s equations, showcasing an advantage in terms of efficiency
and utility.

To better illustrate the generalization ability, we visualize some of the
predicted electromagnetic fields with corresponding errors in different
directions. These results are showcased in Fig. 6 and Fig. 7. From Fig. 6 and
Fig. 7, we see that the network predictions capture the main structures of the
ground-truth. Given that the inputs $\rho$ and $\mathbf{J}$ are composed using
trigonometric functions as basis functions, the predicted electromagnetic
fields may exhibit certain periodicity, which is also evident in Fig. 6.

Figure 6: The predicted electric field and the absolute errors between the
predictions and the ground-truth (at respective time steps of 3.75, 6.25,
8.75, 11.25, 13.75, 16.25 $\text{GeV}^{-1}$). (a) $E_{x}$ in the YOZ plane;
(b) $E_{y}$ in the YOZ plane; (c) $E_{z}$ in the YOZ plane.  Figure 7: The
predicted magnetic field and the absolute errors between the predictions and
the ground-truth (at respective time steps of 3.75, 6.25, 8.75, 11.25, 13.75,
16.25 $\text{GeV}^{-1}$). (a) $B_{x}$ in the YOZ plane; (b) $B_{y}$ in the YOZ
plane; (c) $B_{z}$ in the YOZ plane.

Upon examining samples of the test data, we confirm that the model accurately
fits the entire electromagnetic field without any partial fitting issues. For
simplicity in presentation, we will focus on displaying only the $E_{x}$
component along the x-axis in upcoming results.

###  4.2 Model performance for scenarios with different amplitudes

In Sec. 4.1, we validated the model’s computational accuracy for the
Jefimenko’s equations and its generalization to triangular functions with
varying spatial distribution but consistent amplitude. Building upon the
training discussed in Sec. 4.1, this section focuses on fine-tuning the
network using a single time series with triangular amplitudes that are
significantly larger. We then proceed to evaluate the model’s ability to
generalize to changes in amplitudes. The purpose of this section is to see
network’s capability when the trained network encounters scenarios that are
very different from the training data, and how much new data is required for
fine-tuning.

We begin by randomly selecting three time series from a batch of 500,
amplifying their amplitudes to ten times their original levels. We designated
one series for training and the other two for testing. Continued training of
the model, leveraging the weights obtained in Sec. 4.1, serve to decrease the
amount of time needed for fine-tuning. The results of the testing phase are
detailed in Fig. 8

Figure 8: The predicted electric field and the absolute errors between the
predictions and the ground-truth.(at respective time steps of 3.275, 3.435,
3.595, 3.755, 3.915, 4.075 $\text{GeV}^{-1}$)

The inputs for this set of results, $\rho$ and $\mathbf{J}$, are as follows:

$\begin{array}[]{cc}\rho(x,y,z,t)=\left\\{{\begin{array}[]{*{20}{c}}{\rm{0}}&{{\rm{if\quad
t<0}}}\\\ {10(\text{cos}(t)(\text{cos}(y)\text{cos}(x+10z)-\text{sin}(10y)}\\\
{\text{sin}(x+z))+\text{sin}(t)\text{sin}(y)\text{sin}(10x+z))}&{{\rm{if\quad
t}}\geq{\rm{0}}}\end{array}}\right.\\\
\mathbf{J}(x,y,z,t)=\left\\{{\begin{array}[]{*{20}{c}}{\rm{0}}&{{\rm{if\quad
t<0}}}\\\ {({j_{x}},{j_{y}},{j_{z}})}&{{\rm{if\quad t}}\geq{\rm{0}}}\\\
\end{array}}\right.\end{array}$ (9)

where $j_{x}=\text{sin}(y)\text{cos}(10x+z)\text{cos}(t);\quad
j_{y}=\text{cos}(10y)\text{sin}(x+z)\text{sin}(t);\quad
j_{z}=\text{cos}(y)\text{sin}(x+10z)\text{sin}(t);$ Compared to the results
presented in Sec. 4.1, this set of outputs exhibits a noticeable increase in
complexity as well as larger amplitudes. However, as indicated by Fig. 8, the
model has captured the features compared to the ground-truth, with a maximum
absolute error of $3\times 10^{-4}$. These findings demonstrate the capability
of JefiAtten to rapidly adapt to inputs of very different scenarios (varying
amplitudes) based on its pre-training, and to accommodate more complex input
conditions. This expands the potential applications of JefiAtten.

We continue to probe the limits of JefiAtten’s generalization capabilities. As
previously ascertained, JefiAtten has demonstrated generalization for
amplitudes, and spatial distribution more or less present in the training set.
Next, we test its performance with amplitudes that have never appeared during
the training process. From a pool of 500 datasets, we select five sets with
similar spatial distribution, and augment their amplitudes by factors of 10,
13, 15, 18, and 30, respectively (denoted as set 1-5). We use sets 1 and 3, as
well as 1 and 5 for training, and sets 2 and 4 for testing, the obtained
results are illustrated in Fig. 9

Figure 9: The predicted electric field and the absolute errors between the
predictions and the ground-truth (at respective time points of 3.275, 3.435,
3.595, 3.755, 3.915, 4.075 $\text{GeV}^{-1}$). (a) $E_{x}$ in the YOZ plane.
Set 1 and 3 used as fine-tuning, and set 2 used as testing (i.e., amplitude
for training is 10 and 15, and for testing is 13); (b)$E_{x}$ in the YOZ
plane. Set 1 and 5 used as fine-tuning, and set 4 used as testing (i.e.,
amplitude for training is 10 and 18, and for testing is 15).

The analysis of the figure suggests that the difference between the
predictions of set 4 and the ground-truth values is more pronounced than in
set 2, which has a test error of 0.0147, while set 4 has a test error of
0.0067. Despite this, both sets of predictions confirm JefiAtten’s
generalization ability. This implies a direct relationship between the
accuracy of the model’s outputs and the amplitude discrepancy between the
training and test datasets. Meanwhile, we have the flexibility to balance
computational speed with precision, tailored to the specific needs. For a
quicker training cycle, one could choose to train the model within a
predefined amplitude range; conversely, to achieve higher accuracy in
computations, training with datasets that have amplitudes similar to those of
the test set would be advantageous. These findings further demonstrate
JefiAtten’s ability to quickly adapt to inputs of varying amplitudes,
utilizing its pre-trained foundation.

###  4.3 Model performance for slowly varying scenarios

During the experimental process described above, we assessed the model’s
practicality, generalization ability, and computational speed from various
perspectives. Ultimately, we aim to gain a deeper understanding of the model’s
predictive capabilities. Given the periodic nature of trigonometric functions,
the model inevitably accumulates errors when making predictions on complex
data, and this cumulative error can impact its predictive performance.
However, when the input data has a simpler structure, the model tends to incur
smaller errors during consecutive prediction (i.e., using the network to
predict the subsequent electromagnetic fields based on its own predictions).
Under these circumstances, the model should be capable of making predictions
over a longer time-frame. Therefore, we set the input parameters $\rho$ and
$\mathbf{J}$ as follows:

$\begin{array}[]{cc}\begin{matrix}\rho(x,y,z,t)=\left\\{\begin{matrix}0&\text{if}\quad
t<0\\\ 2(x+y+z)e^{-t}&\text{if}\quad t\geq 0\end{matrix}\right.\\\
\mathbf{J}(x,y,z,t)=\left\\{\begin{matrix}0&\text{if}\quad t<0\\\
(j_{x},j_{y},j_{z})&\text{if}\quad t\geq
0\end{matrix}\right.\end{matrix}\end{array}$ (10)

where $j_{x}=(x^{2}+yz)e^{-t},j_{y}=(y^{2}+xz)e^{-t},j_{z}=(z^{2}+yx)e^{-t}.$
Based on the training conducted in Sec. 4.1, we test this dataset, with the
results displayed in Fig. 10.

Figure 10:  The loss between the model’s predicted value and the ground-truth
value. The model predicts the system from the red dot, and the error is
calculated using the SmoothL1Loss method, details of which can be found in Eq.
7

As depicted in Fig. 10, for simulated data with a time step of $dt=0.005\text{
GeV}^{-1}$ , JefiAtten takes over the traditional equation solving process
from 4s and predicts until 9.5 $\text{GeV}^{-1}$, covering a total of 1100
time steps. The maximum error generated by the predictions is 0.0043. This
outcome further showcases the predictive ability of JefiAtten when dealing
with simple functions.

###  4.4 Ablation Experiments

The JefiAtten model, as depicted in Fig. 4, did not originate in its current
form. The iterative concept behind its design can be referred to in Appendix.
In order to further investigate the rationality of the JefiAtten model’s
structure, we conducted ablation experiments targeting specific components
within the model. The results of these experiments are presented in Tab. 1,
shedding light on the significance of each component in the overall
performance of the model.

Table 1: Experimental results for the necessity of different model blocks.  Network architecture | Deviation
---|---
No inputattention | 0.2873
No FFN | 0.4050
Add Layernorm | 0.4165
No change | 0.0025

Among the ablation experiments conducted, "No inputattention" indicates that
the model eliminates the attention module for capturing temporal and
positional information from the input. "No FFN" denotes the removal of the
Feed-Forward Network (FFN) in each module. "Add Layernorm" signifies the
addition of Layer Normalization in each module. "No change" represents the
JefiAtten model without any modifications. The results demonstrate that the
attention module for temporal and positional information effectively captures
the relevant information. Compared to position masking, this approach offers a
broader application range, enabling rapid acquisition of the temporal and
positional associations within complex input data. As described in Sec. 3.3,
the FFN component provides an effective means of expressing non-linearity, as
confirmed by the results of the ablation experiments. On the other hand, the
inclusion of Layer Normalization in each module can lead to a loss of
information and an increase in the error of the final output results.

##  5 Conclusion

In this paper, we introduced JefiAtten, a novel neural network model designed
to solve Maxwell’s equations through an attention-based mechanism. JefiAtten
leverages the power of self-attention and cross-attention modules to capture
intricate relationships within and between the data inputs, ensuring a
comprehensive understanding of the physical, spatial, and temporal information
embedded in the electromagnetic data. Our model’s architecture adeptly
incorporates fully connected networks for initial data preprocessing, followed
by a series of attention blocks for detailed feature extraction. This design
choice is validated by our ablation studies, which highlight the necessity of
each component in achieving low prediction errors.

JefiAtten has shown improved performance in both pre-training and formal
training phases, outpacing classical computational methods in terms of speed
once adequately trained. The model’s generalization capabilities are evidenced
by its ability to adapt to new scenarios with varying spatial distribution,
amplitudes, and slowly varying conditions with limited additional training.

Our findings open up pathways for future research, where JefiAtten can be
further enhanced to tackle even more complex simulations, potentially serving
as an encouragement for further exploration into the fusion of machine
learning with scientific computing.

##  6 Acknowledgement

The work is supported by the National Natural Science Foundation of China
(NSFC) under the grant number 12105227 and the National Key Research and
Development Program of China under Grant No. 2020YFA0709800.

## Appendix

It is intriguing to consider why we chose to utilize multiple stacked
attention mechanisms, as depicted in Fig. 4, in our network architecture. From
Eq. 1, Eq. 2 (Jefimenko’s equation), it can be observed that when computing
the electromagnetic field, the high temporal requirements imposed by the
presence of a time delay $t_{r}$ necessitate the model’s ability to
effectively capture the temporal relationships between different data points.
This observation emerged during the initial stages of our model exploration.
Initially, we trained the model using only the previous time-step’s
electromagnetic field, current density, and charge density as input data.
However, after training, we noticed that while the model could accurately
predict the electromagnetic field output for shorter time intervals, it
performed poorly for longer time intervals, as illustrated in Fig. 11.

Figure 11:  The error between the model’s predicted value and the Ground-
Truth.

Upon analyzing the reasons behind this phenomenon, we discovered that the
model exhibited a heavy reliance on the input results from the previous time-
step. During the prediction process, the model accumulated significant errors,
leading to substantial deviations in the data. This is precisely why we
combined multiple time series inputs and employed attention mechanisms to
explore the inter-dependencies between the data. Moreover, due to the high
complexity of the input data
($E_{x},E_{y},E_{z},B_{x},B_{y},B_{z},rho,J_{x},J_{y},J_{z},t$), integrating
the positional information with the input data posed considerable challenges.
Consequently, we incorporated the positional information as an input and
utilized attention to identify the correlations between the positional
information and the input data. This is also why our model adopted a structure
involving multiple stacked attention mechanisms, as depicted in Fig. 4.

## Declaration of Generative AI and AI-assisted technologies in the writing
process

During the preparation of this work the authors used GPT-4 in order to improve
readability and language. After using this tool, the authors reviewed and
edited the content as needed and take full responsibility for the content of
the publication.

## References

  * [1] D. R. Nicholson, D. R. Nicholson, Introduction to plasma theory, Vol. 1, Wiley New York, 1983.
  * [2] J. D. Bukowski, D. B. Graves, P. Vitello, Two-dimensional fluid model of an inductively coupled plasma with comparison to experimental spatial profiles, Journal of Applied Physics 80 (5) (1996) 2614–2623.  doi:10.1063/1.363169.
  * [3] R. A. Stewart, P. Vitello, D. B. Graves, Two-dimensional fluid model of high density inductively coupled plasma sources, Journal of Vacuum Science and Technology B: Microelectronics and Nanometer Structures Processing, Measurement, and Phenomena 12 (1) (1994) 478–485.  doi:10.1116/1.587102.
  * [4] T. E. Sheridan, J. Goree, Collisional plasma sheath model, Physics of Fluids B: Plasma Physics 3 (10) (1991) 2796–2804.  doi:10.1063/1.859987.
  * [5] V. Vahedi, M. A. Lieberman, M. V. Alves, J. P. Verboncoeur, C. K. Birdsall, A one-dimensional collisional model for plasma-immersion ion implantation, Journal of Applied Physics 69 (4) (1991) 2008–2014.  doi:10.1063/1.348774.
  * [6] S. Gomez, J. H. Hoyos, J. A. Valdivia, Particle-in-cell method for plasmas in the one-dimensional electrostatic limit, American Journal of Physics 91 (3) (2023) 225–234.  doi:10.1119/5.0135515.
  * [7] B. J. Muga, J. F. Wilson, Particle-in-Cell Method, Springer US, Boston, MA, 1970, pp. 105–116.
  * [8] J. Chen, J. Wang, Y. Tao, Z. Chen, Y. Wang, S. Niu, Simulation of sgemp using particle-in-cell method based on conformal technique, IEEE Transactions on Nuclear Science 66 (5) (2019) 820–826.  doi:10.1109/TNS.2019.2911933.
  * [9] J. Chen, J. Wang, Z. Chen, Z. Ren, H. Qiao, Study of sgemp field-coupling inside and outside reentrant cavity, IEEE Transactions on Electromagnetic Compatibility 64 (4) (2022) 1182–1189.  doi:10.1109/TEMC.2022.3153625.
  * [10] S. A. Berger, An Introduction to Magnetohydrodynamics , Physics Today 55 (1) (2002) 56–57.  doi:10.1063/1.1457269.
  * [11] F. Haas, A magnetohydrodynamic model for quantum plasmas, Physics of Plasmas 12 (6) (2005) 062117.
  * [12] C. Johnson, Numerical solution of partial differential equations by the finite element method, Courier Corporation, 2012.
  * [13] K. Yee, Numerical solution of initial boundary value problems involving maxwell’s equations in isotropic media, IEEE Transactions on Antennas and Propagation 14 (3) (1966) 302–307.  doi:10.1109/tap.1966.1138693.
  * [14] C. Warren, A. Giannopoulos, A. Gray, I. Giannakis, A. Patterson, L. Wetter, A. Hamrah, A cuda-based gpu engine for gprmax: open source fdtd electromagnetic simulation software, Computer Physics Communications 237 (2019) 208–218.  doi:10.1016/j.cpc.2018.11.007.
  * [15] A. F. Oskooi, D. Roundy, M. Ibanescu, P. Bermel, J. Joannopoulos, S. G. Johnson, Meep: A flexible free-software package for electromagnetic simulations by the fdtd method, Computer Physics Communications 181 (3) (2010) 687–702.  doi:10.1016/j.cpc.2009.11.008.
  * [16] H. Wang, T. Fu, Y. Du, W. Gao, K. Huang, Z. Liu, P. Chandak, S. Liu, P. Van Katwyk, A. Deac, et al., Scientific discovery in the age of artificial intelligence, Nature 620 (7972) (2023) 47–60.
  * [17] L. Kuhn, T. Repän, C. Rockstuhl, Exploiting graph neural networks to perform finite-difference time-domain based optical simulations, APL Photonics 8 (3) (Mar 2023).  doi:10.1063/5.0139004.
  * [18] V. Puzyrev, Deep learning electromagnetic inversion with convolutional neural networks, Geophysical Journal International 218 (2) (2019) 817–832.
  * [19] K. Bi, L. Xie, H. Zhang, X. Chen, X. Gu, Q. Tian, Accurate medium-range global weather forecasting with 3d neural networks, Nature 619 (2023) 533 – 538.
  * [20] M. Raissi, P. Perdikaris, G. Karniadakis, Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, Journal of Computational Physics (2019) 686–707.
  * [21] Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, A. Anandkumar, Fourier neural operator for parametric partial differential equations, arXiv: Learning,arXiv: Learning (Oct 2020).
  * [22] S. Wang, H. Wang, P. Perdikaris, Learning the solution operator of parametric partial differential equations with physics-informed deeponets, Science Advances 7 (40) (2021) eabi8605.
  * [23] M. Chen, R. Lupoiu, C. Mao, D.-H. Huang, J. Jiang, P. Lalanne, J. Fan, Physics-augmented deep learning for high-speed electromagnetic simulation and optimization (Aug 2021).  doi:10.21203/rs.3.rs-807786/v1.
  * [24] R. Zhao, Z. Yang, H. Zheng, Y. Wu, F. Liu, Z. Wu, L. Li, F. Chen, S. Q. Song, J. Zhu, W. Zhang, H. Huang, M. Xu, K. Sheng, Q. Yin, J. Pei, G. Li, Y. Zhang, M. Zhao, L. Shi, A framework for the general design and computation of hybrid neural networks, Nature Communications 13 (2022).
  * [25] I. V. Tetko, P. Karpov, R. Van Deursen, G. Godin, State-of-the-art augmented nlp transformer models for direct and single-step retrosynthesis, Nature Communications (Nov 2020).  doi:10.1038/s41467-020-19266-y.
  * [26] Transformers: State-of-the-art natural language processing, Computer Methods in Applied Mechanics and Engineering Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations (2020) 38–45.
  * [27] K. Han, Y. Wang, H. Chen, X. Chen, J. Guo, Z. Liu, Y. Tang, A. Xiao, C. Xu, Y. Xu, Z. Yang, Y. Zhang, D. Tao, A survey on vision transformer, IEEE Transactions on Pattern Analysis and Machine Intelligence 45 (1) (2023) 87–110.
  * [28] Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, B. Guo, Swin transformer: Hierarchical vision transformer using shifted windows, in: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 10012–10022.
  * [29] Z. Li, K. Meidani, A. B. Farimani, Transformer for partial differential equations’ operator learning, Trans. Mach. Learn. Res. 2023 (2022).
  * [30] Z. Hao, Z. Wang, H. Su, C. Ying, Y. Dong, S. Liu, Z. Cheng, J. Song, J. Zhu, GNOT: A general neural operator transformer for operator learning, in: A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato, J. Scarlett (Eds.), Proceedings of the 40th International Conference on Machine Learning, Vol. 202 of Proceedings of Machine Learning Research, PMLR, 2023, pp. 12556–12569.
  * [31] X. Liu, B. Xu, L. Zhang, Mitigating spectral bias for the multiscale operator learning with hierarchical attention (2023).  arXiv:2210.10890.
  * [32] O. Jefimenko, Electricity and Magnetism: An Introduction to the Theory of Electric and Magnetic Fields, New York : Appleton-Century-Crofts, 1995.
  * [33] J.-J. Zhang, J.-N. Chen, G.-L. Peng, T.-J. Du, H.-Y. Xie, Jefigpu: Jefimenko’s equations on gpu, Computer Physics Communications 276 (2022) 108328.  doi:https://doi.org/10.1016/j.cpc.2022.108328.
  * [34] M. Y. Sun, P. Xu, J. J. Zhang, Q. Wang, T. J. Du, J. G. Wang, Rbg-maxwell framework: Simulation of collisional plasma systems via coupled boltzmann-maxwell equations on gpu, arXiv preprint arXiv:2308.04869 (2023).
  * [35] J. K. Tsotsos, S. M. Culhane, W. Y. Kei Wai, Y. Lai, N. Davis, F. Nuflo, Modeling visual attention via selective tuning, Artificial Intelligence (1995) 507–545doi:10.1016/0004-3702(95)00025-9.
  * [36] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. Gomez, L. Kaiser, I. Polosukhin, Attention is all you need, Neural Information Processing Systems,Neural Information Processing Systems (Jun 2017).
  * [37] S. Wolfram, Mathematica: a system for doing mathematics by computer (2nd ed.), Addison-Wesley Longman Publishing Co., Inc. eBooks,Addison-Wesley Longman Publishing Co., Inc. eBooks (Apr 1991).
  * [38] M. A. L., Natural system of units in general relativity (2016).
  * [39] J. Ba, J. Kiros, G. Hinton, Layer normalization, arXiv: Machine Learning,arXiv: Machine Learning (Jul 2016).
  * [40] J. Xu, X. Sun, Z. Zhang, G. Zhao, J. Lin, Understanding and Improving Layer Normalization, Curran Associates Inc., Red Hook, NY, USA, 2019.
  * [41] M. Geva, R. Schuster, J. Berant, O. Levy, Transformer feed-forward layers are key-value memories, in: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021.


# Analytic derivation of the inertial range of compressible turbulence

I. Goldman<EMAIL_ADDRESS>Physics Department Afeka Engineering College,
Tel Aviv, Israel  Astrophysics Department, Tel Aviv University, Tel Aviv,
Israel

###### Abstract

An analytic model for steady state turbulence is employed to obtain the
inertial range power spectrum of compressible turbulence. We assume that for
homogeneous turbulence, the timescales controlling the energy injected at a
given wavenumber from all smaller wave-numbers, are equal for each spatial
component. However, the longitudinal component energy is diverted into
compression, so the rate controlling the energy that is transferred to all
larger wave-numbers by the turbulent viscosity is reduced. The resulting
inertial range is a power law with index -2. Indeed such power spectra were
observed in various astrophysical settings and also in numerical simulations.

Compressible supersonic turbulence characterized by a 1D power-law index of -2
has been observed in molecular clouds Larson (1981); Leung, Kutner, and Mead
(1982); Dame  _et al._ (1986), in interstellar medium of galaxies Roy and
Joncas (1985); Green (1993); Stanimirovic  _et al._ (1999); Goldman (2000), in
a shocked nebula near the Galactic center Contini and Goldman (2011). It has
also been obtained in numerical simulations Passot, Pouquet, and Woodward
(1988); Vázquez-Semadeni, Ballesteros-Paredes, and Rodríguez (1997). Recently,
it was found Besserglik and Goldman (2021) that the intensity of $\gamma$-Ray
emission from the Large Magellanic Cloud exhibits such a power spectrum.

In this Letter, we employ a simple analytic model for steady turbulence
Canuto, Goldman, and Chasnov (1987) to find the power spectrum of the inertial
range of supersonic compressible turbulence. We obtain an inertial range of
the 1D power spectrum which is proportional to $k^{-2}$. Details follow.

The model Canuto, Goldman, and Chasnov (1987) is an extension of a previous
work Canuto and Goldman (1985). It describes homogeneous isotropic steady
state turbulence. Its basic equations are

$\displaystyle n_{s}(k)+\frac{y(k)}{n_{c}(k)}=\nu_{t}(k)k^{2}$ (1)
$\displaystyle y(k)=\int_{k_{0}}^{k}F(k)k^{2}dk$ (2)
$\displaystyle\nu_{t}(k)=\int_{k}^{\infty}\frac{F(K)}{n_{c}(k)}$ (3)
$\displaystyle\nu_{t}(k)k^{2}=\gamma n_{c}(k)$ (4)

with $F(k)=4\pi k^{2}\Phi(k)$ where $k=|\vec{k}|$, and $\Phi(\vec{k})$ is the
3D power spectrum, of the turbulent velocity, $y(k)$ is the k-space mean
square vorticity at wavenumber $k$, $n_{s}(k)$ is the rate controlling the
energy input from the source at $k$, $n_{c}(k)$ is the inverse of the eddy
correlation timescale and $\nu_{t}(k)$ is the turbulent kinematic viscosity at
wavenumber $k$ exerted by all the eddies with larger wavenumbers. Eq.(4)
relates the turbulent viscosity and the eddy correlation rate , with $\gamma$
being a dimensionless constant. The model has been quite successful in
obtaining turbulence spectra for both the large scale spatial scales as well
as the inertial range. An extension of the model Canuto, Goldman, and
Mazzitelli (1996)dealing with stellar turbulent convection was very successful
and has been widely cited.

In the large-eddy range $n_{s}(k)$ is positive and dominates over
$\frac{y(k)}{nc(k)}$. In the dissipation range (small spatial scales)
$n_{s}(k)=-\nu k^{2}$, with $\nu$denoting the microscopic kinematic viscosity.
Depending on $n_{s}(k)$ and $\nu$, there usually exists a mid wavenumber
range– the inertial range, in which both energy input from the source and
energy dissipation are very small compared to the rate of energy transfer.
Thus, in this range

$y(k)=\gamma n_{c}(k)^{2}$ (5)

from which

$y^{\prime}(k)=2\gamma n_{c}(k)n_{c}^{\prime}(k)$ (6)

From Eq.(2) Eq.(3) and Eq.(4) follows that

$\displaystyle y^{\prime}(k)=-k^{2}\nu_{t}^{\prime}(k)=-\gamma
k^{2}\left(n_{c}(k)k^{-2}\right)^{\prime}$ (7)

Eqs.(6) and (7) yield

$n_{c}(k)=Ak^{2/3}$ (8)

with $A$ a constant. Since $F(k)=y^{\prime}(k)k^{-2}$ one gets

$F(k)=\frac{4}{3}\gamma A^{2}k^{-5/3}$ (9)

Thus, the model yields for the inertial range a power spectrum of the
Kolmogorov Kolmogorov (1941) form . This is not surprising as in this model
all the energy injected from smaller wave-numbers is transferred by the
turbulent viscosity to the larger wave-numbers.

However, in compressible turbulence the situation is different: not all the
energy available from the wave-numbers smaller than $k$ is cascaded to smaller
scales, since energy is also being diverted into compression. The longitudinal
component of the turbulence is the one involved in the energy diversion. For
an homogeneous turbulence it is conceivable that the timescales controlling
the input from the wave-numbers smaller than $k$ are equal. Thus, if all its
energy is diverted to compression one gets instead of Eq.(1)

$n_{s}(k)+\frac{2}{3}\frac{y(k)}{n_{c}(k)}=\nu_{t}(k)k^{2}=\gamma n_{c}(k)$
(10)

The assumption that all the energy of the longitudinal component is diverted
into compression, seems justified for supesonic turbulence, which is
ubiquitous in astrophysical turbulence. From Eq.(10) follows the rate equation
the inertial range:

$y(k)=\frac{3}{2}\gamma n_{c}(k)^{2}$ (11)

Eq.(7) is unchanged while differentiation of Eq.(11) yields

$y^{\prime}(k)=3\gamma n_{c}(k)n_{c}^{\prime}(k)$ (12)

Thus, the solution is now

$n_{c}(k)=Bk^{1/2}$ (13)

with $B$ a constant, leading to

$F(k)=\frac{3}{2}\gamma B^{2}k^{-2}$ (14)

Which is the expected form of the inertial range of compressible turbulence.
The present derivation suggests that for sonic turbulence not all of the
longitudinal energy will be diverted. In this case, one may expect that the
inertial range logarithmic slope will be intermediate between $-5/3$ and $-2$.

##  I data availability

This is a theoretical paper. It includes no data.

###### Acknowledgements.

I thank of the Research Authority of Afeka College.

## references

## References

  * Larson (1981) R. B. Larson, “Turbulence and star formation in molecular clouds.” Monthly Notices of the Royal Astronomical Society 194, 809–826 (1981).
  * Leung, Kutner, and Mead (1982) C. M. Leung, M. L. Kutner, and K. N. Mead, “On the origin and structure of isolated dark globules.” Astrophys. J. 262, 583–589 (1982).
  * Dame  _et al._ (1986) T. M. Dame, B. G. Elmegreen, R. S. Cohen, and P. Thaddeus, “The Largest Molecular Cloud Complexes in the First Galactic Quadrant,” Astrophys. J. 305, 892 (1986).
  * Roy and Joncas (1985) J. R. Roy and G. Joncas, “Structure and origin of velocity fluctuations in the HII region Sharpless 142.” Astrophys. J. 288, 142–147 (1985).
  * Green (1993) D. A. Green, “A power spectrum analysis of the angular scale of Galactic neutral hydrogen emission towards L = 140 deg, B = 0 deg,”  (1993).
  * Stanimirovic  _et al._ (1999) S. Stanimirovic, L. Staveley-Smith, J. M. Dickey, R. J. Sault, and S. L. Snowden, “The large-scale HI structure of the Small Magellanic Cloud,” Monthly Notices of the Royal Astronomical Society 302, 417–436 (1999).
  * Goldman (2000) I. Goldman, “Interpretation of the Spatial Power Spectra of Neutral Hydrogen in the Galaxy and in the Small Magellanic Cloud,” Astrophys. J. 541, 701–706 (2000), arXiv:astro-ph/0005023 [astro-ph] .
  * Contini and Goldman (2011) M. Contini and I. Goldman, “Spectra from the shocked nebulae revealing turbulence near the Galactic Centre,” Monthly Notices of the Royal Astronomical Society 411, 792–806 (2011).
  * Passot, Pouquet, and Woodward (1988) T. Passot, A. Pouquet, and P. Woodward, “The plausibility of Kolmogorov-type spectra in molecular clouds,” Astronomy & Astrophysics 197, 228–234 (1988).
  * Vázquez-Semadeni, Ballesteros-Paredes, and Rodríguez (1997) E. Vázquez-Semadeni, J. Ballesteros-Paredes, and L. F. Rodríguez, “A Search for Larson-type Relations in Numerical Simulations of the ISM: Evidence for Nonconstant Column Densities,” Astrophys. J. 474, 292–307 (1997), arXiv:astro-ph/9607175 [astro-ph] .
  * Besserglik and Goldman (2021) D. Besserglik and I. Goldman, “The power spectrum and structure function of the Gamma Ray emission from the Large Magellanic Cloud,” arXiv e-prints , arXiv:2103.09531 (2021), arXiv:2103.09531 [astro-ph.GA] .
  * Canuto, Goldman, and Chasnov (1987) V. M. Canuto, I. Goldman, and J. Chasnov, “A model for fully developed turbulence,” Physics of Fluids 30, 3391–3418 (1987).
  * Canuto and Goldman (1985) V. M. Canuto and I. Goldman, “Analytical model for large-scale turbulence,” PhysRevLett 54, 430–433 (1985).
  * Canuto, Goldman, and Mazzitelli (1996) V. M. Canuto, I. Goldman, and I. Mazzitelli, “Stellar Turbulent Convection: A Self-consistent Model,” Astrophys. J. 473, 550 (1996), arXiv:astro-ph/9609001 [astro-ph] .
  * Kolmogorov (1941) A. Kolmogorov, “The Local Structure of Turbulence in Incompressible Viscous Fluid for Very Large Reynolds’ Numbers,” Akademiia Nauk SSSR Doklady 30, 301–305 (1941).