details appear in Appendix J.

##  Appendix J Detailed Empirical Results

We list the individual accuracies of the teacher and student (along with the
student’s improvement) corresponding to the results of Table 1 in Tables 8-13,
Table 5 in Tables 14-15, Table 2 in Tables 16-21, Table 6 in Tables 22-23 and
Table 7 in Tables 24-25.

Experimental Details: In all the cases, we use SGD with momentum = 0.9 and
batch size = 128 for training. Since we are training only the softmax layer
(i.e., doing logistic regression), we use an exponentially decaying learning
rate scheme with decay parameter = 0.98 (for every epoch) and the initial
learning rate is tuned161616The tuning is done by picking the learning rate
which yields the lowest training loss with the observed (noisy) labels. This
is consistent with our theory setup where we assume convergence to the optimum
of the training loss w.r.t. the observed labels. over
$\\{0.001,0.005,0.01,0.05,0.1,0.5\\}$. The maximum number of epochs is 200.

$\xi$ | Student’s test acc. |  Improvement of student over teacher
---|---|---
0.0 (=Teacher) |  $57.61\pm 0.03$ % |  $0$ %
0.2 |  $59.83\pm 0.12$ % |  $2.22\pm 0.12$ %
0.5 |  $62.79\pm 0.04$ % |  $5.18\pm 0.03$ %
0.7 |  $64.45\pm 0.09$ % |  $6.84\pm 0.06$ %
1.0 |  $66.15\pm 0.27$ % |  $8.54\pm 0.29$ %
1.2 |  ${67.27\pm 0.25}$ % |  ${9.66\pm 0.23}$ %
1.5 |  $\bm{67.65\pm 0.54}$ % |  $\bm{10.04\pm 0.51}$ %
1.7 |  $\bm{67.42\pm 0.58}$ % |  $\bm{9.81\pm 0.55}$ %
2.0 |  $66.17\pm 0.77$ % |  $8.56\pm 0.73$ %

Table 8: Detailed Version of Table 1(a) (50% Random Corruption in Caltech-256
with ResNet-34)

$\xi$ | Student’s test acc. |  Improvement of student over teacher
---|---|---
0.0 (=Teacher) |  $61.15\pm 0.09$ % |  $0$ %
0.5 |  $62.04\pm 0.02$ % |  $0.89\pm 0.10$ %
1.0 |  $63.16\pm 0.06$ % |  $2.01\pm 0.14$ %
1.5 |  $64.28\pm 0.06$ % |  $3.13\pm 0.11$ %
2.0 |  $65.37\pm 0.12$ % |  $4.22\pm 0.20$ %
2.5 |  $66.43\pm 0.05$ % |  $5.28\pm 0.13$ %
3.0 |  $\bm{66.93\pm 0.03}$ % |  $\bm{5.78\pm 0.12}$ %
3.5 |  $\bm{67.01\pm 0.13}$ % |  $\bm{5.86\pm 0.18}$ %
4.0 |  $66.47\pm 0.25$ % |  $5.32\pm 0.33$ %

Table 9: Detailed Version of Table 1(b) (50% Random Corruption in Caltech-256
with VGG-16)

$\xi$ | Student’s test acc. |  Improvement of student over teacher
---|---|---
0.0 (=Teacher) |  $50.80\pm 0.04$ % |  $0$ %
0.2 |  $51.78\pm 0.14$ % |  $0.98\pm 0.12$ %
0.5 |  $53.26\pm 0.14$ % |  $2.46\pm 0.11$ %
0.7 |  $54.18\pm 0.03$ % |  $3.38\pm 0.02$ %
1.0 |  ${54.99\pm 0.08}$ % |  ${4.19\pm 0.09}$ %
1.2 |  $\bm{55.26\pm 0.18}$ % |  $\bm{4.46\pm 0.19}$ %
1.5 |  $\bm{55.26\pm 0.15}$ % |  $\bm{4.46\pm 0.17}$ %
1.7 |  $\bm{55.12\pm 0.16}$ % |  $\bm{4.32\pm 0.18}$ %
2.0 |  $54.32\pm 0.20$ % |  $3.52\pm 0.23$ %

Table 10: Detailed Version of Table 1(c) (50% Hierarchical Corruption in
CIFAR-100 with ResNet-34)

$\xi$ | Student’s test acc. |  Improvement of student over teacher
---|---|---
0.0 (=Teacher) |  $41.60\pm 0.08$ % |  $0$ %
0.2 |  $42.70\pm 0.03$ % |  $1.10\pm 0.09$ %
0.5 |  $44.29\pm 0.06$ % |  $2.69\pm 0.02$ %
0.7 |  $45.32\pm 0.05$ % |  $3.72\pm 0.05$ %
1.0 |  $46.89\pm 0.05$ % |  $5.29\pm 0.11$ %
1.2 |  $47.86\pm 0.06$ % |  $6.26\pm 0.09$ %
1.5 |  $\bm{48.80\pm 0.16}$ % |  $\bm{7.20\pm 0.14}$ %
1.7 |  $\bm{48.83\pm 0.18}$ % |  $\bm{7.23\pm 0.17}$ %
2.0 |  $48.02\pm 0.25$ % |  $6.42\pm 0.26$ %

Table 11: Detailed Version of Table 1(d) (50% Hierarchical Corruption in
CIFAR-100 with VGG-16)

$\xi$ | Student’s test acc. |  Improvement of student over teacher
---|---|---
0.0 (=Teacher) |  $48.93\pm 0.08$ % | 0 %
0.2 |  $49.06\pm 0.05$ % |  $0.13\pm 0.08$ %
0.5 |  $49.90\pm 0.04$ % |  $0.97\pm 0.04$ %
0.7 |  $50.38\pm 0.09$ % |  $1.45\pm 0.01$ %
1.0 |  $\bm{50.78\pm 0.07}$ % |  $\bm{1.85\pm 0.09}$ %
1.2 |  $\bm{50.80\pm 0.06}$ % |  $\bm{1.87\pm 0.06}$ %
1.5 |  $\bm{50.79\pm 0.03}$ % |  $\bm{1.86\pm 0.08}$ %
1.7 |  $\bm{50.73\pm 0.04}$ % |  $\bm{1.80\pm 0.05}$ %
2.0 |  $50.46\pm 0.09$ % |  $1.53\pm 0.02$ %

Table 12: Detailed Version of Table 1(e) (50% Adversarial Corruption in
Food-101 with ResNet-34)

$\xi$ | Student’s test acc. |  Improvement of student over teacher
---|---|---
0.0 (=Teacher) |  $37.01\pm 0.46$ % | 0 %
0.2 |  $37.80\pm 0.23$ % |  $0.79\pm 0.23$ %
0.5 |  $39.15\pm 0.37$ % |  $2.14\pm 0.09$ %
0.7 |  $39.97\pm 0.42$ % |  $2.96\pm 0.04$ %
1.0 |  $40.86\pm 0.51$ % |  $3.85\pm 0.05$ %
1.2 |  $\bm{41.23\pm 0.60}$ % |  $\bm{4.22\pm 0.15}$ %
1.5 |  $\bm{41.40\pm 0.71}$ % |  $\bm{4.39\pm 0.29}$ %
1.7 |  $\bm{41.21\pm 0.76}$ % |  $\bm{4.20\pm 0.34}$ %
2.0 |  $40.54\pm 0.90$ % |  $3.53\pm 0.49$ %

Table 13: Detailed Version of Table 1(f) (50% Adversarial Corruption in
Food-101 with VGG-16)

$\xi$ | Student’s test acc. |  Improvement of student over teacher
---|---|---
0.0 (=Teacher) |  $25.01\pm 0.20$ % | 0 %
0.2 |  $25.90\pm 0.09$ % |  $0.89\pm 0.15$ %
0.5 |  $27.16\pm 0.16$ % |  $2.15\pm 0.06$ %
0.7 |  $27.76\pm 0.21$ % |  $2.75\pm 0.10$ %
1.0 |  $28.33\pm 0.14$ % |  $3.32\pm 0.11$ %
1.2 |  $\bm{28.54\pm 0.11}$ % |  $\bm{3.53\pm 0.16}$ %
1.5 |  $28.39\pm 0.10$ % |  $3.38\pm 0.12$ %
1.7 |  $27.97\pm 0.20$ % |  $2.96\pm 0.24$ %
2.0 |  $26.80\pm 0.27$ % |  $1.79\pm 0.29$ %

Table 14: Detailed Version of Table 5(a) (30% Random Corruption in Stanford
Cars with ResNet-34)

$\xi$ | Student’s test acc. |  Improvement of student over teacher
---|---|---
0.0 (=Teacher) |  $50.34\pm 0.23$ % |  $0$ %
0.5 |  $50.22\pm 0.23$ % |  $-0.12\pm 0.20$ %
1.0 |  $50.88\pm 0.24$ % |  $0.54\pm 0.02$ %
1.5 |  $51.20\pm 0.22$ % |  $0.86\pm 0.01$ %
2.0 |  $51.91\pm 0.41$ % |  $1.57\pm 0.34$ %
2.5 |  $52.39\pm 0.44$ % |  $2.05\pm 0.27$ %
3.0 |  $52.83\pm 0.28$ % |  $2.49\pm 0.25$ %
3.5 |  $52.96\pm 0.28$ % |  $2.62\pm 0.12$ %
4.0 |  $53.21\pm 0.31$ % |  $2.87\pm 0.09$ %
4.5 |  $53.35\pm 0.15$ % |  $3.01\pm 0.22$ %
5.0 |  $\bm{53.55\pm 0.25}$ % |  $\bm{3.21\pm 0.07}$ %
5.5 |  $53.28\pm 0.50$ % |  $2.94\pm 0.33$ %
6.0 |  $53.40\pm 0.28$ % |  $3.06\pm 0.09$ %

Table 15: Detailed Version of Table 5(b) (30% Adversarial Corruption in
Flowers-102 with ResNet-34)

Corruption level | Teacher’s test acc. | Student’s test acc. |  Improvement of student over teacher
---|---|---|---
0% |  $83.97\pm 0.10$ % |  $83.93\pm 0.12$ % |  $-0.04\pm 0.02$ %
10% |  $77.86\pm 0.14$ % |  $80.37\pm 0.04$ % |  $2.51\pm 0.11$ %
30% |  $68.09\pm 0.21$ % |  $74.23\pm 0.08$ % |  $6.14\pm 0.16$ %
50% |  $57.61\pm 0.03$ % |  $66.15\pm 0.27$ % |  $8.54\pm 0.29$ %

Table 16: Detailed Version of Random Corruption in Caltech-256 with ResNet-34
and $\xi=1$ (Table 2(a))

Corruption level | Teacher’s test acc. | Student’s test acc. |  Improvement of student over teacher
---|---|---|---
0% |  $83.97\pm 0.10$ % |  $83.93\pm 0.12$ % |  $-0.04\pm 0.02$ %
10% |  $77.01\pm 0.23$ % |  $79.33\pm 0.13$ % |  $2.32\pm 0.10$ %
30% |  $64.21\pm 0.36$ % |  $69.29\pm 0.12$ % |  $5.08\pm 0.25$ %
50% |  $48.66\pm 0.10$ % |  $54.43\pm 0.29$ % |  $5.77\pm 0.19$ %

Table 17: Detailed Version of Adversarial Corruption in Caltech-256 with
ResNet-34 and $\xi=1$ (Table 2(a))

Corruption level | Teacher’s test acc. | Student’s test acc. |  Improvement of student over teacher
---|---|---|---
0% |  $72.77\pm 0.07$ % |  $72.54\pm 0.07$ % |  $-0.23\pm 0.06$ %
10% |  $70.57\pm 0.14$ % |  $71.20\pm 0.02$ % |  $0.63\pm 0.11$ %
30% |  $66.80\pm 0.06$ % |  $68.14\pm 0.07$ % |  $1.34\pm 0.13$ %
50% |  $62.47\pm 0.10$ % |  $64.58\pm 0.10$ % |  $2.11\pm 0.15$ %

Table 18: Detailed Version of Random Corruption in CIFAR-100 with ResNet-34
and $\xi=1$ (Table 2(b))

Corruption level | Teacher’s test acc. | Student’s test acc. |  Improvement of student over teacher
---|---|---|---
0% |  $72.77\pm 0.07$ % |  $72.54\pm 0.07$ % |  $-0.23\pm 0.06$ %
10% |  $69.39\pm 0.09$ % |  $70.58\pm 0.08$ % |  $1.19\pm 0.08$ %
30% |  $62.18\pm 0.12$ % |  $64.98\pm 0.10$ % |  $2.80\pm 0.06$ %
50% |  $50.80\pm 0.04$ % |  $54.99\pm 0.08$ % |  $4.19\pm 0.09$ %

Table 19: Detailed Version of Hierarchical Corruption in CIFAR-100 with
ResNet-34 and $\xi=1$ (Table 2(b))

Corruption level | Teacher’s test acc. | Student’s test acc. |  Improvement of student over teacher
---|---|---|---
0% |  $63.65\pm 0.08$ % |  $63.28\pm 0.04$ % |  $-0.37\pm 0.10$ %
10% |  $62.44\pm 0.03$ % |  $62.54\pm 0.03$ % |  $0.10\pm 0.04$ %
30% |  $59.38\pm 0.20$ % |  $59.85\pm 0.18$ % |  $0.47\pm 0.04$ %
50% |  $54.76\pm 0.13$ % |  $55.88\pm 0.06$ % |  $1.12\pm 0.08$ %

Table 20: Detailed Version of Random Corruption in Food-101 with ResNet-34 and
$\xi=1$ (Table 2(c))

Corruption level | Teacher’s test acc. | Student’s test acc. |  Improvement of student over teacher
---|---|---|---
0% |  $63.65\pm 0.08$ % |  $63.28\pm 0.04$ % |  $-0.37\pm 0.10$ %
10% |  $61.92\pm 0.13$ % |  $62.16\pm 0.11$ % |  $0.25\pm 0.05$ %
30% |  $57.03\pm 0.16$ % |  $57.80\pm 0.22$ % |  $0.77\pm 0.06$ %
50% |  $48.93\pm 0.08$ % |  $50.78\pm 0.07$ % |  $1.85\pm 0.09$ %

Table 21: Detailed Version of Adversarial Corruption in Food-101 with
ResNet-34 and $\xi=1$ (Table 2(c))

$\xi$ | Student’s test acc. |  Improvement of student over teacher
---|---|---
0.0 (=Teacher) |  $39.78\pm 0.18$ % | 0 %
0.2 |  $41.82\pm 0.06$ % |  $2.04\pm 0.16$ %
0.5 |  $44.83\pm 0.11$ % |  $5.05\pm 0.10$ %
0.7 |  $46.60\pm 0.09$ % |  $6.82\pm 0.16$ %
1.0 |  $48.85\pm 0.06$ % |  $9.07\pm 0.17$ %
1.2 |  $50.21\pm 0.03$ % |  $10.43\pm 0.15$ %
1.5 |  $51.56\pm 0.10$ % |  $11.78\pm 0.16$ %
1.7 |  $52.08\pm 0.03$ % |  $12.30\pm 0.19$ %
2.0 |  $\bm{52.85\pm 0.05}$ % |  $\bm{13.07\pm 0.20}$ %
2.2 |  $\bm{52.67\pm 0.33}$ % |  $\bm{12.89\pm 0.43}$ %
2.5 |  $51.52\pm 0.51$ % |  $11.74\pm 0.60$ %

Table 22: Detailed Version of Table 6(a) (50% Random Corruption in Caltech-256
w/ ResNet-34 and wt. decay $=1\times 10^{-4}$)

$\xi$ | Student’s test acc. |  Improvement of student over teacher
---|---|---
0.0 (=Teacher) |  $54.71\pm 0.05$ % | 0 %
0.2 |  $55.10\pm 0.04$ % |  $0.39\pm 0.09$ %
0.5 |  $56.61\pm 0.04$ % |  $1.90\pm 0.08$ %
0.7 |  $57.51\pm 0.04$ % |  $2.80\pm 0.09$ %
1.0 |  $58.53\pm 0.05$ % |  $3.82\pm 0.04$ %
1.2 |  $58.88\pm 0.03$ % |  $4.17\pm 0.05$ %
1.5 |  $\bm{59.22\pm 0.11}$ % |  $\bm{4.51\pm 0.07}$ %
1.7 |  $\bm{59.27\pm 0.06}$ % |  $\bm{4.56\pm 0.02}$ %
2.0 |  $58.86\pm 0.02$ % |  $4.15\pm 0.07$ %

Table 23: Detailed Version of Table 6(b) (50% Hierarchical Corruption in
CIFAR-100 w/ ResNet-34 and wt. decay $=1\times 10^{-3}$)

Corruption level | Teacher’s test acc. | Student’s test acc. |  Improvement of student over teacher
---|---|---|---
0% |  $82.95\pm 0.02$ % |  $83.20\pm 0.04$ % |  $0.25\pm 0.04$ %
10% |  $74.29\pm 0.12$ % |  $75.68\pm 0.03$ % |  $1.39\pm 0.10$ %
30% |  $54.65\pm 0.13$ % |  $60.96\pm 0.18$ % |  $6.31\pm 0.11$ %
50% |  $39.78\pm 0.18$ % |  $48.85\pm 0.06$ % |  $9.07\pm 0.17$ %

Table 24: Detailed Version of Table 7(a) (Random Corruption in Caltech-256 w/
ResNet-34, $\xi=1$ and wt. decay $=1\times 10^{-4}$)

Corruption level | Teacher’s test acc. | Student’s test acc. |  Improvement of student over teacher
---|---|---|---
0% |  $72.99\pm 0.09$ % |  $72.26\pm 0.01$ % |  $-0.73\pm 0.09$ %
10% |  $70.59\pm 0.04$ % |  $70.62\pm 0.07$ % |  $0.03\pm 0.10$ %
30% |  $64.64\pm 0.12$ % |  $66.41\pm 0.09$ % |  $1.77\pm 0.19$ %
50% |  $54.71\pm 0.05$ % |  $58.53\pm 0.05$ % |  $3.82\pm 0.04$ %

Table 25: Detailed Version of Table 7(b) (Hierarchical Corruption in CIFAR-100
w/ ResNet-34, $\xi=1$ and wt. decay $=1\times 10^{-3}$)


Primary outcome(s): “Our work shows that the FNO is capable of predicting
magnetohy- drodynamic models governing the plasma dynamics, 6 orders of
magnitude faster than the traditional numerical solver, while maintaining
considerable accuracy (NMSE $\sim 10^{5}$).” Supported by table 1.
Baseline: JOREK code. 200 by 200 bi-cubic finite-elements.
Rule 1: ✖ Doesn’t reduce accuracy of JOREK to match accuracy of FNO. See
errors in, e.g., figure 2.
Rule 2: 0.3pt  $\blacktriangle$ !OcFFL1.8ex 160 CPU hours to solve a 2D
advection-dominated time-dependent PDE is unusually long, even at a high
resolution of $200\times 200$. We believe that JOREK is likely using
inefficient numerical methods for advection-dominated flows. We believe the
most likely explanation for the slow implementation is that JOREK uses fully
implicit timestepping, while advection-dominated PDEs are better suited for
explicit timestepping.
Fair comparison: ✖

Article 71: Shit et al. [127]
Title: Semi-Implicit Neural Solver for Time-dependent Partial Differential
Equations
Citations: 0
Fluid-related PDE(s): 2D advection-diffusion, with advection speeds in range
[-2, 2] and diffusion coefficients in range [0.2, 0.8].
Primary outcome(s): “We observe that for a given acceptance, the neural solver
is 19.2% faster.” Supported by figure 5.
Baseline: Semi-implicit scheme (see equation 3), using fixed-point iteration
introduced in section 2.1.
Rule 1: ✓ Figure 5 compares runtime at equal error.
Rule 2: ✖ For this mixed hyperbolic-parabolic PDE, an (explicit) super-time-
stepping method (e.g., [185]) would be more efficient than an implicit method.
Let us explain why. The semi-implicit scheme uses a timestep of $\Delta
t_{\textnormal{implicit}}=0.2$, and each timestep requires a number of fixed
point iterations (up to 25). An explicit method would (since the maximum
diffusion coefficient is $\nu=0.8$ and $\Delta x=0.098$) have a timestep of
$\Delta t_{\textnormal{explicit}}\approx\frac{(\Delta x)^{2}}{2\nu}=0.006$.
Since $\frac{\Delta t_{\textnormal{implicit}}}{\Delta
t_{\textnormal{explicit}}}\approx 33$, but each implicit timestep takes
roughly 25$\times$ more runtime, then implicit timestepping would likely be as
fast or slightly faster (as a rough estimate, as much as $33/25\approx
1.3\times$ faster) than naive explicit timestepping. However, $s$-stage super-
time-stepping allows for timesteps proportional to $s^{2}$ and a runtime
speedup (relative to explicit timestepping) of $s^{2}/s=s$, where the optimal
number of stages $s=\sqrt{\frac{\Delta t_{\textnormal{implicit}}}{\Delta
t_{\textnormal{explicit}}}}$. In this case, the speedup from using super-time-
stepping would be $s=\sqrt{33}=5.75$, which would be faster than the implicit
method.
Fair comparison: ✖

Article 72: Su et al. [128]
Title: Forecasting Variable-Density 3D Turbulent Flow
Citations: 0
Fluid-related PDE(s): 3D compressible Navier-Stokes
Primary outcome(s): “Across flows with different density-ratio, our method
over [sic] 3 orders of magnitude faster than high-fidelity numerical
simulations.” Supported by table 1.
Baseline: Petascale variable-density version of the CFDNS code. Spectral code:
spatial derivatives are evaluated using Fourier transforms. Uses ($64^{3}$)
resolution.
Rule 1: ✖ Table 1 compares the runtime of the ground truth solver to that of
the less accurate ML-based solver. For a fair comparison, reduce the
resolution of the CFDNS code until the accuracy matches the ML-based solver.
Rule 2: ✓ Spectral codes are highly efficient for weakly compressible
turbulence.
Fair comparison: ✖

Article 73: Jeon et al. [129]
Title: Physics-Informed Transfer Learning Strategy to Accelerate Unsteady
Fluid Flow Simulations
Citations: 0
Fluid-related PDE(s): 2D incompressible Navier-Stokes
Primary outcome(s): “The simulation was accelerated by 1.8 times in the
laminar counterflow CFD dataset condition including the parameter-updating
time.”
Baseline: IcoFoam, which is part of OpenFOAM, which uses the PISO algorithm.
Rule 1: ✖ The abstract writes that “Notably, the cross-coupling strategy with
a grid-based network model does not compromise the simulation accuracy for
computational acceleration.” If this were true, then rule 1 would be
satisfied. The problem is that it is the residual of the governing equations
that is “not compromise[d]”, not the accuracy of the solution. As the paper
correctly points out at the top of page 3, a low residual does not imply low
error. Yes, the cross-coupling ML-CFD strategy is $1.8\times$ faster than the
“ground truth” CFD solver, but as figures 9 and 12 show the “ground truth” CFD
solver has zero error while the cross-coupling ML-CFD strategy has positive
error. Thus, rule 1 is not satisfied. To satisfy rule 1, increase the error of
the CFD solver to match that of the cross-coupling strategy. One way to do
this would be to reduce the resolution of the OpenFOAM solution; a second
would be to reduce the residual of OpenFOAM below the default residual.
Rule 2: 0.3pt  $\blacktriangle$ !OcFFL1.8ex We give a warning sign for rule 2
because OpenFOAM is known to be an inefficient general-purpose tool, with high
overhead and slow convergence. See, e.g., [183]. Because OpenFOAM is a general
purpose tool, we also recommend being cautious when evaluating this comparison
(see recommendation 1 in Methods). While we recommend comparing with both FV
and DG methods for the Navier-Stokes equations, we consider FV highly
efficient for problems with fluid-structure interaction.
Fair comparison: ✖

Article 74: Dai et al. [130]
Title: FourNetFlows: An efficient model for steady airfoil flows prediction
Citations: 0
Fluid-related PDE(s): 2D incompressible Navier-Stokes airfoil wing
Primary outcome(s): “We quantitatively shows [sic] the accuracy of
FourNetFlows is matched with the traditional method, running four to five
orders of magnitude faster.” Supported by table 1.
Baseline: SimpleFoam, which uses SIMPLE (semi-implicit method for pressure
linked equations) and is part of OpenFOAM.
Rule 1: ✖ Compares the runtime of ground truth SimpleFOAM to that of the less
accurate ML-based solver.
Rule 2: 0.3pt  $\blacktriangle$ !OcFFL1.8ex We give a warning sign for rule 2
because OpenFOAM is known to be an inefficient general-purpose tool, with high
overhead and slow convergence. See, e.g., [183]. Because OpenFOAM is a general
purpose tool, we also recommend being cautious when evaluating this comparison
(see recommendation 1 in Methods).
Fair comparison: ✖

Article 75: Sun et al. [131]
Title: Local Randomized Neural Networks with Discontinuous Galerkin Methods
for Partial Differential Equations
Citations: 0
Fluid-related PDE(s): 1D Helmholtz, 2D Poisson.
Primary outcome(s): “We compare the proposed methods with the finite element
method and the usual DG method. The LRNN-DG methods can achieve better
accuracy under the same degrees of freedom.” Supported by figure 4.
Baseline: FEM and DG, up to 3rd order. Doesn’t have any other details.
Rule 1: ✓ Compares accuracy at constant number of degrees of freedom, a proxy
for runtime.
Rule 2: ✓ FEM and DG are efficient methods for solving Poisson’s equation.
Fair comparison: ✓

Article 76: Shao et al. [132]
Title: A Poisson’s Equation Solver Based on Neural Network Precondtioned CG
Method
Citations: 0
Fluid-related PDE(s): 2D Poisson
Primary outcome(s): “Numerical examples demonstrate that compared to conjugate
gradient (CG) method, NN-PCG significantly improves convergence performance on
solving 2-D Poisson’s equation.”
Baseline: Conjugate gradient method.
Rule 1: ✓ Figure 2 compares accuracy at constant number of iterations.
Rule 2: ✖ Multigrid methods are state-of-the-art for this PDE. See, e.g.,
figure 4e of article 68 or figure 23 of article 41. Note also that for
sufficiently small 2D problems, direct solves (such as LU decomposition) will
outperform Multigrid methods.
Fair comparison: ✖

Article 77: Discacciati et al. [57]
Title: Controlling oscillations in high-order Discontinuous Galerkin schemes
using artificial viscosity tuned by neural networks
Citations: 54
Fluid-related PDE(s): 1D & 2D advection, 1D & 2D Burgers’, 1D & 2D Euler
Primary outcome(s): “The network-based model is always at par with the best
among the traditional optimized models.” Supported by section 6 and tables 13
and 14.
Baseline: N/A (doesn’t claim superiority)
Rule 1: N/A
Rule 2: N/A
Fair comparison: N/A

Article 78: Magiera et al. [53]
Title: Constraint-aware neural networks for Riemann problems
Citations: 44
Fluid-related PDE(s): 1D Euler, 1D scalar hyperbolic PDE
Primary outcome(s): “Naturally, the case study problems considered in Section
6 do not show a speedup of the computational time by exchanging the analytical
Riemann solver by a surrogate neural network.”
Baseline: N/A
Rule 1: N/A
Rule 2: N/A
Fair comparison: N/A

Article 79: Bezgin et al. [54]
Title: A data-driven physics-informed finite-volume scheme for nonclassical
undercompressive shocks
Citations: 16
Fluid-related PDE(s): Cubic scalar hyperbolic conservation law
Primary outcome(s): “For the weak shock test case which is calculated on a
coarse mesh, the NN scheme is roughly 10 times slower than the WCD scheme.”
Baseline: WCD scheme
Rule 1: N/A (doesn’t claim superiority)
Rule 2: N/A
Fair comparison: N/A

Article 80: Dong and Yang [58]
Title: On computing the hyperparameter of extreme learning machines: Algorithm
and application to computational PDEs, and comparison with classical and high-
order finite elements
Citations: 13
Fluid-related PDE(s): 2D Poisson, 2D non-linear Helmholtz, 1D Burgers.
Primary outcome(s): “It is shown that the current improved ELM far outperforms
the classical FEM. Its computational performance is comparable to that of the
high-order FEM for smaller problem sizes, and for larger problem sizes the ELM
markedly outperforms the high-order FEM.” Supported by figures 16, 27, and 36.
Baseline: FEniCS
Rule 1: N/A (doesn’t claim superiority to stronger baseline)
Rule 2: N/A
Fair comparison: N/A

Article 81: Dresdner et al. [55] (Note: we consider version 1 of this article
on ArXiv, version 2 was uploaded after our systematic review was completed but
before this article was finished.)
Title: Learning to correct spectral methods for simulating turbulent flows
Citations: 8
Fluid-related PDE(s): 1D Kuramoto-Sivashinsky (KS), 1D Burgers’, 2D
incompressible Navier-Stokes
Primary outcome(s): “Overall there is little potential for accelerating 2D
turbulence beyond traditional spectral solvers.”
Baseline: Spectral solver
Rule 1: N/A (doesn’t claim superiority)
Rule 2: N/A
Fair comparison: N/A

Article 82: Toshev et al. [56]
Title: E(3) Equivariant Graph Neural Networks for Particle-Based Fluid
Mechanics
Citations: 0
Fluid-related PDE(s): 3D incompressible Navier-Stokes
Primary outcome(s): “Our main findings are that while being rather slow to
train and evaluate…” Supported by table 1.
Baseline: N/A
Rule 1: N/A (Doesn’t claim superiority)
Rule 2: N/A
Fair comparison: N/A

## References

  *   * Randall and Welser [2018] Randall, D., Welser, C.: The Irreproducibility Crisis of Modern Science: Causes, Consequences, and the Road to Reform. National Association of Scholars, 221 Witherspoon Street 2nd Floor, Princeton, NJ 08542-3215 (2018)
  * Ritchie [2020] Ritchie, S.: Science Fictions: How Fraud, Bias, Negligence, and Hype Undermine the Search for Truth. Vintage, London (2020)
  * Munafò et al. [2017] Munafò, M.R., Nosek, B.A., Bishop, D.V., Button, K.S., Chambers, C.D., Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J.J., Ioannidis, J.: A manifesto for reproducible science. Nature human behaviour 1(1), 1–9 (2017)
  * Ioannidis [2005] Ioannidis, J.P.: Why most published research findings are false. PLoS medicine 2(8), 124 (2005)
  * Collaboration [2015] Collaboration, O.S.: Estimating the reproducibility of psychological science. Science 349(6251), 4716 (2015)
  * Prinz et al. [2011] Prinz, F., Schlange, T., Asadullah, K.: Believe it or not: how much can we rely on published data on potential drug targets? Nature reviews Drug discovery 10(9), 712–712 (2011)
  * Begley and Ellis [2012] Begley, C.G., Ellis, L.M.: Raise standards for preclinical cancer research. Nature 483(7391), 531–533 (2012)
  * Gelman and Loken [2013] Gelman, A., Loken, E.: The garden of forking paths: Why multiple comparisons can be a problem, even when there is no “fishing expedition” or “p-hacking” and the research hypothesis was posited ahead of time. Department of Statistics, Columbia University 348, 1–17 (2013)
  * Baker [2016] Baker, M.: Reproducibility crisis. Nature 533(26), 353–66 (2016)
  * Karagiorgi et al. [2022] Karagiorgi, G., Kasieczka, G., Kravitz, S., Nachman, B., Shih, D.: Machine learning in the search for new fundamental physics. Nature Reviews Physics 4(6), 399–412 (2022)
  * Dara et al. [2022] Dara, S., Dhamercherla, S., Jadav, S.S., Babu, C.M., Ahsan, M.J.: Machine learning in drug discovery: a review. Artificial Intelligence Review 55(3), 1947–1999 (2022)
  * Mater and Coote [2019] Mater, A.C., Coote, M.L.: Deep learning in chemistry. Journal of chemical information and modeling 59(6), 2545–2559 (2019)
  * Carleo et al. [2019] Carleo, G., Cirac, I., Cranmer, K., Daudet, L., Schuld, M., Tishby, N., Vogt-Maranto, L., Zdeborová, L.: Machine learning and the physical sciences. Reviews of Modern Physics 91(4), 045002 (2019)
  * Rajkomar et al. [2019] Rajkomar, A., Dean, J., Kohane, I.: Machine learning in medicine. New England Journal of Medicine 380(14), 1347–1358 (2019)
  * Grimmer et al. [2021] Grimmer, J., Roberts, M.E., Stewart, B.M.: Machine learning for social science: An agnostic approach. Annual Review of Political Science 24, 395–419 (2021)
  * Jumper et al. [2021] Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al.: Highly accurate protein structure prediction with alphafold. Nature 596(7873), 583–589 (2021)
  * Hutson [2018] Hutson, M.: Artificial intelligence faces reproducibility crisis. Science 359(6377), 725–726 (2018) https://doi.org/10.1126/science.359.6377.725
  * Gundersen et al. [2022] Gundersen, O.E., Coakley, K., Kirkpatrick, C., Gil, Y.: Sources of irreproducibility in machine learning: A review. arXiv preprint arXiv:2204.07610 (2022)
  * Sculley et al. [2018] Sculley, D., Snoek, J., Wiltschko, A., Rahimi, A.: Winner’s curse? on pace, progress, and empirical rigor (2018)
  * Armstrong et al. [2009] Armstrong, T.G., Moffat, A., Webber, W., Zobel, J.: Improvements that don’t add up: ad-hoc retrieval results since 1998. In: Proceedings of the 18th ACM Conference on Information and Knowledge Management, pp. 601–610 (2009)
  * Kapoor and Narayanan [2023] Kapoor, S., Narayanan, A.: Leakage and the reproducibility crisis in machine-learning-based science. Patterns 4(9) (2023)
  * Kapoor et al. [2023] Kapoor, S., Cantrell, E., Peng, K., Pham, T.H., Bail, C.A., Gundersen, O.E., Hofman, J.M., Hullman, J., Lones, M.A., Malik, M.M., et al.: Reforms: Reporting standards for machine learning based science. arXiv preprint arXiv:2308.07832 (2023)
  * DeMasi et al. [2017] DeMasi, O., Kording, K., Recht, B.: Meaningless comparisons lead to false optimism in medical machine learning. PloS one 12(9), 0184604 (2017)
  * Roberts et al. [2021] Roberts, M., Driggs, D., Thorpe, M., Gilbey, J., Yeung, M., Ursprung, S., Aviles-Rivero, A.I., Etmann, C., McCague, C., Beer, L., et al.: Common pitfalls and recommendations for using machine learning to detect and prognosticate for covid-19 using chest radiographs and ct scans. Nature Machine Intelligence 3(3), 199–217 (2021)
  * Wynants et al. [2020] Wynants, L., Van Calster, B., Collins, G.S., Riley, R.D., Heinze, G., Schuit, E., Albu, E., Arshi, B., Bellou, V., Bonten, M.M., et al.: Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal. bmj 369 (2020)
  * Whalen et al. [2022] Whalen, S., Schreiber, J., Noble, W.S., Pollard, K.S.: Navigating the pitfalls of applying machine learning in genomics. Nature Reviews Genetics 23(3), 169–181 (2022)
  * Artrith et al. [2021] Artrith, N., Butler, K.T., Coudert, F.-X., Han, S., Isayev, O., Jain, A., Walsh, A.: Best practices in machine learning for chemistry. Nature chemistry 13(6), 505–508 (2021)
  * Thuerey et al. [2021] Thuerey, N., Holl, P., Mueller, M., Schnell, P., Trost, F., Um, K.: Physics-based deep learning. arXiv preprint arXiv:2109.05237 (2021)
  * Brunton and Kutz [2024] Brunton, S.L., Kutz, J.N.: Promising directions of machine learning for partial differential equations. Nature Computational Science, 1–12 (2024)
  * Vinuesa and Brunton [2022] Vinuesa, R., Brunton, S.L.: Enhancing computational fluid dynamics with machine learning. Nature Computational Science 2(6), 358–366 (2022)
  * Karniadakis et al. [2021] Karniadakis, G.E., Kevrekidis, I.G., Lu, L., Perdikaris, P., Wang, S., Yang, L.: Physics-informed machine learning. Nature Reviews Physics 3(6), 422–440 (2021)
  * Cuomo et al. [2022] Cuomo, S., Di Cola, V.S., Giampaolo, F., Rozza, G., Raissi, M., Piccialli, F.: Scientific machine learning through physics–informed neural networks: Where we are and what’s next. Journal of Scientific Computing 92(3), 88 (2022)
  * Duraisamy et al. [2019] Duraisamy, K., Iaccarino, G., Xiao, H.: Turbulence modeling in the age of data. Annual review of fluid mechanics 51, 357–377 (2019)
  * Durran [2013] Durran, D.R.: Numerical Methods for Wave Equations in Geophysical Fluid Dynamics vol. 32. Springer, Berlin (2013)
  * LeVeque [1992] LeVeque, R.J.: Numerical Methods for Conservation Laws vol. 214. Birkhäuser, Basel (1992)
  * Mishra [2018] Mishra, S.: A machine learning framework for data driven acceleration of computations of differential equations. arXiv preprint arXiv:1807.09519 (2018)
  * Kochkov et al. [2021] Kochkov, D., Smith, J.A., Alieva, A., Wang, Q., Brenner, M.P., Hoyer, S.: Machine learning–accelerated computational fluid dynamics. Proceedings of the National Academy of Sciences 118(21), 2101784118 (2021)
  * Kadapa [2021] Kadapa, C.: Machine learning for computational science and engineering–a brief introduction and some critical questions. arXiv preprint arXiv:2112.12054 (2021)
  * Ross et al. [2023] Ross, A., Li, Z., Perezhogin, P., Fernandez-Granda, C., Zanna, L.: Benchmarking of machine learning ocean subgrid parameterizations in an idealized model. Journal of Advances in Modeling Earth Systems 15(1), 2022–003258 (2023)
  * Lippe et al. [2024] Lippe, P., Veeling, B., Perdikaris, P., Turner, R., Brandstetter, J.: Pde-refiner: Achieving accurate long rollouts with neural pde solvers. Advances in Neural Information Processing Systems 36 (2024)
  * Vlachas et al. [2020] Vlachas, P.R., Pathak, J., Hunt, B.R., Sapsis, T.P., Girvan, M., Ott, E., Koumoutsakos, P.: Backpropagation algorithms and reservoir computing in recurrent neural networks for the forecasting of complex spatiotemporal dynamics. Neural Networks 126, 191–217 (2020)
  * Raissi et al. [2019] Raissi, M., Perdikaris, P., Karniadakis, G.E.: Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics 378, 686–707 (2019)
  * Grossmann et al. [2024] Grossmann, T.G., Komorowska, U.J., Latz, J., Schönlieb, C.-B.: Can physics-informed neural networks beat the finite element method? IMA Journal of Applied Mathematics, 143–174 (2024)
  * de la Mata et al. [2023] Mata, F.F., Gijón, A., Molina-Solana, M., Gómez-Romero, J.: Physics-informed neural networks for data-driven simulation: Advantages, limitations, and opportunities. Physica A: Statistical Mechanics and its Applications 610, 128415 (2023)
  * Chuang and Barba [2022] Chuang, P.-Y., Barba, L.A.: Experience report of physics-informed neural networks in fluid simulations: pitfalls and frustration. arXiv preprint arXiv:2205.14249 (2022)
  * Chuang and Barba [2023] Chuang, P.-Y., Barba, L.A.: Predictive limitations of physics-informed neural networks in vortex shedding. arXiv preprint arXiv:2306.00230 (2023)
  * Wang et al. [2022] Wang, S., Yu, X., Perdikaris, P.: When and why pinns fail to train: A neural tangent kernel perspective. Journal of Computational Physics 449, 110768 (2022)
  * Krishnapriyan et al. [2021] Krishnapriyan, A., Gholami, A., Zhe, S., Kirby, R., Mahoney, M.W.: Characterizing possible failure modes in physics-informed neural networks. Advances in Neural Information Processing Systems 34, 26548–26560 (2021)
  * Basir and Senocak [2022] Basir, S., Senocak, I.: Critical investigation of failure modes in physics-informed neural networks. In: AiAA SCITECH 2022 Forum, p. 2353 (2022)
  * Karnakov et al. [2024] Karnakov, P., Litvinov, S., Koumoutsakos, P.: Solving inverse problems in physics by optimizing a discrete loss: Fast and accurate learning without neural networks. PNAS nexus, 005 (2024)
  * Gundersen [2021] Gundersen, O.E.: The fundamental principles of reproducibility. Philosophical Transactions of the Royal Society A 379(2197), 20200210 (2021)
  * Aromataris and Pearson [2014] Aromataris, E., Pearson, A.: The systematic review: an overview. AJN The American Journal of Nursing 114(3), 53–58 (2014)
  * Magiera et al. [2020] Magiera, J., Ray, D., Hesthaven, J.S., Rohde, C.: Constraint-aware neural networks for riemann problems. Journal of Computational Physics 409, 109345 (2020)
  * Bezgin et al. [2021] Bezgin, D.A., Schmidt, S.J., Adams, N.A.: A data-driven physics-informed finite-volume scheme for nonclassical undercompressive shocks. Journal of Computational Physics 437, 110324 (2021)
  * Dresdner et al. [2022] Dresdner, G., Kochkov, D., Norgaard, P., Zepeda-Núñez, L., Smith, J.A., Brenner, M.P., Hoyer, S.: Learning to correct spectral methods for simulating turbulent flows. arXiv preprint arXiv:2207.00556v1 (2022)
  * Toshev et al. [2023] Toshev, A.P., Galletti, G., Brandstetter, J., Adami, S., Adams, N.A.: E ($3$) equivariant graph neural networks for particle-based fluid mechanics. arXiv preprint arXiv:2304.00150 (2023)
  * Discacciati et al. [2020] Discacciati, N., Hesthaven, J.S., Ray, D.: Controlling oscillations in high-order discontinuous galerkin schemes using artificial viscosity tuned by neural networks. Journal of Computational Physics 409, 109304 (2020)
  * Dong and Yang [2022] Dong, S., Yang, J.: On computing the hyperparameter of extreme learning machines: Algorithm and application to computational pdes, and comparison with classical and high-order finite elements. Journal of Computational Physics 463, 111290 (2022)
  * Li et al. [2020] Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A., Anandkumar, A.: Fourier neural operator for parametric partial differential equations. arXiv preprint arXiv:2010.08895 (2020)
  * Lu et al. [2021] Lu, L., Jin, P., Pang, G., Zhang, Z., Karniadakis, G.E.: Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature machine intelligence 3(3), 218–229 (2021)
  * Tompson et al. [2017] Tompson, J., Schlachter, K., Sprechmann, P., Perlin, K.: Accelerating eulerian fluid simulation with convolutional networks. In: International Conference on Machine Learning, pp. 3424–3433 (2017). PMLR
  * Pfaff et al. [2020] Pfaff, T., Fortunato, M., Sanchez-Gonzalez, A., Battaglia, P.W.: Learning mesh-based simulation with graph networks. arXiv preprint arXiv:2010.03409 (2020)
  * Bar-Sinai et al. [2019] Bar-Sinai, Y., Hoyer, S., Hickey, J., Brenner, M.P.: Learning data-driven discretizations for partial differential equations. Proceedings of the National Academy of Sciences 116(31), 15344–15349 (2019)
  * Kim et al. [2019] Kim, B., Azevedo, V.C., Thuerey, N., Kim, T., Gross, M., Solenthaler, B.: Deep fluids: A generative network for parameterized fluid simulations. In: Computer Graphics Forum, vol. 38, pp. 59–70 (2019). Wiley Online Library
  * Wang et al. [2021] Wang, S., Wang, H., Perdikaris, P.: Learning the solution operator of parametric partial differential equations with physics-informed deeponets. Science advances 7(40), 8605 (2021)
  * Um et al. [2020] Um, K., Brand, R., Fei, Y.R., Holl, P., Thuerey, N.: Solver-in-the-loop: Learning from differentiable physics to interact with iterative pde-solvers. Advances in Neural Information Processing Systems 33, 6111–6122 (2020)
  * Cai et al. [2021] Cai, S., Wang, Z., Lu, L., Zaki, T.A., Karniadakis, G.E.: Deepm&mnet: Inferring the electroconvection multiphysics fields based on operator approximation by neural networks. Journal of Computational Physics 436, 110296 (2021)
  * Belbute-Peres et al. [2020] Belbute-Peres, F.D.A., Economon, T., Kolter, Z.: Combining differentiable pde solvers and graph neural networks for fluid flow prediction. In: International Conference on Machine Learning, pp. 2402–2411 (2020). PMLR
  * Li et al. [2021] Li, Z., Zheng, H., Kovachki, N., Jin, D., Chen, H., Liu, B., Azizzadenesheli, K., Anandkumar, A.: Physics-informed neural operator for learning partial differential equations. arXiv preprint arXiv:2111.03794 (2021)
  * Yang et al. [2016] Yang, C., Yang, X., Xiao, X.: Data-driven projection method in fluid simulation. Computer Animation and Virtual Worlds 27(3-4), 415–424 (2016)
  * Hsieh et al. [2019] Hsieh, J.-T., Zhao, S., Eismann, S., Mirabella, L., Ermon, S.: Learning neural pde solvers with convergence guarantees. arXiv preprint arXiv:1906.01200 (2019)
  * Brandstetter et al. [2022] Brandstetter, J., Worrall, D., Welling, M.: Message passing neural pde solvers. arXiv preprint arXiv:2202.03376 (2022)
  * Mao et al. [2021] Mao, Z., Lu, L., Marxen, O., Zaki, T.A., Karniadakis, G.E.: Deepm&mnet for hypersonics: Predicting the coupled flow and finite-rate chemistry behind a normal shock using neural-network approximation of operators. Journal of computational physics 447, 110698 (2021)
  * Greenfeld et al. [2019] Greenfeld, D., Galun, M., Basri, R., Yavneh, I., Kimmel, R.: Learning to optimize multigrid pde solvers. In: International Conference on Machine Learning, pp. 2415–2423 (2019). PMLR
  * Dong and Li [2021] Dong, S., Li, Z.: Local extreme learning machines and domain decomposition for solving linear and nonlinear partial differential equations. Computer Methods in Applied Mechanics and Engineering 387, 114129 (2021)
  * Ray and Hesthaven [2019] Ray, D., Hesthaven, J.S.: Detecting troubled-cells on two-dimensional unstructured grids using a neural network. Journal of Computational Physics 397, 108845 (2019)
  * Xiao et al. [2018] Xiao, X., Zhou, Y., Wang, H., Yang, X.: A novel cnn-based poisson solver for fluid simulation. IEEE transactions on visualization and computer graphics 26(3), 1454–1465 (2018)
  * Wandel et al. [2020] Wandel, N., Weinmann, M., Klein, R.: Learning incompressible fluid dynamics from scratch–towards fast, differentiable fluid models that generalize. arXiv preprint arXiv:2006.08762 (2020)
  * Shan et al. [2020] Shan, T., Tang, W., Dang, X., Li, M., Yang, F., Xu, S., Wu, J.: Study on a fast solver for poisson’s equation based on deep learning technique. IEEE Transactions on Antennas and Propagation 68(9), 6725–6733 (2020)
  * Luz et al. [2020] Luz, I., Galun, M., Maron, H., Basri, R., Yavneh, I.: Learning algebraic multigrid using graph neural networks. In: International Conference on Machine Learning, pp. 6489–6499 (2020). PMLR
  * Zhuang et al. [2021] Zhuang, J., Kochkov, D., Bar-Sinai, Y., Brenner, M.P., Hoyer, S.: Learned discretizations for passive scalar advection in a two-dimensional turbulent flow. Physical Review Fluids 6(6), 064605 (2021)
  * Pathak et al. [2020] Pathak, J., Mustafa, M., Kashinath, K., Motheau, E., Kurth, T., Day, M.: Using machine learning to augment coarse-grid computational fluid dynamics simulations. arXiv preprint arXiv:2010.00072 (2020)
  * Di Leoni et al. [2021] Di Leoni, P.C., Lu, L., Meneveau, C., Karniadakis, G., Zaki, T.A.: Deeponet prediction of linear instability waves in high-speed boundary layers. arXiv preprint arXiv:2105.08697 (2021)
  * Li et al. [2022] Li, Z., Huang, D.Z., Liu, B., Anandkumar, A.: Fourier neural operator with learned deformations for pdes on general geometries. arXiv preprint arXiv:2207.05209 (2022)
  * Stevens and Colonius [2020] Stevens, B., Colonius, T.: Enhancement of shock-capturing methods via machine learning. Theoretical and Computational Fluid Dynamics 34(4), 483–496 (2020)
  * Illarramendi et al. [2020] Illarramendi, E.A., Alguacil, A., Bauerheim, M., Misdariis, A., Cuenot, B., Benazera, E.: Towards an hybrid computational strategy based on deep learning for incompressible flows. In: AIAA Aviation 2020 Forum, p. 3058 (2020)
  * Stachenfeld et al. [2021] Stachenfeld, K., Fielding, D.B., Kochkov, D., Cranmer, M., Pfaff, T., Godwin, J., Cui, C., Ho, S., Battaglia, P., Sanchez-Gonzalez, A.: Learned coarse models for efficient turbulence simulation. arXiv preprint arXiv:2112.15275 (2021)
  * Han et al. [2022] Han, X., Gao, H., Pfaff, T., Wang, J.-X., Liu, L.-P.: Predicting physics in mesh-reduced space with temporal attention. arXiv preprint arXiv:2201.09113 (2022)
  * Stevens and Colonius [2020] Stevens, B., Colonius, T.: Finitenet: A fully convolutional lstm network architecture for time-dependent partial differential equations. arXiv preprint arXiv:2002.03014 (2020)
  * Özbay et al. [2021] Özbay, A.G., Hamzehloo, A., Laizet, S., Tzirakis, P., Rizos, G., Schuller, B.: Poisson cnn: Convolutional neural networks for the solution of the poisson equation on a cartesian mesh. Data-Centric Engineering 2, 6 (2021)
  * Li and Farimani [2022] Li, Z., Farimani, A.B.: Graph neural network-accelerated lagrangian fluid simulation. Computers & Graphics 103, 201–211 (2022)
  * Peng et al. [2022] Peng, W., Yuan, Z., Wang, J.: Attention-enhanced neural network models for turbulence simulation. Physics of Fluids 34(2) (2022)
  * Chen et al. [2021] Chen, L.-W., Cakal, B.A., Hu, X., Thuerey, N.: Numerical investigation of minimum drag profiles in laminar flow using deep learning surrogates. Journal of Fluid Mechanics 919, 34 (2021)
  * Alguacil et al. [2021] Alguacil, A., Bauerheim, M., Jacob, M.C., Moreau, S.: Predicting the propagation of acoustic waves using deep convolutional neural networks. Journal of Sound and Vibration 512, 116285 (2021)
  * Wandel et al. [2021] Wandel, N., Weinmann, M., Klein, R.: Teaching the incompressible navier–stokes equations to fast neural surrogate models in three dimensions. Physics of Fluids 33(4) (2021)
  * List et al. [2022] List, B., Chen, L.-W., Thuerey, N.: Learned turbulence modelling with differentiable fluid solvers: physics-based loss functions and optimisation horizons. Journal of Fluid Mechanics 949, 25 (2022)
  * Cheng et al. [2021] Cheng, L., Illarramendi, E.A., Bogopolsky, G., Bauerheim, M., Cuenot, B.: Using neural networks to solve the 2d poisson equation for electric field computation in plasma fluid simulations. arXiv preprint arXiv:2109.13076 (2021)
  * Wen et al. [2020] Wen, X., Don, W.S., Gao, Z., Hesthaven, J.S.: An edge detector based on artificial neural network with application to hybrid compact-weno finite difference scheme. Journal of Scientific Computing 83, 1–21 (2020)
  * de Lara and Ferrer [2022] Lara, F.M., Ferrer, E.: Accelerating high order discontinuous galerkin solvers using neural networks: 1d burgers’ equation. Computers & Fluids 235, 105274 (2022)
  * Zhao et al. [2022] Zhao, Q., Lindell, D.B., Wetzstein, G.: Learning to solve pde-constrained inverse problems with graph networks. arXiv preprint arXiv:2206.00711 (2022)
  * Illarramendi et al. [2022] Illarramendi, E.A., Bauerheim, M., Cuenot, B.: Performance and accuracy assessments of an incompressible fluid solver coupled with a deep convolutional neural network. Data-Centric Engineering 3, 2 (2022)
  * Holloway et al. [2021] Holloway, I., Wood, A., Alekseenko, A.: Acceleration of boltzmann collision integral calculation using machine learning. Mathematics 9(12), 1384 (2021)
  * Azulay and Treister [2022] Azulay, Y., Treister, E.: Multigrid-augmented deep learning preconditioners for the helmholtz equation. SIAM Journal on Scientific Computing (0), 127–151 (2022)
  * Wu et al. [2022] Wu, T., Maruyama, T., Leskovec, J.: Learning to accelerate partial differential equations via latent global evolution. Advances in Neural Information Processing Systems 35, 2240–2253 (2022)
  * Liu et al. [2022] Liu, X.-Y., Sun, H., Zhu, M., Lu, L., Wang, J.-X.: Predicting parametric spatiotemporal dynamics by multi-resolution pde structure-preserved deep learning. arXiv preprint arXiv:2205.03990 (2022)
  * Zhang et al. [2022] Zhang, E., Kahana, A., Turkel, E., Ranade, R., Pathak, J., Karniadakis, G.E.: A hybrid iterative numerical transferable solver (hints) for pdes based on deep operator network and relaxation methods. arXiv preprint arXiv:2208.13273 (2022)
  * Duarte et al. [2022] Duarte, R., Nemmen, R., Navarro, J.P.: Black hole weather forecasting with deep learning: a pilot study. Monthly Notices of the Royal Astronomical Society 512(4), 5848–5861 (2022)
  * Alguacil et al. [2022] Alguacil, A., Bauerheim, M., Jacob, M.C., Moreau, S.: Deep learning surrogate for the temporal propagation and scattering of acoustic waves. AIAA Journal 60(10), 5890–5906 (2022)
  * Bezgin et al. [2022] Bezgin, D.A., Schmidt, S.J., Adams, N.A.: Weno3-nn: A maximum-order three-point data-driven weighted essentially non-oscillatory scheme. Journal of Computational Physics 452, 110920 (2022)
  * Shang et al. [2022] Shang, Y., Wang, F., Sun, J.: Deep petrov-galerkin method for solving partial differential equations. arXiv preprint arXiv:2201.12995 (2022)
  * Kube et al. [2021] Kube, R., Churchill, R., Sturdevant, B.: Machine learning accelerated particle-in-cell plasma simulations. arXiv preprint arXiv:2110.12444 (2021)
  * Shi et al. [2022] Shi, W., Huang, X., Gao, X., Wei, X., Zhang, J., Bian, J., Yang, M., Liu, T.-Y.: Lordnet: Learning to solve parametric partial differential equations without simulated data. arXiv preprint arXiv:2206.09418 (2022)
  * Ranade et al. [2021] Ranade, R., Hill, C., He, H., Maleki, A., Pathak, J.: A latent space solver for pde generalization. arXiv preprint arXiv:2104.02452 (2021)
  * Chen et al. [2022] Chen, R., Jin, X., Li, H.: A machine learning based solver for pressure poisson equations. Theoretical and Applied Mechanics Letters 12(5), 100362 (2022)
  * Ranade et al. [2021] Ranade, R., Hill, C., He, H., Maleki, A., Chang, N., Pathak, J.: A composable autoencoder-based iterative algorithm for accelerating numerical simulations. arXiv preprint arXiv:2110.03780 (2021)
  * Peng et al. [2023] Peng, W., Yuan, Z., Li, Z., Wang, J.: Linear attention coupled fourier neural operator for simulation of three-dimensional turbulence. Physics of Fluids 35(1) (2023)
  * de Lara and Ferrer [2023] Lara, F.M., Ferrer, E.: Accelerating high order discontinuous galerkin solvers using neural networks: 3d compressible navier-stokes equations. Journal of Computational Physics, 112253 (2023)
  * Ranade et al. [2022] Ranade, R., Hill, C., Ghule, L., Pathak, J.: A composable machine-learning approach for steady-state simulations on high-resolution grids. Advances in Neural Information Processing Systems 35, 17386–17401 (2022)
  * Fang and Tan [2022] Fang, D., Tan, J.: Immersed boundary-physics informed machine learning approach for fluid–solid coupling. Ocean Engineering 263, 112360 (2022)
  * Shukla et al. [2023] Shukla, K., Oommen, V., Peyvan, A., Penwarden, M., Bravo, L., Ghoshal, A., Kirby, R.M., Karniadakis, G.E.: Deep neural operators can serve as accurate surrogates for shape optimization: a case study for airfoils. arXiv preprint arXiv:2302.00807 (2023)
  * Zhang et al. [2022] Zhang, T., Innanen, K., Trad, D.: Learning the elastic wave equation with fourier neural operators. geoconvention 2022, 1–5 (2022)
  * Bezgin et al. [2021] Bezgin, D.A., Buhendwa, A.B., Adams, N.A.: A fully-differentiable compressible high-order computational fluid dynamics solver. arXiv preprint arXiv:2112.04979 (2021)
  * Yang et al. [2023] Yang, Y., Gao, A.F., Azizzadenesheli, K., Clayton, R.W., Ross, Z.E.: Rapid seismic waveform modeling and inversion with neural operators. IEEE Transactions on Geoscience and Remote Sensing 61, 1–12 (2023)
  * Tang et al. [2022] Tang, J., Azevedo, V.C., Cordonnier, G., Solenthaler, B.: Neural green’s function for laplacian systems. Computers & Graphics 107, 186–196 (2022)
  * Nastorg et al. [2022] Nastorg, M., Schoenauer, M., Charpiat, G., Faney, T., Gratien, J.-M., Bucci, M.-A.: Ds-gps: A deep statistical graph poisson solver (for faster cfd simulations). arXiv preprint arXiv:2211.11763 (2022)
  * Gopakumar et al. [2023] Gopakumar, V., Pamela, S., Zanisi, L., Li, Z., Anandkumar, A., Team, M.: Fourier neural operator for plasma modelling. arXiv preprint arXiv:2302.06542 (2023)
  * Shit et al. [2021] Shit, S., Ezhov, I., Mächler, L., Lipkova, J., Paetzold, J.C., Kofler, F., Piraud, M., Menze, B.H., et al.: Semi-implicit neural solver for time-dependent partial differential equations. arXiv preprint arXiv:2109.01467 (2021)
  * Su et al. [2021] Su, X., Walters, R., Aslangil, D., Yu, R.: Forecasting variable-density 3d turbulent flow. In: ICLR 2021 SimDL Workshop (2021)
  * Jeon et al. [2022] Jeon, J., Lee, J., Eivazi, H., Vinuesa, R., Kim, S.J.: Physics-informed transfer learning strategy to accelerate unsteady fluid flow simulations. arXiv preprint arXiv:2206.06817 (2022)
  * Dai et al. [2022] Dai, Y., An, Y., Li, Z.: Fournetflows: An efficient model for steady airfoil flows prediction. arXiv preprint arXiv:2207.04358 (2022)
  * Sun et al. [2022] Sun, J., Dong, S., Wang, F.: Local randomized neural networks with discontinuous galerkin methods for partial differential equations. arXiv preprint arXiv:2206.05577 (2022)
  * Shao et al. [2022] Shao, T., Shan, T., Li, M., Yang, F., Xu, S.: A poisson’s equation solver based on neural network precondtioned cg method. In: 2022 International Applied Computational Electromagnetics Society Symposium (ACES-China), pp. 1–2 (2022). IEEE
  * Richards and Onakpoya [2019] Richards, G.C., Onakpoya, I.J.: Reporting biases. In: Catalogue of Bias (2019). https://catalogofbias.org/biases/reporting-biases/
  * Thornton and Lee [2000] Thornton, A., Lee, P.: Publication bias in meta-analysis: its causes and consequences. Journal of clinical epidemiology 53(2), 207–216 (2000)
  * Boutron and Ravaud [2018] Boutron, I., Ravaud, P.: Misrepresentation and distortion of research in biomedical literature. Proceedings of the National Academy of Sciences 115(11), 2613–2619 (2018)
  * Thomas and Heneghan [2022] Thomas, E.T., Heneghan, C.: Catalogue of bias: selective outcome reporting bias. BMJ Evidence-Based Medicine 27(6), 370–372 (2022)
  * Head et al. [2015] Head, M.L., Holman, L., Lanfear, R., Kahn, A.T., Jennions, M.D.: The extent and consequences of p-hacking in science. PLoS biology 13(3), 1002106 (2015)
  * Saini et al. [2014] Saini, P., Loke, Y.K., Gamble, C., Altman, D.G., Williamson, P.R., Kirkham, J.J.: Selective reporting bias of harm outcomes within studies: findings from a cohort of systematic reviews. Bmj 349 (2014)
  * Altman et al. [2017] Altman, D.G., Moher, D., Schulz, K.F.: Harms of outcome switching in reports of randomised trials: Consort perspective. Bmj 356 (2017)
  * Erasmus et al. [2022] Erasmus, A., Holman, B., Ioannidis, J.P.: Data-dredging bias. BMJ Evidence-Based Medicine 27(4), 209–211 (2022)
  * De Vries et al. [2018] De Vries, Y., Roest, A., Jonge, P., Cuijpers, P., Munafò, M., Bastiaansen, J.: The cumulative effect of reporting and citation biases on the apparent efficacy of treatments: the case of depression. Psychological Medicine 48(15), 2453–2455 (2018)
  * Fanelli [2010] Fanelli, D.: “positive” results increase down the hierarchy of the sciences. PloS one 5(4), 10068 (2010)
  * Sanchez-Gonzalez et al. [2020] Sanchez-Gonzalez, A., Godwin, J., Pfaff, T., Ying, R., Leskovec, J., Battaglia, P.: Learning to simulate complex physics with graph networks. In: International Conference on Machine Learning, pp. 8459–8468 (2020). PMLR
  * Klimesch et al. [2022] Klimesch, J., Holl, P., Thuerey, N.: Simulating liquids with graph networks. arXiv preprint arXiv:2203.07895 (2022)
  * Wang et al. [2022] Wang, S., Wang, H., Perdikaris, P.: Improved architectures and training algorithms for deep operator networks. Journal of Scientific Computing 92(2), 35 (2022)
  * Gupta and Brandstetter [2022] Gupta, J.K., Brandstetter, J.: Towards multi-spatiotemporal-scale generalized pde modeling. arXiv preprint arXiv:2209.15616 (2022)
  * McGreivy and Hakim [2023] McGreivy, N., Hakim, A.: Invariant preservation in machine learned pde solvers via error correction. arXiv preprint arXiv:2303.16110 (2023)
  * Simmons et al. [2011] Simmons, J.P., Nelson, L.D., Simonsohn, U.: False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological science 22(11), 1359–1366 (2011)
  * Wicherts et al. [2016] Wicherts, J.M., Veldkamp, C.L., Augusteijn, H.E., Bakker, M., Van Aert, R., Van Assen, M.A.: Degrees of freedom in planning, running, analyzing, and reporting psychological studies: A checklist to avoid p-hacking. Frontiers in psychology, 1832 (2016)
  * Serra-Garcia and Gneezy [2021] Serra-Garcia, M., Gneezy, U.: Nonreplicable publications are cited more than replicable ones. Science advances 7(21), 1705 (2021)
  * Borji [2018] Borji, A.: Negative results in computer vision: A perspective. Image and Vision Computing 69, 1–8 (2018)
  * Smith et al. [2022] Smith, J.J., Amershi, S., Barocas, S., Wallach, H., Wortman Vaughan, J.: Real ml: Recognizing, exploring, and articulating limitations of machine learning research. In: 2022 ACM Conference on Fairness, Accountability, and Transparency, pp. 587–597 (2022)
  * Gundersen [2021] Gundersen, O.E.: The case against registered reports. AI Magazine 42(1), 88–92 (2021)
  * Schooler [2014] Schooler, J.W.: Metascience could rescue the ‘replication crisis’. Nature 515(7525), 9–9 (2014)
  * Ray and Hesthaven [2018] Ray, D., Hesthaven, J.S.: An artificial neural network as a troubled-cell indicator. Journal of computational physics 367, 166–191 (2018)
  * Wang and Perdikaris [2023] Wang, S., Perdikaris, P.: Long-time integration of parametric evolution equations with physics-informed deeponets. Journal of Computational Physics 475, 111855 (2023)
  * Ovadia et al. [2021] Ovadia, O., Kahana, A., Turkel, E., Dekel, S.: Beyond the courant-friedrichs-lewy condition: Numerical methods for the wave problem using deep learning. Journal of Computational Physics 442, 110493 (2021)
  * Li et al. [2021] Li, Z., Liu-Schiaffini, M., Kovachki, N., Liu, B., Azizzadenesheli, K., Bhattacharya, K., Stuart, A., Anandkumar, A.: Learning dissipative dynamics in chaotic systems. arXiv preprint arXiv:2106.06898 (2021)
  * Ni and Dong [2023] Ni, N., Dong, S.: Numerical computation of partial differential equations by hidden-layer concatenated extreme learning machine. Journal of Scientific Computing 95(2), 35 (2023)
  * Dong and Li [2021] Dong, S., Li, Z.: A modified batch intrinsic plasticity method for pre-training the random coefficients of extreme learning machines. Journal of Computational Physics 445, 110585 (2021)
  * Mueller et al. [2022] Mueller, M., Greif, R., Jenko, F., Thuerey, N.: Leveraging stochastic predictions of bayesian neural networks for fluid simulations. arXiv preprint arXiv:2205.01222 (2022)
  * Wang et al. [2021] Wang, S., Bhouri, M.A., Perdikaris, P.: Fast pde-constrained optimization via self-supervised operator learning. arXiv preprint arXiv:2110.13297 (2021)
  * Schwander et al. [2021] Schwander, L., Ray, D., Hesthaven, J.S.: Controlling oscillations in spectral methods by local artificial viscosity governed by neural networks. Journal of Computational Physics 431, 110144 (2021)
  * Donon et al. [2020] Donon, B., Liu, Z., Liu, W., Guyon, I., Marot, A., Schoenauer, M.: Deep statistical solvers. Advances in Neural Information Processing Systems 33, 7910–7921 (2020)
  * Wan et al. [2023] Wan, Z.Y., Zepeda-Núñez, L., Boral, A., Sha, F.: Evolve smoothly, fit consistently: Learning smooth latent dynamics for advection-dominated systems. arXiv preprint arXiv:2301.10391 (2023)
  * Di Leoni et al. [2023] Di Leoni, P.C., Lu, L., Meneveau, C., Karniadakis, G.E., Zaki, T.A.: Neural operator prediction of linear instability waves in high-speed boundary layers. Journal of Computational Physics 474, 111793 (2023)
  * Kovachki et al. [2021] Kovachki, N., Li, Z., Liu, B., Azizzadenesheli, K., Bhattacharya, K., Stuart, A., Anandkumar, A.: Neural operator: Learning maps between function spaces. arXiv preprint arXiv:2108.08481 (2021)
  * Holl et al. [2020] Holl, P., Koltun, V., Um, K., Thuerey, N.: phiflow: A differentiable pde solving framework for deep learning via physical simulations. In: NeurIPS Workshop, vol. 2 (2020)
  * Nemmen et al. [2020] Nemmen, R., Duarte, R., Navarro, J.P.: The first ai simulation of a black hole. Proceedings of the International Astronomical Union 15(S359), 329–333 (2020)
  * Wandel et al. [2020] Wandel, N., Weinmann, M., Klein, R.: Unsupervised deep learning of incompressible fluid dynamics. arXiv preprint arXiv:2006.08762 (2020)
  * Haridas et al. [2022] Haridas, A., Vadlamani, N.R., Minamoto, Y.: Deep neural networks to correct sub-precision errors in cfd. Applications in Energy and Combustion Science 12, 100081 (2022)
  * Guennebaud et al. [2010] Guennebaud, G., Jacob, B., et al.: Eigen. URl: http://eigen. tuxfamily. org 3(1) (2010)
  * Falgout and Yang [2002] Falgout, R.D., Yang, U.M.: hypre: A library of high performance preconditioners. In: International Conference on Computational Science, pp. 632–641 (2002). Springer
  * Mayr et al. [2018] Mayr, M., Wall, W., Gee, M.: Adaptive time stepping for fluid-structure interaction solvers. Finite Elements in Analysis and Design 141, 55–69 (2018)
  * Reynolds et al. [2014] Reynolds, D.R., Woodward, C.S., Gardner, D.J., Hindmarsh, A.C.: Arkode: a library of high order implicit/explicit methods for multi-rate problems. In: SIAM Conference on Parallel Processing for Scientific Computing (2014)
  * Gottlieb et al. [2001] Gottlieb, S., Shu, C.-W., Tadmor, E.: Strong stability-preserving high-order time discretization methods. SIAM review 43(1), 89–112 (2001)
  * Hakim et al. [2019] Hakim, A., Hammett, G., Shi, E., Mandell, N.: Discontinuous galerkin schemes for a class of hamiltonian evolution equations with applications to plasma fluid and kinetic problems. arXiv preprint arXiv:1908.01814 (2019)
  * Cockburn and Shu [1989] Cockburn, B., Shu, C.-W.: Tvb runge-kutta local projection discontinuous galerkin finite element method for conservation laws. ii. general framework. Mathematics of computation 52(186), 411–435 (1989)
  * Qin et al. [2022] Qin, T., Beatson, A., Oktay, D., McGreivy, N., Adams, R.P.: Meta-pde: Learning to solve pdes quickly without a mesh. arXiv preprint arXiv:2211.01604 (2022)
  * McGreivy and Hakim [2024a] McGreivy, N., Hakim, A.: Data created during random sampling from and systematic review of ML-for-PDE solving research. OSF (2024). https://doi.org/10.17605/OSF.IO/GQ5B3
  * McGreivy and Hakim [2024b] McGreivy, N., Hakim, A.: nickmcgreivy/WeakBaselinesMLPDE: First release. Zenodo (2024). https://doi.org/10.5281/ZENODO.12682908
  * Markert [2022] Markert, J.: Discontinuous galerkin spectral element methods for astrophysical flows in multi-physics applications. PhD thesis, Universität zu Köln (2022)
  * [183] Capuano, F., Beratlis, N., Zhang, F., Peet, Y., Squires, K., Balaras, E.: Cost vs accuracy: Dns of turbulent flow over a sphere using structured immersed-boundary, unstructured finite-volume, and spectral-element methods. Unstructured Finite-Volume, and Spectral-Element Methods
  * Södersten et al. [2019] Södersten, A., Matsunaga, T., Koshizuka, S.: Bucket-based multigrid preconditioner for solving pressure poisson equation using a particle method. Computers & Fluids 191, 104242 (2019)
  * Meyer et al. [2014] Meyer, C.D., Balsara, D.S., Aslam, T.D.: A stabilized runge–kutta–legendre method for explicit super-time-stepping of parabolic and mixed equations. Journal of Computational Physics 257, 594–626 (2014)