# Quasi-streamwise vortices and enhanced dissipation for the incompressible 3D
Navier-Stokes equations

In-Jee Jeong School of Mathematics, Korea Institute for Advanced Study.
E-mail<EMAIL_ADDRESS>Tsuyoshi Yoneda Department of Mathematics,
University of Tokyo. E-mail<EMAIL_ADDRESS>
###### Abstract

We consider the 3D incompressible Navier-Stokes equations under the following
$2+\frac{1}{2}$-dimensional situation: vertical vortex blob (quasi-streamwise
vortices) being stretched by two-dimensional shear flow. We prove enhanced
dissipation induced by such quasi-streamwise vortices.

##  1 Introduction

One of the most important law in the study of developed turbulence is
Kolmogorov’s $4/5$-law, and in the derivation of this law, a significant
assumption is the zeroth law (see Frisch [1] for example). The zeroth law of
turbulence states that, in the limit of vanishing viscosity, the rate of
kinetic energy dissipation for solutions to the incompressible Navier-Stokes
equations becomes nonzero. To formulate this law, let us recall the 3D
incompressible Navier-Stokes equations on
$\mathbb{T}^{3}:=(\mathbb{R}/\mathbb{Z})^{3}$:

$\left\\{\begin{aligned} \partial_{t}u^{\nu}+u^{\nu}\cdot\nabla u^{\nu}+\nabla
p=\nu\Delta u^{\nu}+f,\\\ \nabla\cdot u^{\nu}=0,\\\
u^{\nu}(t=0)=u^{\nu}_{0},\end{aligned}\right.$ (1.1)

where $\nu>0$ is the viscosity and
$u^{\nu}:\mathbb{T}^{3}\rightarrow\mathbb{R}^{3},p:\mathbb{T}^{3}\rightarrow\mathbb{R}$
denote the velocity and pressure of the fluid, respectively. Here
$f:\mathbb{T}^{3}\rightarrow\mathbb{R}^{3}$ is some external force. Assuming
that the solution is sufficiently smooth, taking the dot product of the
equation with $u^{\nu}$ and integrating over the domain $\mathbb{T}^{3}$ gives
the energy balance:

$\begin{split}\frac{d}{dt}\frac{1}{2}\|u^{\nu}(t)\|_{L^{2}}^{2}=\int_{\mathbb{T}^{3}}f(t)\cdot
u^{\nu}(t)dx-\nu\|\nabla u^{\nu}(t)\|_{L^{2}}^{2}.\end{split}$ (1.2)

The zeroth law then postulates that, under the normalization
$\|u_{0}^{\nu}\|_{L^{2}}=1$, the mean energy dissipation rate does not vanish
as $\nu\rightarrow 0^{+}$:

$\begin{split}\liminf_{\nu\rightarrow 0}\nu\langle|\nabla
u^{\nu}|^{2}\rangle>0,\end{split}$

where $\langle\cdot\rangle$ usually denotes some ensemble or long-time, space
averages. Laboratory experiments and numerical simulations of turbulence both
confirm the above zeroth law [6]. Keeping this classical physical phenomena in
mind, in this paper, we compare the Navier-Stokes flow and the heat flow with
the same initial data, and show that vortex stretching is enhancing the
viscosity dissipation. This could be a good example clarifying a mechanism of
the actual zeroth law. Here, by the heat flow, we shall mean the unique
$L^{2}$ solution of

$\left\\{\begin{aligned} \partial_{t}u^{\nu,heat}=\nu\Delta u^{\nu,heat},\\\
u^{\nu,heat}(t=0)=u^{\nu}_{0}\end{aligned}\right.$ (1.3)

in $(t,x)\in\mathbb{R}_{+}\times\mathbb{T}^{3}$.

We now state our main result.

######  Theorem 1.1.

For any $\alpha\in(0,3/4)$, there is a sequence of smooth initial data
$\\{u_{0}^{\nu_{n}}\\}_{n\geq 1}$ on $\mathbb{T}^{3}$ with $\nu_{n}=2^{-2n}$
and $\|u_{0}^{\nu_{n}}\|_{L^{2}}=1$ such that, for the solution $u^{\nu_{n}}$
to the 3D incompressible Navier-Stokes equations (1.1) with zero external
force, we have

$\liminf_{n\rightarrow\infty}\nu_{n}\int_{0}^{\nu_{n}^{2\alpha/3}}\|\nabla
u^{\nu_{n}}(t)\|_{L^{2}}^{2}dt\geq c$ (1.4)

for some absolute constant $c>0$, whereas for the solution to the heat
equation (1.3), we have

$\nu_{n}\int_{0}^{\nu_{n}^{2\alpha/3}}\|\nabla
u^{\nu_{n},heat}(t)\|_{L^{2}}^{2}dt\leq
C\nu_{n}^{2\alpha/3}(1+\nu_{n}^{1-2\max\\{1/2,\alpha\\}}),$ (1.5)

with some absolute constant $C>0$, so that the right hand side vanishes as
$n\rightarrow\infty$.

Recalling the energy balance (1.2), (1.4) implies that at least a fixed
portion of the initial $L^{2}$ energy for $u^{\nu_{n}}$ is lost in the
$O(\nu_{n}^{2\alpha/3})$-timescale. Moreover, the comparison between (1.4) and
(1.5) simply tells us that viscosity alone is not enough to obtain such an
enhanced dissipation.

######  Remark 1.2.

With a different construction, the same authors considered the zeroth law
under the $2+\frac{1}{2}$-dimensional Navier-Stokes flow: small-scale
horizontal vortex blob being stretched by large-scale, anti-parallel pairs of
vertical vortex tubes [3]. It is inspired by the direct numerical simulations
of the 3D Navier-Stokes equations by Goto, Saito and Kawahara [2]. They have
found that sustained turbulence in a periodic box consists of a hierarchy of
antiparallel pairs of vortex tubes.

######  Remark 1.3.

In this paper, we set a sequence of initial data tending to highly oscillating
two-dimensional shear flow, adding oscillating vorticity to the vertical
direction. Let us mention a recent numerical simulation which have inspired
this construction of the initial data. Recently, using direct numerical
simulations of the 3D Navier-Stokes equations, Motoori and Goto [5] considered
the generation mechanism of turbulent boundary layer. They found that small-
scale vortices in the log layer are generated predominantly by the stretching
in a strain-rate field at larger scale rather than by the mean-flow
stretching. On the other hand, large-scale vortices, namely, vortices as large
as the height of the $99\%$ boundary layer thickness are stretched and
amplified directly by the mean flow (we call “qusi-streamwise vortices”). In
particular large-scale vortices tend to align with the stretching direction of
the mean flow, which is inclined at nearly $\pi/4$ from the streamwise
direction. We are inspired by the generation mechanism of this quasi-
streamwise vortices.

Notations. We write $A\lesssim B$ if there exists some constant $C>0$
independent of $\nu$ such that $A\leq CB$. Then, we say $A\approx B$ if
$A\lesssim B$ and $B\lesssim A$. Finally we write $A\simeq B$ if
$A/B\rightarrow 1$ as $\nu\rightarrow 0$. We use the notation
$\langle\cdot,\cdot\rangle$ for the standard $L^{2}$ inner product:

$\begin{split}\langle
f,g\rangle=\int_{\mathbb{T}^{3}}f(x)g(x)\,dx.\end{split}$

Here $f,g$ are real-valued functions. We also have $\|f\|_{L^{2}}^{2}=\langle
f,f\rangle$.

##  2 Proof of the main theorem

We consider solutions of the following form

$\begin{split}u^{\nu}(t,x_{1},x_{2},x_{3})=u^{L,\nu}(t,x_{2})+u^{S,\nu}(t,x_{1},x_{2})\end{split}$

where $u^{L,\nu}$ and $u^{S,\nu}$ only have nontrivial first and third
components, respectively. Furthermore, the solution depends only on $x_{1}$
and $x_{2}$. It is well known that these assumptions (so-called
$2+\frac{1}{2}$ dimensional flow) propagate by the Navier-Stokes equations
(cf. [4]). With some abuse of notation, we shall identify $u^{L,\nu}$ and
$u^{S,\nu}$ with their single nontrivial component, and drop the dependence of
the solution on $\nu$. Then, it is not difficult to see that the 3D Navier-
Stokes equations reduce to two scalar equations

$\begin{split}\partial_{t}u^{L}=\nu\partial_{x_{2}}^{2}u^{L}\end{split}$

and

$\begin{split}\partial_{t}u^{S}+u^{L}\partial_{x_{1}}u^{S}=\nu(\partial_{x_{1}}^{2}+\partial_{x_{2}}^{2})u^{S}.\end{split}$

###  2.1 Choice of $u^{L}$

We take $u_{0}^{L}$ to be a rescaling of smoothed triangular wave

$\begin{split}u_{0}^{L}(x_{2})=T(\nu^{-\frac{1}{2}}x_{2}).\end{split}$ (2.1)

To be precise, we take $T$ to be a $C^{\infty}(\mathbb{T})$-smooth function
satisfying

$T(z)=z\quad\text{for}\quad z\in\left[-\frac{1}{4},\frac{1}{4}\right],$

and $T(z)=-T(1/2+z)$. It is not difficult to see that

$\begin{split}\|u_{0}^{L}\|_{L^{2}}\approx 1.\end{split}$

We may expand $T$ in sine series: we have

$\begin{split}T(z)=\sum_{i\geq 1}a_{i}\sin(2(2i-1)\pi z),\end{split}$

where $\\{a_{i}\\}_{i}$ satisfies $|a_{i}|\lesssim_{\delta}|i|^{-\delta}$ for
any $\delta\geq 0$. Here, it is assumed that $0<\nu\leq 1$ is given in a way
that $\nu^{-\frac{1}{2}}$ is an integer. Then, $u^{L}(t,x_{2})$ is given by
the solution to the one dimensional heat equation with initial data
$u_{0}^{L}$:

$\begin{split}\partial_{t}u^{L}=\nu\partial_{x_{2}}^{2}u^{L}.\end{split}$

From the explicit formula for the initial data in sine series, we have that
the solution is

$\begin{split}u^{L}(t,x_{2})=\sum_{i\geq
1}a_{i}\sin\left(2(2i-1)\pi\nu^{-\frac{1}{2}}x_{2}\right)\exp\left(-{4\pi^{2}}(2i-1)^{2}t\right).\end{split}$

We shall compare it with

$\begin{split}\overline{u}^{L}(t,x_{2}):=u_{0}^{L}(x_{2})\exp\left(-{4\pi^{2}}t\right).\end{split}$

Then by direct computation (using $\\{a_{i}\\}_{i}$ is rapidly decaying) we
see that

$\begin{split}\|u^{L}(t)-\overline{u}^{L}(t)\|_{L^{\infty}}\leq
Ct,\end{split}$ (2.2)

for $t\geq 0$ with $C>0$ independent of $\nu>0$. Indeed,

$\begin{split}&u^{L}(t,x_{2})-\overline{u}^{L}(t,x_{2})=\\\ &\sum_{i\geq
2}a_{i}\sin\left(2(2i-1)\pi\nu^{-\frac{1}{2}}x_{2}\right)\left(\exp\left(-{4\pi^{2}}(2i-1)^{2}t\right)-\exp\left(-{4\pi^{2}}t\right)\right)\end{split}$

so that

$\begin{split}|u^{L}(t,x_{2})-\overline{u}^{L}(t,x_{2})|\leq C\sum_{i\geq
2}|a_{i}|((2i-1)^{2}-1)t\leq Ct.\end{split}$

for any $x_{2}$. The advantage of $\bar{u}^{L}$ over $u^{L}$ is that
$\bar{u}^{L}$ is exactly linear in some region of $\mathbb{T}$, which makes it
easy to analyze the corresponding transport operator.

######  Remark 2.1.

We see that

$\|\nabla u^{L,heat}(t)\|_{L^{2}}^{2}\lesssim\|\nabla
u_{0}^{L}\|_{L^{2}}^{2}\lesssim\nu^{-1}.$

Thus

$\nu\int_{0}^{{\nu^{2\alpha/3}}}\|\nabla
u^{L,heat}(t)\|_{L^{2}}^{2}dt\lesssim{\nu^{2\alpha/3}}.$

###  2.2 Choice of $u^{S}$

For $\alpha\in(0,3/4)$, we assume for simplicity that $\nu^{-\alpha}$ is an
integer (otherwise, we can simply replace $\nu^{-\alpha}$ with its integer
part in the following argument). Then we set

$\begin{split}u^{S}_{0}(x_{1},x_{2})=\sin(\nu^{-\alpha}x_{1})\phi(\nu^{-\frac{1}{2}}x_{2}).\end{split}$
(2.3)

Here $\phi\geq 0$ is a smooth function supported in $(0,1/4)$ with $\phi=1$ in
$(1/8,3/16)$. Note that

$\begin{split}\|u_{0}^{S}\|_{L^{2}}\approx 1.\end{split}$

######  Remark 2.2.

We see that

$\|\nabla u^{S,heat}(t)\|_{L^{2}}^{2}\lesssim\|\nabla
u^{S}_{0}\|_{L^{2}}^{2}\lesssim\nu^{-2\max\\{1/2,\alpha\\}}.$

Thus

$\nu\int_{0}^{{\nu^{2\alpha/3}}}\|\nabla
u^{S,heat}(t)\|_{L^{2}}^{2}dt\lesssim\nu^{1+2\alpha/3-2\max\\{1/2,\alpha\\}}.$

We first consider the linear advection-diffusion equation by rescaled and
smoothed triangular wave, possibly with an additional term $f$:

$\left\\{\begin{aligned}
\partial_{t}\overline{u}^{S}+\overline{u}^{L}\cdot\nabla\overline{u}^{S}=\nu\Delta\overline{u}^{S}+f,\\\
\overline{u}^{S}(t=0)=u^{S}_{0}.\end{aligned}\right.$ (2.4)

Explicitly, the transport operator is given by

$\begin{split}\overline{u}^{L}\cdot\nabla=e^{-c_{0}t}T(\nu^{-\frac{1}{2}}x_{2})\partial_{x_{1}},\quad
c_{0}=4\pi^{2}.\end{split}$

Let us consider the ansatz

$\begin{split}\overline{u}^{S}(t,x_{1},x_{2})=e^{-a(t)}\sin\left(\nu^{-\alpha}(x_{1}-\frac{1-e^{-c_{0}t}}{c_{0}}T(\nu^{-\frac{1}{2}}x_{2}))\right)\phi(\nu^{-\frac{1}{2}}x_{2}).\end{split}$
(2.5)

Here, $a(t)$ is a continuous function of time which will be determined below.
We shall now see that under an appropriate choice of $a(t)$, (2.5) provides an
approximate solution to (2.4) with $f=0$. Let us compute the error. Taking a
time derivative to (2.5),

$\begin{split}&\partial_{t}\overline{u}^{S}=-a^{\prime}(t)\overline{u}^{S}\\\
&-e^{-a(t)-c_{0}t}\nu^{-\alpha}T(\nu^{-\frac{1}{2}}x_{2})\cos\left(\nu^{-\alpha}(x_{1}-\frac{1-e^{-c_{0}t}}{c_{0}}T(\nu^{-\frac{1}{2}}x_{2}))\right)\phi(\nu^{-\frac{1}{2}}x_{2})\end{split}$

so that

$\begin{split}\partial_{t}\overline{u}^{S}+\overline{u}^{L}\cdot\nabla\overline{u}^{S}=-a^{\prime}(t)\overline{u}^{S}.\end{split}$

On the other hand, applying spatial derivatives to (2.5), we have

$\begin{split}\partial_{x_{1}x_{1}}\overline{u}^{S}=-\nu^{-2\alpha}\overline{u}^{S},\end{split}$
$\begin{split}\partial_{x_{2}}\overline{u}^{S}&=e^{-a(t)}\cos\left(\nu^{-\alpha}(x_{1}-\frac{1-e^{-c_{0}t}}{c_{0}}T(\nu^{-\frac{1}{2}}x_{2}))\right)\times\\\
&\ \ \ \ \
\left(-\frac{1-e^{-c_{0}t}}{c_{0}}\nu^{-\frac{1}{2}}\nu^{-\alpha}T^{\prime}(\nu^{-\frac{1}{2}}x_{2})\right)\phi(\nu^{-\frac{1}{2}}x_{2})\\\
&\qquad+e^{-a(t)}\sin\left(\nu^{-\alpha}(x_{1}-\frac{1-e^{-c_{0}t}}{c_{0}}T(\nu^{-\frac{1}{2}}x_{2}))\right)\nu^{-\frac{1}{2}}\phi^{\prime}(\nu^{-\frac{1}{2}}x_{2})\end{split}$

and

$\begin{split}\partial_{x_{2}x_{2}}\overline{u}^{S}&=-\overline{u}^{S}\times\left(-\frac{1-e^{-c_{0}t}}{c_{0}}\nu^{-\frac{1}{2}}\nu^{-\alpha}T^{\prime}(\nu^{-\frac{1}{2}}x_{2})\right)^{2}\\\
&\qquad+e^{-a(t)}\cos\left(\nu^{-\alpha}(x_{1}-\frac{1-e^{-c_{0}t}}{c_{0}}T(\nu^{-\frac{1}{2}}x_{2}))\right)\times\\\
&\qquad\ \ \ \
\left(-\frac{1-e^{-c_{0}t}}{c_{0}}\nu^{-1}\nu^{-\alpha}T^{\prime\prime}(\nu^{-\frac{1}{2}}x_{2})\right)\phi(\nu^{-\frac{1}{2}}x_{2})\\\
&\qquad+2e^{-a(t)}\cos\left(\nu^{-\alpha}(x_{1}-\frac{1-e^{-c_{0}t}}{c_{0}}T(\nu^{-\frac{1}{2}}x_{2}))\right)\times\\\
&\qquad\ \ \ \ \
\left(-\frac{1-e^{-c_{0}t}}{c_{0}}\nu^{-\frac{1}{2}}\nu^{-\alpha}T^{\prime}(\nu^{-\frac{1}{2}}x_{2})\right)\nu^{-\frac{1}{2}}\phi^{\prime}(\nu^{-\frac{1}{2}}x_{2})\\\
&\qquad+e^{-a(t)}\sin\left(\nu^{-\alpha}(x_{1}-\frac{1-e^{-c_{0}t}}{c_{0}}T(\nu^{-\frac{1}{2}}x_{2}))\right)\nu^{-1}\phi^{\prime\prime}(\nu^{-\frac{1}{2}}x_{2}).\end{split}$

Recalling that

$\begin{split}T^{\prime}(z)=1\quad\text{for}\quad
z\in\left[-\frac{1}{4},\frac{1}{4}\right],\end{split}$

we have that $T^{\prime}=1$ and $T^{\prime\prime}=0$ on the support of $\phi$.
With these observations, $\partial_{x_{2}x_{2}}\overline{u}^{S}$ is simply

$\begin{split}\partial_{x_{2}x_{2}}\overline{u}^{S}&=-\overline{u}^{S}\frac{(1-e^{-c_{0}t})^{2}}{c_{0}^{2}}\nu^{-1-2\alpha}\\\
&\qquad-2\nu^{-\alpha-1}e^{-a(t)}\cos\left(\nu^{-\alpha}(x_{1}-\frac{1-e^{-c_{0}t}}{c_{0}}T(\nu^{-\frac{1}{2}}x_{2}))\right)\times\\\
&\qquad\qquad\frac{1-e^{-c_{0}t}}{c_{0}}\phi^{\prime}(\nu^{-\frac{1}{2}}x_{2})\\\
&\qquad+\nu^{-1}e^{-a(t)}\sin\left(\nu^{-\alpha}(x_{1}-\frac{1-e^{-c_{0}t}}{c_{0}}T(\nu^{-\frac{1}{2}}x_{2}))\right)\phi^{\prime\prime}(\nu^{-\frac{1}{2}}x_{2}).\end{split}$

Therefore,

$\begin{split}&\Delta\overline{u}^{S}=(\partial_{x_{1}x_{1}}+\partial_{x_{2}x_{2}})\overline{u}^{S}=-\left(\nu^{-2\alpha}+\frac{(1-e^{-c_{0}t})^{2}}{c_{0}^{2}}\nu^{-1-2\alpha}\right)\overline{u}^{S}+g,\end{split}$

where

$\begin{split}g&=-2\nu^{-\alpha-1}e^{-a(t)}\cos\left(\nu^{-\alpha}(x_{1}-\frac{1-e^{-c_{0}t}}{c_{0}}T(\nu^{-\frac{1}{2}}x_{2}))\right)\times\\\
&\qquad\qquad\frac{1-e^{-c_{0}t}}{c_{0}}\phi^{\prime}(\nu^{-\frac{1}{2}}x_{2})\\\
&\qquad+\nu^{-1}e^{-a(t)}\sin\left(\nu^{-\alpha}(x_{1}-\frac{1-e^{-c_{0}t}}{c_{0}}T(\nu^{-\frac{1}{2}}x_{2}))\right)\phi^{\prime\prime}(\nu^{-\frac{1}{2}}x_{2}).\end{split}$

Hence, defining $a(t)$ to be the solution to the ODE

$\begin{split}a^{\prime}(t)&=\nu^{1-2\alpha}+\frac{(1-e^{-c_{0}t})^{2}}{c_{0}^{2}}\nu^{-2\alpha},\\\
a(0)&=1,\end{split}$

we have that (2.5) is a solution to

$\begin{split}\partial_{t}\overline{u}^{S}+\overline{u}^{L}\cdot\nabla\overline{u}^{S}=\nu\Delta\overline{u}^{S}-\nu
g.\end{split}$ (2.6)

######  Remark 2.3.

Observe that for $O(1)$-small $t>0$, we have

$\begin{split}a^{\prime}(t)\simeq\nu^{1-2\alpha}+t^{2}\nu^{-2\alpha}\end{split}$

and hence

$\begin{split}a(t)\simeq
1+\nu^{1-2\alpha}t+\frac{t^{3}}{3}\nu^{-2\alpha}.\end{split}$

This shows that for $\overline{u}^{S}$, loss of energy occurs within an
$O(\nu^{\frac{2}{3}\alpha})$-timescale. Here, it is required that
$\nu^{1-2\alpha}t\lesssim 1$ for $t=\nu^{\frac{2}{3}\alpha}$, which is
equivalent to $\alpha<\frac{3}{4}$. Indeed, when $0<\alpha<\frac{3}{4}$, we
have that $\exp(-a(t))\gtrsim 1$ for $0\leq t\leq\nu^{2\alpha/3}$, and a
direct computation gives

$\nu\int_{0}^{{\nu^{2\alpha/3}}}\|\partial_{x_{2}}\bar{u}^{S}(t)\|_{L^{2}}^{2}dt\gtrsim\nu\int_{0}^{{\nu^{2\alpha/3}}}t^{2}\nu^{-1-2\alpha}dt\approx
1.$ (2.7)

###  2.3 Difference estimate

We consider the solution of

$\begin{split}\partial_{t}u^{S}+u^{L}\cdot\nabla u^{S}=\nu\Delta
u^{S}\end{split}$ (2.8)

with initial data
$u^{S}_{0}(x_{1},x_{2})=\sin(\nu^{-\alpha}x_{1})\phi(\nu^{-\frac{1}{2}}x_{2})=\overline{u}^{S}_{0}(x_{1},x_{2})$
and compare it with $\overline{u}^{S}$ from (2.5): the difference
$\boldsymbol{\epsilon}=\overline{u}^{S}-u^{S}$ satisfies the equation

$\begin{split}\partial_{t}\boldsymbol{\epsilon}+\overline{u}^{L}\cdot\nabla\boldsymbol{\epsilon}+(u^{L}-\overline{u}^{L})\cdot\nabla
u^{S}=\nu\Delta\boldsymbol{\epsilon}-\nu g.\end{split}$ (2.9)

Taking the $L^{2}$ inner product with $\boldsymbol{\epsilon}$,

$\begin{split}\frac{1}{2}\frac{d}{dt}\|\boldsymbol{\epsilon}\|_{L^{2}}^{2}+\nu\|\nabla\boldsymbol{\epsilon}\|_{L^{2}}^{2}=-\langle\nu
g,\boldsymbol{\epsilon}\rangle-\langle(u^{L}-\overline{u}^{L})\cdot\nabla
u^{S},\boldsymbol{\epsilon}\rangle\end{split}$

We easily estimate

$\begin{split}\left|\langle(u^{L}-\overline{u}^{L})\cdot\nabla
u^{S},\boldsymbol{\epsilon}\rangle\right|&\lesssim\|u^{L}-\overline{u}^{L}\|_{L^{\infty}}\|\partial_{x_{1}}u^{S}\|_{L^{2}}\|\boldsymbol{\epsilon}\|_{L^{2}}\\\
&\lesssim
t\|\partial_{x_{1}}u^{S}_{0}\|_{L^{2}}\|\boldsymbol{\epsilon}\|_{L^{2}}\\\
&\lesssim t\nu^{-\alpha}\|\boldsymbol{\epsilon}\|_{L^{2}}.\end{split}$

Moreover, observing that $\|\nu g\|_{L^{2}}\lesssim 1+t\nu^{-\alpha}$, we have

$\begin{split}\left|\frac{1}{2}\frac{d}{dt}\|\boldsymbol{\epsilon}\|_{L^{2}}^{2}+\nu\|\nabla\boldsymbol{\epsilon}\|_{L^{2}}^{2}\right|\lesssim(1+t\nu^{-\alpha})\|\boldsymbol{\epsilon}\|_{L^{2}}.\end{split}$
(2.10)

Therefore,

$\begin{split}\frac{d}{dt}\|\boldsymbol{\epsilon}\|_{L^{2}}\lesssim
1+t\nu^{-\alpha},\end{split}$

and

$\begin{split}\|\boldsymbol{\epsilon}(t)\|_{L^{2}}\lesssim
t+t^{2}\nu^{-\alpha}.\end{split}$ (2.11)

Finally, we have after integrating (2.10) in $t\in[0,{\nu^{2\alpha/3}}]$ and
applying (2.11),

$\begin{split}\nu\int_{0}^{{\nu^{2\alpha/3}}}\|\nabla\boldsymbol{\epsilon}(t)\|_{L^{2}}^{2}dt&\lesssim\int_{0}^{{\nu^{2\alpha/3}}}\left|\langle\nu
g,\boldsymbol{\epsilon}\rangle\right|+\left|\langle(u^{L}-\overline{u}^{L})\cdot\nabla
u^{S},\boldsymbol{\epsilon}\rangle\right|dt\\\
&\lesssim\int_{0}^{{\nu^{2\alpha/3}}}t(1+t\nu^{-\alpha})^{2}dt\lesssim\nu^{4\alpha/3}+\nu^{2\alpha/3}.\end{split}$
(2.12)

###  2.4 Completion of the proof

We are now in a position to complete the proof. Let us briefly note that
strictly speaking, we only have $\|u^{\nu}\|_{L^{2}}\approx 1$ while the
statement of the Theorem requires $\|u^{\nu}\|_{L^{2}}=1$. This can be fixed
with a simple rescaling of $u^{\nu}$.

By (2.7) from Remark 2.3 and (2.12), we have

$\begin{split}\nu\int_{0}^{\nu^{2\alpha/3}}\|\nabla
u^{\nu}(t)\|_{L^{2}}^{2}dt&\geq\nu\int_{0}^{\nu^{2\alpha/3}}\|\nabla
u^{S,\nu}(t)\|_{L^{2}}^{2}dt\\\
&\geq\nu\int_{0}^{\nu^{2\alpha/3}}\|\partial_{x_{2}}u^{S,\nu}(t)\|_{L^{2}}^{2}dt\\\
&\gtrsim\nu\int_{0}^{\nu^{2\alpha/3}}\|\partial_{x_{2}}\bar{u}^{S,\nu}(t)\|_{L^{2}}^{2}dt-\nu\int_{0}^{\nu^{2\alpha/3}}\|\partial_{x_{2}}\boldsymbol{\epsilon}(t)\|_{L^{2}}^{2}dt\\\
&\gtrsim 1-{\nu^{2\alpha/3}}.\end{split}$

This gives the enhanced dissipation statement (1.4). For the heat flow case,
combining Remark 2.1 and Remark 2.2, we immediately obtain the desired
estimate.

### Acknowledgements

Research of TY was partially supported by Grant-in-Aid for Scientific Research
B (17H02860, 18H01136, 18H01135 and 20H01819), Japan Society for the Promotion
of Science (JSPS). IJ has been supported by a KIAS Individual Grant MG066202
at Korea Institute for Advanced Study, the Science Fellowship of POSCO TJ Park
Foundation, and the National Research Foundation of Korea grant No.
2019R1F1A1058486.

## References

  * [1] U. Frisch, Turbulence, Cambridge University Press, Cambridge 1995.
  * [2] S. Goto, Y. Saito, and G. Kawahara, Hierarchy of antiparallel vortex tubes in spatially periodic turbulence at high Reynolds numbers, Phys. Rev. Fluids 2 (2017), 064603.
  * [3] I.-J. Jeong and T. Yoneda, Vortex stretching and enhanced dissipation for the incompressible 3D Navier-Stokes equations, to appear in Math. Annal.
  * [4] A. Majda and A. Bertozzi, Vorticity and Incompressible Flow, Cambridge University Press, Cambridge 2002.
  * [5] Y. Motoori and S. Goto _Generation mechanism of a hierarchy of vortices in a turbulent boundary layer._ J. Fluid Mech., 865 (2019), 1085-1109.
  * [6] J. Vassilicos, _Dissipation in turbulent flows._ Annual Review of Fluid Mechanics 47:95–114, 2015.


Convolution-Free Medical Image Segmentation
using Transformers

Davood Karimi, Serge Vasylechko , and Ali Gholipour

Department of Radiology, Boston Children’s Hospital, Harvard Medical School,
Boston, MA, USA

###### Abstract

Like other applications in computer vision, medical image segmentation has
been most successfully addressed using deep learning models that rely on the
convolution operation as their main building block. Convolutions enjoy
important properties such as sparse interactions, weight sharing, and
translation equivariance. These properties give convolutional neural networks
(CNNs) a strong and useful inductive bias for vision tasks. In this work we
show that a different method, based entirely on self-attention between
neighboring image patches and without any convolution operations, can achieve
competitive or better results. Given a 3D image block, our network divides it
into $n^{3}$ 3D patches, where $n=3\text{ or }5$ and computes a 1D embedding
for each patch. The network predicts the segmentation map for the center patch
of the block based on the self-attention between these patch embeddings. We
show that the proposed model can achieve segmentation accuracies that are
better than the state of the art CNNs on three datasets. We also propose
methods for pre-training this model on large corpora of unlabeled images. Our
experiments show that with pre-training the advantage of our proposed network
over CNNs can be significant when labeled training data is small.

Index Terms:  Medical image segmentation, deep learning, transformers,
attention

Figure 1: Proposed convolution-free network for 3D medical image segmentation.

Figure 2: Example segmentations predicted by the proposed network and a state
of the art CNN.

##  I Introduction

Image segmentation is a central task in medical image analysis. It is
routinely used for quantifying the size and shape of the volume/organ of
interest, population studies, disease quantification, treatment planning, and
computer-aided intervention. In most medical applications, manual segmentation
by an expert is regarded as the gold standard. However, automatic methods
promise much faster, cheaper, and more reproducible segmentations.

Classical methods in medical image segmentation run the gamut from region
growing [11] and deformable models [36] to atlas-based methods [32], Bayesian
approaches [29], graph cuts [26], clustering methods [12], and more. In the
past few years, deep learning (DL) methods have emerged as a highly
competitive alternative and they have achieved remarkable levels of accuracy
[3] [5] [18] [20] [38] [37] [19]. It appears that DL methods have largely
replaced the classical methods for medical image segmentation.

Due to their success, DL methods for medical image segmentation have attracted
much attention from the research community. Recent surveys of the published
research on this topic can be found in [14] [31]. Some of the main directions
of these studies include improving network architectures, loss functions, and
training strategies. Surprisingly, the one common feature in all of these
works is the use of the convolution operation as the main building block of
the networks. The proposed network architectures differ in terms of the way
the convolutional operations are arranged, but they all build upon the same
convolution operation. There have been several attempts to use recurrent
networks [10] [2] and attention mechanisms [6] for medical image segmentation.
However, all of those models still rely on the convolution operation. Some
recent studies have gone so far as to suggest that in practice most of these
architectures achieve largely similar results [17].

Similar to semantic segmentation, other computer vision tasks such as image
classification and object detection have also been most successfully addressed
with convolutional neural networks (CNNs) [30], [23]. These observations
attest to the importance of the convolution operation to image understanding
and modeling. The effectiveness of the convolution operation can be attributed
to a number of key properties, including: 1) local (sparse) connections, 2)
parameter (weight) sharing, and 3) translation equivariance [25] [24]. These
properties are in large part inspired by the neuroscience of vision [28], and
give CNNs a very strong inductive bias. In fact, a convolutional layer can be
interpreted as a fully connected layer with an “inﬁnitely strong prior” over
its weights [13]. As a result, modern deep CNNs have been able to tackle a
variety of vision tasks with amazing success.

In other important applications, most prominently in natural language
processing, the dominant architectures were those based on recurrent neural
networks (RNNs) [7], [16]. The dominance of RNNs ended with the introduction
of the transformer model, which proposed to replace the recurrence with an
attention mechanism that could learn the relationship between different parts
of a sequence as a whole [34].

The attention mechanism has had a profound impact on the field of natural
language processing. In vision applications, on the other hand, its impact has
been much more limited. A recent survey of applications of transformer
networks in computer vision can be found in [21]. The number of pixels in
typical images is much larger than the number of units of data (e.g., words)
in natural language processing applications. This makes it impossible to apply
standard attention models to images. As a result, despite numerous efforts
(e.g., [35] [15] [9]) attention mechanism has not yet resulted in the type of
sea change that has occurred in natural language processing. The recently-
proposed vision transformer (ViT) appears to be a major step towards adapting
transformer/attention models for computer vision applications [8]. The main
insight in that work is to consider image patches, rather than pixels, to be
the units of information in images. ViT embeds image patches into a shared
space and learns the relation between these embeddings using self-attention
modules. Given massive amounts of training data and computational resources,
ViT was shown to surpass CNNs in image classification accuracy. Subsequent
work has shown that by using knowledge distillation from a CNN teacher and
using standard training data and computational resources, transformer networks
can achieve image classification accuracy levels on par with CNNs [33].

The goal of this work is to explore the potential of self attention-based deep
neural networks for 3D medical image segmentation. We propose a network
architecture that is based on self-attention between linear embeddings of 3D
image patches, without any convolution operations. We train the network on
three medical image segmentation datasets and compare its performance with a
state of the art CNN. The contributions of this work are as follows:

  1. 1.

We propose the first convolution-free deep neural network for medical image
segmentation.

  2. 2.

We show that our proposed network can achieve segmentation accuracies that are
better than or at least on par with a state of the art CNN on three different
medical image segmentation datasets. We show that, unlike recent works on
image classification ([8] [33]), our network can be effectively trained for 3D
medical image segmentation with datasets of $\sim 20-200$ labeled images.

  3. 3.

We propose pre-training methods that can improve our network’s segmentation
accuracy when large corpora of unlabeled training images are available. We
show that when labeled training images are fewer in number, our network
performs better than a state of the art CNN with pre-training.

##  II Materials and methods

###  II-A Proposed network

Figure 1 shows our proposed network for convolution-free medical image
segmentation. The input to the network is a 3D block $B\in{\rm I\\!R}^{W\times
W\times W\times c}$, where $W$ denotes the extent of the block (in voxels) in
each dimension and $c$ is the number of image channels. The block $B$ is
partitioned into $n^{3}$ contiguous non-overlapping 3D patches
$\\{p_{i}\in{\rm I\\!R}^{w\times w\times w\times c}\\}_{i=1}^{N}$, where
$w=W/n$ is the side length of each patch and $N=n^{3}$ denotes the number of
patches in the block. We choose $n$ to be an odd number. In the experiments
presented in this paper $n$ is either 3 or 5, corresponding to 27 or 125
patches, respectively. The network uses the image information in all $N$
patches in $B$ to predict the segmentation for the center patch, as described
below.

Each of the $N$ patches is flattened into a vector of size ${\rm
I\\!R}^{w^{3}c}$ and embedded into ${\rm I\\!R}^{D}$ using a trainable mapping
$E\in{\rm I\\!R}^{D\times w^{3}c}$. Unlike [8], we do not use any additional
tokens since that did not improve the segmentation performance of our network
in any way. The sequence of embedded patches
$X^{0}=[Ep_{1};...;Ep_{N}]+E_{\text{pos}}$ constitutes the input to our
network. The matrix $E_{\text{pos}}\in{\rm I\\!R}^{D\times N}$ is intended to
learn a positional encoding. Without such positional information, the
transformer ignores the ordering of the input sequence because the attention
mechanism is permutation-invariant. Therefore, in most NLP applications such
embedding has been very important. For image classification, on the other
hand, authors of [8] found that positional encoding resulted in relatively
small improvements in accuracy and that simple 1D raster encoding was as good
as more elaborate 2D positional encodings. For the experiments presented in
this paper, we left the positional encoding as a free parameter to be learned
along with the network parameters during training because we do not know a
priori what type of positional encoding would be useful. We discuss the
results of experimental comparisons with different positional encodings below.

The encoder has $K$ stages, each consisting of a multi-head self-attention
(MSA) and a subsequent two-layer fully connected feed-forward network (FFN).
Both the MSA and FFN modules include residual connections, ReLU activations,
and layer normalization. More specifically, starting with the sequence of
embedded and position-encoded patches, $X^{0}$ described above, the
$k^{\text{th}}$ stage of the encoder will perform the following operations to
map $X^{k}$ to $X^{k+1}$:

  1. 1.

$X^{k}$ goes through $n_{h}$ separate heads in MSA. The $i^{\text{th}}$ head:

    1. (a)

Computes the query, key, and value sequences from the input sequence:

$Q^{k,i}=E_{Q}^{k,i}X^{k},K^{k,i}=E_{K}^{k,i}X^{k},V^{k,i}=E_{V}^{k,i}X^{k}$

$\text{where }E_{Q},E_{K},E_{v}\in{\rm I\\!R}^{D_{h}\times D}$

    2. (b)

Computes the self-attention matrix and then the transformed values:

$A^{k,i}=\text{Softmax}(Q^{T}K)/\sqrt{D_{h}}$

$\text{SA}^{k,i}=A^{k,i}V^{k,i}$

  2. 2.

Outputs of the $n_{h}$ heads are stacked and reprojected back onto ${\rm
I\\!R}^{D}$

$\text{MSA}^{k}=E_{\text{reproj}}^{k}[\text{SA}^{k,0};...;\text{SA}^{k,n_{h}}]^{T}$

$\text{where }E_{\text{reproj}}\in{\rm I\\!R}^{D\times D_{h}n_{h}}$

  3. 3.

The output of the MSA module is computed as:

$X_{\text{MSA}}^{k}=\text{MSA}^{k}+X^{k}$

  4. 4.

$X_{\text{MSA}}^{k}$ goes through a two-layer FFN to obtain the output of the
$k^{\text{th}}$ encoder stage:

$X^{k+1}=X_{\text{MSA}}^{k}+E_{2}^{k}\Big{(}\text{ReLU}((E_{1}^{k}X_{\text{MSA}}^{k}+b_{1}^{k})\Big{)}+b_{2}^{k}$

The output of the last encoder stage, $X^{K}$, will pass through a final FFN
layer that projects it onto the ${\rm I\\!R}^{Nn_{\text{class}}}$ and then
reshaped into ${\rm I\\!R}^{n\times n\times n\times n_{\text{class}}}$, where
$n_{\text{class}}$ is the number of classes (for binary segmentation,
$n_{\text{class}}=2$):

$\hat{Y}=\text{Softmax}\Big{(}E_{\text{out}}X^{K}+b_{\text{out}})\Big{)}.$

$\hat{Y}$ is the predicted segmentation for the center patch of the block.

###  II-B Implementation and training

The network was implemented in TensorFlow 1.16 and run on an NVIDIA GeForce
GTX 1080 GPU on a Linux machine with 120 GB of memory and 16 CPU cores. We
compare our model with 3D UNet++ [39], which is a state of the art CNN for
medical image segmentation.

We trained the network parameters to maximize the Dice similarity coefficient,
DSC [27], between $\hat{Y}$ and the ground-truth segmentation of the center
patch using Adam [22]. We used a batch size of 10 and a learning rate of
$10^{-4}$, which was reduced by half if after a training epoch the validation
loss did not decrease.

Pre-training: Manual segmentation of complex structures such as the brain
cortical plate can take several hours of a medical expert’s time for a single
image. Therefore, methods that can achieve high accuracy with fewer labeled
training images are highly advantageous. To improve the network’s performance
when labeled training images are insufficient, we propose to first pre-train
the network on unlabeled data for denoising or inpainting tasks. For denoising
pre-training, we add Gaussian noise with $\text{SNR}=10\text{ dB}$ to the
input block $B$, whereas for inpainting pre-training we set the center patch
of the block to constant 0. In both cases, the target is the noise-free center
patch of the block. For pre-training, we use a different output layer (without
the softmax operation) and train the network to minimize the $\ell_{2}$ norm
between the prediction and the target. To fine-tune the pre-trained network
for the segmentation task, we introduce a new output layer with softmax and
train the network on the labeled data as explained in the above paragraph. We
fine-tune the entire network, rather than only the output layer, on the
labeled images because We have found that fine-tuning the entire network for
the segmentation task leads to better results.

###  II-C Data

Table I shows the datasets used in this work. We used $\sim 1/5$ of the
training images for validation. The final models were evaluated on the test
images. The images were split into train/validation/test at random.

TABLE I: Datasets used for experiments in this work. target organ |  image modality |  $[n_{train},n_{test}]$
---|---|---
Brain cortical plate |  T2 MRI |  [18, 9]
Pancreas |  CT |  [231, 50]
Hippocampus |  MRI |  [220, 40]

##  III Results and Discussion

Table II presents test segmentation accuracy of the propsoed method and UNet++
in terms of DSC, the 95 percentile of the Hausdorff Distance (HD95) and
Average Symmetric Surface Distance (ASSD). In these experiment, we used these
parameter settings for our network: $K=7,W=24,n=3,D=1024,D_{h}=256,n_{h}=4$.
We used the same parameter settings for all experiments reported in the rest
of the paper, unless otherwise stated. For each dataset and each of the three
criteria, we ran paired t-tests to see if the differences were statistically
significant. As shown in the table, segmentation accuracy for the proposed
convolution-free method was significantly better than, or at least on part
with, UNet++. Figure 3 shows example slices from test images in each dataset
and the segmentations predicted by the proposed method and UNet++. We have
observed that the propsoed method is capable of accurately segmenting fine and
complex structures such as the brain cortical plate. In terms of training
time, our network converged in approximately 24 hours of GPU time.

TABLE II: Segmentation accuracy of the proposed method and UNet++. Better results for each dataset/criterion are in bold type. Asterisks denote statistically significant difference ($p<0.01$ in paired t-test.) Dataset |  Method |  DSC |  HD95 (mm) |  ASSD (mm)
---|---|---|---|---
Brain cortical plate |  Proposed |  $\bm{0.879\pm 0.026^{*}}$ |  $0.92\pm 0.04$ |  $\bm{0.24\pm 0.03}$
|  UNet++ |  $0.860\pm 0.024$ |  $\bm{0.91\pm 0.04}$ |  $0.25\pm 0.04$
Pancreas |  Proposed |  $\bm{0.826\pm 0.020^{*}}$ |  $\bm{5.72\pm 1.61^{*}}$ |  $\bm{2.13\pm 0.24^{*}}$
|  UNet++ |  $0.808\pm 0.021$ |  $6.67\pm 1.80$ |  $2.45\pm 0.21$
Hippocampus |  Proposed |  $\bm{0.881\pm 0.017^{*}}$ |  $\bm{1.10\pm 0.19}^{*}$ |  $\bm{0.40\pm 0.04}$
|  UNet++ |  $0.852\pm 0.022$ |  $1.33\pm 0.26$ |  $0.41\pm 0.07$
Figure 3: Example segmentations predicted by the proposed method and UNet++.

To assess the segmentation accuracy with reduced number of labeled training
images, we trained our method and UNet++ with $n_{\text{train}}=5,10,\text{
and }15$ labeled training images from cortical plate and pancreas datasets.
For cortical plate segmentation, we used 500 unlabeled images from the dHCP
dataset [4] for pre-training. For pancreas segmentation, we used the remaining
training images (i.e., $231-n_{\text{train}}$) for pre-training. We pre-
trained our model as explained in Section II-B. To pre-train UNet++, we used
the method propsoed in [1]. Figure 4 shows the results of this experiment.
With the propsoed pre-training, the convolution-free network achieves
significantly more accurate segmentations with fewer labeled training images.

Figure 4: Segmentation accuracy (in DSC) for the proposed network and UNet++
with reduced labeled training data on the cortical plate (left) and pancreas
(right) datasets.

Figure 5 shows example attention maps of the proposed network for pancreas
segmentation. As mentioned above, to process a test image of a desired size,
we apply our network in a sliding window fashion. To generate the attention
maps for the entire image, at each location of the sliding window the
attention matrices (which are of size ${\rm I\\!R}^{N\times N}$) are summed
along the column to determine the total attention paid to each of the $N$
patches by the other patches in the block. Performing this computation in a
sliding window fashion and computing the voxel-wise average gives us the
attention maps shown in these figures. They indicate how much attention is
paid to every part of the image.

Figure 5: Example attention maps for two pancreas images.

The attention maps shown in Figure 5 show that, in general, the early stages
of the network have a wider attention scope. They attend to other structures
and anatomical features that surround the organ of interest (here, the
pancreas). The deeper stages of the network are more focused on the pancreas
itself. A similar behaviour can be seen for cortical plate segmentation in
Figure 6. In the earlier stage the network attends to the entire brain, while
in the deeper layers the network’s attention is more focused to the regions
around the cortical plate. Another observation from these figures, especially
Figure 5 is the variability between the attention pattern in different heads
of the multi-head self-attention mechanism. In each stage, the four separate
heads of the MSA module adopt quite different attention patterns. This
indicates that the multi-head design of the MSA gives the network more
flexibility to learn the attention patterns that help improve the segmentation
accuracy. The importance of multi-head design is well documented in natural
language processing applications [34] and our results show it is important for
3D medical image segmentation as well, as we show further below.

Figure 6: Example attention maps for a cortical plate image.

Table III effects of some of the network design choices on the segmentation
accuracy on the pancreas dataset. In this table, the baseline (first row)
corresponds to the settings that we have used in the experiments reported
above, i.e., $K=7,W=24,n=3,D=1024,D_{h}=256,n_{h}=4$. We selected these
settings based on preliminary experiments and we have found them to be good
settings for different datasets. Increasing the number/size of the patches or
increasing the network depth typically leads to slight improvements in
accuracy. Furthermore, using a fixed positional encoding or no positional
encoding slightly reduces segmentation accuracy compared with free-
parameter/learnable positional encoding. Finally, using a single-head
attention significantly reduces the segmentation accuracy, which indicates the
importance of the multi-head design to enable the network to learn a more
complex relation between neighboring patches.

TABLE III: Effect of some of the network hyperparameters on the segmentation accuracy on the pancreas dataset. The baseline (first row) corresponds to these settings: $K=7,W=24,n=3,D=1024,D_{h}=256,n_{h}=4$, which are the hyperparameter values used in all experiments reported in this paper other than in this table.  Parameter settings |  DSC |  HD95 (mm) |  ASSD (mm)
---|---|---|---
baseline |  $0.826\pm 0.020$ |  $5.72\pm 1.61$ |  $2.13\pm 0.24$
larger blocks, $n=5$ |  $0.828\pm 0.023$ |  $5.68\pm 1.63$ |  $2.01\pm 0.18$
no positional encoding |  $0.818\pm 0.026$ |  $5.84\pm 1.74$ |  $2.26\pm 0.27$
fixed positional encoding |  $0.823\pm 0.021$ |  $5.81\pm 1.54$ |  $2.16\pm 0.23$
deeper network, $K=10$ |  $0.827\pm 0.018$ |  $5.50\pm 1.48$ |  $2.10\pm 0.21$
shallower network, $K=4$ |  $0.810\pm 0.023$ |  $6.14\pm 1.80$ |  $2.29\pm 0.38$
more heads, $n_{h}=8$ |  $0.824\pm 0.017$ |  $5.68\pm 1.56$ |  $2.14\pm 0.21$
single head, $n_{h}=1$ |  $0.802\pm 0.026$ |  $6.82\pm 1.40$ |  $2.31\pm 0.35$

##  IV Conclusions

The convolution operation has a strong basis in the structure of the mammalian
primary visual cortex and it is well suited for developing powerful techniques
for image modeling and image understanding. In recent years, CNNs have been
shown to be highly effective in tackling various computer vision problems.
However, there is no reason to believe that no other model can outperform CNNs
on a specific vision task. Medical image analysis applications, in particular,
pose specific challenges such as 3D nature of the images and small number of
labeled images. In such applications, other models can be more effective than
CNNs. In this work we presented a new model for 3D medical image segmentation.
Unlike all recent models that use convolutions as their main building block,
our model is based on self-attention between neighboring 3D patches. Our
results show that the proposed network can outperform a state of the art CNN
on three medical image segmentation datasets. With pre-training for denoising
and in-painting tasks on unlabeled images, our network also performed better
than a CNN when only 5-15 labeled training images were available. We expect
that the network proposed in this paper should be effective for other tasks in
medical image analysis such as anomaly detection and classification.

## References

  * [1] Bai, W., Oktay, O., Sinclair, M., Suzuki, H., Rajchl, M., Tarroni, G., Glocker, B., King, A., Matthews, P.M., Rueckert, D.: Semi-supervised learning for network-based cardiac mr image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 253–260. Springer (2017)
  * [2] Bai, W., Suzuki, H., Qin, C., Tarroni, G., Oktay, O., Matthews, P.M., Rueckert, D.: Recurrent neural networks for aortic image sequence segmentation with sparse annotations. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 586–594. Springer (2018)
  * [3] Bakas, S., Reyes, M., Jakab, A., Bauer, S., Rempfler, M., Crimi, A., Shinohara, R.T., Berger, C., Ha, S.M., Rozycki, M., et al.: Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the brats challenge. arXiv preprint arXiv:1811.02629 (2018)
  * [4] Bastiani, M., et al.: Automated processing pipeline for neonatal diffusion mri in the developing human connectome project. NeuroImage 185, 750–763 (2019)
  * [5] Bernard, O., Lalande, A., Zotti, C., Cervenansky, F., Yang, X., Heng, P.A., Cetin, I., Lekadir, K., Camara, O., Ballester, M.A.G., et al.: Deep learning techniques for automatic mri cardiac multi-structures segmentation and diagnosis: is the problem solved? IEEE transactions on medical imaging 37(11), 2514–2525 (2018)
  * [6] Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., Lu, L., Yuille, A.L., Zhou, Y.: Transunet: Transformers make strong encoders for medical image segmentation. arXiv preprint arXiv:2102.04306 (2021)
  * [7] Chung, J., Gulcehre, C., Cho, K., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555 (2014)
  * [8] Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)
  * [9] Dou, H., Karimi, D., Rollins, C.K., Ortinau, C.M., Vasung, L., Velasco-Annis, C., Ouaalam, A., Yang, X., Ni, D., Gholipour, A.: A deep attentive convolutional neural network for automatic cortical plate segmentation in fetal mri. arXiv preprint arXiv:2004.12847 (2020)
  * [10] Gao, Y., Phillips, J.M., Zheng, Y., Min, R., Fletcher, P.T., Gerig, G.: Fully convolutional structured lstm networks for joint 4d medical image segmentation. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018). pp. 1104–1108 (2018). https://doi.org/10.1109/ISBI.2018.8363764
  * [11] Gibbs, P., Buckley, D.L., Blackband, S.J., Horsman, A.: Tumour volume determination from mr images by morphological segmentation. Physics in Medicine & Biology 41(11), 2437 (1996)
  * [12] Goldszal, A.F., Davatzikos, C., Pham, D.L., Yan, M.X., Bryan, R.N., Resnick, S.M.: An image-processing system for qualitative and quantitative volumetric analysis of brain images. Journal of computer assisted tomography 22(5), 827–837 (1998)
  * [13] Goodfellow, I., Bengio, Y., Courville, A., Bengio, Y.: Deep learning, vol. 1. MIT press Cambridge (2016)
  * [14] Hesamian, M.H., Jia, W., He, X., Kennedy, P.: Deep learning techniques for medical image segmentation: achievements and challenges. Journal of digital imaging 32(4), 582–596 (2019)
  * [15] Ho, J., Kalchbrenner, N., Weissenborn, D., Salimans, T.: Axial attention in multidimensional transformers. arXiv preprint arXiv:1912.12180 (2019)
  * [16] Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural computation 9(8), 1735–1780 (1997)
  * [17] Isensee, F., Kickingereder, P., Wick, W., Bendszus, M., Maier-Hein, K.H.: No new-net. In: International MICCAI Brainlesion Workshop. pp. 234–244. Springer (2018)
  * [18] Kamnitsas, K., Ledig, C., Newcombe, V.F., Simpson, J.P., Kane, A.D., Menon, D.K., Rueckert, D., Glocker, B.: Efficient multi-scale 3d cnn with fully connected crf for accurate brain lesion segmentation. Medical image analysis 36, 61–78 (2017)
  * [19] Karimi, D., Samei, G., Kesch, C., Nir, G., Salcudean, S.E.: Prostate segmentation in mri using a convolutional neural network architecture and training strategy based on statistical shape models. International Journal of Computer Assisted Radiology and Surgery 13(8), 1211–1219 (Aug 2018). https://doi.org/10.1007/s11548-018-1785-8, https://doi.org/10.1007/s11548-018-1785-8
  * [20] Karimi, D., et al.: Accurate and robust deep learning-based segmentation of the prostate clinical target volume in ultrasound images. Medical Image Analysis 57, 186 – 196 (2019). https://doi.org/https://doi.org/10.1016/j.media.2019.07.005, http://www.sciencedirect.com/science/article/pii/S1361841519300623
  * [21] Khan, S., Naseer, M., Hayat, M., Zamir, S.W., Khan, F.S., Shah, M.: Transformers in vision: A survey. arXiv preprint arXiv:2101.01169 (2021)
  * [22] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. In: Proceedings of the 3rd International Conference on Learning Representations (ICLR) (2014)
  * [23] Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems. pp. 1097–1105 (2012)
  * [24] Le Cun, Y., Boser, B., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W., Jackel, L.D.: Handwritten digit recognition with a back-propagation network. In: Proceedings of the 2nd International Conference on Neural Information Processing Systems. pp. 396–404 (1989)
  * [25] LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. nature 521(7553), 436 (2015)
  * [26] Mahapatra, D., Buhmann, J.M.: Prostate mri segmentation using learned semantic knowledge and graph cuts. IEEE Transactions on Biomedical Engineering 61(3), 756–764 (March 2014). https://doi.org/10.1109/TBME.2013.2289306
  * [27] Milletari, F., Navab, N., Ahmadi, S.A.: V-net: Fully convolutional neural networks for volumetric medical image segmentation. In: 3D Vision (3DV), 2016 Fourth International Conference on. pp. 565–571. IEEE (2016)
  * [28] Olshausen, B.A., Field, D.J.: Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381(6583), 607–609 (1996)
  * [29] Prince, J.L., Pham, D., Tan, Q.: Optimization of mr pulse sequences for bayesian image segmentation. Medical Physics 22(10), 1651–1656 (1995)
  * [30] Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: Towards real-time object detection with region proposal networks. In: Advances in neural information processing systems. pp. 91–99 (2015)
  * [31] Taghanaki, S.A., Abhishek, K., Cohen, J.P., Cohen-Adad, J., Hamarneh, G.: Deep semantic segmentation of natural and medical images: A review. Artificial Intelligence Review pp. 1–42 (2020)
  * [32] Thompson, P.M., Toga, A.W.: Detection, visualization and animation of abnormal anatomic structure with a deformable probabilistic brain atlas based on random vector field transformations. Medical image analysis 1(4), 271–294 (1997)
  * [33] Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., Jégou, H.: Training data-efficient image transformers & distillation through attention. arXiv preprint arXiv:2012.12877 (2020)
  * [34] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. arXiv preprint arXiv:1706.03762 (2017)
  * [35] Wang, H., Zhu, Y., Green, B., Adam, H., Yuille, A., Chen, L.C.: Axial-deeplab: Stand-alone axial-attention for panoptic segmentation. In: European Conference on Computer Vision. pp. 108–126. Springer (2020)
  * [36] Wang, Y., Guo, Q., Zhu, Y.: Medical image segmentation based on deformable models and its applications. In: Deformable Models, pp. 209–260. Springer (2007)
  * [37] Zeng, Q., Karimi, D., Pang, E.H., Mohammed, S., Schneider, C., Honarvar, M., Salcudean, S.E.: Liver segmentation in magnetic resonance imaging via mean shape fitting with fully convolutional neural networks. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 246–254. Springer (2019)
  * [38] Zeng, Q., Samei, G., Karimi, D., Kesch, C., Mahdavi, S.S., Abolmaesumi, P., Salcudean, S.E.: Prostate segmentation in transrectal ultrasound using magnetic resonance imaging priors. International journal of computer assisted radiology and surgery 13(6), 749–757 (2018)
  * [39] Zhou, Z., Siddiquee, M.M.R., Tajbakhsh, N., Liang, J.: Unet++: A nested u-net architecture for medical image segmentation. In: Deep learning in medical image analysis and multimodal learning for clinical decision support, pp. 3–11. Springer (2018)


Summary Report of the Topical Group on Physics Education
Community Engagement Frontier (CEF4/CommF4)
Snowmass 2021

Sijbrand J. de Jong1, Sudhir Malik2, Randal Ruchti3

1Faculty of Science, Radboud University, 6525 AJ Nijmegen, The Netherlands

2Physics Department, University of Puerto Rico Mayaguez, PR 00682, USA

3Department of Physics and Astronomy, University of Notre Dame, Notre Dame, IN
46556 USA

> ABSTRACT
>
>
>
>
> An essential companion to the development and advancement of the field of
> Particle Physics is a strong program in physics education at all levels,
> that can attract entry level students across the full demographic spectrum
> and provide them with the education, training and skills needed to advance
> to successful careers in Science, Technology, Engineering and Mathematics
> (STEM) and other fields. This report summarizes the work of several
> investigative teams that have reviewed and assessed current opportunities in
> physics education across K-12, undergraduate, graduate and postdoctoral
> domains, including national and international linkages. From these
> assessments, recommendations have been put forward aimed to innovate
> educationally in strategic ways to strengthen ties between the research
> community and teachers, between the academic community and the private
> sector, and through both domestic and international connections.

Submitted to the Proceedings of the US Community Study

on the Future of Particle Physics (Snowmass 2021)

##  1 Introduction

To have in place the needed educated and trained workforce for currently
planned and future experiments in high energy particle physics and ancillary
fields, the attraction to and education of students for a broad range of
careers in STEM is both necessary and essential.

The Challenge: Traditional STEM educational efforts have provided a high-
quality workforce, but with rather selective demographics. And if left
unchanged, the character of the workforce will likely remain stagnant and
generally discouraging to the participation of women and those from
underrepresented groups.

The Opportunity: The Snowmass 2021 Process, with its 10-year planning and
20-year vision, offers a creative opportunity to assess the challenges, build
on what is currently working very well, and frame a structure to broader
opportunity for young researchers to join the exciting particle physics field
through an expanded range of educational opportunities.

The Approach and Organization of this Topical Group: To identify what is
working and the shortfalls, and to recommend actions to be taken, the
Community Engagement Frontier Topical Group on Physics Education has viewed
the physics education process in a systemic way as indicated schematically in
Figure 1. The schema highlights the educational process from the level of
incoming K-12 students and then upward through undergraduate and graduate
education, postdoctoral and faculty education.

Figure 1: A Schematic Representation of Particle Physics Education

A very large and demographically diverse population of students who might
consider and potentially enter STEM fields enters at the base of the pyramid.
For a variety of reasons, educational, cultural and societal, the fraction of
students that choose STEM careers and are able proceed upward to research
careers becomes reduced at each step. To assess these issues, the Physics
Education Topical Group formed four Working groups (A through D) organized by
community interest, all of which have submitted contributed papers to the
arXiv and include: Group A. Opportunities for Particle Physics Engagement in
K-12 Schools and Undergraduate Education [1]; Group B. Transforming U.S.
Particle Physics Education: A Snowmass 2021 Study [2]; Group C. Broadening the
Scope of Education, Career and Open Science in HEP [3]; and Group D. The
Necessity of International Particle Physics Opportunities for American
Education [4].

The Organization of this Report: In this Topical Group Summary Report, we
follow the schematic (and systemic) flow up the Pyramid of Figure 1 and
highlight briefly important issues identified by each group including selected
recommendations. The education process arises naturally at the K-12 and
undergraduate levels where the challenge is to attract and engage young
students to interest in STEM and physics, and then to hold and enrich their
interest and provide needed scaffolding to help sustain their upward path to
research careers. Greater depth and insight will be found in each of the
associated and referenced Contributed Papers indicated above [1, 2, 3, 4]. At
the end of this report, we provide potential linkages of this Physics
Education Topical Group with the other Topical Groups of the Community
Engagement Frontier of Snowmass 2021, and conclude with a final comment.

##  2 Opportunities for Particle Physics Engagement in K-12 Schools and
Undergraduate Education

While many (particularly young) students might show an early interest and
aptitude for science and mathematics at the elementary level, the structures
are not necessarily in place to capture, nurture and develop such nascent
interest. In the contributed paper [1] it was found that joint activities of
academia and K-12 educators and pupils should focus on how scientists develop
knowledge and the essence of the knowledge acquired so far. It should connect
to sister STEM fields, to their successful outreach programs and with an
emphasis of a holistic view on STEM. It is important that K-12 activities
address all students and teachers, adapting to their appropriate level of
scientific literacy, and be mindful of biases in program designs that might
filter student participation by race, gender, or socioeconomic background. The
paper [1] makes a number of recommendations that can be condensed into:

  1. 1.

Formation at the local level of collaborative communities (”fora”) of STEM
experts of all backgrounds (physicists, engineers, technicians, students and
K-12 teachers and students - to form direct linkages among researchers and
educators for dialog, collaboration, mutual enrichment and mentoring among
members. These fora should be responsive to local community need.

  2. 2.

These fora should not be isolated or self-isolating. To aid in interaction and
information transfer, they should be supported by a nationally, or even
internationally, organised online repository for sharing resources;

  3. 3.

The fora can only be created and sustained if a minimal amount of support for
coordination and logistics is made available. For the sustainability, it is
important to have a steady source for this support, which can come from
colleges, universities or institutes, but might also come from or be
supplemented by outreach support in research grants.

  4. 4.

An important ingredient for sustainable fora is that the efforts of the
participants are appropriately and regularly recognised.

##  3 Educational Opportunities at the Undergraduate, Graduate and
Postdoctoral Level

This critical time domain for the development and pursuit of careers in
particle physics and ancillary fields has traditionally been the purview of
university and laboratory groups in the US and abroad. In the words of the
engaged group of young researchers working on this topic: ”Graduate school
(and undergraduate school to a lesser extent) is where researchers acquire
most of the technical skills required for research, develop scientific problem
solving abilities, learn how to establish themselves in their field, and begin
developing their career. It is unfortunate, then, that the skills gained by
physicists during their formal education are often mismatched with the skills
actually required for a successful career in physics.” These words suggest
that there is clearly work to do to strengthen this key educational domain.

###  3.1 Online Survey of Educational Experiences in the HEP Community

The group performed an online survey of the U.S. particle physics community to
gain background on how practitioners in the field at all levels assessed their
own physics education and training, with the aim to identify areas of
challenge and inform potential improvements for the future. The group’s report
[2] is detailed and replete with findings and recommendations, from which we
have identified the following succinct elements:

  1. 4.

Graduate programs in particle physics should normalize training for a broad
range of STEM careers with appropriate formal courses, rather than forcing
students to resort prematurely to self-teaching or peer learning. The courses
should provide strong grounding in particle physics and mathematics, but also
computation, statistics and instrumentation, with the aim to benefit careers
in physics, industry or education.

  2. 5.

Correspondingly, Universities should provide undergraduate students with a
more complete picture of what particle physics trained researchers do. A
realistic view of common career paths post baccalaureate and post graduate
school should be presented, including for theoretical and experimental
positions as well as non-academic careers.

  3. 6.

Undergraduate participation in the survey was rather minimal, driven in part
by lack of information. With support from Professional Societies and Physics
Departments, connections and networking opportunities should be developed to
strengthen connections for undergraduate students with HEPA community
activities. And the HEPA Community should actively plan to perform a future
survey with emphasis on undergraduate participation, to assess and strengthen
links to students at that academic level. .

###  3.2 A New Perspective on the Masters Degree

As noted in several of the recommendations derived from the comprehensive
survey cited above and detailed in the Contributed Paper [2], a potentially
important professional level post baccalaureate, the Masters Degree, is often
overlooked or ignored by the physics community which tends to be ”Ph D”
driven. The Masters level has several important attributes worthy of community
consideration. (1) It is a level where more extensive cross-disciplinary
(elective) course work is possible, providing potential branches to applied
math, statistics, computer science, engineering and nuclear medicine. (2) It
is a level of potential participation by and engagement with students from the
private sector who enroll for technical advancement with support from their
companies. This nexus can provide a bridge between students following an
academic path with those already in commercial applications sectors, opening
up dialog and potential career opportunities that might otherwise be
overlooked. (3) The Masters Level also affords an intermediate (and
potentially achievable) academic target for students from groups traditionally
underrepresented in physics and for whom a PhD in physics might seem an
unlikely goal. And (4) enriched programs at the Masters Level can lead to
collaborative opportunities across academia, which is the central topic of the
next section.

  1. 7.

Universities, especially also non-research universities, should consider
setting up Masters Degree programs in particle physics and related areas, such
as hardware and software technology for Big Science experiments.

##  4 Collaborative Opportunities Across Academia

The emphases of this group were to review the challenges and develop
strategies to help transform the particle physics field into a stronger and
more diverse ecosystem of talent and expertise, with the expectation of long-
lasting scientific and societal benefits. Regarded as central were the
building of collaborative bridges between faculty and experimental programs at
R1 institutions (major research universities and national laboratories) with
those at R2 institutions (that provide training up to masters degrees),
Predominantly Undergraduate Institutions (PUIs) and Community Colleges (CC)).
The intended aims over the next decade are to:

  1. 8.

Expand the benefits of faculty collaboration and research opportunities across
the broad spectrum of academia and give equivalence: opportunities for all in
technical and scientific leadership on projects and with appropriate
recognition for contributions.

  2. 9.

In corollary, conduct a study of new models of collaboration or cooperation
that would allow R2/PUI/CC faculty and their students to collaborate in
demonstrably effective ways in experiments. This includes addressing the
challenges of teaching loads, student training and funding availability that
directly impact consequential participation.

  3. 10.

Broadly accessible data and analysis platforms will benefit student access and
participation. In the true spirit of Open Science, the HEP community should
define, with cogent arguments, what should be the scope of making our data and
resources publicly available, and the hardware, software and person-power
costs associated with such implementation.

Machine Learning (ML) is becoming an integral part of physics research. Many
critical HEP algorithms for triggering, reconstruction, and analysis rely on
ML and there are entire conferences and summer schools dedicated to this
crosscutting field. Despite the relevance and importance of this research,
pursuing a career at the intersection of these fields remains a tenuous and
undefined endeavor. The current mindset in the field is that highly
specialized skills such as software and firmware development and
instrumentation development are not broadly recognized as “physics” work. This
perception of what it means to ‘be a physicist’ must be challenged lest it
continue to be an impediment. Confronting and addressing these issues would
encourage an influx of new workforce into the field, help retain those who are
in the field, and equip those who might seek careers outside of HEP.

  1. 11.

Qualification for HEP faculty jobs should not be based solely on physics
analysis but must be expanded to to include computing, software and/or
hardware contributions.

##  5 International Opportunities for Particle Physics Education

Particle physics is a global endeavor. No one institution or nation can
assemble the resources or expertise needed to explore the frontiers of the
field. The diversity of national, social and cultural backgrounds present in
the experiments and labs enriches the pool of intellectual thought and
solidifies the validity of their scientific findings. And no one country or
region has a monopoly on good ideas or approaches to strengthen the
educational experience for studwnts. But there are noteworthy examples of
excellent ongoing efforts, which this working group reviewed in detail. Their
line of thought has been worked out in a contributed paper [4] and hasled to
several noteworthy recommendations.

  1. 12.

U.S. based pre-university particle physics collaborations, such as QuarkNet
and other outreach programmes, should expand collaboration with international
partners, such as the International Particle Physics Outreach Group (IPPOG),
the CERN Beamline for Schools (BL4S) and Teacher summer school programmes in
Europe, and should collaborate with partners in the developing world, such as
the African School of Fundamental Physics and Applications. The participation
in the Global Cosmics portal should be enhanced by developing low-cost cosmic
ray detectors for educational use.

  2. 13.

Student exchange programs should be fostered and supported, such as the NSF
Research Experience for Undergraduates (REU) program, which funds
participation of U.S. students in the CERN Summer Student program, and the
DoE-INFN summer student exchange program between the U.S. and Italy. Where
possible these should be extended, in particular with student exchange
programs and summer schools in developing countries, such as the African
School of Fundamental Physics and Applications.

##  6 Interconnections and Synergies with Community Engagement Topical Groups

Physics Education (CEF4) has important interconnections to other Snowmass 2021
Topical Groups within the Community Engagement Frontier.

With CEF1, the Topical Group on Applications and Industry:. Opportunties are
potentially vast, linking education with technological innovation. National
Laboratories are a natural training ground for student research opportunities.
And at the Masters Degree level, universities and colleges are a fertile
training ground to mix students moving up to STEM careers with students from
industry who seek to further enrich their technological training. This is a
melting pot worth pursiuing energetically and behooves universities and
colleges to consider the Masters Degree in Physics in a new light.

With CEF2, the Topical Group on Career Pipeline and Development: Physics
Education (CEF4) is central to creating a skilled workforce pipeline to all
HEP Frontiers and beyond for STEM areas in industry. All recommendations in
CEF2 are strongly endorsed by this CEF4 Working Group. Beyond regular course
curriculum, Software Training programs and Open Science activities [3] can go
a long way to attracting talent in HEP as well as preparing HEP talent for
STEM industry jobs. Strong participation by faculties and students of PUIs and
CCs in HEP programs [5] will enable a diverse and inclusive STEM workforce for
HEP and industry.

With CEF3, the Topical Group on Diversity and Inclusion: With the inclusion of
a well-educated and demographically and geographically diverse group of
students, the person power needed for current and future experimental and
scientific challenges will be in place to meet the needs of particle physics
community over the next two decades.

With CEF5, the Topical Group on Public Education and Outreach: K/12 education
is fundamentally public education at its most fundamental level. With students
excited and engaged, and their bringing that interest and excitement home to
their families, the thrill of particle physics and the possibilities of future
discoveries will help build the strong supportive community needed to sustain
the science.

With CEF6, the Topical Group on Public Policy and Government Engagement:
Particle Physics is an exciting, yet esoteric field. It is therefore
challenging to translate that excitement to those who make public policy and
provide funding support, when they are under myriad pressures from across the
political spectrum. However, hand-in-hand with the expansive educational
opportunities that discovery science affords, and through mention of the
excitement and participation of students across the nation, the conversational
optics can be refocused in constructive ways, to the benefit of the field and
nation.

With CEF7 the Topical Group on Environmental and Societal Impacts: At the time
of Fermilab’s founding near the village of Weston, Illinois, the land was
prairie. Since that time, the laboratory has had programs on environment and
ecology that are particularly accessible to younger students and to engage
them in the natural world. These programs, coordinated through the Lederman
Science Center at Fermilab, are a wonderful model of the interconnection of
particle physicists, education, and local schools and communities [6], [7].
While there are outreach aspects to such programs, the latter reference is
intrinsically educational by design.

##  7 Conclusions

To support a compelling program of scientific discovery in the long run, a
robust education program in physics, mathematics and the sciences is an
essential companion. Such a program should provide students across the
demographic spectrum ample basis of opportunity to enter particle physics and
ancillary fields to engage in and benefit from the science.

The ten-year program and twenty-year vision of Snowmass 2021 affords a
strategic opportunity and time window within which both the science and the
education process can evolve holistically and constructively. This requires
Community Engagement at all educational levels, which can only be sustained
when the people involved are appropriately recognised, credited and supported
for their contributions.

## References

  * [1] M.G. Bardeen, O.M. Bitter, M. Glover, S.J. de Jong, T.R. Lewis, M. Fetsko et al., _Particle Physics Outreach to K-12 Schools and Opportunities in Undergraduate Education_ , 2203.10953.
  * [2] O. Bitter, E.V. Hansen, S. Kravitz, V. Velan and Y. You, _Transforming U.S. Particle Physics Education: A Snowmass 2021 Study_ , 2204.08983.
  * [3] S. Malik, D. DeMuth, S. de Jong, R. Ruchti, S. Thais, G. Fidalgo et al., _Broadening the scope of Education, Career and Open Science in HEP_ , 2203.08809.
  * [4] E. Arce-Larreta, K. Assamagan, E. Barzi, U. Bilow, K. Cecire, S. de Jong et al., _The Necessity of International Particle Physics Opportunities for American Education_ , 2203.09336.
  * [5] M. Bellis, B. Bhattacharya, J. Hogan, S. Malik, B. Pearson, D. DeMuth et al., _Enhancing HEP research in predominantly undergraduate institutions and community colleges_ , 2203.11662.
  * [6] _Tallgrass-Prairie_ , https://ecology.fnal.gov/tallgrass-prairie/.
  * [7] _Prairie Resources for Students_ , https://ed.fnal.gov/data/prairie_resources.shtml.