# Directional transport of active particles confined in $3D$(three
dimensional) smooth corrugated channel

Bing Wang, Wenfei Wu  School of Mechanics and Optoelectronics Physics, Anhui
University of Science and Technology, Huainan, 232001, P.R.China
<EMAIL_ADDRESS>
###### Abstract

The transport phenomenon of active particles confined in $3D$(three
dimensional) corrugated confined channel with Gaussian noises is investigated.
Large noise intensity perpendicular to the symmetry axis is good for the
diffusion and current along the axis. The generalized resonance transport
phenomenon appears with increasing noise intensity parallel to the symmetry
axis. Large noise intensity parallel to axis can suppress the diffusion. The
diffusion coefficient has a maximum with increasing polar angle noise
intensity. There exits an optimal value of parameter $f$ that result in
maximum movement speed. Large $f$ is good for the diffusion. Transport reverse
phenomenon appears with increasing channel parameter $\varepsilon$ and
$\Delta$. Too large or too small values of $\varepsilon$ and $\Delta$ can
suppress the diffusion.

  *

Active particles are able to take up energy from environment and convert the
energy to directional motion. Investigation of active particles system has
enabled to mimic and dissect mechanisms in biological systems[1]. These
investigations have grown substantially both in theory and application[2, 3,
4, 5, 6, 7, 8, 9], especially, particles confined in spatial space. There are
numerous realizations of confine active particles in nature ranging from
bacteria and spermatozoa to artificial colloidal micro-swimmers, e.g.,
biological cells[10, 11], ion channels[12], nanoporous materials[13, 14],
zeolites[15], microfluidic channels[16], artificial nanopores[17], and
ionpumps [18].

Confined active particles shows a series of interesting phenomenons, e.g.
current reversal[19, 20, 21], self-organization[22, 23] and so on. Ghosh _et
al_. investigated active overdamped microswimmers in a two-dimensional
periodically compartmentalized channel and proved that ratcheting of Janus
particles can be orders of magnitude stronger than for ordinary thermal
potential ratchets and thus experimentally accessibl[24]. Ao _et al_.
investigated the transport diffusivity of Janus particles in the absence of
external biases with reflecting walls and found the self-diffusion can be
controlled by tailoring the compartment geometry[25]. Bechinger _et al_. given
an overview of experimental achievements connected to the realization of
artificial microswimmers and nanoswimmers[26]. Murali _et al_. showed that
geometric constraints are a route to affect the emergent noise properties of a
single active particle[27]. Liu _et al_. investigated the entropic stochastic
resonance when a self-propelled Janus particle moves in a double-cavity
container[28]. Li _et al_. studied the transport of noninteracting anisotropic
particles in a narrow two dimensional left-right and up-down asymmetrical
channel[29]. Pototsky _et al_. considered a colony of point like self-
propelled surfactant particles without direct interactions that cover a thin
liquid layer on a solid support[30].

Previous research considers the particles confined in two dimensions ($2D$)
corrugated channel. In this paper, we investigate the directional transport of
active particles confined in a three dimensions($3D$) corrugated channel. The
paper is structured as follows. Section 1 gives the model considered in this
paper. Section 2 analyses the effects of the channel and noise on the system.
A concluding discussion is offered in section 3.

##  1 Basic model and methods

In this work, we consider active particles confined in a $3D$ smooth
corrugated channel with Gaussian white noises. The dynamic of the particle is
governed by the following dimensionless equations[31, 32, 33]

$\frac{\partial{x}}{\partial
t}=v_{0x}+\xi_{x}(t)=v_{0}\sin\theta(t)\cos\varphi(t)+\xi_{x}(t),$ (1)

$\frac{\partial{y}}{\partial
t}=v_{0y}+\xi_{y}(t)=v_{0}\sin\theta(t)\sin\varphi(t)+\xi_{y}(t),$ (2)

$\frac{\partial{z}}{\partial
t}=v_{0z}+\xi_{z}(t)=v_{0}\cos\theta(t)+\xi_{z}(t),$ (3)

$\frac{\partial{\theta(t)}}{\partial t}={\xi}_{\theta}(t),$ (4)

$\frac{\partial{\varphi(t)}}{\partial t}={\xi}_{\varphi}(t).$ (5)

Here, $v_{0}$ is the self-propelled speed of the active particle. The polar
angle between ${v}_{0}$ and $z$ axis is $\theta(t)$. The azimuth angle between
projection of ${v}_{0}$ on $xoy$ plane with $x$ axis is $\varphi(t)$.
$\xi_{x}$, $\xi_{y}$ and $\xi_{z}$ are the noises and parallel to $x$ axis,
$y$ axis and $z$ axis, respectively. $\xi_{\theta}$ and $\xi_{\varphi}$ are
the angle noises, respectively. $\xi_{i}(i=x,y,z,\theta,\varphi)$ satisfies
the following relations.

$\langle\xi_{i}(t)\rangle=0,(i=x,y,z,\theta,\varphi),$ (6)
$\langle\xi_{i}(t)\xi_{j}(s)\rangle=\delta_{ij}{Q_{i}}\delta(t-t^{\prime}),(i=x,y,z,\theta,\varphi),$
(7)

$\langle\cdots\rangle$ denotes an ensemble average over the distribution of
the random forces. $Q_{i}(i=x,y,z,\theta,\varphi)$ is the noise intensity.
Normally, the noise intensity relates to temperature $T$. For a charged
particle, $\xi_{i}$ can be effected by the external electric field intensity.
As electric field intensity is a vector quantity, so the noise intensity
$Q_{x}$, $Q_{y}$, $Q_{z}$, $Q_{\theta}$ and $Q_{\varphi}$ can take different
value.

Figure 1: (a)Illustrations of the smooth $3D$ corrugated channel with $L=1.0$,
$\varepsilon=1.0$, $f=1.8$ and $\Delta=-0.8$; (b)Front view of the channel
with $L=1.0$, $\varepsilon=1.0$, $f=1.8$ and $\Delta=-0.8$; (c)Illustrations
of the smooth $3D$ corrugated channel with $L=1.0$, $\varepsilon=1.0$, $f=1.8$
and $\Delta=0.8$; (d)Front view of the channel with $L=1.0$,
$\varepsilon=1.0$, $f=1.8$ and $\Delta=0.8$; (e)The particle with self-
propelled velocity $\vec{v}_{0}$ confined in the channel.

In this paper, the particles are confined in a $3D$ smooth corrugated channel.
The channel is periodic in space along the $z$-axis as shown in Fig.1 and is
defined by the following sinusoidal function[34]

$W(z)=\varepsilon[\sin(\frac{2\pi{z}}{L})+\frac{\Delta}{4}\sin(\frac{4\pi{z}}{L})]+f.$
(8)

The shape of the channel are controlled by the parameters $\varepsilon$, $f$
and $\Delta$. In order the channel can not becomes a enclosure space and the
particles can transport the channel, the parameter $f>0$. The diameter of the
pore is $f-|\varepsilon|$(Fig.1) and $f-|\varepsilon|>0$.

A central practical question in the theory of Brownian motors is the over all
long time behavior of the particle, and the key quantities of particle
transport is the particle velocity $\langle V\rangle$ and the time-independent
diffusion coefficient $D$[35]. Because particles along the $x$ and $y$
directions are confined, we only calculate the $z$ direction average velocity

$\langle
V\rangle=\lim_{t\to\infty}\frac{\langle{z(t)-z(t_{0})}\rangle}{t-t_{0}},$ (9)

$z(t_{0})$ is the position of the particle at time $t_{0}$.

The time-independent normal diffusion coefficient is given by[36]

$D=\lim_{t\to\infty}\frac{\langle{z^{2}(t)\rangle-\langle
z(t)}\rangle^{2}}{2t}.$ (10)

##  2 Results and discussion

In this letter, we demonstrate the transport phenomenon of active particles
confined in a $3D$ smooth corrugated channel. In order to give a clear
analysis of the system. Eqs.(1, 2, 3, 4, 5) are integrated using the Euler
algorithm. The integration step time $\Delta t=10^{-4}$ and the total
integration time is more than $10^{5}$. The stochastic averages are obtained
as ensemble averages over $10^{5}$ trajectories.

Figure 2: The average velocity $\langle V\rangle$ and diffusion coefficient
$D$ as functions of $Q_{x}$ for different self-propelled speed $v_{0}$. The
other parameters are $L=1.0$, $\varepsilon=1.0$, $f=1.8$, $Q_{y}=Q_{z}=1.0$,
$Q_{\theta}=Q_{\varphi}=0.5$: (a)$\Delta=-0.8$, (b)$\Delta=-0.8$,
(c)$\Delta=0.8$, (d)$\Delta=0.8$.

The average velocity $\langle V\rangle$ and diffusion coefficient $D$ as
functions of $x$ axis noise intensity $Q_{x}$ is reported in Fig.2. In
Fig.2(a), $\langle V\rangle>0$ when $x$ axis noise does not exist($Q_{x}=0$).
$\langle V\rangle<0$ when $Q_{x}\neq 0$, so the particle moves in $-z$
direction when $x$ axis noise exist. The average speed $|\langle V\rangle|$
increases with increasing $Q_{x}$, and the slope of $\langle V\rangle-Q_{x}$
curve decreases with increasing $Q_{x}$ and changes to zero when $Q_{x}$ is
large. So small $x$ axis noise intensity will inhibit current in $-z$
direction, and large $x$ axis noise intensity is good for this current. The
effect will become weak when $x$ axis noise intensity is too large. In
Fig.2(b), the diffusion coefficient $D$ increases with increasing $Q_{x}$, so
large $x$ axis noise is good for the diffusion. In Fig.2(c), the average
velocity $\langle V\rangle$($\langle V\rangle>0$) increases with increasing
$Q_{x}$. The slope of $\langle V\rangle-Q_{x}$ curve decreases with increasing
$Q_{x}$ and changes to zero when $Q_{x}$ is large. So large $x$ axis noise
intensity is good for the current in $+z$ direction. In Fig.2(d), just like
the result of Fig.2(b), $D$ increases with increasing $Q_{x}$, namely large
$x$ axis noise is good for the diffusion too.

Figure 3: The average velocity $\langle V\rangle$ and diffusion coefficient
$D$ as functions of $Q_{y}$ for different self-propelled speed $v_{0}$. The
other parameters are $L=1.0$, $\varepsilon=1.0$, $f=1.8$, $Q_{x}=Q_{z}=1.0$,
$Q_{\theta}=Q_{\varphi}=0.5$ : (a)$\Delta=-0.8$, (b)$\Delta=-0.8$,
(c)$\Delta=0.8$, (d)$\Delta=0.8$.

The average velocity $\langle V\rangle$ and diffusion coefficient $D$ as
functions of $y$ axis noise intensity $Q_{y}$ is reported in Fig.3. Just like
the effect of $x$ axis noise(Fig.2), when $\Delta=-0.8$(Fig.3(a) and
Fig.3(b)), $\langle V\rangle>0$ when $Q_{y}=0$. When $Q_{y}\neq 0$, the
particle moves in $-z$ direction. The average speed $|\langle V\rangle|$ and
diffusion coefficient $D$ monotonic increases with increasing $Q_{y}$.
$\langle V\rangle-Q_{y}$ and $D-Q_{y}$ curves change to horizon when $Q_{y}$
is large. When $\Delta=0.8$(Fig.3(c) and Fig.3(d)), the particle moves in $+z$
direction. $|\langle V\rangle|$ and $D$ monotonic increase with increasing
$Q_{y}$. The $\langle V\rangle-Q_{y}$ and $D-Q_{y}$ curves change to horizon
when $Q_{y}$ is large. So large $y$ axis noise intensity is good for diffusion
and the current in $-z$ and $+z$ direction, but the effects will become weak
when the noise intensity is large. From Figs.2 and 3, we find $x$ axis noise
and $y$ axis noise have the same effect on the system. This is because the
channel is axis-symmetric, so $x$ axis and $y$ axis noises are equivalent for
the system. In Figs.2 and 3, we know noise perpendicular to the symmetry axis
have non-negligible influence on the directional movement and diffusion.

Figure 4: The average velocity $\langle V\rangle$ and diffusion coefficient
$D$ as functions of $Q_{z}$ for different self-propelled speed $v_{0}$. The
other parameters are $L=1.0$, $\varepsilon=1.0$, $f=1.8$, $Q_{x}=Q_{y}=1.0$,
$Q_{\theta}=Q_{\varphi}=0.5$: (a)$\Delta=-0.8$, (b)$\Delta=-0.8$,
(c)$\Delta=0.8$, (d)$\Delta=0.8$.

$\langle V\rangle$ and $D$ as functions of $z$ axis noise intensity $Q_{z}$ is
reported in Fig.4. Unlike the effects of $x$ axis and $y$ axis noises, we find
the average speed $|\langle V\rangle|$ has an obvious maximum with increasing
$Q_{z}$($\langle V\rangle<0$ when $\Delta=-0.8$, and $\langle V\rangle>0$ when
$\Delta=0.8$). So the generalized resonance transport phenomenon appears in
the system. There exists an optimum value of $Q_{z}$, and the current is very
obvious at this point. Too large or too small $Q_{z}$ will inhibit the
directional movement speed. In Figs.4(b) and 4(d), we find the diffusion
coefficient $D$ monotonic decreases with increasing $Q_{z}$. So large $Q_{z}$
will inhabit the diffusion. From Figs. 2, 3 and 4, we find an interesting
phenomenon, that is the diffusion becomes obvious when the noise intensity
perpendicular to the axis is large, but can be suppressed if the noise
intensity parallel to the axis is large.

Figure 5: The average velocity $\langle V\rangle$ and diffusion coefficient
$D$ as functions of polar angle noise intensity $Q_{\theta}$ for different
self-propelled speed $v_{0}$. The other parameters are $L=1.0$,
$\varepsilon=1.0$, $f=1.8$, $Q_{x}=Q_{y}=Q_{z}=1.0$, $Q_{\varphi}=0.5$:
(a)$\Delta=-0.8$, (b)$\Delta=-0.8$, (c)$\Delta=0.8$, (d)$\Delta=0.8$.

$\langle V\rangle$ and $D$ as functions of polar angle noise intensity
$Q_{\theta}$ is reported in Fig.5. We find whenever $\Delta=-0.8$ or
$\Delta=0.8$, $\langle V\rangle$ monotonic decreases with increasing
$Q_{\theta}$. In the case of $\Delta=-0.8$(Fig.5(a)), the particle moves in
$+z$ direction when $Q_{\theta}=0.0$, and the direction changes from in $+z$
direction($\langle V\rangle>0$) to in $-z$ direction($\langle V\rangle<0$)
with increasing $Q_{\theta}$. So the current reverse phenomenon appears with
increasing $Q_{\theta}$. In the case of $\Delta=0.8$, the particle moves
always in $+z$ direction, and the speed becomes smaller and smaller with
increasing $Q_{\theta}$. Whenever $\Delta=-0.8$ or $\Delta=0.8$, there exists
an optimum value of $Q_{\theta}$, and the diffusion is very obvious at this
point(Fig.5(b) and Fig.5(d)).

Figure 6: The average velocity $\langle V\rangle$ and diffusion coefficient
$D$ as functions of azimuth angle noise intensity $Q_{\varphi}$ for different
self-propelled speed $v_{0}$. The other parameters are $L=1.0$,
$\varepsilon=1.0$, $f=1.8$, $Q_{x}=Q_{y}=Q_{z}=1.0$, $Q_{\theta}=0.5$:
(a)$\Delta=-0.8$, (b)$\Delta=-0.8$, (c)$\Delta=0.8$, (d)$\Delta=0.8$.

The average velocity $\langle V\rangle$ and diffusion coefficient $D$ as
functions of the azimuth angle noise intensity $Q_{\varphi}$ is reported in
Fig.6. We find whenever $\Delta=-0.8$ or $\Delta=0.8$, the average
speed$|\langle V\rangle|$ and diffusion coefficient $D$ increase with
increasing noise intensity $Q_{\varphi}$. So large azimuth angle noise
intensity will help to the current and the diffusion. $\langle V\rangle-
Q_{\varphi}$ and $D-Q_{\varphi}$ curves change to horizon when $Q_{\varphi}$
is large.

Figure 7: The average velocity $\langle V\rangle$ and diffusion coefficient
$D$ as functions of parameter $f$ for different $v_{0}$. The other parameters
are $L=1.0$, $\varepsilon=1.0$, $Q_{x}=Q_{y}=Q_{z}=1.0$,
$Q_{\theta}=Q_{\varphi}=0.5$: (a)$\Delta=-0.8$, (b)$\Delta=-0.8$,
(c)$\Delta=0.8$, (d)$\Delta=0.8$.

The average velocity $\langle V\rangle$ and diffusion coefficient $D$ as
functions of the channel parameter $f$ with different $v_{0}$ is reported in
Fig.7. We set $f\geq 1.0$ to avoid the channel becomes a enclosure space
because the parameter $\varepsilon$ is fixed($\varepsilon=1.0$). In Figs.7(a),
we find $\langle V\rangle\rightarrow 0$ when $f=1.0$, the reason is that the
channel becomes a enclosure space and the particles are sealed in the
enclosure space. $\langle V\rangle<0$ and $\langle V\rangle$ has a minimum(The
average speed $|\langle V\rangle|$ has a maximum) with increasing $f$. So
there exits an optimal value of $f$ that results in the most obvious current
in $-z$ direction. In Fig.7(b), $D$ monotonic increases with increasing $f$.
So large $f$ is good for the diffusion. In Fig.7(c), contrary to the result of
Fig.7(a), we find $\langle V\rangle>0$ and $\langle V\rangle$ has a maximum
with increasing $f$. So there exits an optimal value of $f$ that results in
the maximum moving speed in $z$ direction. Just like Fig.7(b), we find $D$
monotonic increases with increasing $f$ in Fig.7(d), so large $f$ is good for
diffusion. From the definition Eq.8 of the channel, we know the larger of $f$,
the larger of the pore size is. In this case, too large pore size can not
promote the directional moving speed but inhabit this phenomenon. However,
large pore size is good for the diffusion of the particle.

Figure 8: The average velocity $\langle V\rangle$ and diffusion coefficient
$D$ as functions of parameter $\varepsilon$ for different $v_{0}$. The other
parameters are $L=1.0$, $f=1.8$, $Q_{x}=Q_{y}=Q_{z}=1.0$,
$Q_{\theta}=Q_{\varphi}=0.5$: (a)$\Delta=-0.8$, (b)$\Delta=-0.8$,
(c)$\Delta=0.8$, (d)$\Delta=0.8$.

The average velocity $\langle V\rangle$ and diffusion coefficient $D$ as
functions of the channel parameter $\varepsilon$ with different $v_{0}$ is
reported in Fig.8. In Fig.8(a), $\langle V\rangle$ has a maximum($\langle
V\rangle_{max}>0$) and a minimum($\langle V\rangle_{min}<0$) with increasing
$\varepsilon$. The $\langle V\rangle_{max}$ is on the left and the $\langle
V\rangle_{min}$ is on the right. $\langle V\rangle\rightarrow 0$ when
$|\varepsilon|>1.8$ because channel will become a enclosure space. In
Fig.8(b), $D$ has a maximum with increasing $\varepsilon$($\varepsilon=0$).
The channel changes to a straight tube when $\varepsilon=0$, so straight tube
is better for diffusion then corrugated pipe. In Fig.8(c), $\langle V\rangle$
has a minimum($\langle V\rangle_{min}<0$) and a maximum($\langle
V\rangle_{max}>0$) with increasing $\varepsilon$. $\langle V\rangle_{min}$ is
on the left and $\langle V\rangle_{max}$ is on the right. Just like Fig.8(a),
$\langle V\rangle\rightarrow 0$ because the channel becomes a enclosure space
when $|\varepsilon|>1.8$. In this figure, we find the transport reverse
phenomenon appears with increasing $\varepsilon$. In Fig.8(d), we find $D$ has
a maximum when the parameter $\varepsilon=0$. That is, the diffusion is very
obvious when the corrugated channel becomes a straight pipe.

Figure 9: The ratio $\langle V\rangle/v_{0}$ and diffusion coefficient $D$ as
functions of $v_{0}$ for different $\Delta$. The other parameters are $L=1.0$,
$\varepsilon=1.0$, $f=1.8$, $Q_{x}=Q_{y}=Q_{z}=1.0$,
$Q_{\theta}=Q_{\varphi}=0.5$.

The ratio $\langle V\rangle/v_{0}$ and diffusion coefficient $D$ as functions
of self-propelled speed $v_{0}$ with different adjust parameter $\Delta$ is
reported in Fig.9. In Fig.9(a), the ratio $\langle V\rangle/v_{0}$ exists a
minimum and a maximum with increasing $v_{0}$. The particle moves in $-z$
direction when $v_{0}$ is small. The ratio $\langle V\rangle/v_{0}$ reaches a
minimum first, then increases with increasing $v_{0}$ and reaches a maximum,
then decreases with increasing $v_{0}$ again, and $\langle
V\rangle/v_{0}\rightarrow 0$ in the end. The current reverse phenomenon
appears with increasing $v_{0}$. In Fig.9(c), we find $\langle V\rangle>0$ and
$\langle V\rangle/v_{0}$ has a maximum with increasing self-speed $v_{0}$.
$\langle V\rangle/v_{0}\rightarrow 0$ when $v_{0}$ is large. In Fig.9(b) and
Fig.9(d), we find $D$ has maximum with increasing $v_{0}$, so proper value of
$v_{0}$ is good for the diffusion, too small or too large $v_{0}$ will inhabit
the diffusion.

Figure 10: The average velocity $\langle V\rangle$ and the diffusion
coefficient $D$ as functions of adjust parameter $\Delta$ for different self-
propelled speed $v_{0}$. The other parameters are $L=1.0$, $\varepsilon=1.0$,
$f=1.8$, $Q_{x}=Q_{y}=Q_{z}=1.0$, $Q_{\theta}=Q_{\varphi}=0.5$.

The average velocity $\langle V\rangle$ and diffusion coefficient $D$ as
functions of the adjust parameter $\Delta$ is reported in Fig.10. In
Fig.10(a), we find $\langle V\rangle$ first decreases with increasing $\Delta$
and reaches a minimum($\langle V\rangle_{min}<0$), then increases with
increasing $\Delta$ and reaches a maximum($\langle V\rangle_{max}>0$), then
decreases with increasing $\Delta$ in the end. There exist a minimum and a
maximum in the $\langle V\rangle-\Delta$ curve, and the current reverse
phenomenon appears with increasing $\Delta$. In Fig.10(b), we find the
diffusion coefficient $D$ reaches a maximum when the parameter
$\Delta\rightarrow 0$.

##  3 Conclusions

In this paper, we numerically investigated the transport phenomenon of active
particles confined in a $3D$ smooth channel with Gaussian noise. We find large
noise intensity perpendicular to the symmetry axis is good for the current in
the axis direction and the diffusion. The average speed has a maximum with
increasing noise intensity parallel to the symmetry axis. The transport
reverse phenomenon appears with increasing polar angle noise intensity when
the adjust parameter $\Delta$ is negative. Large azimuth noise intensity is
good for the current and diffusion. Changing the parameters $\varepsilon$ and
$\Delta$ can make different shape, and the transport reverse phenomenon
appears with increasing $\varepsilon$ and $\Delta$. Proper size of pore is
good for the directional current, but too large or too small pore size can
inhabit this phenomenon. The diffusion is very obvious when the corrugated
channel becomes a straight pipe.

##  4 Acknowledgments

Project supported by Natural Science Foundation of Anhui Province(Grant
No:1408085QA11) and College Physics Teaching Team of Anhui Province(Grant
No:2019jxtd046).

##  5 Referencing

## References

  * [1] S. Ramaswamy, 2010 Annu. Rev. Condens. Matter Phys. 1 323\.
  * [2] M. C. Marchetti, J. F. Joanny, S. Ramaswamy, et al., 2013 Rev. Mod. Phys. 85 1143\.
  * [3] D. Needleman, Z. Dogic, 2017 Nat. Rev. Mater. 2 17048\.
  * [4] E. Pince, S. K. P. Velu, A. Callegari, et al., 2016 Nat. Commun. 7 10907\.
  * [5] S. Palagi, P. Fischer, 2018 Nat. Rev. Mater. 3 113\.
  * [6] P. Pietzonka, E. Fodor, C. Lohrmann, M. E. Cates, U. Seifert, 2019 Phys. Rev. X 9 041032\.
  * [7] A. Kulkarni, S. P. Thampi, M. V. Panchagnula, 2019 Phys. Rev. Lett. 122 048002\.
  * [8] J. C. Moreno, M. L. Rubio Puzzo, W. Paul, 2020 Phys. Rev. E 102 022307\.
  * [9] G. Gompper, R. G. Winkler, T. Speck, A. Solon, C. Nardini, F. Peruani, et al., 2020 J. Phys.: Condens. Matter 32 193001\.
  * [10] S. Tang, F. Zhang, H. Gong, F. Wei, J. Zhuang, et al., 2020 Sci. Robot. 5 eaba6137.
  * [11] H. X. Zhou, G. Rivas, A. P. Minton, 2008 Annu. Rev. Biophys. 37 375\.
  * [12] B. Hille. Ion Channels of Excitable Membranes. Sinauer Associates, 3rd edition, 2001.ISBN 0878933212.
  * [13] E. Beerdsen, D. Dubbeldam, B. Smit, 2005 Phys. Rev. Lett. 95 164505\.
  * [14] E. Beerdsen, D. Dubbeldam, B. Smit, 2006 Phys. Rev. Lett. 96 044501\.
  * [15] F. J. Keil, R. Krishna, M. O. Coppens, 2000 Rev. Chem. Eng. 16 71\.
  * [16] T. M. Squires, S. R. Quake, 2005 Rev. Mod. Phys. 77 977\.
  * [17] D. Pedone, M. Langecker, G. Abstreiter, U. Rant. 2011 Nano Lett. 11 1561\.
  * [18] Z. Siwy, I. D. Kosinska, A. Fulinski, C. R. Martin, 2005 Phys. Rev. Lett. 94 048102\.
  * [19] C. Hu, Y. Ou, J. Wu, Q. Chen, B. Ai, 2015 J. Stat. Mech. 2015 05025\.
  * [20] H. W. Hu, L. Du, L. H. Qu, Z. L. Cao, Z. C. Deng , Y. C. Lai, 2021 Phys. Rev. Research 3 033162\.
  * [21] B. Wang, H. Chen, Y. Wu, 2020 Physica A 537 122779\.
  * [22] J. Richardi, M. P. Pileni, J. J. Weis, 2009 J. Chem. Phys. 130 124515\.
  * [23] C. Iss, D. Midou, A. Moreau, D. Held, et al., 2019 Soft Matter 15 2971\.
  * [24] P. K. Ghosh, V. R. Misko, F. Marchesoni, F. Nori, 2013 Phys. Rev. Lett. 110 268301\.
  * [25] X. Ao, P. K. Ghosh, Y. Li, G. Schmid, P. Hänggi, F. Marchesoni, 2015 EPL 109 10003\.
  * [26] C. Bechinger, R. Di Leonardo, H. Löwen, C. Reichhardt, G. Volpe, G. Volpe, 2016 Rev. Mod. Phys. 88 045006\.
  * [27] A. Murali, P. Dolai, A. Krishna, K. Vijay Kumar, S. Thutupalli, 2022 Phys. Rev. Research 4 013136\.
  * [28] Z. Liu, L. Du , W. Guo , D. Mei, 2016 Eur. Phys. J. B 89 222\.
  * [29] F. Li, B. Ai, 2017 Physica A 484 27\.
  * [30] A. Pototsky, U. Thiele, H. Stark, 2016 Eur. Phys. J. E 39 51\.
  * [31] P. Hänggi, F. Marchesoni, S. Savelev, G. Schmid, 2010 Phys. Rev. E 82 041121\.
  * [32] Y. Han, A. M. Alsayed, M. Nobili, J. Zhang, T. C. Lubensky, A. G. Yodh, 2006 Science 314 626\.
  * [33] M. Pu, H. Jiang, Z. Hou, 2017 Soft Matter 13 4112\.
  * [34] Y. He, B. Ai, 2010 Phys. Rev. E 81 021110\.
  * [35] L. Machura, M. Kostur, P. Talkner, J. Luczka, F. Marchesoni, P. Hänggi, 2004 Phys. Rev. E 70 061105\.
  * [36] P. Reimann, C. Van den Broeck, H. Linke, P. Hänggi, J. M. Rubi, A. Perez-Madrid, 2001 Phys. Rev. Lett. 87 010602\.


# Towards Non-I.I.D. and Invisible Data with FedNAS:
Federated Deep Learning via Neural Architecture Search

Chaoyang He Murali Annavaram Salman Avestimehr
University of Southern California
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Federated Learning (FL) has been proved to be an effective learning framework
when data cannot be centralized due to privacy, communication costs, and
regulatory restrictions. When training deep learning models under an FL
setting, people employ the predefined model architecture discovered in the
centralized environment. However, this predefined architecture may not be the
optimal choice because it may not fit data with non-identical and independent
distribution (non-IID). Thus, we advocate automating federated learning
(AutoFL) to improve model accuracy and reduce the manual design effort. We
specifically study AutoFL via Neural Architecture Search (NAS), which can
automate the design process. We propose a Federated NAS (FedNAS) algorithm to
help scattered workers collaboratively searching for a better architecture
with higher accuracy. We also build a system based on FedNAS. Our experiments
on non-IID dataset show that the architecture searched by FedNAS can
outperform the manually predefined architecture 111This paper was accepted to
CVPR 2020 workshop on neural architecture search and beyond for representation
learning. We released the source code at https://fedml.ai.

##  1 Introduction

While CNN (Convolutional Neural Network) is capable of performing various
computer vision tasks, it consumes a significant amount of data to achieve
state-of-the-art performance. In the case of COVID-19, results from CT
imaging, an effective method used to diagnose coronavirus-induced pneumonia,
varies with the age and immunity status of the patient, as well as the disease
stage at the time of scanning, underlying health conditions, and other drug
interventions used at the time. Thus, significant amounts of CT image data
from patients across hospitals around the world are essential for rapid and
accurate diagnosis and treatment. However, centralizing such small and
scattered heterogeneous data from various organizations is challenging due to
privacy and confidentiality concerns, high communication and storage costs,
protection of intellectual property, regulatory restrictions, and legal
constraints. Consequently, a promising approach to solve this problem is
Federated Learning (FL) [10], a decentralized learning framework which can
train models without sharing the raw data with the collaborating nodes. For
example, in Figure 1, each hospital can each learn a CNN model without
disclosing its own raw CT images.

Figure 1: Federated Neural Architecture Search (step 1: search locally; step
2: sending the gradient of $\alpha$ and $\omega$ to the server side; step 3:
merge gradient to get global $\alpha$ and $\omega$; step 4: synchronize
updated $\alpha$ and $\omega$ to each client.)

However, as [8] points out, when training deep neural networks under the FL
setting where the data is non-IID (non-identical and independent
distribution), using the predefined model architecture may not be the optimal
design choice. Since the data distribution is invisible to researchers, to
find a better model architecture with higher accuracy, developers must design
or choose multiple architectures, then tune hyperparameters remotely to fit
the scattered data. This process is extremely expensive because attempting
many rounds of training on edge devices results in a remarkably higher
communication cost and on-device computational burden than the data center
environment.

Due to these challenges, we advocate automating federated learning, which we
term as AutoFL (Automated Federated Learning). In this work, we specifically
study AutoFL via Neural Architecture Search (NAS), which automates the design
process of the model architecture without inefficient manual attempts. In the
centralized data environment, NAS significantly outperforms many manually
designed architectures [7]. However, it is still not clear whether NAS can
really help boost the model performance in a heterogeneous distributed data
environment like FL. Therefore, we aim to answer a fundamental question: can
we design a collaborative NAS framework that helps improve model performance
in the federated deep learning setting, in which the data distribution is non-
identical and independent?

To answer this question, we consider the cross-organization scenario, in which
each client in the network is a GPU-equipped edge server located in an
organization (e.g., the hospital shown in Figure 1). We then propose Federated
NAS (FedNAS) to search architectures among edge servers collaboratively. As
the process shown in Figure 1, in FedNAS, we first utilize the gradient-based
method MiLeNAS [4] as local searcher on the local data of each worker. MiLeNAS
can be easily distributed, and it is efficient in terms of search time.
Formally, it formulates NAS as a mixed-level problem:
$w=w-\eta_{w}\nabla_{w}\mathcal{L}_{\mathrm{tr}}(w,\alpha),\alpha=\alpha-\eta_{\alpha}\left(\nabla_{\alpha}\mathcal{L}_{\mathrm{tr}}(w,\alpha)+\lambda\nabla_{\alpha}\mathcal{L}_{\mathrm{val}}(w,\alpha)\right)$,
where $w$ represents the network weight and $\alpha$ determines the neural
architecture. $\mathcal{L}_{tr}(w,\alpha)$ and $\mathcal{L}_{val}(w,\alpha)$
denote the loss with respect to training data and validation data with $w$ and
$\alpha$, respectively. After the local search, each worker then synchronizes
weights $w$ and architecture $\alpha$ with other workers by weighted
aggregation.

We design an AutoFL system based on the FedNAS to evaluate our idea. We
construct a non-IID image dataset based on CIFAR10 and deploy our system in a
distributed computing environment, which consists of 16 clients and 1 server.
Our experiments show that Fed-NAS can search for a better architecture with a
higher accuracy in only a few hours compared to FedAvg [10], which purely
utilizes the state-of-the-art manually designed architecture.

We also summarize our research directions that have the potential to further
improve the efficiency and effectiveness of our proposed FedNAS.

##  2 Related Works

Recently, Neural Architecture Search (NAS) [7] has attracted widespread
attention due to its advantages over manually designed models. There are three
major NAS methods: evolutionary algorithms, reinforcement learning-based
methods, and gradient-based methods [4]. While in the Federated Learning (FL)
domain [10, 3], using designed model architectures and optimizing by FedAvg
[10] is the main method to improve model performance. To our knowledge, NAS is
rarely studied in FL setting. Although [8] first proposed the concept of
automating FL via NAS, the concrete method and details are never given. In our
work, we propose a FedNAS algorithm and demonstrate its efficacy. Our AutoFL
system design is relevant to [1]. However, its system only supports manually
designed architectures.

##  3 Proposed Method

###  3.1 Problem Definition

In federated learning setting, there are $K$ nodes in the network. Each node
has a dataset
$\mathcal{D}_{k}:=\left\\{\left(x_{i}^{k},y_{i}\right)\right\\}_{i=1}^{N_{k}}$
which is non IID. When collaboratively training a deep neural networks (DNN)
model with $K$ nodes, the objective function is defined as:

$\min_{w}f(w,\underbrace{\alpha}_{fixed})\stackrel{{\scriptstyle\text{ def
}}}{{=}}\min_{w}\sum_{k=1}^{K}\frac{N_{k}}{N}\cdot\frac{1}{N_{k}}\sum_{i\in\mathcal{D}_{k}}\ell(x_{i},y_{i};w,\underbrace{\alpha}_{fixed})$
(1)

where $w$ represents the network weight, $\alpha$ determines the neural
architecture, and $\ell$ is the loss function of the DNN model. To minimize
the objective function above, previous works choose a fixed model architecture
$\alpha$ and then design variant optimization techniques to train the model
$w$.

We propose to optimize the federated learning problem from a completely
different angle, optimizing $w$ and $\alpha$ simultaneously. Formally, we can
reformulate the objective function as:

$\min_{w,\alpha}f(w,\alpha)\stackrel{{\scriptstyle\text{ def
}}}{{=}}\min_{w,\alpha}\sum_{k=1}^{K}\frac{N_{k}}{N}\cdot\frac{1}{N_{k}}\sum_{i\in\mathcal{D}_{k}}\ell(x_{i},y_{i};w,\alpha)$
(2)

In other words, for the non-IID dataset scattered across many workers, our
goal is to search for an optimal architecture $\alpha$ and related model
parameters $w$ to fit the dataset more effectively thus achieve better model
performance. In this work, we consider searching for CNN architecture to
improve the performance of the image classification task.

###  3.2 Search Space

Figure 2: Search Space

Normally, NAS includes three consecutive components: the search space
definition, the search algorithm, and the performance estimation strategy [7].
Our search space follows the mixed-operation search space defined in DARTS [9]
and MiLeNAS [4], where we search in two shared convolutional cells and then
build it up as an entire model architecture (as shown in Figure 2). Inside the
cell, to relax the categorical candidate operations between two nodes (e.g.,
convolution, max pooling, skip connection, zero) to a continuous search space,
mixed operation using softmax over all possible operations is proposed:

$\bar{o}^{(i,j)}(x)=\sum_{k=1}^{d}\underbrace{\frac{\exp(\alpha_{k}^{(i,j)})}{\sum_{k^{\prime}=1}^{d}\exp(\alpha_{k^{\prime}}^{(i,j)})}}_{p_{k}}o_{k}(x)$
(3)

where the weight $p_{k}$ of the mixed operation $\bar{o}^{(i,j)}(x)$ for a
pair of nodes $(i,j)$ is parameterized by a vector $\alpha^{i,j}$. Thus, all
architecture operation options inside a network (model) can be parameterized
as $\alpha$. More details are introduced in Appendix A.1.

###  3.3 Local Search

Following the above-mentioned search space, each worker searches locally by
utilizing the mixed-level optimization technique MiLeNAS [4]:

$\displaystyle w=w-\eta_{w}\nabla_{w}\mathcal{L}_{\mathrm{tr}}(w,\alpha)$ (4)
$\displaystyle\alpha=\alpha-\eta_{\alpha}\left(\nabla_{\alpha}\mathcal{L}_{\mathrm{tr}}(w,\alpha)+\lambda\nabla_{\alpha}\mathcal{L}_{\mathrm{val}}(w,\alpha)\right)$

where $\mathcal{L}_{tr}(w,\alpha)$ and $\mathcal{L}_{val}(w,\alpha)$ denote
the loss with respect to the local training data and validation data with $w$
and $\alpha$, respectively.

###  3.4 Federated Neural Architecture Search

We propose FedNAS, a distributed neural architecture search algorithm that
aims at optimizing the objective function in Equation 2 under the FL setting.
We introduce FedNAS corresponding to four steps in Figure 1: 1) The local
searching process: each worker optimizes $\alpha$ and $w$ simultaneously using
Eq. 4 for several epochs; 2) All clients send their $\alpha$ and $w$ to the
server; 3) The central server aggregates these gradients as follows:

$\displaystyle w_{t+1}\leftarrow\sum_{k=1}^{K}\frac{N_{k}}{N}w_{t+1}^{k}$ (5)
$\displaystyle{\alpha}_{t+1}\leftarrow\sum_{k=1}^{K}\frac{N_{k}}{N}{\alpha}_{t+1}^{k}$

4) The server sends back the updated $\alpha$ and $w$ to clients, and each
client updates its local $\alpha$ and $w$ accordingly, before running the next
round of searching. This process is summarized in Algorithm 1. After
searching, an additional evaluation stage is conducted by using a traditional
federated optimization method such as FedAvg [10]. Merging search and
evaluation into one stage is one of future works (check Appendix B for more
details).

Algorithm 1 FedNAS Algorithm.

1: Initialization: initialize $w_{0}$ and $\alpha_{0}$; $K$ clients are
selected and indexed by $k$; $E$ is the number of local epochs; $T$ is the
number of rounds.

2: Server executes:

3: for each round $t=0,1,2,...,T-1$ do

4: for each client $k$ in parallel do

5:
$w_{t+1}^{k},{\alpha}_{t+1}^{k}\leftarrow\text{ClientLocalSearch}(k,w_{t},\alpha_{t})$

6: $w_{t+1}\leftarrow\sum_{k=1}^{K}\frac{N_{k}}{N}w_{t+1}^{k}$

7: ${\alpha}_{t+1}\leftarrow\sum_{k=1}^{K}\frac{N_{k}}{N}{\alpha}_{t+1}^{k}$

8:

9: ClientLocalSearch($k$, $w$, $\alpha$): // Run on client $k$

10: for $e$ in epoch do

11: for minibatch in training and validation data do

12: Update $w=w-\eta_{w}\nabla_{w}\mathcal{L}_{\mathrm{tr}}(w,\alpha)$

13: Update

14:
$\alpha=\alpha-\eta_{\alpha}\left(\nabla_{\alpha}\mathcal{L}_{\mathrm{tr}}(w,\alpha)+\lambda\nabla_{\alpha}\mathcal{L}_{\mathrm{val}}(w,\alpha)\right)$

15: return $w$, $\alpha$ to server

###  3.5 AutoFL System Design

We design an AutoFL system using FedNAS based on FedML [2], an open-source
research library for federated learning. The system architecture is shown in
Figure 3. This design separates the communication and the model training into
two core components shared by the server and clients. The first is the
communication protocol component, which is responsible for low-level
communication among the server and clients. The second is the on-device deep
learning component, which is built based on the popular deep learning
framework PyTorch. These two components are encapsulated as ComManager,
Trainer, and Aggregator, providing high-level APIs for the above layers. With
the help of these APIs, in ClientManager, the client can train or search for
better architectures and then send its results to the server-side, while in
ServerManager, the server can aggregate and synchronize the model architecture
and the model parameters with the client-side.

Figure 3: Abstract System Architecture of AutoFL

##  4 Experiments and Results

###  4.1 Experimental Setup

Implementation and Deployment. We set up our experiment in a distributed
computing network equipped with GPUs. There are 17 nodes in total, one
representing the server-side, and the other 16 nodes representing clients,
which can be organizations in the real world (e.g., the hospitals). Each node
is a physical server that has an NVIDIA RTX 2080Ti GPU card inside. We
deployed the FedNAS system described in Appendix 3.5 on each node. Our code
implementation is based on PyTorch 1.4.0, MPI4Py
222https://pypi.org/project/mpi4py/ 3.0.3 , and Python 3.7.4. For simplicity,
we assume all the clients join the training process for every communication
round.

Task and Dataset. Our training task is image classification on the CIFAR10
dataset, which consists of 60000 32x32 color images in 10 classes, with 6000
images per class. We generate non-IID (non identical and independent
distribution) dataset by splitting the 50000 training images into K clients in
an unbalanced manner: sampling $\mathbf{p}_{c}\sim\operatorname{Dir}_{J}(0.5)$
and allocating a $\mathbf{p}_{c,k}$ proportion of the training samples of
class $c$ to local client $k$. The 10000 test images are used for a global
test after the aggregation of each round. For different methods, we record the
global test accuracy as the metric to compare model performance. Since the
model performance is sensitive to the data distribution, we fix the non-IID
dataset in all experiments for a fair comparison. The actual data distribution
used for experiments can be found in Appendix A.2.

Model and Baseline. We compare FedNAS with FedAvg. FedAvg runs on a manually
designed architecture, DenseNet[6], which extends ResNet [5], but higher
performance and fewer model parameters than ResNet.

###  4.2 Results on non-IID dataset

(a) FedAvg on DenseNet

(b) The search stage of FedNAS

Figure 4: Test Accuracy on Non-IID Dataset (multiple runs)

Figure 4(a) shows the results in a specific non-IID data distribution. For a
fair comparison, results are obtained by fine-tuning hyperparameters of each
method, and each method is run three times. Figure 4(b) shows the test
accuracy during the searching process. Interestingly, FedNAS can achieve
higher accuracy than FedAvg during the searching process (81.24% in Figure
4(b); 77.78% in Figure 4(a)). This further confirms the efficacy of FedNAS. We
also evaluate the searched architecture under this data distribution. We found
that each run of FedNAS can obtain a higher test accuracy than each run of
FedAvg. On average, the architecture searched by FedNAS obtains a test
accuracy 4% higher than FedAvg.

Hyperparameters and visualization of the searched architecture can be found in
Appendix A.3 and A.4, respectively.

Remark. We also run experiments on other distributions of non-IID datasets, in
which FedNAS is also demonstrated to beat FedAvg, confirming that FedNAS
searches for better architectures with a higher model performance.

###  4.3 Evaluation of the Efficiency

Method | Search | Parameter | Hyperparameter
---|---|---|---
Time | Size
FedAvg | $>$ 3 days | - | rounds = 100
(single) | local epochs=20
FedAvg | 12 hours | 20.01M | batch size=64
(distributed) |
FedNAS | 33 hours | - | rounds = 50
(single) | local epochs=5
FedNAS | $<$ 5 hours | 1.93M | batch size=64
(distributed) |

Table 1: Efficiency Comparison (16 RTX2080Ti GPUs as clients, and 1 RTX2080Ti
as server)

In order to more comprehensively reflect our distributed search overhead, we
developed the single-process and distributed version of FedNAS and FedAvg. The
single-process version simulates the algorithm by performing a client-by-
client search on a single GPU card. As shown in Table 1, compared to FedAvg
and manually designed DenseNet, FedNAS can find better architecture with fewer
parameters in less time. FedAvg spends more time because it requires more
local epochs to converge.

##  5 Future Works

We introduce our future works in Appendix B.

##  6 Conclusion

We study automating federated learning (AutoFL) via Neural Architecture Search
(NAS) by proposing a Federated NAS (FedNAS) algorithm that can help scatter
workers collaboratively searching for a better architecture with higher
accuracy. We build an AutoFL system based on FedNAS. Our experiments on the
non-IID dataset show that the architecture searched by FedNAS can outperform
FedAvg training on the manually designed architecture.

## References

  * [1] Keith Bonawitz, Hubert Eichner, Wolfgang Grieskamp, Dzmitry Huba, Alex Ingerman, Vladimir Ivanov, Chloe Kiddon, Jakub Konecny, Stefano Mazzocchi, H Brendan McMahan, et al.  Towards federated learning at scale: System design.  arXiv preprint arXiv:1902.01046, 2019.
  * [2] Chaoyang He, Songze Li, Jinhyun So, Mi Zhang, Hongyi Wang, Xiaoyang Wang, Praneeth Vepakomma, Abhishek Singh, Hang Qiu, Li Shen, Peilin Zhao, Yan Kang, Yang Liu, Ramesh Raskar, Qiang Yang, Murali Annavaram, and Salman Avestimehr.  Fedml: A research library and benchmark for federated machine learning.  arXiv preprint arXiv:2007.13518, 2020.
  * [3] Chaoyang He, Conghui Tan, Hanlin Tang, Shuang Qiu, and Ji Liu.  Central server free federated learning over single-sided trust social networks.  arXiv preprint arXiv:1910.04956, 2019.
  * [4] Chaoyang He, Haishan Ye, Li Shen, and Tong Zhang.  Milenas: Efficient neural architecture search via mixed-level reformulation.  In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
  * [5] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  * [6] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger.  Densely connected convolutional networks.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4700–4708, 2017.
  * [7] Frank Hutter, Lars Kotthoff, and Joaquin Vanschoren, editors.  Automatic Machine Learning: Methods, Systems, Challenges.  Springer, 2019.
  * [8] Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Keith Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al.  Advances and open problems in federated learning.  arXiv preprint arXiv:1912.04977, 2019.
  * [9] Hanxiao Liu, Karen Simonyan, and Yiming Yang.  Darts: Differentiable architecture search.  arXiv preprint arXiv:1806.09055, 2018.
  * [10] H Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, et al.  Communication-efficient learning of deep networks from decentralized data.  arXiv preprint arXiv:1602.05629, 2016.

Client ID | Numbers of samples in the classes | Distribution
---|---|---
$c_{0}$ | $c_{1}$ | $c_{2}$ | $c_{3}$ | $c_{4}$ | $c_{5}$ | $c_{6}$ | $c_{7}$ | $c_{8}$ | $c_{9}$
k=0 | 144 | 94 | 1561 | 133 | 1099 | 1466 | 0 | 0 | 0 | 0 |
k=1 | 327 | 28 | 264 | 16 | 354 | 2 | 100 | 20 | 200 | 3 |
k=2 | 6 | 6 | 641 | 1 | 255 | 4 | 1 | 2 | 106 | 1723 |
k=3 | 176 | 792 | 100 | 28 | 76 | 508 | 991 | 416 | 215 | 0 |
k=4 | 84 | 1926 | 1 | 408 | 133 | 24 | 771 | 0 | 0 | 0 |
k=5 | 41 | 46 | 377 | 541 | 7 | 235 | 54 | 1687 | 666 | 0 |
k=6 | 134 | 181 | 505 | 720 | 123 | 210 | 44 | 58 | 663 | 221 |
k=7 | 87 | 2 | 131 | 1325 | 1117 | 704 | 0 | 0 | 0 | 0 |
k=8 | 178 | 101 | 5 | 32 | 1553 | 10 | 163 | 9 | 437 | 131 |
k=9 | 94 | 125 | 0 | 147 | 287 | 100 | 23 | 217 | 608 | 279 |
k=10 | 379 | 649 | 106 | 90 | 35 | 119 | 807 | 819 | 3 | 85 |
k=11 | 1306 | 55 | 681 | 227 | 202 | 34 | 0 | 648 | 0 | 0 |
k=12 | 1045 | 13 | 53 | 6 | 77 | 70 | 482 | 7 | 761 | 494 |
k=13 | 731 | 883 | 15 | 161 | 387 | 552 | 4 | 1051 | 0 | 0 |
k=14 | 4 | 97 | 467 | 899 | 0 | 407 | 50 | 64 | 1098 | 797 |
k=15 | 264 | 2 | 93 | 266 | 412 | 142 | 806 | 2 | 243 | 1267 |
Table 2: The actual heterogeneous data distribution (non-IID) used in
experiments

##  Appendix A More Experiment Details

###  A.1 Details of the Search Space Definition

We adopt the following 7 operations in our CIFAR-10 experiments: 3 $\times$ 3
and 5 $\times$ 5 separable convolutions, 3 $\times$ 3 and $5\times 5$ dilated
separable convolutions, 3 $\times$ 3 max pooling, 3 $\times$ 3 average
pooling, identity, and zero.

The network is formed by stacking convolutional cells multiple times. Cell $k$
takes the outputs of cell $k-2$ and cell $k-1$ as its input. Each cell
contains seven nodes: two input nodes, one output node, and the other four
intermediate nodes inside the cell. The input of the first intermediate node
is set equal to two input nodes, and the other intermediate nodes take all
previous intermediate nodes’ output as input. The output node concatenates all
intermediate nodes’ output depth-wise. There are two types of cells: the
normal cell and the reduction cell. The reduction cell is designed to reduce
the spatial resolution of feature maps, located at 1/3 and 2/3 of the total
depth of the network. Architecture parameters determine the discrete operation
value between two nodes. All normal cells and all reduction cells share the
same architecture parameters $\bm{\alpha}_{\text{n}}$ and
$\bm{\alpha}_{\text{r}}$, respectively. By this definition, our method
alternatively optimizes architecture parameters ($\bm{\alpha}_{\text{n}}$,
$\bm{\alpha}_{\text{r}}$) and model weight parameters $\bm{w}$.

###  A.2 Details of the heterogeneous distribution on each client (non-IID)

Table 2 shows the actual data distribution used in our experiments. We can see
that the sample number of each class in each worker is highly unbalanced. Some
classes in a worker even have no samples, and some classes take up most of the
proportion (highlighted in the table).

###  A.3 Hyperparameter Setting

We report important well-tuned hyperparameters used in our experiments. FedNAS
searches 50 communication rounds using 5 local searching epochs, with a batch
size of 64. For FedAvg, DenseNet201 is used for training, with 100
communication rounds, 20 local epochs, a learning rate of 0.08, and a batch
size of 64. Both methods use the same data augmentation techniques that are
used in image classification, such as random crops, flips, and normalization.
More details and other parameter settings can be found in our source code.

###  A.4 Visualization of the Search Architecture

Figure 5: Normal Cell Architecture Figure 6: Reduction Cell Architecture

We report the architecture searched based on the above non-IID dataset and
hyper-parameter setting. Figure 5 and 6 show the normal cell architecture and
the reduction cell architecture, respectively. We can see that the reduction
cell uses more pooling operations while the normal cell has more convolutional
operations.

##  Appendix B Future Works

Our future work aims to improve the FedNAS framework from form the following
perspectives.

  * •

Local NAS under Resource Constraint. Our current search space fits for cross-
organization federated learning, where the edge device can be equipped with
powerful GPU devices. But when used in resource-constrained environments such
as smartphones or IoT devices, the memory of our search space is too large.
Searching on compact search space or using sampling methods are potential
solutions to this challenge.

  * •

One Stage NAS. In our FedNAS framework, we first search the architecture and
then train it from scratch. This two-stage training process can be optimized
by being merged into a single process. In FedNAS, a naive way to perform one-
stage NAS is to continue training the model after the search process is
finished. More advanced methods to further improve efficiency are open
problems.

  * •

Asynchronous Aggregation. The searching or training time can be reduced by
asynchronous aggregation when each worker has a different number of samples,
communication abilities, and computational resources. For FedNAS, the impact
of the asynchronous aggregation on model performance should be evaluated.

  * •

Personalized NAS. Since the data distribution in each client is heterogeneous,
a personalized model in each worker may have better accuracy than a global
shared model. To meet this requirement, searching for personalized
architectures is a challenging task.


11institutetext: Australian Institute for Machine Learning, University of
Adelaide 22institutetext: Macquarie University 33institutetext: University of
Wollongong 44institutetext: Flinders University

# Structural Attention: Rethinking Transformer for Unpaired Medical Image
Synthesis

Vu Minh Hieu Phan 11 0000-0003-3861-0296 Yutong Xie 11 Bowen Zhang 11 Yuankai
Qi 22 Zhibin Liao 11 Antonios Perperidis 11 Son Lam Phung 33 Johan W. Verjans
11 Minh-Son To 44

###### Abstract

Unpaired medical image synthesis aims to provide complementary information for
an accurate clinical diagnostics, and address challenges in obtaining aligned
multi-modal medical scans. Transformer-based models excel in imaging
translation tasks thanks to their ability to capture long-range dependencies.
Although effective in supervised training settings, their performance falters
in unpaired image synthesis, particularly in synthesizing structural details.
This paper empirically demonstrates that, lacking strong inductive biases,
Transformer can converge to non-optimal solutions in the absence of paired
data. To address this, we introduce UNet Structured Transformer (UNest) — a
novel architecture incorporating structural inductive biases for unpaired
medical image synthesis. We leverage the foundational Segment-Anything Model
to precisely extract the foreground structure and perform structural attention
within the main anatomy. This guides the model to learn key anatomical
regions, thus improving structural synthesis under the lack of supervision in
unpaired training. Evaluated on two public datasets, spanning three
modalities, i.e., MR, CT, and PET, UNest improves recent methods by up to
19.30% across six medical image synthesis tasks. Our code is released at
https://github.com/HieuPhan33/MICCAI2024-UNest.

##  1 Introduction

Multi-modal medical imaging, including modalities such as computed tomography
(CT), magnetic resonance imaging (MRI) and positron emission tomography (PET),
can be a valuable tool with multiple applications in clinical practice, aiding
in accurate disease diagnosis [4, 7], lesion detection [5] and treatment
planning [1]. While these modalities provide complementary insights, obtaining
multiple scans can be time-consuming, costly, and, most importantly,
potentially harmful to patients through additional radiation exposure [21]
(i.e., CT and PET). Medical image synthesis has emerged as a potential
solution to enable comprehensive patient evaluation without requiring multiple
scans.

Most synthesis methods employ a supervised Pix2Pix [9, 6, 31], showing
effectiveness given paired data between two domains. Yet, acquiring
substantial paired data is challenging due to variations in resolution and
patient positioning. This highlights the advantage of unpaired image
synthesis. CycleGAN [33] is a pioneering work in unpaired image translation,
which imposes domain cycle consistency using two generators. Subsequent
approaches further incorporate structural consistency loss, either by aligning
pixel-wise correlations [27, 28, 17] or preserving shape fidelity [23, 11]
between original and synthesized images. Previous methods adopt convolution
operators with the local inductive biases, guiding the model to extract local
features. This limits their ability to capture long-range spatial contexts.

Figure 1: (a) Synthetic MR-to-CT results of different ViT methods: ResViT [6],
UNETR [12], and our proposed UNest, where our UNest most accurately preserves
the structural carvity. (b) Attention maps for two patches in a smooth brain
region (highlighted by a purple star) a structural nasal cavity (indicated by
a red star). Transformer methods tend to focus on less relevant background
features.

Recently, Vision Transformer (ViT) [8] discards the inductive biases to learn
features globally. It has achieved state-of-the-art results for many
supervised image-to-image translation tasks such as medical semantic
segmentation [12, 24], and image super-resolution [3, 29]. ResViT [6] has
advanced the use of hybrid Convolution-Transformer models for paired image
translation. While several unpaired methods [22, 25] develop hybrid models,
their architectures primarily rely on convolutional operators.

Unfortunately, ViT models [12, 6] struggle when applied to unpaired medical
image synthesis. Fig. 1(a) shows examples that current ViT methods fail to
synthesize intricate anatomical structures in the nasal cavities. Without
convolution-like inductive biases, recent studies [16, 20] show that ViT is
less sample-efficient and fail to attend to discriminative features in low
data regime. In this paper, we empirically discover that the ViT’s performance
is non-optimal in unpaired image translation scenarios, where the lack of
supervision signals exaggerates the problem. Fig. 1(b) shows that UNETR and
ResViT attend to less relevant background regions when learning foreground
structures in the brain and nasal cavity (respectively denoted by the pink and
red stars), suggesting non-optimal solution under unpaired training. Solutions
such as Swin UNETR [24] use local attention on specific windows; yet,
confining the context within a restricted window can exclude crucial
anatomical regions.

This paper introduces a strong inductive bias into the Transformer and
proposes a simple-yet-effective architecture, called UNEt Structural
Transformer (UNest) for effective unpaired medical image synthesis. At the
core of UNest, our Structural Transformer (ST) block segregates foreground and
background tokens, and injects structural and local inductive biases
respectively on each region. When synthesizing foreground, our structural
attention aggregates contexts within the anatomy to encode relationships
between anatomical regions. Fig. 1(b) shows that UNest adaptively focuses on
local areas when synthesizing structural cavities (pink) and more globally
when generating smooth brain (red).

Our contributions can be summarized as follows. 1) We show empirically that
injecting a structural inductive bias enables Transformer to focus on
discriminative areas, thus enhancing the synthesis of anatomical structures in
unpaired image synthesis. 2) We introduce a simple-yet-effective architecture,
coined UNet Structural Transformer (UNest), applying a dual attention
strategy: structural attention for the foreground and local attention for the
background. 3) Evaluated across six translation tasks covering three
modalities: MR, CT, and PET, our method significantly improves the accuracy
across various anatomical structures, from head to whole-body images.

##  2 Method

CycleGAN’s overview. Let $\\{\mathbf{x}_{n}\\}_{i=1}^{n}$ and
$\\{\mathbf{y}_{m}\\}_{m=1}^{M}$ denote unpaired training samples from domains
$X$ and $Y$. Based on CycleGAN [33], our framework consists of two generators
$G_{XY}:X\mapsto Y$ and $G_{YX}:Y\mapsto X$, learning forward and backward
mappings between the two domains, as shown in Fig. 2(c). Here, $G_{XY}$ and
$G_{YX}$ are trained to fool the respective discriminator $D_{Y}$ and $D_{X}$
via adversarial loss

$\mathcal{L}_{\text{adv,Y}}(G_{\text{X}},D_{\text{Y}})=\mathbb{E}_{\mathbf{y}\sim
p(\mathbf{y})}\big{[}\log
D_{\text{Y}}(\mathbf{y})\big{]}+\mathbb{E}_{\mathbf{x}\sim
p(\mathbf{x})}\big{[}\log(1-D_{\text{Y}}(G_{\text{X}}(\mathbf{x}))).$ (1)

Adversarial loss $\mathcal{L}_{\text{adv,X}}$ for training generator $G_{Y}$
is defined similarly. For unpaired training, CycleGAN imposes the cycle
consistency loss

$\mathcal{L}_{\text{cycle}}=\mathbb{E}_{\mathbf{x}\sim
p(\mathbf{x})}\|\mathbf{x}-G_{Y}(G_{X}(\mathbf{x}))\|+\mathbb{E}_{\mathbf{y}\sim
p(\mathbf{y})}\|\mathbf{y}-G_{X}(G_{Y}(\mathbf{y}))\|.$ (2)

Figure 2: (a) UNest architecture uses Structural Transformer blocks (pink
block), as shown in (b), to perform dual-attention strategy on foreground and
background tokens separately. The decoder upsamples features using
deconvolutional layers and skip-connections from early encoder layers. (c)
CycleGAN with UNest generators.

Analysis of Transformer models on unpaired image synthesis. Given a feature
map $F\in\mathbb{R}^{H\times W\times K}$, the attention mechanism in
Transformer [26] first projects a feature map $F$ into query $Q$, key $K$, and
value $V$. Given a query token $\mathbf{q}\in\mathbb{R}^{K}$, it aggregates
the contextual information from a set of $T$ surrounding key-value tokens
$K_{S}\in\mathbb{R}^{T\times K}$, and $V_{S}\in\mathbb{R}^{T\times K}$. The
global attention methods (e.g., UNETR [12], ResViT [6]) consider aggregating
the entire image with a global scope $S$:

$S=\\{(i,j)|i\in[1,H],j\in[1,W]\\}.$ (3)

Learning on a global scope, Transformer is considered lacking local inductive
bias. In contrast, the local attention methods (e.g., Swin UNETR) incorporate
a local inductive bias, inspired by CNN. They aggregate tokens from a local
$m\times m$ window surrounding the query token:

$S=\\{(i,j)|i\in[i_{q}-\frac{m}{2},i_{q}+\frac{m}{2}],j\in[j_{q}-\frac{m}{2},j_{q}+\frac{m}{2}]\\},$
(4)

where $(i_{q},j_{q})$ is the index of the query token $q$ in the image. The
self-attention module $\text{SA}(\cdot,\cdot)$ learns a new query feature by
aggregating the key $K_{S}$ and value vectors $V_{S}$ within the pre-defined
scope $S$ as $\tilde{F}=\text{SA}(F,S)=\text{Softmax}(QK^{T}_{S})V_{S}$.

Despite achieving high accuracy on supervised learning tasks [6, 32, 30, 12],
current Transformer models falter under unpaired image synthesis. Fig. 3 shows
that global attention adopted in ResViT [6] and UNETR [12] deforms the hip
structures, while the local attention in Swin UNETR [24] generates artifacts.
This indicates that lacking strong inductive biases makes current Transformer
models converge to a non-optimal solution under unpaired training.

Figure 3: Visual results for PET-to-CT on AutoPET dataset [10].

Our UNet Structural Transformer. Given the lack of supervision in unpaired
image translation, we propose to inject the structural inductive bias,
enforcing the attention on discriminative anatomical structures. We introduce
a UNest architecture, for unpaired medical image synthesis. UNest consists of
Structural Transformer (ST) blocks and a convolutional decoder with skip
connections, as shown in Fig. 2(a).

As shown in Fig. 2(a), Transformer-based encoder $E(\cdot)$ first splits the
input image $\mathbf{x}$ of shape $H\times W$ into a sequence of $HW/p^{2}$
non-overlapping patches with patch size $p\times p$. We flatten this sequence
and denote it as $X_{p}\in\mathbb{R}^{(H^{\prime}\cdot W^{\prime})\times P}$,
where $H^{\prime}=\frac{H}{p}$, $W^{\prime}=\frac{W}{p}$, and $P=p^{2}$ is the
total size of each patch token. Each patch token
$\mathbf{x}^{i}_{p}\in\mathbb{R}^{P}$ is projected into a $K$ dimensional
embedding space via linear layer $\theta\in\mathbb{R}^{P\times K}$:

$\mathbf{f}_{0}=[\mathbf{x}^{1}_{p}\theta,\mathbf{x}^{2}_{p}\theta,\ldots,\mathbf{x}^{H^{\prime}W^{\prime}}_{p}\theta].$
(5)

Different from previous global [12, 2] and local attention [24], our
structural attention aggregates contexts within the main anatomy. In other
words, the query scope $S$ consists of tokens in the foreground anatomy. To
realize this, we use a lightweight patch classifier to classify each token as
foreground or background. The classifier uses a $1\times 1$ convolution,
followed by a Sigmoid function. The result is a binary mask
$M\in\mathbb{R}^{H^{\prime}\times W^{\prime}}$, where each element $M_{h,w}$
represents the probability of the token at position $(h,w)$ being foreground.

To train the patch classifier, the ground-truth binary mask $\hat{M}$ is
extracted by leveraging foundational Segment-Anything Model (SAM) [14]. As SAM
is a class-agnostic segmentation, we use it to extract all masks in the image
and choose the one covering the largest foreground area, indicative of the
primary anatomy. We compute the ground-truth masks offline. During training,
they are loaded to compute the binary cross-entropy (BCE) loss between ground-
truth and predicted masks to optimize the patch classifier:

$\mathcal{L}_{\text{mask}}=-\frac{1}{H^{\prime}\times
W^{\prime}}\displaystyle\sum_{i=1}^{H^{\prime}}\displaystyle\sum_{j=1}^{W^{\prime}}\left[\hat{M}_{ij}\log(M_{ij})+(1-\hat{M}_{ij})\log(1-M_{ij})\right].$
(6)

The decoder receives the contextual features from ST blocks and generates the
synthetic image $\tilde{\mathbf{y}}$ with skip connections. Each decoder layer
gradually upsamples encoded features using a sequence of $2\times 2$
deconvolution and a $3\times 3$ convolution.

The final generator loss combines the CycleGAN’s loss [33], including cycle
consistency and adversarial losses, and the mask loss, weighted by $\lambda$:

$\mathcal{L}=\mathcal{L}_{\text{adv}}+\mathcal{L}_{\text{cycle}}+\lambda\mathcal{L}_{\text{mask}}.$
(7)

Details of our Structural Transformer block. To guide the Transformer under
unpaired training, our Structural Transform applies dual attention strategy.
For the foreground, we perform structural attention to learn relationships
between anatomical regions. For the background, the local attention is
performed, enabling effective information exchange between foreground and
background features. The ST block uses the binary mask $M$ to classify
foreground tokens given a threshold $\sigma$. For a query token
$q_{\text{fg}}$ in the foreground, our model aggregates key-value tokens
within a scope $S_{\text{fg}}$ within the foreground anatomy:

$S_{\text{fg}}=\\{(i,j)\mid M_{i,j}>\sigma\\}.$ (8)

A default value $\sigma=0.5$ is used to classify foreground tokens. The
structural attention consists of a layer normalization (LN) followed by a
multi-head self-attention (MSA) layer [26] and a multi-layer perceptron (MLP)
with skip connections to facilitate the preservation of low-level features.
The foreground token output $\mathbf{f}^{\text{fg}}_{t+1}$ from the structural
attention in the $t$-th ST block is formulated as:

$\displaystyle\mathbf{f^{\prime}}^{\text{fg}}_{t+1}$
$\displaystyle=\text{SA}(\text{LN}(\mathbf{f}^{\text{fg}}_{t}),S_{\text{fg}})+\mathbf{f}^{\text{fg}}_{t},\text{
and}$ (9) $\displaystyle\mathbf{f}^{\text{fg}}_{t+1}$
$\displaystyle=\text{MLP}(\text{LN}(\mathbf{f^{\prime}}^{\text{fg}}_{t+1}))+\mathbf{f^{\prime}}^{\text{fg}}_{t+1}.$
(10)

For a query token $q_{\text{bg}}$ in the background, we aggregate tokens
around a local window $m\times m$:

$S_{\text{bg}}=\\{(i,j)|i\in[i_{q}-\frac{m}{2},i_{q}+\frac{m}{2}],j\in[j_{q}-\frac{m}{2},j_{q}+\frac{m}{2}]\\}.$
(11)

Similarly, the background token output $\mathbf{f}^{\text{bg}}_{t+1}$ from the
local attention is formulated as:

$\displaystyle\mathbf{f^{\prime}}^{\text{bg}}_{t+1}$
$\displaystyle=\text{SA}(\text{LN}(\mathbf{f}^{\text{bg}}_{t}),S_{b})+\mathbf{f}^{\text{bg}}_{t},\text{
and}$ (12) $\displaystyle\mathbf{f}^{\text{bg}}_{t+1}$
$\displaystyle=\text{MLP}(\text{LN}(\mathbf{f^{\prime}}^{\text{bg}}_{t}))+\mathbf{f^{\prime}}^{\text{bg}}_{t}.$
(13)

Lastly, the dual-attention ST block consolidates the output $\mathbf{f}_{t+1}$
based on the indices $(h,w)$ designated for the foreground $S_{\text{fg}}$,
and the background scope $S_{\text{bg}}$.

##  3 Experimental Results

Datasets and implementation details. For MR-to-CT and MR-to-PET translation
tasks, experiments are conducted on the MRXFDG [18] dataset, a public
neuroimaging set containing cerebral scans from 37 adult subjects. Sagittal
slices are extracted and resized to $224\times 224$ pixels. For the PET-to-CT
translation task, models are evaluated on the AutoPET dataset [10], comprising
310 whole-body PET and CT scans acquired from a Siemens Biograph mCT scanner.
Coronal views are extracted and resized to $256\times 256$ pixels. Both
datasets are partitioned into training, validation, and test sets with the
proportion of 8:1:1.

All models are trained using the Adam optimizer for 100 epochs, with a
learning rate of 1e-4 which linearly decays to zero over the last 50 epochs.
We train with a batch size of 16 on two NVIDIA RTX 3090 GPUs.

Evaluation metrics. Employing standard performance metrics from prior studies
[27, 15], we compute mean absolute error (MAE), peak signal-to-noise ratio
(PSNR), and structural similarity (SSIM) between the real and synthesized
images. The reported results are averaged over five runs. A paired student’s
$t$-test is conducted between UNest and the compared methods to test the
significance of performance difference ($p=0.05$).

Table 1: Comparison of different methods on head scan, MRXFDG dataset [18] for
four translation tasks. The improvement of UNest over all compared methods is
statistically significant at $p=0.05$.

Methods | MR-to-PET | PET-to-MR | MR-to-CT | CT-to-MR
---|---|---|---|---
MAE $\downarrow$ | PSNR $\uparrow$ | SSIM $\uparrow$ | MAE $\downarrow$ | PSNR $\uparrow$ | SSIM $\uparrow$ | MAE $\downarrow$ | PSNR $\uparrow$ | SSIM $\uparrow$ | MAE $\downarrow$ | PSNR $\uparrow$ | SSIM $\uparrow$
Convolution-based |  |  |  |  |  |  |  |  |  |  |  |
CycleGAN [33] | 7.88±.1 | 33.11±.01 | 79.53±.2 | 11.21±.1 | 31.89±.02 | 62.47±.3 | 7.20±.1 | 34.67±.04 | 82.56±.5 | 10.75±.2 | 32.15±.06 | 70.54±.7
sc-CycleGAN [27] | 7.54±.1 | 33.20±.02 | 80.06±.3 | 10.64±.2 | 31.93±.03 | 62.87±.3 | 7.12±.2 | 34.82±.05 | 83.06±.7 | 10.34±.4 | 32.33±.11 | 70.77±.8
AttentionGAN [23] | 7.21±.1 | 33.54±.03 | 81.09±.2 | 10.23±.1 | 33.02±.02 | 70.97±.1 | 6.84±.1 | 34.97±.05 | 83.88±.6 | 9.92±.4 | 33.12±.09 | 72.09±.9
MaskGAN [19] | 6.95±.1 | 33.98±.02 | 81.76±.2 | 9.81±.1 | 33.10±.02 | 71.35±.2 | 6.59±.2 | 35.10±.05 | 84.53±.7 | 9.37±.3 | 33.39±.09 | 72.53±1.0
Hybrid Conv-Trans |  |  |  |  |  |  |  |  |  |  |  |
TransUNet [2] | 13.10±.2 | 32.68±.04 | 70.81±.6 | 13.70±.3 | 31.78±.03 | 59.25±.5 | 10.25±.2 | 33.47±.04 | 75.52±.8 | 12.48±.3 | 31.22±.06 | 62.32±.9
ResViT [6] | 10.44±.1 | 32.99±.03 | 76.84±.5 | 14.66±.2 | 30.62±.03 | 56.24±.4 | 7.58±.2 | 34.72±.03 | 81.89±.7 | 11.72±.3 | 31.85±.07 | 64.51±1.3
Cytran [22] | 8.11±.1 | 33.23±.02 | 78.29±.5 | 11.05±.3 | 32.22±.04 | 60.74±.5 | 6.98±.1 | 34.85±.04 | 82.68±.7 | 9.85±.2 | 32.45±.06 | 70.80±1.0
Pure Transformers |  |  |  |  |  |  |  |  |  |  |  |
PTNet [32] | 11.43±.3 | 32.76±.07 | 70.53±.8 | 13.63±.4 | 32.33±.08 | 63.21±.9 | 9.42±.4 | 33.90±.10 | 75.88±.9 | 12.08±.7 | 31.43±.13 | 62.89±1.2
UNETR [12] | 7.35±.1 | 33.60±.04 | 81.90±.4 | 8.98±.1 | 33.06 ±.03 | 72.18±.2 | 7.95±.4 | 34.55±.11 | 81.18±.9 | 10.45±.3 | 32.98±.05 | 68.33±1.0
Swin-UNETR [24] | 7.54±.2 | 33.53±.03 | 81.72±.6 | 9.11±.2 | 33.03±.07 | 71.49±.8 | 8.80±.7 | 34.28±.18 | 80.53±1.11 | 11.15±.5 | 32.87±.05 | 67.93±1.3
UNest (Ours) | 6.32±.2 | 34.24±.04 | 82.97±.4 | 8.10±.3 | 33.91±.05 | 74.13±.3 | 6.18±.3 | 35.76±.09 | 85.48±.5 | 8.72±.4 | 34.17±.07 | 72.97±.9

Figure 4: Visual results of different methods for MRI-to-PET and MRI-to-CT
translation on MRXFDG dataset.

Comparisons with state-of-the-art. Table 1 benchmarks four synthesis tasks on
the MRXFDG dataset. Without inductive bias, UNETR is inferior to convolution
AttentionGAN on a small-sized dataset, which is common in the medical domain.
Injecting strong learning biases, the proposed UNest outperforms Swin-UNETR by
19.30% and 3.95% respectively in terms of MAE and PSNR. Fig. 4 presents visual
results on MR-to-PET and MR-to-CT. Without inductive bias, UNETR tends to
produce blurrier details, while Swin-UNETR distorts the details in the
subcortical brain structures. In contrast, our structural attention method,
accurately synthesizes the structural regions in PET. Table 2 presents the
benchmark of different synthesis methods on AutoPET dataset [10] for PET-to-CT
and CT-to-PET tasks. Incorporating a structural inductive bias, our UNest
significantly improves upon UNETR and Swin-UNETR by 14.21% and 11.73%
respectively in terms of MAE.

Table 2: Quantitative comparison of different methods on the AutoPET dataset
for translating between PET and CT. The improvement of UNest over all compared
methods is statistically significant at $p=0.05$. FG-S + BG-S: an ablated
version of UNest with structural attention applying on both foreground and
background regions.

Methods | PET-to-CT | CT-to-PET
---|---|---
MAE $\downarrow$ | PSNR $\uparrow$ | SSIM $\uparrow$ | MAE $\downarrow$ | PSNR $\uparrow$ | SSIM $\uparrow$
Convolution-based |  |  |  |  |  |
CycleGAN [33] | 14.34$\pm.2$ | 32.30$\pm.04$ | 66.85$\pm.6$ | 12.80$\pm.1$ | 31.95$\pm.02$ | 69.53$\pm.5$
sc-CycleGAN [27] | 14.59$\pm.2$ | 32.42$\pm.06$ | 67.43$\pm.8$ | 12.77$\pm.1$ | 32.05$\pm.04$ | 70.53$\pm.8$
AttentionGAN [23] | 13.62$\pm.2$ | 32.97$\pm.06$ | 67.12$\pm.6$ | 11.87$\pm.1$ | 32.61$\pm.03$ | 72.54$\pm.6$
QS-Attention [13] | 13.21$\pm.4$ | 33.70$\pm.08$ | 71.49$\pm 1.1$ | 9.75$\pm.2$ | 33.21$\pm.07$ | 77.42$\pm 1.0$
Hybrid Convolution-Transformer |  |  |  |  |  |
TransUNet [2] | 16.65$\pm.5$ | 32.68$\pm.10$ | 62.68$\pm 1.1$ | 13.52$\pm.4$ | 31.94$\pm.09$ | 67.88$\pm 1.1$
Cytran [22] | 14.77$\pm.3$ | 32.24$\pm.08$ | 64.37$\pm.9$ | 9.05$\pm.2$ | 33.56$\pm.06$ | 74.65$\pm.8$
ResViT [6] | 13.55$\pm.3$ | 33.28$\pm.07$ | 70.53$\pm.7$ | 9.90$\pm.1$ | 32.13$\pm.06$ | 68.29$\pm.7$
Pure Transformers |  |  |  |  |  |
PTNet [32] | 14.11$\pm.6$ | 32.62$\pm.10$ | 66.53$\pm 1.3$ | 9.24$\pm.5$ | 32.77$\pm.09$ | 70.53$\pm 1.1$
UNETR [12] | 11.08$\pm.3$ | 33.70$\pm.08$ | 76.06$\pm.8$ | 7.77$\pm.2$ | 33.58$\pm.05$ | 80.30$\pm.8$
Swin-UNETR [24] | 10.93$\pm.4$ | 33.78$\pm.08$ | 76.42$\pm.9$ | 7.42$\pm.3$ | 33.78$\pm.07$ | 80.53$\pm.8$
UNest (Ours) | 9.57$\pm.3$ | 34.05$\pm.07$ | 78.47$\pm.8$ | 6.55$\pm.2$ | 34.27$\pm.05$ | 81.55$\pm.7$
UNest w/ FG-S + BG-S | 9.73 $\pm.4$ | 33.85 $\pm.07$ | 77.85 $\pm.8$ | 6.87 $\pm.3$ | 34.11 $\pm.05$ | 81.21 $\pm.8$

Figure 5: Left: Error maps of synPET produced by UNest with FG-S \+ BG-S
attention, and our hybrid S-L attention. Right: Attention maps of global and
structural attention.

Ablation study. Table 2 shows that using dual attention with structural
attention FG-S + BG-S improves MAE of UNETR and Swin UNETR by 12.18% and
10.98%, respectively, on PET-CT. Separating attentions on BG and FG injects a
structural inductive bias, useful for unpaired training. Notably, using our
hybrid attention with FG-S and BG-L achieves the best results, which is
further supported by visual results in Fig. 5-Left. Fig. 5-Right shows that
global attention attends to less relevant BG tokens, while our structural
attention adaptively attends to anatomical features. It incorporates a long-
range context for smooth brain regions (top) and a localized focus for a
structural sinus (bottom).

##  4 Conclusion

In this paper, we introduce the UNet Structural Transformer (UNest) for
unpaired medical image synthesis. Traditional Transformer methods often
underperform in unpaired image synthesis, due to the absence of appropriate
learning priors. Injecting structural priors as in our UNest significantly
reduces the synthesis error of previous Transformer models by 19.3% on the
small-sized MRXFDG. The structural inductive bias also boosts synthesis
performance of current models by 14.2% on the large-scale AutoPET dataset.

## References

  * [1] Learning with radiomics for disease diagnosis and treatment planning: A review. Frontiers in Oncology 12, 773840 (2022)
  * [2] Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., Lu, L., Yuille, A.L., Zhou, Y.: TransUNet: Transformers make strong encoders for medical image segmentation. arXiv preprint arXiv:2102.04306 (2021)
  * [3] Chen, X., Wang, X., Zhou, J., Qiao, Y., Dong, C.: Activating more pixels in image super-resolution transformer. In: CVPR. pp. 22367–22377 (2023)
  * [4] Cui, C., Yang, H., Wang, Y., Zhao, S., Asad, Z., Coburn, L.A., Wilson, K.T., Landman, B., Huo, Y.: Deep multi-modal fusion of image and non-image data in disease diagnosis and prognosis: a review. Prog. in Biomed. Engineer. (2023)
  * [5] Dai, Y., Gao, Y., Liu, F.: Transmed: Transformers advance multi-modal medical image classification. Diagnostics 11(8), 1384 (2021)
  * [6] Dalmaz, O., Yurt, M., Çukur, T.: Resvit: residual vision transformers for multimodal medical image synthesis. IEEE Trans. Med. Imag. 41(10), 2598–2614 (2022)
  * [7] Doherty, D., Millen, K.J., Barkovich, A.J.: Midbrain and hindbrain malformations: advances in clinical diagnosis, imaging, and genetics. The Lancet Neurology 12(4), 381–393 (2013)
  * [8] Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. In: Proc. Int. Conf. Learn. Represent. (2021)
  * [9] Emami, H., Dong, M., Nejad-Davarani, S.P., Glide-Hurst, C.K.: SA-GAN: Structure-aware gan for organ-preserving synthetic ct generation. In: MICCAI. pp. 471–481. Springer (2021)
  * [10] Gatidis, S., Hepp, T., Früh, M., La Fougère, C., Nikolaou, K., Pfannenberg, C., Schölkopf, B., Küstner, T., Cyran, C., Rubin, D.: A whole-body fdg-pet/ct dataset with manually annotated tumor lesions. Scientific Data 9(1), 601 (2022)
  * [11] Ge, Y., Wei, D., Xue, Z., Wang, Q., Zhou, X., Zhan, Y., Liao, S.: Unpaired MR to CT synthesis with explicit structural constrained adversarial learning. In: IEEE Int. Symposium Biomed. Imaging. IEEE (2019)
  * [12] Hatamizadeh, A., Tang, Y., Nath, V., Yang, D., Myronenko, A., Landman, B., Roth, H.R., Xu, D.: Unetr: Transformers for 3d medical image segmentation. In: WACV. pp. 574–584 (2022)
  * [13] Hu, X., Zhou, X., Huang, Q., Shi, Z., Sun, L., Li, Q.: Qs-attn: Query-selected attention for contrastive learning in i2i translation. In: CVPR. pp. 18291–18300 (2022)
  * [14] Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., Lo, W.Y., et al.: Segment anything. arXiv preprint arXiv:2304.02643 (2023)
  * [15] Liu, Y., Chen, A., Shi, H., Huang, S., Zheng, W., Liu, Z., Zhang, Q., Yang, X.: CT synthesis from MRI using multi-cycle GAN for head-and-neck radiation therapy. Computer. Med. Imag. Graphic. 91, 101953 (2021)
  * [16] Lu, Z., Xie, H., Liu, C., Zhang, Y.: Bridging the gap between vision transformers and convolutional neural networks on small datasets. Proc. Adv. Neural Inform. Process. Syst. 35, 14663–14677 (2022)
  * [17] Matsuo, H., Nishio, M., Nogami, M., Zeng, F., Kurimoto, T., Kaushik, S., Wiesinger, F., Kono, A.K., Murakami, T.: Unsupervised-learning-based method for chest MRI-CT transformation using structure constrained unsupervised generative attention networks. Scientific reports 12(1), 11090 (2022)
  * [18] Mérida, I., Jung, J., Bouvard, S., Le Bars, D., Lancelot, S., Lavenne, F., Bouillot, C., Redouté, J., Hammers, A., Costes, N.: Cermep-idb-mrxfdg: a database of 37 normal adult human brain [18f] fdg pet, t1 and flair mri, and ct images available for research. EJNMMI research 11(1), 1–10 (2021)
  * [19] Phan, V.M.H., Liao, Z., Verjans, J.W., To, M.S.: Structure-preserving synthesis: Maskgan for unpaired mr-ct translation. In: MICCAI. pp. 56–65. Springer (2023)
  * [20] Raghu, M., Unterthiner, T., Kornblith, S., Zhang, C., Dosovitskiy, A.: Do vision transformers see like convolutional neural networks? Proc. Adv. Neural Inform. Process. Syst. 34, 12116–12128 (2021)
  * [21] Richardson, D.B., Cardis, E., Daniels, R.D., Gillies, M., O’Hagan, J.A., Hamra, G.B., Haylock, R., Laurier, D., Leuraud, K., Moissonnier, M., et al.: Risk of cancer from occupational exposure to ionising radiation: retrospective cohort study of workers in france, the united kingdom, and the united states. BMJ 351 (2015)
  * [22] Ristea, N.C., Miron, A.I., Savencu, O., Georgescu, M.I., Verga, N., Khan, F.S., Ionescu, R.T.: Cytran: A cycle-consistent transformer with multi-level consistency for non-contrast to contrast ct translation. Neurocomputing (2023)
  * [23] Tang, H., Liu, H., Xu, D., Torr, P.H., Sebe, N.: Attentiongan: Unpaired image-to-image translation using attention-guided generative adversarial networks. IEEE Trans. Neu. Netw. Learn. Syst. (2021)
  * [24] Tang, Y., Yang, D., Li, W., Roth, H.R., Landman, B., Xu, D., Nath, V., Hatamizadeh, A.: Self-supervised pre-training of swin transformers for 3d medical image analysis. In: CVPR (2022)
  * [25] Torbunov, D., Huang, Y., Yu, H., Huang, J., Yoo, S., Lin, M., Viren, B., Ren, Y.: Uvcgan: Unet vision transformer cycle-consistent gan for unpaired image-to-image translation. In: WACV. pp. 702–712 (2023)
  * [26] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need 30 (2017)
  * [27] Yang, H., Sun, J., Carass, A., Zhao, C., Lee, J., Prince, J.L., Xu, Z.: Unsupervised MR-to-CT synthesis using structure-constrained CycleGAN. IEEE Trans. Med. Imag. 39(12), 4249–4261 (2020)
  * [28] Yang, H., Sun, J., Carass, A., Zhao, C., Lee, J., Xu, Z., Prince, J.: Unpaired brain MR-to-CT synthesis using a structure-constrained CycleGAN. In: MICCAI Workshop: Deep Learn. Med. Image Analysis, pp. 174–182. Springer (2018)
  * [29] Yu, F., Wang, X., Cao, M., Li, G., Shan, Y., Dong, C.: Osrt: Omnidirectional image super-resolution with distortion-aware transformer. In: CVPR. pp. 13283–13292 (2023)
  * [30] Zhang, B., Liu, L., Phan, M.H., Tian, Z., Shen, C., Liu, Y.: Segvitv2: Exploring efficient and continual semantic segmentation with plain vision transformers. Int. J. Comput. Vis. (2023)
  * [31] Zhang, J., Cui, Z., Jiang, C., Zhang, J., Gao, F., Shen, D.: Mapping in cycles: Dual-domain PET-CT synthesis framework with cycle-consistent constraints. In: MICCAI. pp. 758–767. Springer (2022)
  * [32] Zhang, X., He, X., Guo, J., Ettehadi, N., Aw, N., Semanek, D., Posner, J., Laine, A., Wang, Y.: Ptnet3d: A 3d high-resolution longitudinal infant brain mri synthesizer based on transformers. IEEE Trans. Med. Imag. 41(10), 2925–2940 (2022)
  * [33] Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: ICCV. pp. 2223–2232 (2017)


11institutetext: University of JyvÃskylÃ, Department of Physics, P.O. Box 35,
40014 University of JyvÃskylÃ, Finland 22institutetext: Helsinki Institute of
Physics, P.O. Box 64, 00014 University of Helsinki, Finland

# Illuminating the impact-parameter dependence of UPC dijet photoproduction

Kari J. Eskola  Vadim Guzey  Ilkka Helenius  Petja Paakkinen Speaker,
<EMAIL_ADDRESS>Hannu Paukkunen

###### Abstract

We present new NLO pQCD predictions for photoproduction of dijets in
ultraperipheral PbPb collisions at 5.02 TeV with a realistic photon flux and
up-to-date nuclear PDFs. Our calculation of the impact parameter dependence of
the photon flux includes the effects of the nuclear form factor in the photon-
emitting nucleus and the spatial dependence of nuclear PDFs of the target
nucleus, which are estimated using the Wood-Saxon nuclear density profile. We
show that a significant portion of the measured dijets at large $z_{\gamma}$
originate from events with impact parameters of the order of a few nuclear
radii, and that the cross section predictions therefore become sensitive to
the modelling of the nuclear geometry and photon flux close to the source
nucleus.

##  1 Introduction

In ultra-peripheral heavy-ion collisions (UPCs), two nuclei pass each other at
an impact parameter larger than the sum of their radii, suppressing the
hadronic strong interactions. Hard interactions of one nucleus with the
electromagnetic field of the other are still allowed, providing a way to probe
the nuclear contents through photon-induced processes Bertulani:2005ru . For
example, dijet photoproduction has been promoted as a way to probe the nuclear
PDFs Strikman:2005yv ; Guzey:2018dlm . Previous NLO pQCD predictions for this
process have been performed by using a point-like charge distribution for the
nucleus to simplify the calculations. We show that this picture receives
corrections due to geometrical effects originating from the finite size of the
involved nuclei.

##  2 Impact-parameter dependent factorization of dijet cross section

In this work, we follow the approach of Refs. Baron:1993nk ; Greiner:1994db ;
Krauss:1997vr , where the photon-induced process is treated in the equivalent-
photon approximation with explicit transverse-plane dependence and keeping
track of the finite-size effects. For the dijet production we have

$\begin{split}{\rm d}\sigma^{AB\rightarrow A+{\rm
dijet}+X}=\sum_{i,j,X^{\prime}}\int&{\rm d}^{2}{\bf b}\,\Gamma_{AB}({\bf
b})\int{\rm d}^{2}{\bf r}\,f_{\gamma/A}(y,{\bf r})\otimes
f_{i/\gamma}(x_{\gamma},Q^{2})\\\ \otimes\int&{\rm d}^{2}{\bf
s}\,f_{j/B}(x,Q^{2},{\bf s})\otimes{\rm d}\hat{\sigma}^{ij\rightarrow{\rm
dijet}+X^{\prime}}\delta({\bf r}\\!-\\!{\bf s}\\!-\\!{\bf b}).\end{split}$ (1)

Here, $\Gamma_{AB}({\bf b})$ gives the probability of no hadronic interaction
between the involved nuclei at an impact parameter ${\bf b}$,
$f_{\gamma/A}(y,{\bf r})$ is the number of photons at a transverse distance
$|{\bf r}|$ from the center of the emitting nucleus $A$ carrying a fraction
$y$ of the beam energy, and $f_{i/\gamma}(x_{\gamma},Q^{2})$ the density of
partons $i\in\\{q,\bar{q},g,\gamma\\}$ with a momentum fraction $x_{\gamma}$
in the photon at the scale $Q^{2}$. Here,
$f_{\gamma/\gamma}(x_{\gamma},Q^{2})=\delta(1-x_{\gamma})$ accounts for the
direct contribution. Finally, $f_{j/B}(x,Q^{2},{\bf s})$ is the parton density
for a parton $j\in\\{q,\bar{q},g\\}$ with a momentum fraction $x$ in the
target nucleus $B$ at a transverse distance $|{\bf s}|$ from its center, and
${\rm d}\hat{\sigma}^{ij\rightarrow{\rm dijet}+X^{\prime}}$ is the partonic
cross section.

Assuming that we can factorize the target parton distribution further as
$f_{j/B}(x,Q^{2},{\bf s})=\frac{1}{B}\,T_{B}({\bf s})\times f_{j/B}(x,Q^{2})$,
where $f_{j/B}(x,Q^{2})$ is the ordinary spatially averaged nuclear PDF and
$T_{B}({\bf s})$ the nuclear thickness function, we get

${\rm d}\sigma^{AB\rightarrow A+{\rm
dijet}+X}=\sum_{i,j,X^{\prime}}f_{\gamma/A}^{\rm eff}(y)\otimes
f_{i/\gamma}(x_{\gamma},Q^{2})\otimes f_{j/B}(x,Q^{2})\otimes{\rm
d}\hat{\sigma}^{ij\rightarrow{\rm dijet}+X^{\prime}}$ (2)

where

$f_{\gamma/A}^{\rm eff}(y)=\int{\rm d}^{2}{\bf r}\,f_{\gamma/A}^{\rm
eff}(y,{\bf r})=\frac{1}{B}\int{\rm d}^{2}{\bf r}\int{\rm d}^{2}{\bf
s}\,f_{\gamma/A}(y,{\bf r})\,T_{B}({\bf s})\,\Gamma_{AB}({\bf r}\\!-\\!{\bf
s})$ (3)

is an _effective_ photon flux containing now all the transverse-plane
integrals (cf. Ref. ATLAS:2022cbd ).

##  3 Effective photon flux

Figure 1: The bare (left) and effective (right) photon flux for different
photon energies $k=y\sqrt{s_{\rm NN}}/2$ in PbPb collisions at $\sqrt{s_{\rm
NN}}=5.02\ {\rm TeV}$ under different approximations (see text).

Figure 1 shows the bare ($f_{\gamma/A}$) and effective ($f_{\gamma/A}^{\rm
eff}$) photon flux in PbPb collisions at 5.02 TeV in three different
approximations:

[WS]

PL

refers to the pointlike approximation, where the bare flux is taken to be that
of a point source with a charge $Z$ (see e.g. Ref. Guzey:2018dlm ),

$\textstyle f_{\gamma/A}^{{\sf PL}}(y,{\bf r})=\frac{Z^{2}\alpha_{\rm
e.m.}}{\pi^{2}}m_{p}^{2}y[K_{1}^{2}(\xi)+\frac{1}{\gamma_{L}}K_{0}^{2}(\xi)],\quad\xi=ym_{p}|{\bf
r}|,\quad\gamma_{L}=\sqrt{s_{\rm NN}}/(2m_{p}),$ (4)

where $m_{p}$ is the proton mass. This bare PL flux is shown as dotted lines
in Figure 1 (left). The target nucleus is also taken as a pointlike object
with $T_{B}({\bf s})=B\delta({\bf s})$, and the requirement of no hadronic
interaction between the nuclei is implemented with a simple cut in the impact
parameter, $\Gamma_{AB}({\bf b})=\theta(|{\bf b}|-b_{\rm min})$, where $b_{\rm
min}=2R_{{\sf PL}}$ is a tuneable parameter which we take as 14.2 fm in
accordance with Ref. Guzey:2018dlm . Since we have ${\bf b}={\bf r}$ in this
approximation, this amounts to having a cut in the ${\bf r}$-dependent
effective flux as shown in Figure 1 (right).

WSδ(s)

improves upon the previous one by deriving the bare flux through the nuclear
form factor $F^{{\sf WS}}$ calculated from the Woods-Saxon nuclear density
profile (cf. Ref. Krauss:1997vr ),

$\textstyle f_{\gamma/A}^{{\sf WS}}(y,{\bf r})=\frac{Z^{2}\alpha_{\rm
e.m.}}{\pi^{2}}\frac{1}{y}\left|\int_{0}^{\infty}\frac{{\rm
d}k_{\perp}k_{\perp}^{2}}{k_{\perp}^{2}+(ym_{p})^{2}}F^{{\sf
WS}}(k_{\perp}^{2}+(ym_{p})^{2})\,J_{1}(|{\bf r}|k_{\perp})\right|^{2},$ (5)

see the solid lines in Figure 1 (left). This only affects the distribution
within the radius of the emitting nucleus $|{\bf r}|<R_{{\sf PL}}$, and thus a
more important improvement comes from taking $\Gamma_{AB}({\bf
b})=\exp[-\,\sigma_{\rm NN}\,T_{AB}({\bf b})]$, where $T_{AB}({\bf b})$ is the
nuclear overlap function calculated from the Woods-Saxon distribution, and
$\sigma_{\rm NN}=90\ {\rm mb}$ is the nucleon-nucleon total cross section at
$\sqrt{s_{\rm NN}}=5.02\ {\rm TeV}$. The effect is demonstrated in Figure 1
(right) with dashed lines. We still keep $T_{B}({\bf s})=B\delta({\bf s})$ in
Eq. (3) for this approximation.

WS

is the most realistic approximation, where, on top of the previous one, also
the target profile $T_{B}({\bf s})$ is calculated from the Woods-Saxon
distribution. The resulting effective flux is shown as solid lines in Figure 1
(right), where we see that the additional integration over finite ${\bf s}$
gives a non-negligible contribution in the region $|{\bf r}|<2R_{{\sf PL}}$ in
contrast to the two other approximations. This effect becomes increasingly
important for large photon energies where the bare flux is very steeply
falling as a function of $|{\bf r}|$.

For the events with $|{\bf r}|>3R_{{\sf PL}}$, i.e. ‘far-passing’ nuclei, the
PL approximation works fine, but producing high-$p_{\rm T}$ jets requires
sufficient energy from the photon, which enhances sensitivity to the ‘near-
encounter’ $|{\bf r}|<3R_{{\sf PL}}$ region, as we discuss next.

##  4 UPC dijet cross section in PbPb collisions at 5.02 TeV

Figure 2: The effective flux (left) and UPC dijet cross section (right) in
PbPb collisions at $\sqrt{s_{\rm NN}}=5.02\ {\rm TeV}$ under different
approximations (see text).

We compare in Figure 2 the effective flux integrated over the transverse-plane
variables (left) and the inclusive dijet cross section as a function of
$z_{\gamma}=M_{\rm jets}\exp(y_{\rm jets})/\sqrt{s_{\rm NN}}$ (right) in PbPb
collisions at $\sqrt{s_{\rm NN}}=5.02\ {\rm TeV}$, where $M_{\rm jets}$,
$y_{\rm jets}$ are the invariant mass and rapidity of the jet system. Jets are
defined using the anti-$k_{\rm T}$ algorithm with $R=0.4$ and are required to
satisfy $|\eta_{\rm jet}|<4.4$ with at least one of them having $p_{\rm
T,jet}>20\ {\rm GeV}$ and the rest $p_{\rm T,jet}>15\ {\rm GeV}$ with $M_{\rm
jets}>35\ {\rm GeV}$, in accordance with Ref. ATLAS:2017kwa . At leading
order, this observable would resolve to $z_{\gamma}=yx_{\gamma}$. Here, we
calculate the cross section in NLO pQCD with the Frixione & Ridolfi jet
photoproduction code Frixione:1997ks  using the EPPS21 nuclear Eskola:2021nhw
and the GRV photon PDFs Gluck:1991jc .

The upper panels of the figure show the absolute effective flux and cross
section, whereas in the middle panels we show the fraction of flux and cross
section coming from the ‘near-encounter’ $|{\bf r}|<3R_{{\sf PL}}$ region with
respect to that integrated over the full phase space. This fraction is small
for low-energy photons, but increases with growing energy, reaching unity at
$y\approx 0.1$. Consequently, the fraction of the cross section coming from
‘near-encounter’ events rises from 0.2 to 0.8 between the lowest and highest
$z_{\gamma}$ bin. This indicates that the UPC dijets with large $z_{\gamma}$
are potentially sensitive to the modelling of collision geometry. Indeed, as
we plot in the lower panels, the flux in WSδ(s) and WS deviate from the PL
approximation for high photon energies, causing a 40% suppression in WSδ(s)
compared to PL in the cross section in the largest $z_{\gamma}$ bin. The PL
calculation does a rather good job in approximating the full WS one with some
10% difference at maximum, which is of the order of the expected nuclear-PDF
effects Guzey:2018dlm  and thus also important, but the large difference
between WS and WSδ(s) shows that taking into account the full collision
geometry with finite ${\bf s}$ is important for an accurate physical
interpretation of this observable.

##  5 Conclusions

Requiring an energetic photon in the initial state, a significant part of the
UPC dijet cross section at large $z_{\gamma}$ comes from configurations where
the nuclei pass each other at relatively small impact parameters. We have
shown that this leads to a sensitivity to the nuclear transverse profile with
a significant effect in the largest studied $z_{\gamma}$ bins. In the above,
we have assumed that one can factorize $f_{j/B}(x,Q^{2},{\bf
s})=\frac{1}{B}\,T_{B}({\bf s})\times f_{j/B}(x,Q^{2})$, but this is of course
a simplification and one should use impact-parameter dependent nuclear PDFs
Frankfurt:2011cs ; Helenius:2012wd  instead. Furthermore, we have neglected
the possibility of electromagnetic breakup through Coulomb excitations. These
matters will be discussed in an upcoming publication inprep .

This research was funded through the Academy of Finland project No. 330448, as
a part of the Center of Excellence in Quark Matter of the Academy of Finland
(projects No. 346325 and No. 346326) and as a part of the European Research
Council project ERC-2018-ADG-835105 YoctoLHC. We acknowledge computing
resources from the Finnish IT Center for Science (CSC), utilised under the
project jyy2580.

## References

  * (1) C. A. Bertulani, S. R. Klein, J. Nystrand, Ann. Rev. Nucl. Part. Sci. 55, 271 (2005)
  * (2) M. Strikman, R. Vogt, S. N. White, Phys. Rev. Lett. 96, 082001 (2006)
  * (3) V. Guzey, M. Klasen, Phys. Rev. C 99, 065202 (2019)
  * (4) N. Baron, G. Baur, Phys. Rev. C 48, 1999 (1993)
  * (5) M. Greiner, M. Vidovic, C. Hofmann, A. Schafer, G. Soff, Phys. Rev. C 51, 911 (1995)
  * (6) F. Krauss, M. Greiner and G. Soff, Prog. Part. Nucl. Phys. 39, 503 (1997)
  * (7) ATLAS Collaboration, ATLAS-CONF-2022-021
  * (8) ATLAS Collaboration, ATLAS-CONF-2017-011
  * (9) S. Frixione, G. Ridolfi, Nucl. Phys. B 507, 315 (1997)
  * (10) K. J. Eskola, P. Paakkinen, H. Paukkunen, C. A. Salgado, Eur. Phys. J. C 82, 413 (2022)
  * (11) M. Gluck, E. Reya, A. Vogt, Phys. Rev. D 46, 1973 (1992)
  * (12) L. Frankfurt, V. Guzey, M. Strikman, Phys. Rept. 512, 255 (2012)
  * (13) I. Helenius, K. J. Eskola, H. Honkanen, C. A. Salgado, JHEP 07, 073 (2012)
  * (14) K. J. Eskola, V. Guzey, I. Helenius, P. Paakkinen, H. Paukkunen, in preparation