# Nonequilibrium capillary self-assembly

S. J. Thomson${}^{1},2$, J.-W. Barotta2, and Daniel M. Harris2 Corresponding
author<EMAIL_ADDRESS>1Department of Engineering Mathematics,
University of Bristol, Ada Lovelace Building, University Walk, Bristol, BS8
1TW  2School of Engineering, Center for Fluid Mechanics, Brown University, 184
Hope Street, Providence, RI 02912

Macroscopic objects supported by surface tension at the fluid interface can
self-assemble through the action of capillary forces arising from interfacial
deformations. The resulting self-assembled structures are ordered but remain
trapped in one of potentially many metastable states in the capillary energy
landscape. This contrasts with microscopic colloidal self-assembly where
thermal fluctuations excite transitions between geometrically distinct ground-
state configurations. We herein utilize supercritical Faraday waves to drive
structural rearrangements between metastable states of few-particle clusters
of millimetric spheres bound by capillary attractions at the fluid interface.
Using a combination of experiments and theoretical modelling, we demonstrate
how the occupation probabilities of different cluster topologies and
transition statistics are controlled by the level of the vibrational forcing
and the spatial extent of long-range capillary forces. Our results demonstrate
how self-assembly dynamics and statistics may be manipulated across scales by
controlling the strength of fluctuations and by tuning the properties of the
particle interaction-potential.

Self-assembly processes are ubiquitous in nature and technology from the
development and self-organization of living matter [1, 2], to the directed
assembly of passive and active colloids [3, 4, 5] and control of macroscopic
robot collectives [6, 7]. In the context of fluid mechanics, self-assembly may
be realized by exploiting curvature-induced forces generated by passive
objects supported by surface tension at the air-fluid interface [8, 9, 10,
11]. Existing, well-established design principles of capillary self-assembly
focus on the behavior of such systems at equilibrium [12] where numerous
assembly schemes have been devised that exploit particle shape and contact
line geometry [13, 14, 15, 16], surface chemistry [17] and external directing
fields [18] to drive particular structure formation. When combined with multi-
particle interactions, this catalogue of techniques gives rise to potentially
many metastable states in the capillary energy landscape that the assembling
particles can occupy. However passive, non-Brownian particles are irreversibly
locked-in to the resulting self-assembled structure in the absence of an
external force or driving field. This lack of dynamic reconfigurability
contrasts with the microscopic realm where thermal fluctuations are sufficient
to excite transitions between geometrically distinct ground-state
configurations of colloidal clusters held together by purely short-range
depletion forces [19, 20]. The transitions between metastable states and
accompanying emergent statistics has thus not been studied experimentally in
capillary systems whose constituent particles are non-Brownian and are subject
to long-range forces whose spatial extent is set by the capillary length of
the fluid.

Figure 1: Oblique perspective of a cluster of six spherical particles, each of
radius 0.079 cm, momentarily occupying a triangle configuration in a field of
supercritical Faraday waves of characteristic wavelength $\lambda=0.84$ cm
[21]. The Faraday waves are visualized using a reflected color pattern. The
complex fluid environment due to the Faraday waves and horizontal turbulent
surface flows plays the role of an active bath. Figure 2: Faraday-wave driven
structural rearrangements of capillary-bound six-particle clusters. (a)
Oblique images of the unique ground-state configurations formed when fewer
than six spherical particles of radius 0.079 cm are placed on the static
surface of a water-glycerol mixture. (b) Adding a sixth particle forms one of
three metastable configurations: a triangle (T), chevron (C, ground state), or
parallelogram (P). (c) Schematic of our experiment. Four clusters are confined
to square-shaped corrals filled with a water-glycerol mixture. The entire set-
up is vibrated vertically with acceleration $\Gamma$ and frequency $f$ above
the Faraday threshold $\Gamma_{F}=0.9g$. Typical experimental cluster
trajectories (solid lines) over a 30 second interval are shown for (d)
$\Gamma=1g$ and (e) $\Gamma=1.2g$. The (pink) circles in (d) and (e) mark the
location of transitions (totalling 16 and 24 for $\Gamma=1g$ and
$\Gamma=1.2g$, respectively), the white arrows point to the start of each
trajectory and the dashed line marks the boundary of the confining corral. The
inset of both (d) and (e) plots the long-time experimental distribution of
$v_{c}=\lVert\sum_{i=1}^{6}\dot{\mathbf{x}}_{i}(t)\rVert_{2}$, the
instantaneous speed of the cluster center of mass (light grey). The components
of $\dot{\mathbf{x}}_{i}$ are approximately Gaussian [22] and hence a gamma
distribution is fit to $v_{c}$ (solid cyan). The mode of the gamma
distribution is 0.17 cm/s for $\Gamma=1g$ and 1.14 cm/s for $\Gamma=1.2g$
showing a speed enhancement of the particles as $\Gamma$ is increased. (f)
Example of a T-to-P transition _via_ a hinge-like motion (traced by the arrow
in the left- and middle-pane) when $\Gamma=1.2g$.

We here substitute the role of a heat bath with a field of periodically forced
supercritical Faraday waves to drive structural rearrangements in macroscopic
clusters of spherical particles bound by capillary attractions (Fig. 1). We
demonstrate experimentally how the cluster occupation probabilities and
transition statistics are controlled by the strength of the vibrational
forcing and use theoretical modelling to interrogate the role of long-range
capillary forces on the assembly statistics. Our results shed light on how
reconfigurable self-assembly may be manipulated across length-scales by
exploiting fine control over the strength of fluctuations and properties of
the particle interaction-potential.

When placed on the quiescent surface of a water-glycerol mixture (8:2 by
volume, density $\rho=1.0563$ g/cm3 and surface tension $\sigma=71.8$ dyn/cm),
fewer than six spherical Polytetrafluoroethylene (PTFE) particles (radius
$R=0.079$ cm) coated with a commercially available spray (Rust-Oleum NeverWet)
self-assemble into a unique ground-state configuration through the
equilibration of capillary and contact forces (Fig. 2(a)). However, the
addition of a sixth particle gives rise to three geometrically distinct,
metastable cluster configurations, referred to herein as chevron (C), triangle
(T), and parallelogram (P) (Fig. 2(b)). In the limit of small interfacial
deformations and small Bond number, the capillary interaction-energy between
six spherical particles floating at the fluid interface may be approximated by
$U=-U_{0}\sum_{i=1}^{6}\sum_{j>i}K_{0}(|\mathbf{x}_{i}-\mathbf{x}_{j}|/l_{c})$
where $K_{0}$ is the zeroth-order modified Bessel function of the second kind,
$\mathbf{x}_{i}$ denotes the horizontal position of the $i$th particle in the
cluster, the capillary length of the fluid $l_{c}=\sqrt{\sigma/\rho g}=0.264$
cm and $g=981$ cm/s2 is acceleration due to gravity [9, 10]. Hence
$U\rightarrow 0$ for particle spacings much greater than $l_{c}$. The energy
parameter $U_{0}=2\pi\sigma R^{2}\text{Bo}^{2}\Sigma^{2}\approx 45$ mJ where
the Bond number $\text{Bo}=(R/l_{c})^{2}=0.1\ll 1$ and $\Sigma\approx 1$ is
the dimensionless resultant weight of the particle [10, 22]. Computing $U$ for
the C, T, and P configurations we find $U^{\text{C}}=-412.9$ mJ,
$U^{\text{T}}=-409.3$ mJ and $U^{\text{P}}=-403.6$ mJ. The C configuration is
thus the true ground state of the six-particle system. In the limit the
potential becomes infinitely narrow and infinitely deep ($l_{c}\rightarrow 0$)
corresponding to short-range nearest-neighbor coupling, the C, T and P
configurations are energetically degenerate [23, 20].

Figure 3: Control of six-particle assembly and transition statistics with
vibrational forcing. (a) Experimental statistical steady-state occupation
probabilities for six-particle clusters for $\Gamma/g=(1,1.05,\ldots,1.2)$
(points). Theoretical occupation probabilities predicted from our Langevin
simulations are plotted as a function of the noise strength $\gamma$ (curves).
Note that $\gamma$ and $\Gamma$ are nonlinearly related. Experimental
transition probability matrices for (b) $\Gamma=1$g and (c) $\Gamma=1.2$g. The
colors in (b) and (c) correspond to the conditional probability
Pr(end$\lvert$start) of finishing in a particular end state following a
transition from one of three starting states, with numerical values given in
the bottom right of each square. At the top left of each square is the count
of each transition summed over all four clusters over the 1 hour experimental
time-scale. Transitions that terminate in a C state have higher probability
regardless of start state. (d) Plots of the variation in capillary energy,
$U$, when transitioning from a C state through the breaking of a single bond.
The lowest activation energies belong to C-C transitions where one or two
particles move around the periphery of the cluster in a hinge-like motion,
illuminating why the C-C transitions are favored.

Our experimental set-up is depicted in Fig. 2(c) with full details of our
experimental protocol provided in the Supplemental Material [22]. To gather
statistically significant amounts of data, four sets of six-particle clusters
are confined to four separate but identical illuminated square chambers, each
filled with 9.0 mL of water-glycerol solution corresponding to a depth of 5 mm
in the middle of each chamber. A small meniscus on the perimeter of each
chamber repelled the clusters away from the boundary. The entire set-up is
mounted on an in-house electromagnetic shaker vibrating vertically with
acceleration $\Gamma\cos(2\pi ft)$ where $\Gamma$ is the maximum acceleration,
$f$ is the vibrational frequency, and $t$ is time. An external air-bearing
guides the motion along a single axis and a closed-loop feedback system
ensures that $\Gamma$ remains within a tolerance of $\pm 0.001g$ [24].
Informed by [25, 26], $f=60$ Hz for all our experiments and for the fluid
parameters and corral geometry reported here, the Faraday threshold (the
critical value of $\Gamma$ for which supercritical Faraday waves spontaneously
appear on the fluid surface) of the fluid is $\Gamma_{F}=0.90\pm 0.03g$.
Forcing frequencies $f\gtrsim 80$ Hz were found to increase the tendency of
clusters to disintegrate, significantly inhibiting the collection of reliable
statistical data of the particle dynamics. For each experimental run, the
particles are filmed for 1 hour using a camera recording at 40 frames-per-
second mounted directly over the bath surface, capturing thousands of
transitions over the lifetime of the experiment.

Agitation of the particles by the Faraday waves and accompanying turbulent
surface flows for $\Gamma>\Gamma_{F}$ [27, 26, 28] causes the clusters to
translate across the fluid surface (Fig. 2(d) and (e)) and drives structural
rearrangements between the C, T, and P cluster topologies (Fig. 2(f) and
Supplemental Video). Cluster configurations, and hence transitions between
cluster configurations, are identified from each cluster’s adjacency matrix in
every frame of post-processed particle-tracking data [20, 22]. A typical
cluster rearrangement involves a single bond between two neighbouring
particles breaking before the cluster settles into a new state (Fig. 2(f) and
Supplemental Video), akin to thermally driven colloids [20]. As $\Gamma$ is
increased there is a concomitant speed enhancement of the particles (inset
Figs. 2(d) and (e)) and rearrangements can involve the breaking of multiple
bonds, the particles taking a more circuitous route between states thereafter
(see Supplemental Video). On occasion a particle can detach from the cluster
[29] and freely translate around the bath surface before eventually docking
onto the cluster once more under the action of attractive capillary forces.

We quantify the steady-state occupation probabilities $\Pi$ of the C, T and P
cluster configurations for varying $\Gamma$ by counting the number of times
each state is formed following a transition, divided by the total number of
observations of all states (Fig. 3(a)). Across the range of $\Gamma$
considered, we find that C states are more propitious followed by P and then
T. When $\Gamma=1g$ there are few instances of T and P configurations.
However, as $\Gamma$ is increased incrementally to $\Gamma=1.2g$, there is a
monotonic reduction in $\Pi^{\text{C}}$ and a corresponding increase in the
yield of both T and P configurations. Similar statistics emerge if the
occupation time of each configuration is used instead of the number of
occupations.

The transition count matrices and resulting transition probability matrices
for $\Gamma=1g$ and $\Gamma=1.2g$ are shown in Figures 3(b) and (c),
respectively. For $\Gamma=1g$, the majority of transitions occur between two
chevron states (transitions between chevron chiral enantiomers) that involve
one or two particles moving around the perimeter of the cluster in a hinge-
like motion. As the level of vibrational of forcing is increased, driving the
system further from equilibrium, there is a corresponding increase in the
number of transitions that occur between any two configurations and hence the
total number of transitions. However, we find that transition pathways
terminating in a C state are favoured regardless of whether the cluster starts
in a C, P or T state. The appreciable number of T-T transitions for higher
$\Gamma$ is due in part to the aforementioned detachment events where one or
two particles leave the cluster only to later rejoin. Across the range of
vibrational forcing, but particularly for higher $\Gamma$, we find that
Pr(C$\lvert$P)Pr(P$\lvert$T)Pr(T$\lvert$C)$\neq$
Pr(C$\lvert$T)Pr(T$\lvert$P)Pr(P$\lvert$C) violating detailed balance [30], a
hallmark of the nonequilibrium nature of the driven-dissipative Faraday-wave
system.

Our experiments reveal that C states have the highest occupation probability
with significantly more C-C transitions than between any other state pair. We
explore the physical origins of the occupation and transition statistics by
first performing Langevin simulations of the particle dynamics on the energy
landscape $U$. Specifically, the motion of each particle in the cluster is
modelled as

$m\ddot{\mathbf{x}}_{i}+\mu\dot{\mathbf{x}}_{i}=-\nabla_{\mathbf{x}_{i}}U+\mathbf{F}^{{\text{contact}}}_{i}-\nabla
V|_{\mathbf{x}=\mathbf{x}_{i}}+\gamma\boldsymbol{\eta}_{i},$ (1)

where $m$ is the mass of each particle and $\mu=0.0379\ \text{kg/s}$ is an
experimentally determined drag coefficient (see [22]). The forces on the
right-hand side of equation (1) from left-to-right are: the attractive
capillary force between particles $-\nabla_{\mathbf{x}_{i}}U=F_{c}\sum_{j\neq
i}K_{1}\left(|\mathbf{x}_{i}-\mathbf{x}_{j}|/l_{c}\right)\hat{\mathbf{x}}_{ij}$
where
$\hat{\mathbf{x}}_{ij}=\mathbf{x}_{i}-\mathbf{x}_{j}/|\mathbf{x}_{i}-\mathbf{x}_{j}|$,
$K_{1}$ is the first-order modified Bessel function of the second kind and the
interaction strength $F_{c}=U_{0}/l_{c}$; a purely dissipative contact force
to maintain particle separation upon contact [31]; a confining force that
mimics the square-shaped corral; and a fluctuating force that we assume takes
the form of an uncorrelated white noise with strength $\gamma$ determined by
matching the centre of mass velocity of the clusters for each value of
$\Gamma$ plotted in Fig. 3(a) [22]. The functional forms of
$\mathbf{F}^{\text{contact}}$ and $-\nabla V$ are provided in [22]. Equations
(1) are solved numerically using the Euler-Maruyama method [32, 33, 22].

While our theoretical model over-predicts the total number of transitions over
the experimental time scale due to the assumption of white noise (see [22] for
a comparison), the theoretical occupation probabilities show excellent
agreement with experiments. Specifically, our numerical simulations predict
$\Pi^{\text{C}}>\Pi^{\text{P}}>\Pi^{\text{T}}$ across the range of $\gamma$
and there is a similar variation in the occupation probabilities as the
strength of driving (controlled by $\Gamma$ in experiments and $\gamma$ in our
theoretical model) is increased (Fig. 3(a)). The success of our theoretical
model prompts us to more closely analyze the role of the interaction potential
$U$ on the emergent statistics.

Figure 4: Theoretical assembly statistics for varying interaction length
$\ell$ and $\gamma=0.03$ N$\sqrt{\text{s}}$. Transitions probability matrices
for (a) $\ell/l_{c}=1$ and (b) $\ell/l_{c}=0.4$. The colors in (a) and (b)
indicate conditional probabilities Pr(end$\lvert$ start), with numerical
values overlayed in text. (c) Theoretical statistical steady-state occupation
probabilities versus $\ell$. Below $\ell/l_{c}\approx 0.4$, the strength of
the capillary force cannot hold the cluster together at the chosen value of
$\gamma$. Inset: plots of the modified capillary force for varying $\ell$ with
the direction of decreasing $\ell/l_{c}$ indicated by the left-most arrow.

In both thermal colloids held together by depletion forces [20] and acoustic
assembly [29], short-range (much less than a particle radius) interactions
render the C, T and P cluster topologies energetically degenerate. In contrast
the range of interactions is set by $l_{c}\sim R$ in the present capillary
system. Consequently, $U$ varies along the entire reaction coordinate as the
particles traverse transition pathways. To gain further insight into why C
states are more propitious, we consider the one-dimensional energy landscape
formed by a subset of possible transition paths wherein a single bond breaks
in a C cluster and the particles go on to form a C, P or T configuration (Fig.
3(d)). In addition to the C configuration being the true ground state of the
system, we find that lower activation energies are required to excite hinge-
like C-C transitions than any other pathway and hence these pathways will be
most favored. Symmetry also plays a role: for example, C-P transitions are
more favored than C-T transitions despite having slightly higher activation
energy since there are 4 ways to enter into a C-P transition and only 1 for
C-T transitions in this simplified landscape.

How does the interaction length influence the assembly statistics? We modify
the attractive capillary force in our theoretical model (1) to
$\overline{F}_{c}\sum_{j\neq
i}K_{1}\left(|\mathbf{x}_{i}-\mathbf{x}_{j}|/\ell\right)\hat{\mathbf{x}}_{ij}$
where $\overline{F}_{c}=F_{c}K_{1}(2R/l_{c})/K_{1}(2R/\ell)$ and $\ell$ is an
adjustable interaction-length parameter controlling the spatial extent of the
force. The magnitude of the force when particles are in contact is thus
constant as $\ell$ varies (inset of Fig. 4). For decreasing $\ell$ and
shorter-range forces, the particles more readily escape an increasingly
narrower potential well, decreasing the relative proportion of transitions
that terminate in a C state in favour of other transition pathways that end in
P or T (Fig. 4(a) and (b)). This is reflected in the occupation probability
statistics: as $\ell$ decreases there is a corresponding monotonic decrease in
$\Pi^{\text{C}}$ and a pronounced increase in $\Pi^{\text{P}}$ (Fig. 4(c)).
Indeed, $\Pi^{\text{C}}$ and $\Pi^{\text{P}}$ converge as $\ell$ decreases,
resembling the statistics of colloids bound by short-range depletion forces
[20].

We have studied the emergent assembly statistics for clusters of millimetric
particles at the fluid interface driven by a field of supercritical Faraday
waves. From an experimental perspective, it would be interesting to explore
the implications of our results for colloidal assembly where fine control may
be exerted over both the temperature (strength of fluctuations) and details of
the interaction potential. Modern colloidal-synthesis techniques allow the
properties of the interaction potential to be tuned [34, 35, 36], for example
by varying the depletant concentration [37] or indeed by incorporating
capillarity [12], thereby offering control mechanisms over the assembly
statistics. The results of such experiments could be readily corroborated
using recently developed techniques from numerical continuation [38]. Finally,
our results prompt investigation into how assembly statistics are affected
when particles are submerged in a wider class of athermal environments, for
example active bacterial baths [39, 40], or when the particles themselves are
living or active [41, 11, 42].

This work is partially supported by the Office of Naval Research (ONR
N00014-21-1-2816). J.-W.B. is supported by the Department of Defense through
the National Defense Science and Engineering Graduate (NDSEG) Fellowship
Program.

## References

  * Grzybowski  _et al._ [2009] B. A. Grzybowski, C. E. Wilmer, J. Kim, K. P. Browne, and K. J. M. Bishop, Self-assembly: from crystals to cells, Soft Matter 5, 1110 (2009).
  * Tan  _et al._ [2022] T. H. Tan, A. Mietke, J. Li, Y. Chen, H. Higinbotham, P. J. Foster, S. Gokhale, J. Dunkel, and N. Fakhri, Odd dynamics of living chiral crystals, Nature 607, 287 (2022).
  * Chakraborty  _et al._ [2022] I. Chakraborty, D. J. G. Pearce, R. W. Verweij, S. C. Matysik, L. Giomi, and D. J. Kraft, Self-assembly dynamics of reconfigurable colloidal molecules, ACS Nano 16, 2471 (2022).
  * Tang and Grover [2022] X. Tang and M. A. Grover, Control of microparticle assembly, Annual Review of Control, Robotics, and Autonomous Systems 5, 491 (2022).
  * Bishop  _et al._ [2023] K. J. M. Bishop, S. L. Biswal, and B. Bharti, Active colloids as models, materials, and machines, Annual Review of Chemical and Biomolecular Engineering 14, 1 (2023).
  * Slavkov  _et al._ [2018] I. Slavkov, D. Carrillo-Zapata, N. Carranza, X. Diego, F. Jansson, J. Kaandorp, S. Hauert, and J. Sharpe, Morphogenesis in robot swarms, Science Robotics 3, eaau9178 (2018).
  * Yu  _et al._ [2023] H. Yu, Y. Fu, X. Zhang, L. Chen, D. Qi, J. Shi, and W. Wang, Programmable active matter across scales, Programmable Materials 1, e7 (2023).
  * Nicolson [1949] M. M. Nicolson, The interaction between floating particles, in  _Mathematical Proceedings of the Cambridge Philosophical Society_ , Vol. 45 (Cambridge University Press, 1949) pp. 288–295.
  * Kralchevsky and Nagayama [2000] P. A. Kralchevsky and K. Nagayama, Capillary interactions between particles bound to interfaces, liquid films and biomembranes, Advances in Colloid and Interface Science 85, 145 (2000).
  * Vella and Mahadevan [2005] D. Vella and L. Mahadevan, The “Cheerios effect”, American Journal of Physics 73, 817 (2005).
  * Protière [2023] S. Protière, Particle rafts and armored droplets, Annual Review of Fluid Mechanics 55, 459 (2023).
  * Liu  _et al._ [2018] I. B. Liu, N. Sharifi-Mood, and K. J. Stebe, Capillary assembly of colloids: Interactions on planar and curved interfaces, Annual Review of Condensed Matter Physics 9, 283 (2018).
  * Choi  _et al._ [2000] I. S. Choi, M. Weck, B. Xu, N. L. Jeon, and G. M. Whitesides, Mesoscopic, templated self-assembly at the fluid- fluid interface, Langmuir 16, 2997 (2000).
  * Loudet  _et al._ [2005] J.-C. Loudet, A. M. Alsayed, J. Zhang, and A. G. Yodh, Capillary interactions between anisotropic colloidal particles, Physical Review Letters 94, 018301 (2005).
  * Poty  _et al._ [2014] M. Poty, G. Lumay, and N. Vandewalle, Customizing mesoscale self-assembly with three-dimensional printing, New Journal of Physics 16, 023013 (2014).
  * Wang  _et al._ [2017] W. Wang, J. Giltinan, S. Zakharchenko, and M. Sitti, Dynamic and programmable self-assembly of micro-rafts at the air-water interface, Science Advances 3, e1602522 (2017).
  * Bowden  _et al._ [1997] N. Bowden, A. Terfort, J. Carbeck, and G. M. Whitesides, Self-assembly of mesoscale objects into ordered two-dimensional arrays, Science 276, 233 (1997).
  * Cavallaro Jr  _et al._ [2011] M. Cavallaro Jr, L. Botto, E. P. Lewandowski, M. Wang, and K. J. Stebe, Curvature-driven capillary migration and assembly of rod-like particles, Proceedings of the National Academy of Sciences 108, 20923 (2011).
  * Perry  _et al._ [2012] R. W. Perry, G. Meng, T. G. Dimiduk, J. Fung, and V. N. Manoharan, Real-space studies of the structure and dynamics of self-assembled colloidal clusters, Faraday Discussions 159, 211 (2012).
  * Perry  _et al._ [2015] R. W. Perry, M. C. Holmes-Cerfon, M. P. Brenner, and V. N. Manoharan, Two-dimensional clusters of colloidal spheres: Ground states, excited states, and structural rearrangements, Physical Review Letters 114, 228301 (2015).
  * Bush [2015] J. W. M. Bush, Pilot-wave hydrodynamics, Annual Review of Fluid Mechanics 47, 269 (2015).
  * [22] See Supplemental Material at [URL will be inserted by publisher] for a description of our experimental protocol and theoretical modeling.
  * Holmes-Cerfon  _et al._ [2013] M. Holmes-Cerfon, S. J. Gortler, and M. P. Brenner, A geometrical approach to computing free-energy landscapes from short-ranged potentials, Proceedings of the National Academy of Sciences 110, E5 (2013).
  * Harris and Bush [2015] D. M. Harris and J. W. M. Bush, Generating uniaxial vibration with an electrodynamic shaker and external air bearing, Journal of Sound and Vibration 334, 255 (2015).
  * Francois  _et al._ [2015] N. Francois, H. Xia, H. Punzmann, and M. Shats, Wave-particle interaction in the faraday waves, The European Physical Journal E 38, 106 (2015).
  * Xia  _et al._ [2019] H. Xia, N. Francois, H. Punzmann, and M. Shats, Tunable diffusion in wave-driven two-dimensional turbulence, Journal of Fluid Mechanics 865, 811 (2019).
  * Sanlı  _et al._ [2014] C. Sanlı, K. Saitoh, S. Luding, and D. Van Der Meer, Collective motion of macroscopic spheres floating on capillary ripples: Dynamic heterogeneity and dynamic criticality, Physical Review E 90, 033018 (2014).
  * Francois  _et al._ [2020] N. Francois, H. Xia, H. Punzmann, and M. Shats, Nonequilibrium thermodynamics of turbulence-driven rotors, Physical Review Letters 124, 254501 (2020).
  * Lim  _et al._ [2019] M. X. Lim, A. Souslov, V. Vitelli, and H. M. Jaeger, Cluster formation by acoustic forces and active fluctuations in levitated granular matter, Nature Physics 15, 460 (2019).
  * Grimmett and Stirzaker [2020] G. Grimmett and D. Stirzaker,  _Probability and random processes_ (Oxford University Press, 2020).
  * Deen  _et al._ [2007] N. Deen, M. V. S. Annaland, M. A. Van der Hoef, and J. A. M. Kuipers, Review of discrete particle modeling of fluidized beds, Chemical Engineering Science 62, 28 (2007).
  * Higham [2001] D. J. Higham, An algorithmic introduction to numerical simulation of stochastic differential equations, SIAM Review 43, 525 (2001).
  * Higham  _et al._ [2002] D. J. Higham, X. Mao, and A. M. Stuart, Strong convergence of Euler-type methods for nonlinear stochastic differential equations, SIAM Journal on Numerical Analysis 40, 1041 (2002).
  * Gil-Villegas  _et al._ [1997] A. Gil-Villegas, A. Galindo, P. J. Whitehead, S. J. Mills, G. Jackson, and A. N. Burgess, Statistical associating fluid theory for chain molecules with attractive potentials of variable range, The Journal of Chemical Physics 106, 4168 (1997).
  * Lu and Weitz [2013] P. J. Lu and D. A. Weitz, Colloidal particles: crystals, glasses, and gels, Annual Review of Condensed Matter Physics 4, 217 (2013).
  * Al Harraq  _et al._ [2022] A. Al Harraq, A. A. Hymel, E. Lin, T. M. Truskett, and B. Bharti, Dual nature of magnetic nanoparticle dispersions enables control over short-range attraction and long-range repulsion interactions, Communications Chemistry 5, 72 (2022).
  * Edwards and Bevan [2012] T. D. Edwards and M. A. Bevan, Depletion-mediated potentials and phase behavior for micelles, macromolecules, nanoparticles, and hydrogel particles, Langmuir 28, 13816 (2012).
  * Trubiano and Holmes-Cerfon [2020] A. Trubiano and M. Holmes-Cerfon, From canyons to valleys: Numerically continuing sticky-hard-sphere clusters to the landscapes of smoother potentials, Physical Review E 101, 042608 (2020).
  * Heinonen  _et al._ [2022] V. Heinonen, A. Abraham, J. Słomka, K. J. Burns, P. J. Sáenz, and J. Dunkel, Emergent universal statistics in nonequilibrium systems with dynamical scale selection, arXiv preprint arXiv:2205.01627 (2022).
  * Grober  _et al._ [2023] D. Grober, I. Palaia, M. C. Uçar, E. Hannezo, A. Šarić, and J. Palacci, Unconventional colloidal aggregation in chiral bacterial baths, Nature Physics (2023).
  * Ko  _et al._ [2022] H. Ko, M. Hadgu, K. Komilian, and D. L. Hu, Small fire ant rafts are unstable, Physical Review Fluids 7, 090501 (2022).
  * Zöttl and Stark [2023] A. Zöttl and H. Stark, Modeling active colloids: From active brownian particles to hydrodynamic and chemical fields, Annual Review of Condensed Matter Physics 14, 109 (2023).


# SoQal: Selective Oracle Questioning in Active Learning

Dani Kiyasseh
Department of Engineering Science
University of Oxford
Oxford, UK
<EMAIL_ADDRESS>
&Tingting Zhu
Department of Engineering Science
University of Oxford
Oxford, UK
<EMAIL_ADDRESS>
&David A. Clifton11footnotemark: 1
Department of Engineering Science
University of Oxford
Oxford, UK
<EMAIL_ADDRESS>
equal contribution

###### Abstract

Large sets of unlabelled data within the healthcare domain remain
underutilized. Active learning offers a way to exploit these datasets by
iteratively requesting an oracle (e.g. medical professional) to label
instances. This process, which can be costly and time-consuming is overly-
dependent upon an oracle. To alleviate this burden, we propose SoQal, a
questioning strategy that dynamically determines when a label should be
requested from an oracle. We perform experiments on five publically-available
datasets and illustrate SoQal’s superiority relative to baseline approaches,
including its ability to reduce oracle label requests by up to 35%. SoQal also
performs competitively in the presence of label noise: a scenario that
simulates clinicians’ uncertain diagnoses when faced with difficult
classification tasks.

##  1 Introduction

The success of modern-day deep learning algorithms in the medical domain has
been contigent upon the availability of large, labelled datasets (Poplin et
al., 2018; Tomašev et al., 2019; Attia et al., 2019). The curation of such
datasets, however, is a challenge due to the time-consuming nature of and high
costs associated with labelling. This is particularly the case in the medical
domain where the input of expert medical professionals is required. One way of
overcoming this challenge and exploiting large, unlabelled datasets is via
active learning (AL) (Settles, 2009). In this setting, a learner is tasked
with iteratively acquiring a subset of unlabelled instances and asking an
oracle to label it, before adding it to the set of labelled instances. By
presenting the most informative instances to the oracle, AL aims to improve
the performance of algorithms while minimizing the burden of labelling on the
oracle.

Although shown to be data-efficient, current AL approaches are overly reliant
on the presence of an oracle. Namely, an oracle is always assumed to be
present. Such over-reliance is detrimental for two reasons. Firstly, it
negatively affects the applicability of AL algorithms to scenarios where an
oracle is either unavailable or is ill-trained for the task at hand. This is
prevalent, for instance, in low-resource healthcare settings where there is a
shortage of qualified medical professionals. Secondly, over-reliance can still
inundate experts with a significant number of label requests, the very goal AL
is supposed to minimize. This is particularly consequential for expert medical
professionals who have limited bandwidth and who are increasingly suffering
from ’burnout’ (West et al., 2016; Shanafelt et al., 2017).

Decreasing the dependence of AL algorithms on the oracle and further
alleviating their associated labelling burden can significantly improve the
manner in which a physician is involved while also overcoming limitations
inherent in the oracle. Although Kiyasseh et al. (2020) suggest performing
oracle-free active learning, they only consider the extreme scenarios where an
oracle is either unavailable or always available. We hypothesize that an
algorithm capable of finding a middle-ground in terms of an oracle strategy
could lead to less dependence on an oracle while not compromising, or
potentially improving, performance. To design such an algorithm, we take
inspiration from work in selective classification (Chow, 1970; El-Yaniv and
Wiener, 2010) where algorithms learn to abstain from making a prediction.

Our Contributions. In this paper, we challenge the traditional assumptions of
active learning, namely the availability of noise-free oracles and propose a
dynamic strategy to deal with this.

  1. 1.

Selective Oracle Questioning (SoQal): a dynamic strategy that learns when to
request a label from an oracle during active learning.

  2. 2.

A novel objective function that helps a network predict the zero-one
classification loss incurred on the main task. We use this prediction to
control the dependence of the network on an oracle.

##  2 Related Work

Active learning and healthcare have been relatively under-explored. A recent
review of active learning methodologies can be found in Settles (2009). In the
healthcare domain, Gong et al. (2019) propose to acquire instances from an
electronic health record (EHR) database using a Bayesian deep latent Gaussian
model to improve mortality prediction. Smailagic et al. (2018, 2019) introduce
MedAL, a method that actively acquires unannotated medical images by measuring
their distance in a latent space to images in the training set. Such
similarity metrics, however, are sensitive to the original amount of labelled
training data. The work of Wang et al. (2019) is similar to ours in that they
focus on the electrocardiogram. Gal et al. (2017) adopt BALD (Houlsby et al.,
2011) in the context of Monte Carlo Dropout to acquire datapoints that
maximize the Jensen-Shannon divergence (JSD) across MC samples. There have
been several attempts at learning from multiple or imperfect labelers (Dekel
et al., 2012; Zhang and Chaudhuri, 2015; Sinha et al., 2019). Urner et al.
(2012) propose choosing the oracle that should label a particular instance.
Unlike our approach, they do not explore independence from an oracle. Yan et
al. (2016) do consider abstention in an AL setting, yet it is performed by the
labeler. Instead, our approach places the decision of abstention under the
control of the learner. To the best of our knowledge, previous work, in
contrast to ours, has assumed the existence of an oracle and has not explored
a dynamic oracle selection strategy.

Selective classification and healthcare fit well with one another given the
high-stakes scenarios present in the latter. Early work in selective
classification by Chow (1970) introduces the risk-coverage trade-off whereby
the empirical risk of a model is inversely related to its rate of abstentions.
El-Yaniv and Wiener (2010) define perfect learning as an empirical risk of
zero that corresponds to non-zero coverage of instances and propose the
Consistent Selective Strategy (CSS) to achieve this. Wiener and El-Yaniv
(2011) use a support vector machine (SVM) to rank and reject instances based
on the degree of disagreement between hypotheses. In some frameworks, these
happen to be the same instances that active learning treats as being most
informative. More recently, Cortes et al. (2016) outline an objective function
that penalizes inappropriate abstentions alongside the rate at which they are
performed. Instead of using uncertainty-based heuristics such as the entropy
of the posterior predictive distribution or the Softmax-Response (Geifman and
El-Yaniv, 2017). Liu et al. (2019) exploit portfolio theory and propose the
gambler’s loss in order to learn a selection function that determines whether
instances are rejected. However, this approach requires a significant amount
of hyperparameter tuning. Most similar to our work is SelectiveNet (Geifman
and El-Yaniv, 2019) where a multi-head neural architecture is used in
conjunction with an empirical selective risk (ESR) objective function and a
percentile threshold. In contrast, our work proposes a different objective
function, a thresholding mechanism, and specifically considers oracle
selection. Moreover, ESR assumes that the labels associated with instances are
known. In contrast, we extend the idea of selective classification to the
setting where labels are unknown.

##  3 Methods

###  3.1 Active Learning

In this work, we consider a learner $f_{\omega}:X\rightarrow Y$, a neural
network parameterized by $\omega$ that maps inputs $X\in\mathbb{R}^{m}$ to
outputs $Y\in[1\ldots\textit{C}]$, where C is the number of classes. After
training on a pool of labelled data $\textit{L}=(X_{L},Y_{L})$ for $\tau$
epochs, the learner is tasked with querying the unlabelled pool of data
$U=(X_{U},Y_{U})$ and acquiring the top $b\%$ of instances, $x_{b}\sim X_{U}$,
that it deems to be most informative.

The degree of informativeness of an instance is determined by an acquisition
function, $\alpha$, such as that found in Bayesian Active Learning by
Disagreement (BALD) (Houlsby et al., 2011) or ALPS (Kiyasseh et al., 2020).
Such approaches when used in conjunction with Monte Carlo Dropout (MCD) (Gal
and Ghahramani, 2016) identify instances that lie in the region of
classification uncertainty. This is a region in which hypotheses disagree the
most about instances. One forward pass of MCD outputs a softmax posterior
distribution $p(y|x,\omega_{t})$ where $\omega_{t}\sim q_{\theta}(\omega)$
represents parameters sampled from the MC distribution. To obtain an accurate
approximation of the hypothesis space, this is repeated T times resulting in
$G\in\mathbb{R}^{T\text{x}C}$ for each instance.

$\begin{split}\mathrm{BALD_{MCD}}=\mathrm{JSD}(p_{1},p_{2},\ldots,p_{T})&=\mathrm{H}(p(y|x))-\mathbb{E}_{p(w|D_{train})}\left[\mathrm{H}(p(y|x,w))\right]\\\
&\approx\mathrm{H}\left(\frac{1}{T}\sum_{t=1}^{T}p(y|x,\omega_{t})\right)-\frac{1}{T}\sum_{t=1}^{T}\left[\mathrm{H}(p(y|x,\omega_{t}))\right]\end{split}$
(1)

where JSD is the Jensen-Shannon Divergence and H represents the entropy
function. Once instances are acquired, they are provided to an oracle, who is
assumed to be available and noise-free, for labelling before being added to
the pool of labelled instances. This process is repeated until the performance
of an algorithm is considered to be sufficient.

###  3.2 Selective Oracle Questioning

Traditionally, in AL, requesting a label from an oracle automatically follows
the act of selecting an unlabelled instance. We challenge this convention and
treat these two processes as independent of one another. This section
describes how to choose whether or not to request a label after an unlabelled
instance has been chosen.

Architecture. We assume the existence of a prediction network, $f_{\omega}$,
which for each instance, $x$, generates posterior class probabilities,
$p(y|x,\omega)$, and an oracle selection network, $g_{\theta}:X\rightarrow
o\in[0,1]$ parameterized by $\theta$ that maps that same instance to a scalar,
as shown in Fig. 1.

Figure 1: Selective Oracle Questioning Framework.

Objective Function. We interpret the scalar, $o$, as approximating the
probability that an oracle is requested for a label. Ideally, a network should
only be reliant on an oracle when it cannot classify an instance correctly
itself. Therefore, high values of $o$ should be associated with incorrect
network predictions. Conversely, low values of $o$ should be associated with
correct network predictions. We encourage this behaviour by assigning the
zero-one loss, e, of $f_{\omega}$ as the ground truth label for $g_{\theta}$.

We note that, for each instance, this ground truth label will inevitably shift
during training as the network becomes more adept at classifying it. Early in
training, the ratio of misclassified to correctly classified instances will be
high. Late in training, the opposite is true. If such ratios are left
unaccounted for, with e being used as the ground truth label, the majority of
the outputs of $g_{\theta}$ will be high early during training and low near
the end. Therefore, distinguishing between individual instances based solely
on the output of $g_{\theta}$ would be difficult and thus deem it an
unreliable signal for oracle selection. This scenario is equivalent to that of
class imbalance. We describe how to mitigate this effect below.

Our objective function for a mini-batch of size, $B$, thus consists of two
terms: 1) a cross-entropy class prediction loss for the main task, and 2) a
weighted binary cross-entropy loss for the oracle selection network.

$\begin{split}\mathcal{L}=&\sum_{i=1}^{B}\overbrace{-\log\left(p(y_{i}=c|x_{i},\omega)\right)}^{\text{Class
Prediction Loss}}-\overbrace{\beta
e_{i}\log\left(g_{\theta}(o|x_{i})\right)-(1-e_{i})\log\left(1-g_{\theta}(o|x_{i})\right)}^{\text{Oracle
Selection Loss}}\\\ \end{split}$ (2)

where c is the target class. To offset the aforementioned class imbalance, we
introduce a dynamic hyperparameter,
$\beta=\frac{\sum\delta_{e=0}}{\sum\delta_{e=1}}$, which changes according to
the ratio of correctly classified to misclassified instances within a mini-
batch, where $\delta$ is the Kronecker delta function. As training progresses,
$\beta<1\rightarrow\beta>1$.

Thresholding. As we are dealing with unlabelled instances, we are interested
in exploiting the output of $g_{\theta}$ as a proxy for whether an instance is
correctly classified ($e=0$) or not ($e=1$). The separability of these two
states determine the reliability of such a proxy. In Fig. 2(b), we illustrate
the distribution of the $o$ values that correspond to $e=0$ and $e=1$ on the
labelled training data.

(a) Early in Training

(b) Late in Training

(c) Hellinger Distance

Figure 2: Density of the outputs of the oracle selection network $g_{\theta}$
conditioned on the zero-one classification error (a) early in training and (b)
late in training. (c) Hellinger distance, $\mathcal{D}_{H}$, between
distributions of outputs of selection function during training. Delegation of
oracle questioning to the network occurs when $\mathcal{D}_{H}\geq S$. Notice
the improved separability of the two distributions as a result of the training
procedure.

At the end of each training epoch, the $o$ values in Fig. 2(b) were fit to two
unimodal Gaussian distributions. This generates
$\mathcal{N}_{0}(\mu_{0},\sigma_{0}^{2})$ and
$\mathcal{N}_{1}(\mu_{1},\sigma_{1}^{2})$ for $e=0$ and $e=1$, respectively.
We quantify the separability of these two distributions using the Hellinger
distance, $\mathcal{D}_{H}\in[0,1]$.

$\mathcal{D}_{H}=\sqrt{1-\sqrt{\frac{2\sigma_{0}\sigma_{1}}{\sigma_{0}^{2}\sigma_{1}^{2}}}e^{-\frac{1}{4}\frac{(\mu_{0}-\mu_{1})^{2}}{\sigma_{0}^{2}\sigma_{1}^{2}}}}$
(3)

If, at a particular acquisition epoch, $\mathcal{D}_{H}$ does not exceed some
threshold S, then $g_{\theta}$ cannot be relied upon and an oracle is always
requested for a label. The value of S can be altered depending on the degree
of trust one has in the network and labeller. When $\mathcal{D}_{H}\geq S$,
$\mathcal{N}_{0}$ and $\mathcal{N}_{1}$ are evaluated using the $o$ value for
each acquired unlabelled instance. We outline the probability of asking an
oracle, p(A), in Eq. 4. Algorithms LABEL:algo:al and LABEL:algo:soqal in
Appendix B illustrate the entire active learning procedure.

$p(\text{A})=\begin{dcases}1,&\mathcal{D}_{H}<S\\\
1,&\mathcal{N}\left(o|\mu_{1},\sigma_{1}^{2},e=1\right)>\mathcal{N}\left(o|\mu_{0},\sigma_{0}^{2},e=0\right)\text{and}\
\mathcal{D}_{H}\geq S\\\ 0,&\text{otherwise}\\\ \end{dcases}$ (4)

###  3.3 Chernoff Bound on Error Rate of Selection Network

Given that the selection network is tasked with making a binary decision, we
can obtain a theoretical upper bound on its probability of making an error
(via the overlap of density functions in Fig. 2). An error in this context can
be interpreted as stubbornness, where the network does not ask for help when
it should have, and over-reliance, where the network asks for help when it
should not have. The Chernoff upper bound on the error rate is as follows. The
full derivation can be found in Appendix C.

$\begin{split}P(\text{error})&\leq
P(e=0)^{\beta^{*}}P(e=1)^{1-\beta^{*}}e^{-\left[\frac{\beta^{*}(1-\beta^{*})(\mu_{0}-\mu_{1})^{2}}{2(\beta^{*}\sigma_{0}^{2}+(1-\beta^{*})\sigma_{1}^{2})}+\frac{1}{2}log\frac{\beta^{*}\sigma_{0}^{2}+(1-\beta^{*})\sigma_{1}^{2}}{\sigma_{0}^{2\beta^{*}}\sigma_{1}^{2(1-\beta^{*})}}\right]}\end{split}$
(5)

where $P(e=0)$ and $P(e=1)$ represent the prior probabilities of each class
corresponding to the zero-one loss. $\beta^{*}$ is obtained by minimizing the
exponent term.

##  4 Experimental Design

###  4.1 Datasets

Experiments were implemented in PyTorch (Paszke et al., 2019) and were
conducted on five publically-available datasets. These datasets consist of
physiological time-series data such as the photoplethysmogram (PPG) and the
electrocardiogram (ECG) alongside available cardiac arrhythmia labels. We use
$\mathcal{D}_{1}$ = PhysioNet 2015 PPG, $\mathcal{D}_{2}$ = PhysioNet 2015 ECG
(Clifford et al., 2015) (5-way), $\mathcal{D}_{3}$ = PhysioNet 2017 ECG
(Clifford et al., 2017) (4-way), $\mathcal{D}_{4}$ = Cardiology ECG (Hannun et
al., 2019) (12-way), and $\mathcal{D}_{5}$ = PTB ECG (Bousseljot et al., 1995)
(2-way).

###  4.2 Baselines

We experiment with baselines that exhibit varying degrees of oracle
dependence. No Oracle was explored by Kiyasseh et al. (2020) where 0% of label
are oracle-based and are instead based on network predictions. Epsilon Greedy
is a stochastic strategy from the reinforcement learning literature Watkins
(1989) where the degree of network exploration, performed with probability
$\epsilon$, is decayed exponentially throughout training. In our case, we
exponentially decay the reliance of the network on an oracle as a function of
the number of acquisition epochs. Entropy Response assumes that high entropy
predictions generated by a network are indicative of instances of which the
network is unsure. Therefore, we introduce a threshold,
$S_{\mathrm{Entropy}}$, such that if it is exceeded, an oracle is requested to
label the instance chosen. The most dependent baseline is 100% Oracle, a
traditionally-employed strategy in AL where 100% of labels are oracle-based.

We do not compare our methods to Softmax Response (Geifman and El-Yaniv, 2017)
and SelectiveNet (Geifman and El-Yaniv, 2019), despite their strong
performance for selective classification, as they do not trivially extend to
the setting in which labels are unavailable.

###  4.3 Hyperparameters

Active Learning. For all experiments, we follow the hyperparameter choices in
Kiyasseh et al. (2020). Namely, we chose the number of MC samples T = 20.
Acquisitions of unlabelled instances were performed at pre-defined epochs of
$\tau=5n$, $n\in\mathbb{N}^{+}$. Moreover, the amount of instances acquired
during each acquisition epoch is $b=2\%$ of the remaining unlabelled
instances. Lastly, we chose the temporal period $\Delta t$ = 1 for all
experiments involving temporal variants of acquisition functions, as described
later.

Selective Oracle Questioning. In order for selective oracle questioning to be
delegated to the network, we must have $\mathcal{D}_{H}\geq S$. Given that
$\mathcal{D}_{H}$ was observed to have an increasing trend, as seen in Fig.
2(c), we chose $S=0.15$ to balance between reliability of the selection
function and independence from an oracle. We also explore the sensitivity of
SoQal to this choice of $S$.

##  5 Experiments

###  5.1 Selective Oracle Questioning with Noise-Free Oracle

The ability of a learner to appropriately determine when to request labels
from an oracle can significantly alleviate the associated labelling burden. In
this section, we evaluate this ability amongst the various oracle selection
strategies. In Fig. 3, we illustrate the validation AUC of SoQal during
training compared to that of the proposed baselines. We show that the 100%
oracle strategy outperforms the remaining methods. This can be seen by the
$\mathrm{AUC}\approx 0.70$ and $\approx 0.80$ in Figs. 3(a) and 3(b),
respectively. We expect this behaviour as labels from a noise-free oracle are
likely to be accurate. Conversely, the no oracle strategy struggles, as seen
by an $\mathrm{AUC}\approx 0.64$ and $\approx 0.62$ in Figs. 3(a) and 3(b),
respectively. This can be explained by the idea that complete independence
from an oracle, whereby labels are network-generated, is likely to lead to
noisy labels and thus hinder performance. Based on these findings, it is clear
that a dynamic oracle questioning strategy can offer a balance.

(a) $\mathcal{D}_{2}$, BALDMCP

(b) $\mathcal{D}_{3}$, BALDMCD

Figure 3: Mean validation AUC as a function of oracle selection strategies on
(a) $\mathcal{D}_{2}$ using BALDMCP and (b) $\mathcal{D}_{3}$ using BALDMCD.
Results are averaged across 5 seeds.

We illustrate, in Table 1, the test AUC of the oracle questioning strategies
on all datasets. Across $\mathcal{D}_{1}$ \- $\mathcal{D}_{3}$, we show that
SoQal consistently outperforms its counterparts. For instance, while using
BALDMCD on $\mathcal{D}_{2}$, SoQal achieves an $\mathrm{AUC}=0.707$ compared
to $\mathrm{AUC}=0.609$ and $0.584$ for Epsilon Greedy and Entropy Response,
respectively. These findings suggest that SoQal is better equipped to know
when and for which instance a label is requested from an oracle. However, we
observe that SoQal performs on par and relatively worse than the remaining
methods on $\mathcal{D}_{4}$ and $\mathcal{D}_{5}$, respectively. We
hypothesize that the former result is due to the cold-start problem
(Konyushkova et al., 2017) whereby AL algorithms fail to learn due to few
available labelled training data. We support this claim with experiments in
Appendix I. As for the worse performance on $\mathcal{D}_{5}$, we believe this
is due to the high degree of independence endowed upon the learner given the
choice of $S$. Increasing the value of $S$ will cede control to the oracle and
thus improve performance, an effect we quantify in Section 5.4.

Table 1: Mean test AUC of oracle questioning strategies in the presence of a
noise-free oracle. Results are shown for datasets
$\mathcal{D}_{1}-\mathcal{D}_{5}$ and all acquisition functions. Mean and
standard deviation values are shown across five seeds. ’No AL’ is the strategy
that does not employ active learning.

Dataset | Ac. Function $\alpha$ | Oracle Questioning Method |
---|---|---|---
No Oracle | Entropy Response | Epsilon Greedy | SoQal (ours) | 100% Oracle | No AL
$\mathcal{D}_{1}$ | $\mathrm{BALD_{MCD}}$ | 0.465 $\pm$ 0.017 | 0.496 $\pm$ 0.039 | 0.491 $\pm$ 0.028 | 0.621 $\pm$ 0.021 | 0.653 $\pm$ 0.013 | 0.577 $\pm$ 0.014
$\mathrm{BALD_{MCP}}$ | 0.464 $\pm$ 0.023 | 0.517 $\pm$ 0.043 | 0.501 $\pm$ 0.043 | 0.645 $\pm$ 0.015 | 0.676 $\pm$ 0.020
$\mathrm{BALC_{KLD}}$ | 0.500 $\pm$ 0.023 | 0.548 $\pm$ 0.034 | 0.548 $\pm$ 0.042 | 0.598 $\pm$ 0.055 | 0.634 $\pm$ 0.030
Temporal $\mathrm{BALC_{KLD}}$ | 0.496 $\pm$ 0.024 | 0.536 $\pm$ 0.040 | 0.521 $\pm$ 0.059 | 0.646 $\pm$ 0.067 | 0.659 $\pm$ 0.033
$\mathcal{D}_{2}$ | $\mathrm{BALD_{MCD}}$ | 0.573 $\pm$ 0.063 | 0.584 $\pm$ 0.041 | 0.609 $\pm$ 0.071 | 0.707 $\pm$ 0.038 | 0.713 $\pm$ 0.053 | 0.679 $\pm$ 0.040
$\mathrm{BALD_{MCP}}$ | 0.589 $\pm$ 0.045 | 0.638 $\pm$ 0.043 | 0.637 $\pm$ 0.044 | 0.677 $\pm$ 0.042 | 0.735 $\pm$ 0.028
$\mathrm{BALC_{KLD}}$ | 0.602 $\pm$ 0.044 | 0.582 $\pm$ 0.017 | 0.643 $\pm$ 0.033 | 0.677 $\pm$ 0.024 | 0.722 $\pm$ 0.018
Temporal $\mathrm{BALC_{KLD}}$ | 0.575 $\pm$ 0.017 | 0.612 $\pm$ 0.050 | 0.605 $\pm$ 0.019 | 0.648 $\pm$ 0.057 | 0.735 $\pm$ 0.011
$\mathcal{D}_{3}$ | $\mathrm{BALD_{MCD}}$ | 0.581 $\pm$ 0.014 | 0.588 $\pm$ 0.013 | 0.673 $\pm$ 0.015 | 0.721 $\pm$ 0.025 | 0.802 $\pm$ 0.008 | 0.716 $\pm$ 0.012
$\mathrm{BALD_{MCP}}$ | 0.623 $\pm$ 0.020 | 0.676 $\pm$ 0.058 | 0.665 $\pm$ 0.028 | 0.720 $\pm$ 0.044 | 0.798 $\pm$ 0.007
$\mathrm{BALC_{KLD}}$ | 0.631 $\pm$ 0.010 | 0.629 $\pm$ 0.004 | 0.643 $\pm$ 0.041 | 0.731 $\pm$ 0.033 | 0.787 $\pm$ 0.008
Temporal $\mathrm{BALC_{KLD}}$ | 0.600 $\pm$ 0.005 | 0.630 $\pm$ 0.014 | 0.654 $\pm$ 0.019 | 0.730 $\pm$ 0.024 | 0.794 $\pm$ 0.002
$\mathcal{D}_{4}$ | $\mathrm{BALD_{MCD}}$ | 0.486 $\pm$ 0.011 | 0.489 $\pm$ 0.030 | 0.474 $\pm$ 0.037 | 0.468 $\pm$ 0.021 | 0.585 $\pm$ 0.011 | 0.486 $\pm$ 0.023
$\mathrm{BALD_{MCP}}$ | 0.493 $\pm$ 0.030 | 0.504 $\pm$ 0.026 | 0.492 $\pm$ 0.024 | 0.499 $\pm$ 0.029 | 0.605 $\pm$ 0.024
$\mathrm{BALC_{KLD}}$ | 0.505 $\pm$ 0.032 | 0.504 $\pm$ 0.039 | 0.473 $\pm$ 0.010 | 0.495 $\pm$ 0.012 | 0.588 $\pm$ 0.033
Temporal $\mathrm{BALC_{KLD}}$ | 0.511 $\pm$ 0.030 | 0.496 $\pm$ 0.023 | 0.496 $\pm$ 0.023 | 0.503 $\pm$ 0.010 | 0.532 $\pm$ 0.027
$\mathcal{D}_{5}$ | $\mathrm{BALD_{MCD}}$ | 0.717 $\pm$ 0.006 | 0.715 $\pm$ 0.005 | 0.718 $\pm$ 0.006 | 0.661 $\pm$ 0.105 | 0.937 $\pm$ 0.004 | 0.710 $\pm$ 0.097
$\mathrm{BALD_{MCP}}$ | 0.719 $\pm$ 0.009 | 0.678 $\pm$ 0.074 | 0.774 $\pm$ 0.047 | 0.453 $\pm$ 0.136 | 0.705 $\pm$ 0.013
$\mathrm{BALC_{KLD}}$ | 0.679 $\pm$ 0.056 | 0.664 $\pm$ 0.064 | 0.726 $\pm$ 0.019 | 0.638 $\pm$ 0.145 | 0.900 $\pm$ 0.036
Temporal $\mathrm{BALC_{KLD}}$ | 0.720 $\pm$ 0.010 | 0.689 $\pm$ 0.061 | 0.741 $\pm$ 0.028 | 0.571 $\pm$ 0.161 | 0.708 $\pm$ 0.002

###  5.2 Selective Oracle Questioning with Noisy Oracle

In healthcare, physicians may be ill-trained, fatigued, or unable to diagnose
a case due to its difficulty. We simulate these scenarios by introducing two
types of label noise. We stochastically flip each label 1) to any other label
randomly (Random), or 2) to its nearest neighbour from a different class in a
compressed subspace (Nearest Neighbour). Whereas the first form of noise is
extreme, the latter form is more realistic as it may represent uncertainty in
physician diagnoses. To simulate various magnitudes of noise, we chose the
probability of introducing noise, $\gamma=[0.05,0.1,0.2,0.4,0.8]$. In Fig. 4,
we illustrate the effect of label noise on the test AUC of oracle questioning
strategies. The remaining results can be found in Appendix H.

(a) $\mathcal{D}_{1}$, $\mathrm{BALD_{MCP}}$

Figure 4: Average AUC of the oracle questioning strategies in the absence and
presence of various magnitudes of label noise on $\mathcal{D}_{1}$ using
$\mathrm{BALD_{MCP}}$. With up to 80% random or nearest neighbour label noise,
SoQal still outperforms its counterpart methods that are trained without label
noise.

In Fig. 4, we show that SoQal outperforms the remaining strategies across all
noise types and levels (except with 40% random noise). For instance, with 5%
random noise, SoQal achieves an $\mathrm{AUC}\approx 0.66$ compared to
$\mathrm{AUC}\approx 0.56$ and $\approx 0.53$ for Epsilon Greedy and Entropy
Response, respectively. Secondly, SoQal is better able to deal with label
noise than its counterparts. Specifically, SoQal with 80% random noise
achieves $\mathrm{AUC}\approx 0.53$ which is greater than $\mathrm{AUC}\approx
0.50$ and $\approx 0.52$ achieved by Epsilon Greedy and Entropy Response with
no noise, respectively. This effect, which is even more pronounced when
dealing with nearest neighbour noise, indicates the utility of SoQal in the
presence of a noisy oracle. Finally, we observe that the introduction of label
noise occasionally improves performance. This can be seen by the increase in
SoQal’s AUC from 0.64 (no noise) to 0.66 (5% random noise). We hypothesize
that this is due to inherent label noise in the public datasets. Therefore, by
introducing further noise, we may be nudging these labels towards their ground
truth values.

###  5.3 Degree of Dependence of SoQal on Oracle

It could be argued that the superiority of SoQal is simply due to high oracle
dependence, as would be naively expected. In this section, we quantify SoQal’s
dependence on an oracle using the oracle ask-rate: the proportion of all
instance acquisitions whose labels are requested from an oracle. In Fig. 5(a),
we illustrate this oracle ask-rate for different label noise scenarios.

(a) Oracle ask-rate for different label noise scenarios

(b) Correlation between oracle ask-rate and generalization performance

Figure 5: (a) SoQal’s oracle ask-rate and (b) correlation between oracle ask-
rate and average test AUC. Results are averaged across five seeds and all
datasets, $\mathcal{D}_{1}-\mathcal{D}_{5}$, and are shown for each
acquisition function and label noise scenario.

In Fig. 5(a), we show that the oracle ask-rate varies based on the acquisition
function used. For instance, at 20% random noise, $\mathrm{BALD_{MCP}}$
requests labels 65% of the time whereas the remaining acquisition functions do
so approximately 77% of the time. We hypothesize that this variability in the
oracle ask-rate is due to the variability in the difficulty of the instances
acquired by the acquisition functions. In other words, decreased dependence by
$\mathrm{BALD_{MCP}}$ could be indicative of the acquisition of instances that
are relatively farther away from the hyperplane. Thus, they are easier to
classify and require less oracle guidance.

In the presence of label noise, decreased dependence is indeed associated with
improved generalization performance. This claim is supported by the negative
correlation between the oracle ask-rate and the test AUC observed in Fig.
5(b). In other words, networks are requesting labels less and performing
better. Such findings reaffirm the conclusion that SoQal knows when to request
a label from an oracle.

###  5.4 Controlling Oracle Dependence via Hellinger Threshold,
$\boldsymbol{S}$

When $\mathcal{D}_{H}<S$, all label requests are sent to the oracle.
Therefore, the value of $S$ should control the oracle ask-rate and thus
performance. We illustrate the performance of SoQal for a range of values of
$S$ in Table 2. We confirm the expected positive relationship between $S$ and
the oracle ask-rate where as $S=0.1\rightarrow 0.4$, the oracle ask-rate
increases from 86% to 100%. Moreover, for this particular dataset and
acquisition function, $S=0.200$ is the optimal value as it achieves an
$\mathrm{AUC}=0.768$ with an oracle ask-rate below 100%. This finding
reaffirms our previous hypothesis that the original labels in the dataset may
be noisy. Therefore, not requesting these particular labels from the oracle is
advantageous.

Table 2: Mean test AUC of SoQal and oracle ask rate in response to various threshold values, $S$. Results are shown for $\mathcal{D}_{3}$ and BALDMCD across five seeds. Experiments are performed with a noise-free oracle. Threshold, $S$ | 0.100 | 0.125 | 0.150 | 0.175 | 0.200 | 0.300 | 0.400
---|---|---|---|---|---|---|---
Average Oracle Ask Rate % | 86 | 85 | 89 | 90 | 94 | 100 | 100
AUC | 0.716 | 0.744 | 0.721 | 0.753 | 0.768 | 0.743 | 0.755

##  6 Discussion and Future Work

In this work, we proposed a dynamic oracle questioning strategy, SoQal, in the
context of active learning and healthcare. We showed that while striking a
balance between independence from and over-reliance on an oracle, SoQal
outperforms strong baseline methods. Furthermore, in the presence of noisy
oracles which represent ill-trained or fatigued physicians, SoQal decreases
its dependence by 35% and continues to outperform its counterparts. Indeed, we
showed that this decreased dependence was appropriate and was correlated with
improved generalization performance. We now mention several exciting avenues
worth exploring.

Incorporating Prior Information. As it stands, and in the absence of a priori
information, the default mode for SoQal is deferral to an oracle. If, however,
relevant a priori information is available (such as the extent of the noise
inherent in the oracle’s labels), then either the default mode or the
Hellinger threshold, $S$, can be altered accordingly. The latter can also be
changed during training if a dynamic label noise detector is present.

Incorporating Multiple Oracles. In this work, we explored a dynamic oracle
questioning strategy in the presence of a single oracle. Realistic clinical
scenarios may include multiple experts of various levels of competency.
Therefore, an interesting line of research could focus on a strategy that
dynamically determines the ideal expert for the instance at hand.

##  7 Broader Impact

The exploration of less burdensome active learning algorithms in the context
of healthcare can alleviate the exigent burden placed on medical
practitioners. This is particularly acute in environments where burnout of
physicians and nurses is increasingly observed due to a lack of electronic
health record usability (Melnick et al., 2020) and increased expectations
(Ferket, 2020). The decreased dependence of SoQal on an expert, at appropriate
times, could prevent the disruption of clinical workflows and thus put patient
care at centre stage. On the other hand, inappropriate independence of an
algorithm from an expert can lead to incorrect learning signals that are
reinforced throughout the algorithm’s lifetime. Consequently, a high degree of
misdiagnoses can occur, thus negatively impacting clinical decision making and
patient outcomes. SoQal attempts to balance this independence against
diagnostic accuracy.

As for low-resource clinical settings where physicians are either ill-trained
or unavailable and labelled data is scarce, SoQal offers a scalable approach
for operating in such environments. It labels data iteratively without over-
dependence on potentially unreliable ‘experts’. This, in turn, generates
large, labelled datasets that can be successfully leveraged by data-hungry
deep learning algorithms.

## References

  * Poplin et al. (2018) Ryan Poplin, Avinash V Varadarajan, Katy Blumer, Yun Liu, Michael V McConnell, Greg S Corrado, Lily Peng, and Dale R Webster.  Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning.  _Nature Biomedical Engineering_ , 2(3):158, 2018\.
  * Tomašev et al. (2019) Nenad Tomašev, Xavier Glorot, Jack W Rae, Michal Zielinski, Harry Askham, Andre Saraiva, Anne Mottram, Clemens Meyer, Suman Ravuri, Ivan Protsyuk, et al.  A clinically applicable approach to continuous prediction of future acute kidney injury.  _Nature_ , 572(7767):116–119, 2019.
  * Attia et al. (2019) Zachi I Attia, Suraj Kapa, Francisco Lopez-Jimenez, Paul M McKie, Dorothy J Ladewig, Gaurav Satam, Patricia A Pellikka, Maurice Enriquez-Sarano, Peter A Noseworthy, Thomas M Munger, et al.  Screening for cardiac contractile dysfunction using an artificial intelligence–enabled electrocardiogram.  _Nature Medicine_ , 25(1):70–74, 2019.
  * Settles (2009) Burr Settles.  Active learning literature survey.  Technical report, University of Wisconsin-Madison, Department of Computer Sciences, 2009.
  * West et al. (2016) Colin P West, Liselotte N Dyrbye, Patricia J Erwin, and Tait D Shanafelt.  Interventions to prevent and reduce physician burnout: a systematic review and meta-analysis.  _The Lancet_ , 388(10057):2272–2281, 2016.
  * Shanafelt et al. (2017) Tait D Shanafelt, Lotte N Dyrbye, and Colin P West.  Addressing physician burnout: the way forward.  _Jama_ , 317(9):901–902, 2017.
  * Kiyasseh et al. (2020) Dani Kiyasseh, Tingting Zhu, and David A. Clifton.  ALPS: Active Learning via Perturbations.  _arXiv preprint arXiv:2004.09557_ , 2020.
  * Chow (1970) C Chow.  On optimum recognition error and reject tradeoff.  _IEEE Transactions on Information Theory_ , 16(1):41–46, 1970.
  * El-Yaniv and Wiener (2010) Ran El-Yaniv and Yair Wiener.  On the foundations of noise-free selective classification.  _Journal of Machine Learning Research_ , 11(May):1605–1641, 2010.
  * Gong et al. (2019) Wenbo Gong, Sebastian Tschiatschek, Richard Turner, Sebastian Nowozin, and José Miguel Hernández-Lobato.  Icebreaker: element-wise active information acquisition with bayesian deep latent gaussian model.  _arXiv preprint arXiv:1908.04537_ , 2019.
  * Smailagic et al. (2018) Asim Smailagic, Pedro Costa, Hae Young Noh, Devesh Walawalkar, Kartik Khandelwal, Adrian Galdran, Mostafa Mirshekari, Jonathon Fagert, Susu Xu, Pei Zhang, et al.  Medal: Accurate and robust deep active learning for medical image analysis.  In _IEEE International Conference on Machine Learning and Applications_ , pages 481–488, 2018.
  * Smailagic et al. (2019) Asim Smailagic, Pedro Costa, Alex Gaudio, Kartik Khandelwal, Mostafa Mirshekari, Jonathon Fagert, Devesh Walawalkar, Susu Xu, Adrian Galdran, Pei Zhang, et al.  O-medal: Online active deep learning for medical image analysis.  _arXiv preprint arXiv:1908.10508_ , 2019.
  * Wang et al. (2019) Guijin Wang, Chenshuang Zhang, Yongpan Liu, Huazhong Yang, Dapeng Fu, Haiqing Wang, and Ping Zhang.  A global and updatable ecg beat classification system based on recurrent neural networks and active learning.  _Information Sciences_ , 501:523–542, 2019.
  * Gal et al. (2017) Yarin Gal, Riashat Islam, and Zoubin Ghahramani.  Deep Bayesian active learning with image data.  In _Proceedings of the 34th International Conference on Machine Learning-Volume 70_ , pages 1183–1192. JMLR. org, 2017.
  * Houlsby et al. (2011) Neil Houlsby, Ferenc Huszár, Zoubin Ghahramani, and Máté Lengyel.  Bayesian active learning for classification and preference learning.  _arXiv preprint arXiv:1112.5745_ , 2011.
  * Dekel et al. (2012) Ofer Dekel, Claudio Gentile, and Karthik Sridharan.  Selective sampling and active learning from single and multiple teachers.  _Journal of Machine Learning Research_ , 13(Sep):2655–2697, 2012.
  * Zhang and Chaudhuri (2015) Chicheng Zhang and Kamalika Chaudhuri.  Active learning from weak and strong labelers.  In _Advances in Neural Information Processing Systems_ , pages 703–711, 2015.
  * Sinha et al. (2019) Samarth Sinha, Sayna Ebrahimi, and Trevor Darrell.  Variational adversarial active learning.  In _Proceedings of the IEEE International Conference on Computer Vision_ , pages 5972–5981, 2019.
  * Urner et al. (2012) Ruth Urner, Shai Ben David, and Ohad Shamir.  Learning from weak teachers.  In _Artificial intelligence and statistics_ , pages 1252–1260, 2012\.
  * Yan et al. (2016) Songbai Yan, Kamalika Chaudhuri, and Tara Javidi.  Active learning from imperfect labelers.  In _Advances in Neural Information Processing Systems_ , pages 2128–2136, 2016.
  * Wiener and El-Yaniv (2011) Yair Wiener and Ran El-Yaniv.  Agnostic selective classification.  In _Advances in Neural Information Processing Systems_ , pages 1665–1673, 2011.
  * Cortes et al. (2016) Corinna Cortes, Giulia DeSalvo, and Mehryar Mohri.  Learning with rejection.  In _International Conference on Algorithmic Learning Theory_ , pages 67–82. Springer, 2016.
  * Geifman and El-Yaniv (2017) Yonatan Geifman and Ran El-Yaniv.  Selective classification for deep neural networks.  In _Advances in neural information processing systems_ , pages 4878–4887, 2017.
  * Liu et al. (2019) Ziyin Liu, Zhikang Wang, Paul Pu Liang, Russ R Salakhutdinov, Louis-Philippe Morency, and Masahito Ueda.  Deep gamblers: Learning to abstain with portfolio theory.  In _Advances in Neural Information Processing Systems_ , pages 10622–10632, 2019.
  * Geifman and El-Yaniv (2019) Yonatan Geifman and Ran El-Yaniv.  Selectivenet: A deep neural network with an integrated reject option.  _arXiv preprint arXiv:1901.09192_ , 2019.
  * Gal and Ghahramani (2016) Yarin Gal and Zoubin Ghahramani.  Dropout as a bayesian approximation: Representing model uncertainty in deep learning.  In _International Conference on Machine Learning_ , pages 1050–1059, 2016.
  * Paszke et al. (2019) Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al.  Pytorch: An imperative style, high-performance deep learning library.  In _Advances in Neural Information Processing Systems_ , pages 8024–8035, 2019.
  * Clifford et al. (2015) Gari D Clifford, Ikaro Silva, Benjamin Moody, Qiao Li, Danesh Kella, Abdullah Shahin, Tristan Kooistra, Diane Perry, and Roger G Mark.  The physionet/computing in cardiology challenge 2015: reducing false arrhythmia alarms in the icu.  In _2015 Computing in Cardiology Conference_ , pages 273–276, 2015\.
  * Clifford et al. (2017) Gari D Clifford, Chengyu Liu, Benjamin Moody, H Lehman Li-wei, Ikaro Silva, Qiao Li, AE Johnson, and Roger G Mark.  Af classification from a short single lead ECG recording: the physionet/computing in cardiology challenge 2017.  In _2017 Computing in Cardiology_ , pages 1–4, 2017.
  * Hannun et al. (2019) Awni Y Hannun, Pranav Rajpurkar, Masoumeh Haghpanahi, Geoffrey H Tison, Codie Bourn, Mintu P Turakhia, and Andrew Y Ng.  Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network.  _Nature Medicine_ , 25(1):65, 2019.
  * Bousseljot et al. (1995) R Bousseljot, D Kreiseler, and A Schnabel.  Nutzung der ekg-signaldatenbank cardiodat der ptb über das internet.  _Biomedizinische Technik/Biomedical Engineering_ , 40(s1):317–318, 1995.
  * Watkins (1989) Christopher John Cornish Hellaby Watkins.  Learning from delayed rewards.  1989\.
  * Konyushkova et al. (2017) Ksenia Konyushkova, Raphael Sznitman, and Pascal Fua.  Learning active learning from data.  In _Advances in Neural Information Processing Systems_ , pages 4225–4235, 2017.
  * Melnick et al. (2020) Edward R Melnick, Liselotte N Dyrbye, Christine A Sinsky, Mickey Trockel, Colin P West, Laurence Nedelec, Michael A Tutty, and Tait Shanafelt.  The association between perceived electronic health record usability and professional burnout among us physicians.  In _Mayo Clinic Proceedings_ , volume 95, pages 476–487. Elsevier, 2020.
  * Ferket (2020) Kathleen Ferket.  Burnout in nurses across practice domains: Implications and correlations to physician burnout.  _The Resilient Healthcare Organization: How to Reduce Physician and Healthcare Worker Burnout_ , 2020.


# Early Evaluation of Intel Optane Non-Volatile Memory with HPC I/O
Workloads††thanks: notes on title

Kai Wu<EMAIL_ADDRESS>Frank Ober†<EMAIL_ADDRESS>

Shari Hamlin†<EMAIL_ADDRESS>Qiang Guan⋆<EMAIL_ADDRESS>Dong Li
<EMAIL_ADDRESS>

University of California, Merced Intel Corp., USA† Los Alamos National Lab⋆

###### Abstract

High performance computing (HPC) applications have a high requirement on
storage speed and capacity. Non-volatile memory is a promising technology to
replace traditional storage devices to improve HPC performance. Earlier in
2017, Intel and Micron released first NVM product – Intel Optane SSDs. Optane
is much faster and more durable than the traditional storage device. It
creates a bridge to narrow the performance gap between DRAM and storage. But
is the existing HPC I/O stack still suitable for new NVM devices like Intel
Optane? How does HPC I/O workload perform with Intel Optane?

In this paper, we analyze the performance of I/O intensive HPC applications
with Optane as a block device and try to answer the above questions. We study
the performance from three perspectives: (1) basic read and write bandwidth of
Optane, (2) a performance comparison study between Optane and HDD, including
checkpoint workload, MPI individual I/O vs. POSIX I/O, and MPI individual I/O
vs. MPI collective I/O, and (3) the impact of Optane on the performance of a
parallel file system, PVFS2.

11footnotetext: LA-UR-17-27890

##  I Introduction

The modern HPC applications can be very demanding for storage capacity and
performance. For example, Blue Brain project aims to simulate the human brain
with a daunting 100PB memory that needs to be revisited by the solver at every
time step; The cosmology simulation studying Q continuum works on 2PM per
simulation. To meet storage and performance requirement of those HPC
applications, non-violate memory (NVM) technology is a promising building
block for future HPC.

According to some recent literatures [1], the emerging NVM techniques, such as
phase change memory (PCM) [2] and STT-RAM [3], have shown many interesting
performance characteristics. First, the performance of NVM is much better than
that of a hard drive, and even closer to that of DRAM. Second, NVM has better
density than DRAM while remaining non-volatile. Third, NVM has byte
addressability which can largely improve load/store efficiency. Table I
summarizes the features of different NVM techniques and compares them to the
traditional DRAM and storage technologies.

TABLE I: Memory Technology Summary [1] |  Read time (ns) |  Write time (ns) |  Read BW (MB/s) |  Write BW (MB/s)
---|---|---|---|---
DRAM |  10 |  10 |  1,000 |  900
PCRAM |  20-200 |  80-$10^{4}$ |  200-800 |  100-800
SLC Flash |  $10^{4}$-$10^{5}$ |  $10^{4}$-$10^{7}$ |  0.1 |  $10^{-3}$-$10^{-1}$
ReRAM |  5-$10^{5}$ |  5-$10^{8}$ |  1-1000 |  0.1-1000
Hard drive |  $10^{6}$ |  $10^{6}$ |  50-120 |  50-120

Earlier in 2017, Intel and Micron released their joint NVM product Intel
Optane SSD. Optane is based on 3D Xpoint technology. The 3D XPoint is 1000x
faster and 1000x more endurable than NAND flash memory. To leverage Optane for
future HPC, we must understand the performance impact of Optane on HPC
applications. In this paper, we study the Optane performance with HPC
applications under various test environments. This is the first work to
evaluate performance of Intel Optane in the HPC field. In this paper, we
conduct the following performance study. We first test Optane with Intel Open
Storage Toolkit. We get preliminary understanding on this new NVM device’s
performance (particular read/write latency and bandwidth), and compare it with
the performance of a traditional hard drive. Then, we analyze the performance
of some I/O intensive HPC applications with Optane as a high-speed block
device. We particularly compare the performance of MPI I/O and POSIX I/O based
on Opetane. Furthermore, we compare the performance of two different MPI I/O
techniques, MPI individual I/O and collective I/O. Collective I/O is an
optimization technique based on the assumption of poor I/O performance. We
want to know that whether it is still necessary to use MPI collective I/O for
Optane. Lastly, we investigate MPI I/O performance on Optane with a a parallel
file system, PVFS. PVFS is widely used in HPC. How does Optane device affect
the performance of a parallel file system? We study the performance with
different configurations of PVFS.

The rest of the paper is organized as follows. In Section 2 we present
background information for this paper. Section 3 describes our test
environment and benchmarks we use for evaluation. A detailed evaluation of
Intel Optane on HPC I/O workloads is shown in Section 4. Finally, in Section 5
we conclude this paper and discuss our future work.

##  II Background

###  II-A Non-violate memory usage model

There are two major approaches to integrate NVM into the system. One approach
is based on a memory-based model, and the other is based on a storage-based
model. Figure 1 depicts the two approaches.

Figure 1: Two types of NVM usage model

(1) Storage-based Model. Similar to traditional HDD and SSD, NVM is used as a
block device in this usage model. The data path 1 in Figure 1 depicts such
usage model. All operations including read and write are managed by I/O
controller. Users can use regular block I/O interface, such as PCI-e or SATA,
to access such storage-based NVM. However, using the storage-based model
cannot easily achieve byte-addressability offered by NVM. Optane offers us an
opportunity to use NVM based on the storage-based model.

(2) Memory-based Model. In this model, NVM is attached to a high-speed memory
bus as DIMMs. All load and store instructions are operated by the memory
controller. The data path 2 in Figure 1 depicts such usage model.

###  II-B Intel Optane Memory

Optane Memory [4, 5] is a 3D XPoint non-volatile memory jointly developed by
Intel and Micron. The 3D XPoint technology is a new class of non-volatile
memory that provides unprecedented performance. It can provide up to 1,000
times lower latency at the media level and many times greater endurance than
NAND. The 3D XPoint technology can deliver game-changing performance for data-
intensive applications. Its ability to enable high-speed, high-capacity data
storage creates new possibilities for system architects to enable entirely new
applications.

##  III EXPERIMENTAL METHODOLOGY

###  III-A Benchmarks

In our evaluation tests, we use four benchmarks, which are Intel Open Storage
Toolkit, HACC-IO, IOR, and mpi-io-test. The first three benchmarks are tested
under a network file system (NFS) environment, and the fourth one is tested in
a Parallel Virtual File System (PVFS) environment.

(1) Intel® Open Storage Toolkit. This toolkit is used to measure I/O
bandwidth, latency and IOPS of block device for different I/O workloads. The
Intel Open Storage Toolkit supports various user configurable options, such as
read/write ratio, random/sequential ratio, and request size.

(2) HACC-IO Benchmark. This benchmark is the I/O kernel of HACC (an HPC
application based on N-body simulation). It has random I/O write operations
with all-to-all communication patterns.

(3) IOR Benchmark. IOR is a popular benchmark to study parallel I/O. It
leverages the scalability of MPI to easily and accurately calculate the
aggregate bandwidth of a user-defined number of client machines. IOR supports
POSIX, MPI-IO (including individual I/O and collective I/O), and HDF5 I/O
interfaces. Using IOR, the user can easily configure different I/O patterns,
including “sequential” and “random offset” file access patterns. We use the
sequential pattern in our tests.

(4) MPI-IO-test Benchmark. This benchmark is shipped with PVFS2. It is used to
test MPI I/O and functionality implemented by ROMIO [6]. mpi-io-test can
produce aggregate bandwidth numbers by varying block sizes, number of
processors, number of iterations and MPI I/O methods (independent I/O or
collective I/O).

###  III-B PVFS

PVFS can provide high performance access in a large-scale cluster computing
environment. It is modularity, flexibility and with ROMIO support. In a PVFS
cluster architecture, each node is used as one or more of: compute node, I/O
node and metadata node. I/O nodes hold file data; metadata nodes hold metadata
include stat-info, attributes, and datafile-handles as well as directory-
entries; compute nodes run applications that utilize the file system by
sending requests to the servers over the network. We use PVFS2 [7], the second
version of PVFS, for our tests.

###  III-C Evaluation Environment

We have two experimental environment setups: one is with NFS, and the other is
with PVFS2. The NFS environment has four nodes and each node has an Intel
E52690v4 processor, 128 GB memory, 1 TB regular hard drive (attached by SATA),
and 370 GB Optane SSD (attached by M.2 PCIe). The PVFS environment has six
nodes which have the same hardware configuration as the four nodes in the NFS
environment. All nodes are provided by Intel, and we only compare the
performance of Optane device and a regular hard drive (noted as ”HDD“ in this
paper). We cannot compare the performance of Optane and SSD, because there is
no SSD device on those nodes. All of the nodes use Linux kernel 3.10. We use
MPICH-2.6 for MPI. We use 10GB Ethernet in our test environment.

##  IV PERFORMANCE STUDY

###  IV-A Basic Performance

We measure the raw bandwidth of Optann and HDD with Intel Open Storage Toolkit
on a single node. Table II shows the results. The table reveals that both read
and write bandwidths of Optane device are much higher than those of HDD. For
example, Optane achieves 357% and 847% of HDD bandwidth for random write and
random read, respectively. We also notice that Optane has no big performance
difference between random and sequential read/write. This is quite different
from HDD that has big performance difference between random and sequential
read/write.

TABLE II: Read and Write Performance (MB/s) on raw block device |  Write Random Bandwidth |  Write Sequential Bandwidth |  Read Random Bandwidth |  Read Sequential Bandwidth
---|---|---|---|---
Optane |  2174.08 |  2172.62 |  2286.15 |  2568.53
HDD |  6.08 |  200.25 |  2.7 |  204.30

###  IV-B Checkpoint Workload Performance

Checkpoint is a common fault tolerance mechanism in current production
supercomputers and industrial data centers. In order to evaluate checkpoint
performance on Optane, we use HACC-IO benchmark. In our tests, we use three
nodes as compute node and one node as storage node. The compute nodes access
storage nodes through NFS. We change the number of MPI tasks per compute node
in our tests. Figure 2 shows the results.

We first notice that running HACC, the performance of Optane device is still
much better than that of HDD, but the performance difference between Optane
and HDD is smaller than the bandwidth difference we get in the basic
performance evaluation (Section IV-A). The maximum performance difference
between Optane and HDD when running HACC is 560%, while the bandwidth
difference between Optane and HDD in Section IV-A is at least 1000%. We
attribute such smaller performance difference to the network effect when doing
remote I/O with NFS. The network latency is typically longer than I/O access
latency in our test platforms. Hence the network becomes a performance
bottleneck, overshadowing the performance benefit provided by Optane.

Another interesting result from Figure 2 is that the Optane performance seems
not very scalable in our tests. Ideally, the maximum bandwidth per MPI task
shown in Figure 2 should remain stable as we increase the number of MPI tasks
per node. However, we see slightly decrease of the bandwidth per MPI task when
the number of MPI tasks is larger than four.

Figure 2: Performance evaluation with HACC-IO Benchmark

###  IV-C POSIX I/O and MPI Individual I/O

POSIX I/O is widely used in HPC, and MPI I/O adds another layer on top of
POSIX I/O to improve the performance of parallel I/O. The functionality of MPI
I/O includes data validation and data re-arrangement to achieve better I/O
performance. However, MPI I/O introduces overhead. In the traditional storage
system, such overhead is much smaller than the I/O performance benefit,
justifying the necessity of using MPI I/O. However, given the excellent
performance of Optane, the I/O performance benefit of MPI I/O could become
smaller. We expect that the overhead of MPI I/O will become more pronounced.

Figure 3: Performance evaluation with IOR benchmark (POSIX I/O vs. MPI
individual I/O)

In this test, we run IOR benchmark with three compute nodes and one storage
node, and evaluate the performance of POSIX I/O and MPI individual I/O. We use
NFS for remote storage access. The results are shown in Figure 3. We find that
when the number of MPI tasks per node is smaller (i.e., 1 or 4), the
performance difference between POSIX I/O and MPI individual I/O on Optane is
larger than that on HDD. This result is consistent with our expectation: the
overhead of MPI I/O is more pronounced on Optane. However, when the number of
MPI tasks per node is larger (i.e., 16), we see almost no performance
difference between POSIX I/O and MPI individual I/O. We attribute such smaller
performance difference to the increasing performance benefit of MPI individual
I/O when the number of MPI tasks per node is larger.

###  IV-D MPI Collective I/O and MPI Individual I/O

We compare performance of two MPI I/O methods, MPI collective I/O and MPI
individual I/O, with IOR benchmark. We run IOR benchmark with three compute
nodes and one storage node, and use NFS for remote storage access. MPI
collective I/O is an optimization of MPI individual I/O. MPI collective I/O
re-arranges data distribution between MPI tasks before I/O operations to
improve I/O performance. MPI collective I/O has two phases, i.e., data shuffle
and I/O operation. By re-arranging data in the data shuffle phase, MPI
collective I/O can combine multiple I/O operations, reduce the number of I/O
transactions, and avoid unnecessary data fetching. However, the data shuffle
brings overhead when re-arranging data between MPI tasks. In the traditional
storage system, such overhead can be easily overweighted by the performance
benefit because of the bad performance of the storage system. However, with
Optane, we expect that the data shuffle overhead cannot be easily overshadowed
by the performance benefit. Hence, MPI collective I/O may become an
unnecessary performance optimization.

Figure 4 shows the results. Surprisingly, we do not see any performance
benefit of MPI collective I/O in any test (neither Optane nor HDD), and MPI
collective I/O always performs worse than MPI individual I/O. We attribute
such result to the bad network performance in our test platform. The network
performance becomes a bottleneck in the data shuffle phase. No matter how much
I/O performance benefit MPI collective I/O can bring, it is always smaller
than the overhead of the data shuffle.

Figure 4: Performance evaluation with IOR benchmark (MPI individual I/O vs.
MPI collective I/O)

###  IV-E Performance Evaluation with PVFS

In this section, we change the configurations of PVFS, and then evaluate the
performance of Optane. To use PVFS, we need to have I/O node, metadata node,
and compute node. We change the numbers of those nodes in our evaluation. We
use mpi-io-test during the evaluation.

Figure 5: PVFS2 performance sensitivity to data block size on Optane

We first change the data block size and study the performance sensitivity of
Optane to the data block size. We use one compute node, three I/O nodes and
one metadata node for this test. We use 24 clients on the compute node. Figure
5 show the results. The results show that write bandwidth of HDD continues
growing when the block size increases. The similar story happens on Optane,
but there is an interesting performance drop as we increase the block size
from 120 KB to 2MB. We suspect such drop is because of the PVFS data alignment
problem across multiple I/O nodes, but this performance drop needs to be
further investigated.

Figure 6: PVFS2 performance sensitivity to the number of clients per compute
node

We further study the performance sensitivity of Optane to the number of
clients per compute node. We use two compute nodes, three I/O nodes and one
metadata node for this test. Figure 6 show the results. We notice that Optane
has pretty good scalability. In particular, as we increase the number of
clients, the write bandwidth keeps increasing. However, HDD does not have such
good scalability: the write bandwidth remains relatively stable.

Figure 7: PVFS2 performance sensitivity to the number of of I/O nodes

We then study the performance sensitivity of Optane to the number of I/O
nodes. We use one compute node and one metadata node for this test. We use 24
clients on the compute node. Figure 7 show the results. We notice that write
bandwidth of both Optane and HDD increases as we increase the number of I/O
nodes. However, their bandwidths increase at different rates. For Optane, the
bandwidth increases less than 15%, while for HDD, the bandwidth increases
2.6x. This indicates that for Optane with better I/O performance, the storage
bandwidth may not be the major performance bottleneck. This performance result
for Optane is different from HDD which suffers from the limited storage
bandwidth.

Figure 8: Comparing the performance of MPI individual I/O and MPI collective
I/O with PVFS2

Lastly, we compare the performance of MPI individual I/O and MPI collective
I/O on PVFS2. We use three I/O nodes, one metadata node and one compute node
(24 clients on the compute node) for this tests. In Section IV-D, we have
compared the performance of MPI individual I/O and MPI collective I/O in an
NFS-based test environment, and find that MPI individual I/O always performs
better than MPI collective I/O because of the bad network performance.
However, in the PVFS2-based test environment, we find a different story. MPI
Collective I/O performs better than MPI Independent I/O on both Optane and HDD
(see Figure 8). We reason that PVFS2 uses some performance optimization
techniques to reduce the negative impact of network communication. The
reduction of the impact of network communication reduces the overhead of the
data shuffle which makes the benefit of MPI collective I/O more pronounced.

##  V Related Work

###  V-A Non-Volatile Memory

Non-volatile memory (NVM) technology is under quick development and has
attracted a large body of research. A comprehensive survey about NVM can be
found in a prior study [1]. There are some earlier works which focus on the
architecture-level design issues of NVM [8, 9, 10, 11], such as wear-leveling,
read-write disparity issues, etc. Some of those studies [12, 13, 14, 15] use
NVM as a replacement of DRAM at the architecture level. Many studies also
consider NVM as a storage device, such as Onyx [16], Moneta [17], and PMBD
[18]. Liu et al. [19] investigate the current IO mechanisms on NVM but do not
consider the effect of the file system. The Intel Optane product also uses NVM
as a storage device.

Some prior studies have explored file systems for NVM. For example, BPFS [20]
uses shadow paging techniques for fast and reliable updates to critical file
system metadata structures. SCMFS [21] adopts a scheme similar to page tables
in memory management for file management in NVM. PMFS [22] allows using memory
mapping (mmap) for directly accessing NVM space and avoids redundant data
copies. Some programing models for NVM are developed to take the benefit of
byte-addressability and persistence. For example, Mnemosyne [23] gives a
simple programming interface for NVM, such as the interface to declare non-
volatile data objects. CDDS [24] attempts to provide consistent and durable
data structures. NV-Heaps [25] gives a simple model with support of
transactional semantics. SoftPM [26] offers a memory abstraction similar to
`malloc` for allocating objects in NVM.

In this study, we use Optane as a storage device and deploy both conventional
file system and parallel file system atop for HPC applications. According to
our experimental results, we find that the high-speed Intel Optane could
significantly improve HPC application performance on both regular NFS and
PVFS. However, the end-to-end effect is workload dependent and related to a
variety of factors in the I/O stack.

###  V-B PVFS

There are also some prior works studies the PVFS performance. For example,
[27] analyze the performance of PVFS2 under different hardware deployment and
explores the performance bottleneck; [28] tries to leverage additional local
storage to boost the local data read performance for parallel file systems;
[29] improves performances of collective I/O of MPI-IO on PVFS by avoiding the
data-redistribution.

##  VI Conclusions

We study the performance impact of the recent Intel Optane with HPC I/O
intensive applications. Given the distinct performance characteristics of
Optane, some of existing I/O stacks must be re-examined to optimize
performance. According to our experimental results, we find Optane device has
very high bandwidth but needs improve scalability under certain conditions;
Collective I/O optimization does not always work. In the future, we may build
a model to choose appropriate I/O method based on application workload
characteristics and storage performance. Our work lays some foundation for the
deployment of NVM products in the future HPC.

## References

  * [1] K. Suzuki and S. Swanson, “The Non-Volatile Memory Technology Database (NVMDB),” Department of Computer Science & Engineering, University of California, San Diego, Tech. Rep. CS2015-1011, 2015, http://nvmdb.ucsd.edu.
  * [2] J. Y. Y. Z. B. Z. E. I. O. M. D. B. Benjamin C. Lee, Ping Zhou, “Phase-Change Technology and the Future of Main Memory,” in _IEEE Micro, vol. 30, no. 1, pp. 143–143_ , 2010.
  * [3] e. a. Dieny, B., “Spin-dependent phenomena and their implementation in spintronic devices,” in _International Symposium on VLSI Technology, Systems and Applications_ , 2008.
  * [4] Intel, “Revolutionizing Memory and Storage,” https://www.intel.com/content/www/us/en/architecture-and-technology/intel-optane-technology.html.
  * [5] B. Tallis, “The Intel Optane Memory(SSD) Preview: 32GB of Kaby Lake Caching,” http://www.anandtech.com/show/11210/the-intel-optane-memory-ssd-review-32gb-of-kaby-lake-caching.
  * [6] R. L. R. L. B. G. P. A. C. g. Rajeev Thakur, Rob Ross, “ROMIO: A High-Performance, Portable MPI-IO Implementation,” http://www.mcs.anl.gov/projects/romio/.
  * [7] N. M. e. a. Latham, R., “A next-generation parallel file system for Linux cluster. ,” in _LinuxWorld Mag. 2.1_ , 2004.
  * [8] B. Lee, E. Ipek, O. Mutlu, and D. Burger, “Architecturing Phase Change Mmemorhy as a Scalable DRAM Architecture,” in _International Symposium on Computer Architecture_ , 2009.
  * [9] M. K. Qureshi, M. Franchescini, V. Srinivasan, L. Lastras, B. Abali, and J. Karidis, “Enhancing Lifetime and Security of PCM-Based Main Memory with Start-Gap Wear Leveling,” in _MICRO_ , 2009.
  * [10] M. K. Qureshi, V. Srinivasan, and J. A. Rivers, “Scalable High-Performance Main Memory System Using Phase-Change Memory Technology,” in _ISCA_ , 2009\.
  * [11] J. Y. Y. Z. Ping Zhou, Bo Zhao, “A durable and energy efficient main memory using phase change memory technology ,” in _Proceedings of the 36th annual international symposium on Computer architecture(ISCA’09)_ , 2009.
  * [12] S. Yang, K. Wu, Y. Qiao, D. Li, and J. Zhai, “Algorithm-directed crash consistence in non-volatile memory for hpc,” in _2017 IEEE International Conference on Cluster Computing (CLUSTER)_ , 2017.
  * [13] P. Wu, D. Li, Z. Chen, J. Vetter, and S. Mittal, “Algorithm-Directed Data Placement in Explicitly Managed No-Volatile Memory,” in _ACM Symposium on High-Performance Parallel and Distributed Computing (HPDC)_ , 2016.
  * [14] S. R. Dulloor, A. Roy, Z. Zhao, N. Sundaram, N. Satish, R. Sankaran, J. Jackson, and K. Schwan, “Data Tiering in Heterogeneous Memory Systems,” in _Proceedings of the Eleventh European Conference on Computer Systems (EuroSys)_ , 2016.
  * [15] K. Wu, Y. Huang, and D. Li, “Unimem: Runtime data managementon non-volatile memory-based heterogeneous main memory,” in _Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis_ , ser. SC ’17, 2017.
  * [16] T. I. M. R. K. G. S. S. Ameen Akel, Adrian M. Caulfield, “Onyx: A Prototype Phase Change Memory Storage Array,” in _Proceedings of the 3rd USENIX Workshop on Hot Topics in Storage and File Systems (HotStorage 2011)_ , 2011.
  * [17] J. C. T. I. M. R. K. G. S. S. Adrian M. Caulfield, Arup De, “Moneta: A High-performance Storage Array Architecture for Next-generation, Non-volatile Memories,” in _Proceedings of the 43rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO 2010)_ , 2010.
  * [18] F. Chen, M. P. Mesni, and S. Hahn, “A protected block device for persistent memory,” in _IEEE Symposium on Mass Storage Systems and Technologies (MSST)_ , 2014.
  * [19] W. Liu, K. Wu, J. Liu, F. Chen, and D. Li, “Performance evaluation and modeling of hpc i/o on non-volatile memory,” in _2017 International Conference on Networking, Architecture, and Storage (NAS)_ , 2017.
  * [20] J. Condit, E. B. Nightingale, C. Frost, E. Ipek, B. Lee, D. Burger, and D. Coetzee, “Better I/O Through Byte-addressable, Persistent Memory,” in _Symposium on Operating Systems Principles (SOSP)_ , 2009.
  * [21] X. Wu and A. L. N. Reddy, “SCMFS: A File System for Storage Class Memory,” in _International Conference for High Performance Computing, Networking, Storage and Analysis (SC)_ , 2011.
  * [22] Intel, “Persistent Memory File System,” https://github.com/linux.pmfs/pmfs.
  * [23] H. Volos, A. J. Tack, and M. M. Swift, “Mnemosyne: Lightweight Persistent Memory,” in _Architectural Support for Programming Languages and Operating Systems (ASPLOS)_ , 2011.
  * [24] S. Venkataraman, N. Tolia, P. Ranganathan, and R. H. Campbell, “Consistent and Durable Data Structures for Non-volatile Byte-addressable Memory,” in _FAST_ , 2011.
  * [25] J. Coburn, A. Caulfield, A. Akel, L. Grupp, R. Gupta, R. Jhala, and S. Swanson, “NV-heaps: Making Persistent Objects Fast and Safe with Next-generation, Non-volatile Memories,” in _Architectural Support for Programming Languages and Operating Systems (ASPLOS)_ , 2011.
  * [26] D. C. C. C. R. R. J. W. Jorge Guerra, Leonardo M´armol, “Software Persistent Memory,” in _Proceedings of the 2012 USENIX Annual Technical Conference_ , 2012.
  * [27] T. L. Julian M. Kunkel, “Performance Evaluation of the PVFS2 Architecture,” in _15th EUROMICRO International Conference on Parallel, Distributed and Network-Based Processing(PDP’07)_ , 2007.
  * [28] R. R. Peng Gu, Jun Wang, “Bridging the Gap Between Parallel File Systems and Local File Systems: A Case Study with PVFS,” in _37th International Conference on Parallel Processing, Portland, OR_ , 2008.
  * [29] G. U. Jonathan Ilroy, Cyril Randriamaro, “Improving MPI-I/O Performance on PVFS ,” in _7th International Euro-Par Conference(Euro-Par ’01)_ , 2001\.