# Inferring pointwise diffusion properties of single trajectories with deep
learning

Borja Requena  ICFO – Institut de Ciències Fotòniques, The Barcelona Institute
of Science and Technology, Av. Carl Friedrich Gauss 3, 08860 Castelldefels
(Barcelona), Spain  Sergi Masó  Facultat de Ciències, Tecnologia i
Enginyeries, Universitat de Vic – Universitat Central de Catalunya (UVic-UCC),
C. de la Laura,13, 08500 Vic, Spain  Joan Bertran  Facultat de Ciències,
Tecnologia i Enginyeries, Universitat de Vic – Universitat Central de
Catalunya (UVic-UCC), C. de la Laura,13, 08500 Vic, Spain  Maciej Lewenstein
ICFO – Institut de Ciències Fotòniques, The Barcelona Institute of Science and
Technology, Av. Carl Friedrich Gauss 3, 08860 Castelldefels (Barcelona), Spain
ICREA, Pg. Lluís Companys 23, 08010 Barcelona, Spain  Carlo Manzo  Facultat de
Ciències, Tecnologia i Enginyeries, Universitat de Vic – Universitat Central
de Catalunya (UVic-UCC), C. de la Laura,13, 08500 Vic, Spain  Gorka Muñoz-Gil
<EMAIL_ADDRESS>Institute for Theoretical Physics, University of
Innsbruck, Technikerstr. 21a, A-6020 Innsbruck, Austria

###### Abstract

In order to characterize the mechanisms governing the diffusion of particles
in biological scenarios, it is essential to accurately determine their
diffusive properties. To do so, we propose a machine learning method to
characterize diffusion processes with time-dependent properties at the
experimental time resolution. Our approach operates at the single-trajectory
level predicting the properties of interest, such as the diffusion coefficient
or the anomalous diffusion exponent, at every time step of the trajectory. In
this way, changes in the diffusive properties occurring along the trajectory
emerge naturally in the prediction, and thus allow the characterization
without any prior knowledge or assumption about the system. We first benchmark
the method on synthetic trajectories simulated under several conditions. We
show that our approach can successfully characterize both abrupt and
continuous changes in the diffusion coefficient or the anomalous diffusion
exponent. Finally, we leverage the method to analyze experiments of single-
molecule diffusion of two membrane proteins in living cells: the pathogen-
recognition receptor DC-SIGN and the integrin $\alpha 5\beta 1$. The analysis
allows us to characterize physical parameters and diffusive states with
unprecedented accuracy, shedding new light on the underlying mechanisms.

##  I Introduction

Advances in optical imaging have made it possible to observe single molecules
in living biological systems [1]. When combined with particle tracking
algorithms, these techniques allow tracing the movement of individual
molecules, viruses, and organelles with nanometric precision, enabling the
study of transport mechanisms in complex biological environments. Through the
biophysical characterization of trajectories, we can extract meaningful
parameters to describe physical and biological processes. However, accurately
quantifying the trajectories remains a challenging task due to the stochastic
nature of these processes and to experimental drawbacks, such as imaging noise
and the emitter’s photophysics [2].

In cellular systems, a widespread diffusion feature is the occurrence of time-
dependent changes of motion [3]. Typically, these changes are associated with
transient interactions with other components [4, 5, 6] resulting in the sudden
variation of a parameter, e.g., the diffusion coefficient, which can switch
between a discrete [7], or continuous set of levels [8, 9, 10]. Furthermore,
they can induce smooth changes such as those associated with the
spatiotemporal heterogeneity of the environment [11]. Examples of trajectories
undergoing this kind of diffusion are schematically depicted in Fig. 1A.

Figure 1: Heterogeneous trajectories and the STEP pipeline. (A) Examples of
trajectories and the corresponding diffusion coefficient $D$ as a function of
time for: constant $D$; changes within a discrete set of states with fixed
$D$; continuous and monotonous change of $D$; switch between random $D$s. (B)
Schematic of the pipeline of STEP: an input trajectory is fed to the
architecture, which consists of a stack of CNN, a transformer encoder, and a
pointwise feedforward layer. The model’s output is the pointwise prediction of
the diffusion parameter of interest (in this case $D$).

Trajectories with time-dependent diffusion properties pose an additional
challenge to characterize the motion of individual particles, which has been
tackled with different approaches. For trajectories displaying abrupt changes,
segmentation methods [12, 13, 14, 3] are a valuable strategy but cannot deal
with long-range correlations and often offer limited time resolution due to
temporal averaging. On the other hand, model-dependent methods such as the
hidden Markov model (HMM) have been quite successful in describing
heterogeneous diffusion [15, 16, 17], although they require prior knowledge
about the diffusive states involved and their kinetic scheme. Recently, data-
driven approaches have shown remarkable capabilities to extract information
from individual stochastic trajectories, even in the presence of changes of
diffusion properties [18, 19, 20].

In this work, we propose STEP, a method based on state-of-the-art deep-
learning architectures to extract pointwise diffusion features from individual
trajectories without any prior information (see Fig. 1B). STEP features the
most recent advances in sequence-to-sequence learning [21], which have shown
impressive results in natural language processing tasks and beyond [22, 23,
24]. It combines convolutional [25] and attention layers [26] to cope with the
presence of short and long-range correlations, providing remarkable
performance over trajectories of any length and in the presence of noise.

The article is structured as follows: first, we show the ability of STEP to
predict diffusion properties, such as the diffusion coefficient and the
anomalous diffusion exponent, on simulated data reproducing experimentally
relevant scenarios. Then, we analyze simulated trajectories with smoothly
varying diffusion coefficients, showing that STEP correctly finds the expected
scaling. Finally, we use STEP to study two experimental data sets obtained by
the tracking of single-molecule live-cell imaging experiments and reporting
the motion of two membrane receptors: _i_) the pathogen-recognition receptor
DC-SIGN, which has been associated with random changes of diffusion
coefficients [10]; and _ii_) the $\alpha 5\beta 1$ integrin, expected to be
transiently arrested by binding to the cytoskeleton and the extracellular
matrix [27].

##  II Results

### The STEP architecture

Recently, we have witnessed an enormous effort in the development of deep-
learning approaches to study diffusive processes [18]. Previous works usually
focused on characterizing diffusive properties of single trajectories, i. e.,
predicting an overall or average diffusive parameter for each input trajectory
[28, 29, 30, 31, 32]. Recent works have proven the suitability of this
approach to study complex phenomena in different experimental scenarios [33,
34]. With STEP, we propose a sequence-to-sequence approach that translates
position coordinates into diffusion properties providing their pointwise
prediction at every time step of the input trajectory [20]. STEP translates an
input trajectory of arbitrary length $T$ into a sequence of $T$ elements
containing the diffusive properties of interest, as illustrated in Fig. 1B.
While it is effectively impossible to characterize diffusion from a single
displacement, STEP uses the whole trajectory as context to perform the
prediction at every point.

This approach allows us to study trajectories where diffusion properties can
vary over time with different patterns: from trajectories with constant
diffusive properties to trajectories that switch between discrete diffusive
states or where diffusive parameters change continuously over time (see Fig.
1A for examples). Unlike previous works, where expert input is needed in order
to choose an appropriate method, STEP can be seamlessly applied to any
diffusive data. Importantly, it does not rely on prior assumptions, such as
the number of changepoints [35, 36] or the properties of the expected
diffusive states [37].

State-of-the-art architectures for diffusion characterization rely on very
different approaches [38, 39, 40]. Several models obtain outstanding
performance by combining recurrent neural networks (RNNs) that account for
long-range correlations [30, 36] with convolutional neural networks (CNNs)
focusing on local features [41, 34, 42]. In STEP, we use a similar scheme but
replace the RNN with a transformer encoder [26], the cutting-edge architecture
for sequence modeling, and we stack XResNet blocks [43] to build the
convolutional part (see Methods). Local features are extracted through a CNN
and fed to the transformer, thus processing local and long-range features
simultaneously (see Fig. 1B). Finally, we use a pointwise fully-connected
layer of non-linear neurons to obtain the appropriate output dimension. We
provide a detailed description of the method in Section B.1.

Figure 2:  Time-dependent diffusion properties prediction. (A) 2D histogram of
the predicted diffusion coefficient $D$ compared to the ground truth. The mean
relative error (MRE) over the whole test set is 0.226. (B) Relative error for
the prediction of $D$ as a function of the segment length. (C) 2D histogram of
the predicted anomalous diffusion exponent $\alpha$ of FBM segments compared
to the ground truth. The MAE over the test set with all the diffusion models
is 0.271. (D) Mean absolute error (MAE) for the prediction of $\alpha$ as
function of the segment length. (E to H) Jaccard index for the changepoint
detection problem as a function of: (E) the ratio between consecutive segment
$D$s , (F) the changepoint position for $D$, (G) the difference between
consecutive segment $\alpha$s, and (H) the changepoint position for $\alpha$.

### Pointwise prediction of diffusion properties

We first validate STEP on the task of inferring the diffusion coefficient
pointwise from simulated trajectories reproducing transient Brownian motion
with abrupt changes of diffusion coefficient. The diffusion coefficient can
randomly vary in the range $D\in[10^{-3},10^{3}]$ and the dwell time in each
diffusion coefficient is drawn from an exponential distribution. Further
details of the simulations and the data sets are described in Section B.3. A
2D histogram of the ground truth versus the predicted diffusion coefficient
shows that STEP can precisely determine the diffusion coefficient on the whole
range of $D$ values included in the simulations (Fig. 2A) with an overall
relative error $|D_{\text{true}}-D_{\text{pred}}|/D_{\text{true}}=0.226$.

To further explore the performance of STEP, we calculate the relative error as
a function of the segment length (i.e. the dwell time for each diffusion
coefficient) that expectedly shows an improvement at longer length (blue line
in Fig. 2B). As a baseline to compare STEP’s precision in estimating the
diffusion coefficient, we use the linear fit of the time-averaged mean squared
displacement (TA-MSD) (see Appendix A). However, while STEP can obtain
pointwise predictions of $D$ from heterogeneous trajectories, the TA-MSD needs
trajectories of constant $D$ and was thus fed with pre-segmented trajectories.
In spite of this disadvantage, STEP provides better results (blue vs yellow
line in Fig. 2B). Notably, when STEP is provided with pre-segmented
trajectories, we observe a further improvement, with a nearly 2-fold reduction
of the error at short segment length (red line in Fig. 2B), demonstrating
outstanding prediction capabilities.

We then examine the ability of STEP to predict the anomalous diffusion
exponent $\alpha$. We consider heterogeneous trajectories simulated according
to different underlying models (further details are provided in Appendix A)
composed of segments with $\alpha$ suddenly changing in the range
$\alpha\in[0.05,2]$. The 2D histogram of the ground truth vs the predicted
$\alpha$ for trajectories undergoing fractional Brownian motion (FBM) (Fig.
2C) shows that STEP successfully predicts the anomalous diffusion exponent
with a mean absolute error $|\alpha_{\rm true}-\alpha_{\rm pred}|=0.271$,
obtaining results in line with the top-scoring approaches for this task [36,
41] in the Anomalous Diffusion challenge [18].

In Fig. 2D, we further report the mean absolute error for the inference of
$\alpha$ as a function of the segment length and in comparison to the results
obtained through the linear fit of the TA-MSD in logarithmic space and CONDOR
[35], the best-in-class approach in the corresponding task of the AnDi
Challenge [18]. As discussed earlier, both methods were provided with pre-
segmented trajectories, whereas STEP deals directly with heterogeneous
trajectories. STEP strongly outperforms the TA-MSD approach and shows
performance comparable to CONDOR. For this task, providing pre-segmented data
to STEP marginally improves its performance for long segments, whereas it even
reduced it for short ones. This result suggests that segment length is more
important than the exact knowledge of the segment edges and STEP effectively
combines local and global information.

In Appendix C, we extend the assessment of the performance of STEP as a
function of the localization precision and the dwell-time duration/number of
segments of the trajectories, showing that the method can be efficiently
applied in a wide range of experimental conditions. Additionally, in Appendix
D, we extend the analysis about anomalous diffusion showing the performance of
STEP as a function of the underlying diffusion models and the localization
precision.

### Detecting diffusive changepoints in heterogeneous trajectories

For trajectories undergoing sudden changes of diffusion properties, the exact
knowledge of the points at which these properties’ changes occur is crucial to
infer temporal properties and kinetic rates of the system and fully
characterizing the underlying physical process. While STEP does not explicitly
detect changepoints, its output provides a precise estimation of the diffusion
property which is supposed to change, hence simplifying the task of
changepoint detection and location with respect to the use of raw data. To
highlight this capability, we compare the results obtained by a state-of-the-
art kernel changepoint detection (KCPD) method [44, 45] when applied to STEP’s
predictions and to the timetrace of trajectory displacements. To assess the
performance of the method, we compute the Jaccard index (JI) considering as a
true positive any changepoint predictions lying within a threshold distance
$\mathcal{E}$ from the corresponding ground truth.

We first quantify the performance of the method to detect changes of the
diffusion coefficient in heterogeneous trajectories performing Brownian
motion. We use a benchmark data set with trajectories exhibiting a single
changepoint with $\mathcal{E}=5$. Through the application of the KCPD
algorithm on the prediction of STEP, we can successfully detect the
changepoints with high accuracy even for consecutive segments whose diffusion
coefficients are just one order of magnitude apart, as we show in Fig. 2E
(blue line). Furthermore, the method is robust with respect to the changepoint
position within the trace (Fig. 2F (blue line)). In contrast, when we apply
KCPD directly over the trajectory displacements (dashed purple lines) we
observe a decrease in performance over the whole range of diffusion
coefficient ratio. On average, STEP produces a 20% reduction in error reaching
an average JI of 0.833 compared to 0.796 obtained with the raw displacements.

We perform a similar analysis to detect changes in the anomalous diffusion
exponent in FBM trajectories. Since the anomalous diffusion exponent is an
asymptotic property and cannot be easily calculated from the raw data, to
build our baseline, we compute $\alpha$ with a linear fit of the TA-MSD on a
log-log scale using a sliding window of 30 time steps, which we then feed into
the KCPD algorithm (dashed yellow lines). Expectedly, the larger the
differences between segment parameters, the better we can detect the
changepoints, as we show in Fig. 2G. We obtain a 30% reduction of error by
using STEP with respect to the baseline method, achieving an average JI of
0.515 and 0.297, respectively, with $\mathcal{E}=20$. However, the plot of the
performance shows that finding changes in $\alpha$ is a harder task than
finding changes in $D$. Moreover, we also observe a performance drop when the
changepoints are near the trajectory edges (Fig. 2H). In these cases, we deal
with short segments whose anomalous diffusion exponent can be hard to
determine, as they rely on the arising of long-range correlations.

### Revealing continuous changes of diffusion properties

When considering heterogeneous trajectories in the biological context, the
typical behavior one expects is represented by particles undergoing diffusion
with piecewise constant properties that can suddenly change, e.g., as the
result of specific interactions with other biological components. However, the
presence of molecular crowding and gradients of concentration can produce a
continuous variation of diffusion properties over time. These changes might be
challenging to detect due to the limited spatiotemporal resolution of the
experiments or the lack of specific approaches for trajectory analysis. Since
STEP predicts pointwise diffusion properties in a model-free fashion, it
inherently features the capability to perform this kind of analysis, even
without dedicated training.

Figure 3: Continuous changes of diffusion properties. Predictions of the
diffusion coefficient of two sets of SBM trajectories with $\alpha=0.1$ (blue)
and $0.5$ (red). The thin continuous lines correspond to predictions for
exemplary single trajectories, bold continuous lines show the average over
3000 trajectories, and the dashed lines show the theoretically expected
scaling for every set. Figure 4: Switch between random diffusive states of the
pathogen-recognition receptor DC-SIGN. (A) Characteristic features of the ATTM
model: an exemplary trajectory undergoing changes of diffusion coefficient,
the distribution of diffusion coefficient, and the relation between the
diffusion coefficient and the dwell time. (B) Predictions of the diffusion
coefficient obtained by applying STEP to simulated ATTM trajectories (dots)
and the result of applying the changepoint analysis (black line). (C)
Distribution of $D$ obtained through the analysis described in (B), showing
the expected power law behavior at small $D$. (D) Relation between $D$ and the
dwell time $\tau$ obtained through the analysis described in (B), showing the
expected power law behavior. (E) Examples of experimental trajectories of DC-
SIGN with the corresponding predictions obtained for $D$ (dots) and the
changepoint analysis (black line). (F) Histogram of the distribution of $D$
obtained for the experimental trajectories. Inset: power-law fit at small $D$.
(G) 2D histogram of $D$ and $\alpha$ obtained for the experimental
trajectories.

To evaluate the performance of STEP on smoothly-varying trajectories, we rely
on simulations of scaled Brownian motion (SBM) [11]. SBM trajectories are
characterized by a time-dependent diffusion coefficient with a power-law
relationship $D(t)\sim t^{\alpha-1}$, where $\alpha$ is the anomalous
diffusion exponent. Further details about simulations are given in Section
B.3.

In Fig. 3, we show the predictions obtained for the diffusion coefficient at
every time step of trajectories with $\alpha=0.1$ and $0.5$. The shaded lines
represent the predictions obtained for individual trajectories which, despite
the fluctuations, already indicate the decreasing trend. Averaging over 3000
trajectories with the same $\alpha$ (bold lines) reveals the correct scaling
as compared with the expected power law (dashed lines). Furthermore, the
inference of $\alpha$ correctly provides a nearly constant value throughout
the trajectory, as expected (see Appendix D for additional details), and
points towards a low false positive rate with respect to changes in anomalous
diffusion.

### Characterizing anomalous diffusion from changes of normal diffusive
properties

To test the potential of STEP for the analysis of experimental trajectories,
we use it to study the motion of the pathogen-recognition receptor DC-SIGN
expressed in Chinese hamster ovarian cells [46]. Previous analysis of these
experiments revealed the occurrence of anomalous diffusion and weak ergodicity
breaking as a consequence of changes of diffusion coefficient [10]. This
behavior was described in the framework of the annealed transit-time model
(ATTM) [47], whose main features are schematically summarized in Fig. 4A.

In brief, ATTM depicts Brownian diffusion randomly switching diffusion
coefficient $D$, with values sampled from a distribution with a power-law
behavior $D^{\sigma-1}$ for small $D$ and a fast decay for
$D\rightarrow\infty$. The model further assumes a correlation between $D$ and
the dwell time $\tau$ of the form $\tau(D)\sim D^{-\gamma}$ to predict an
anomalous diffusion exponent $\alpha=\sigma/\gamma$. Therefore, the correct
characterization of both the distribution of $D$ and $\tau$ is crucial to
corroborate the compatibility with the underlying model. In the original work,
changes of diffusivity were detected through a changepoint analysis [48] but
the sensitivity and the time-resolution of the method did not allow a thorough
investigation of this behavior.

To demonstrate that STEP enables better characterization of these data, we
first use simulated ATTM trajectories. We segment the trajectories applying
the KCPD algorithm introduced in the previous sections over the STEP
predictions, as we show in Fig. 4B. Thus, we assign to each segment a single
$D$, taking the average segment prediction, and a $\tau$. We successfully
recover the power-law behavior of $D$ (Fig. 4C) and the power-law relationship
between $\tau$ and $D$ (Fig. 4D).

Then, we apply this approach to the DC-SIGN trajectories of Ref. [10]. The
results confirm the occurrence of diffusivity changes between segments of
nearly-constant diffusion coefficient and with variable duration, as we show
in Fig. 4E. Interestingly, our approach reveals twice as many changepoints as
the previous analysis.

Figure 5: Multi-state diffusion of the integrin $\bm{\alpha 5\beta 1}$. (A)
Examples of experimental trajectories of the integrin $\alpha$5$\beta$1 with
the corresponding predictions obtained for $D$ (dots) and the changepoint
analysis (black line). (B) 2D histogram of $D$ and $\alpha$ with the
respective marginal distributions. (C) Scatter plot of the predictions
obtained for $D$ and $\alpha$ at the segment level, color-coded according to a
clustering analysis performed with a _k_ -means algorithm. (D) Distribution of
the turning angle for the four clusters of segments obtained as in (C). (E)
Distribution of the confinement radius for the clusters showing restrained
diffusion.

The distribution of $D$ obtained for trajectory segments spans several orders
of magnitude, as we show in the histogram of Fig. 4F. For small $D$, it
displays a behavior compatible with a power-law with exponent $\sigma\approx
0.37$ over nearly three decades (inset of Fig. 4F), compatible with the ATTM.
Notably, this behavior could not be directly verified in the original article.
In principle, our method would allow us to verify the correlation between $D$
and dwell time, as we have shown in the simulations. However, this task is
limited by the variable trajectory length [49] and by the lack of statistics,
in particular for segments at small $D$.

As a further test, we predict the anomalous diffusion exponent with STEP. We
assign a single $\alpha$ by taking the average prediction of each segment. The
results reported in Fig. 4G show an interesting correlation between $D$ and
$\alpha$ that suggests a more complex diffusion pattern, involving the
occurrence of anomalous diffusion also at the level of individual segments.

### Characterizing multi-state diffusion processes

We use STEP to analyze experimental trajectories of the integrin
$\alpha$5$\beta$1 diffusing in the membrane of HeLa cells. Integrins are
transmembrane receptors for the extracellular matrix (ECM) in focal adhesions,
which mechanically link the ECM and actin filaments in the cytoplasm and
activate signaling pathways involved in cell migration, proliferation, or
apoptosis [27]. The dynamics of the integrin $\alpha$5$\beta$1 is influenced
by interactions with fibronectin and actin-binding proteins [50, 49]. Its
motion has been reported to switch from fast free-diffusion to slow free-
diffusion and immobilization, as well as exhibiting rearward actin-driven
movement.

We use STEP to predict both the diffusion coefficient and the anomalous
diffusion exponent for the integrin $\alpha$5$\beta$1 trajectories. Then, we
segment the trajectories by applying the KCPD method to both predictions at
once. In this way, we assign every segment a unique $D$ and $\alpha$ by taking
the average prediction over the segment. Examples of the results are shown in
Fig. 5A and the joint distribution of $D$ and $\alpha$ in Fig. 5B. The visual
inspection of Fig. 5B reveals two main clusters centered around
($D=10^{-6}\mu$m2/s, $\alpha=0.25$) and ($D=0.1\,\mu$m2/s, $\alpha=1$). The 2D
histogram of the same parameters calculated at the pointwise level (pre-
segmentation) does not show any major differences with respect to Fig. 5B.

However, the unsupervised clustering of the data using a _k_ -means algorithm
in combination with model selection performed with the elbow method [51]
reveals four clusters of segments (Fig. 5C) characterized by different motion
features. The first two clusters show a rather restrained motion, with
integrins spending 40% of the time in a state characterized by
$D=1.2^{-5}\mu$m2/s and $\alpha=0.23$, and 14% of the time with
$D=0.06\,\mu$m2/s and $\alpha=0.46$. For both clusters, the distribution of
angles between successive steps shows a peak centered at 180$\degree$,
indicating backward movements due to reflection at potential boundaries, as we
show in Fig. 5D. The confinement radius of the first cluster has a median of
$14.9\,$nm (st. dev. $\pm 13.3\,$nm), which is comparable to the localization
precision of these experiments. This allows us to associate it with protein
immobilization. The second cluster shows confined motion within areas with a
broad distribution of sizes, as we see in Fig. 5E, and a median radius of
$90.4\,$nm (st. dev. $\pm 98.2\,$nm). The third cluster represents the 29% of
the total recording and shows minor deviations from Brownian motion with
$\alpha=0.88$ and a nearly uniform angle distribution, and has an average
$D=0.10\,\mu$m2/s, close to the value typically reported for this protein.
Interestingly, the analysis pinpoints a fourth population, corresponding to a
20% of the total recording, undergoing superdiffusion with $\alpha=1.3$ and
$D=0.14\,\mu$m2/s, and with a persistent direction of motion between
consecutive steps (Fig. 5D).

##  III Discussion

In this work we present STEP, a machine learning method to predict diffusion
properties from individual trajectories at every time step. The method relies
on a combination of state-of-the-art machine learning architectures that take
into account correlations at different time scales. The presented approach is
especially appealing to analyze trajectories from particles undergoing
heterogeneous motion, where changes in diffusion properties occur over time.
Moreover, it does not require prior knowledge of the underlying physical
process or the temporal resolution at which changes in diffusion occur.

To illustrate the predictive power of STEP, we benchmark it on simulated
trajectories in various conditions. We show its ability to predict piecewise
constant diffusion properties, such as the diffusion coefficient or the
anomalous diffusion exponent, in noisy and short trajectories. Furthermore, we
demonstrate that STEP boosts up the accuracy of a changepoint detection
algorithm to detect the time at which diffusion changes take place.
Importantly, we also prove the suitability of our method to study continuous
changes of diffusion. To further showcase the potential applications of the
method, we study trajectories obtained by tracking live-cell single-molecule
imaging experiments of proteins of the plasma membrane. First, we characterize
the motion of the pathogen recognition receptor DC-SIGN, which was shown to
exhibit random changes in the diffusion coefficient. Our analysis confirms
such a hypothesis and improves the accuracy with which we detect these
changes. Moreover, our results suggest the occurrence of more complex
phenomena that need further investigation. Secondly, we study the diffusion of
the integrin $\alpha_{5}\beta_{1}$. In agreement with previous works, our
analysis confirms the existence of different diffusion modes and allows their
precise classification according to the diffusion coefficient, the anomalous
diffusion exponent, and the levels of spatial constraint.

We believe that STEP represents a first step towards a new class of machine
learning algorithms to study dynamic systems through a sequence-to-sequence
approach. Focusing on the instantaneous prediction of the property of interest
enables the characterization of the trajectories at experimental time
resolution without averaging and filtering and minimizes the prior knowledge
needed to perform the analysis. As such, the information retrieved with STEP
can provide information about diffusion properties with unprecedented
resolution and thus shed light on the underlying physical processes of a
variety of systems.

## Materials and Methods

### Machine learning model

In diffusion phenomena, we deal with complex statistical signals that can
exhibit various types of time correlations. Therefore, we need a machine
learning (ML) model that can capture correlations at different time scales.
Furthermore, we often encounter trajectories with very different lengths, even
in the same experiment. Hence, it is crucial that the model is length-
independent to ensure that it is as applicable as possible. In this work, we
propose to use a ML technique based on the advances of sequence-to-sequence
learning [21], which has shown impressive results in natural language
processing tasks and beyond [52, 53, 54]. Analogously to a translation task,
we implement a model that takes a particle trajectory as input and outputs the
diffusion properties of interest at each time step. In this way, the input and
the output have the same length.

In our framework, the input trajectory, $\mathbf{x}$, is a $d$-dimensional
vector of arbitrary length $T$ whose elements, $x_{t}$, correspond to the
particle position at each time step, $t\in[1,T]$. In this work, we consider
two-dimensional trajectories ($d=2$), but the method is easily adapted to any
dimension. On the other hand, the output is a one-dimensional vector of length
$T$ whose elements contain the value of a diffusion property of interest. For
instance, if we study the behavior of the diffusion coefficient along a
trajectory, the output elements would be the diffusion coefficient at each
time step, $D_{t}$. We refer the reader to Fig. 1 for a schematic
representation of the procedure. In case we are interested in more than one
property, we can either extend the output dimension, e.g., predicting
$[D_{t},\alpha_{t}]$, or implement an independent model for each one of them.
In this work, we have opted for the second option.

We propose an architecture combining convolutional and self-attention
mechanisms. Interestingly, very recent works have shown analogous strategies,
both with supervised [20] and unsupervised approaches [55]. First, the input
trajectory is processed by a series of convolutional layers that capture the
short-range correlations. We implement them following the XResNet architecture
[43]. Then, the result follows through a transformer encoder [26], which can
capture global correlations. Finally, we use pointwise fully-connected layers
to reach the desired output dimension.

We refer to Appendix B for a detailed description of the architecture and its
training procedure. We also provide a full library containing the code and
detailed explanations on how to reproduce the results of this work in Ref.
[56].

### Cell culture and plating

For the live-cell single-molecule imaging experiments involving the integrin
${\rm\alpha 5\beta 1}$, HeLa cells were cultured in DMEM (Gibco, 11960-044),
supplemented with 10% (v/v) fetal bovine serum (FBS, Sigma). Cells were tested
for mycoplasma contamination using PCR (Biotools kit, 4542). For fluorescence
imaging, glass-bottom dishes (IBIDI, 81158) were coated with fibronectin (FN,
Sigma, F2008) by placing 10$\,\mu$g/mL FN on the glass for 1 h at
37$\,\degree$C, and then blocked with BSA 2 mg/mL for 1 h at 37$\,\degree$C.
Cells were plated at a density of 5$\times$10${}^{4}\,$cells/dish and cultured
for 24 h prior to use.

### Preparation of half-antibody fragments

Half-antibody fragments were obtained following a protocol similar to the one
used in [6]. Briefly, mouse anti-human integrin ${\rm\alpha 5}$ antibody
(50$\,\mu$L; BD Biosciences, 610633) was dialyzed (ThermoFisher, Slide-A-Lyze
MINI Dialysis Device, 2K) against PBS overnight at room temperature to replace
the commercial buffer. Then, antibodies were reduced with 1 mM DTT for 30 min
at room temperature and dialyzed again, using Slide-A-Lyze MINI Dialysis
Device 2K, for 4 h at room temperature against PBS to remove DTT. To avoid
reassociation of reduced antibodies, sulfhydryl groups were blocked by
incubating with iodoacetamide 20 mM for 1 h at 4$\,\degree$C with agitation.
Iodoacetamide was then removed from the reaction by dialysis overnight at
4$\,\degree$C. Finally, reduced antibodies were biotinylated with a 10-fold
molar excess of EZ-Link Sulfo-NHS_LC_Biotin (Thermo Scientific) for 30 min at
room temperature with agitation and stored at 4$\,\degree$C until use.

### Single-molecule labeling

Biotinylated half-antibody fragments were conjugated to streptavidin-coated
quantum dots (QD655 streptavidin conjugate, Invitrogen, Q10123mp). Cells were
washed 3 times with washing buffer (PBS with 6% BSA) and labeled with half
antibody-quantum dots (about 1 nM) in washing buffer (200$\,\mu$L per dish)
for 15 min at 37$\,\degree$C, followed by two washes.

### Live-cell single-molecule imaging

Imaging was performed using a Leica DMi8 fluorescence microscope. Samples were
illuminated in total internal reflection fluorescence (TIRF) geometry.
Excitation was achieved with a CW laser (Obis, Coherent, $\lambda$=488 nm,
$<$1 kW/cm2). Fluorescence was recorded using an oil-immersion objective
(Leica, 100X, NA=1.47) and an sCMOS camera (Photometrics 95B) with appropriate
filters (Chroma). Movies were recorded at a frame rate of 33 Hz. A microscope
environment chamber (Okolab) was used to keep cells in a 5% CO2 atmosphere
while recording.

### Single-particle tracking

Particle detection and tracking were performed using u-track [57]. The
detection (Gaussian Mixture-Model Fitting) and tracking parameters were
optimized based on visual inspection and performance diagnostic of the
resulting detection and tracking. All image and data analysis tasks were
performed in MATLAB 2020a and more recent versions (The MathWorks, Natick,
MA). Videos were loaded into MATLAB using Bio-Formats [58].

## References

  * [1] Leonhard Möckl, Don C Lamb, and Christoph Bräuchle.  Super-resolved fluorescence microscopy: nobel prize in chemistry 2014 for eric betzig, stefan hell, and william e. moerner.  Angewandte Chemie International Edition, 53(51):13972–13977, 2014\.
  * [2] Carlo Manzo and Maria F Garcia-Parajo.  A review of progress in single particle tracking: from methods to biophysical insights.  Reports on progress in physics, 78(12):124601, 2015.
  * [3] Shuhui Yin, Nancy Song, and Haw Yang.  Detection of velocity and diffusion coefficient change points in single-particle trajectories.  Biophysical journal, 115(2):217–229, 2018.
  * [4] Suvrajit Saha, Il-Hyung Lee, Anirban Polley, Jay T Groves, Madan Rao, and Satyajit Mayor.  Diffusion of gpi-anchored proteins is influenced by the activity of dynamic cortical actin.  Molecular biology of the cell, 26(22):4033–4045, 2015.
  * [5] Nirmalya Bag, Shuangru Huang, and Thorsten Wohland.  Plasma membrane organization of epidermal growth factor receptor in resting and ligand-bound states.  Biophysical journal, 109(9):1925–1936, 2015.
  * [6] Shalini T Low-Nam, Keith A Lidke, Patrick J Cutler, Rob C Roovers, Paul MP van Bergen en Henegouwen, Bridget S Wilson, and Diane S Lidke.  Erbb1 dimerization is promoted by domain co-confinement and stabilized by ligand binding.  Nature structural & molecular biology, 18(11):1244–1249, 2011\.
  * [7] Adal Sabri, Xinran Xu, Diego Krapf, and Matthias Weiss.  Elucidating the origin of heterogeneous anomalous diffusion in the cytoplasm of mammalian cells.  Physical Review Letters, 125(5):058101, 2020.
  * [8] Jae-Hyung Jeon, Matti Javanainen, Hector Martinez-Seara, Ralf Metzler, and Ilpo Vattulainen.  Protein crowding in lipid bilayers gives rise to non-gaussian anomalous lateral diffusion of phospholipids and proteins.  Physical Review X, 6(2):021006, 2016.
  * [9] Thomas J Lampo, Stella Stylianidou, Mikael P Backlund, Paul A Wiggins, and Andrew J Spakowitz.  Cytoplasmic rna-protein particles exhibit non-gaussian subdiffusive behavior.  Biophysical journal, 112(3):532–542, 2017.
  * [10] Carlo Manzo, Juan A Torreno-Pina, Pietro Massignan, Gerald J Lapeyre Jr, Maciej Lewenstein, and Maria F Garcia Parajo.  Weak ergodicity breaking of receptor motion in living cells stemming from random diffusivity.  Physical Review X, 5(1):011021, 2015.
  * [11] Jae-Hyung Jeon, Aleksei V Chechkin, and Ralf Metzler.  Scaled brownian motion: a paradoxical process with a time dependent diffusivity for the description of anomalous diffusion.  Physical Chemistry Chemical Physics, 16(30):15811–15817, 2014.
  * [12] Anthony R Vega, Spencer A Freeman, Sergio Grinstein, and Khuloud Jaqaman.  Multistep track segmentation and motion classification for transient mobility analysis.  Biophysical journal, 114(5):1018–1025, 2018.
  * [13] Marloes Arts, Ihor Smal, Maarten W Paul, Claire Wyman, and Erik Meijering.  Particle mobility analysis using deep learning and the moment scaling spectrum.  Scientific reports, 9(1):1–10, 2019.
  * [14] Yann Lanoiselée and Denis S Grebenkov.  Unraveling intermittent features in single-particle trajectories by a local convex hull method.  Physical Review E, 96(2):022144, 2017.
  * [15] Jonathan E Bronson, Jingyi Fei, Jake M Hofman, Ruben L Gonzalez Jr, and Chris H Wiggins.  Learning rates and states from biophysical time series: a bayesian approach to model selection and single-molecule fret data.  Biophysical journal, 97(12):3196–3205, 2009.
  * [16] Fredrik Persson, Martin Lindén, Cecilia Unoson, and Johan Elf.  Extracting intracellular diffusive states and transition rates from single-molecule tracking data.  Nature methods, 10(3):265–269, 2013.
  * [17] Nilah Monnier, Zachary Barry, Hye Yoon Park, Kuan-Chung Su, Zachary Katz, Brian P English, Arkajit Dey, Keyao Pan, Iain M Cheeseman, Robert H Singer, et al.  Inferring transient particle transport dynamics in live cells.  Nature methods, 12(9):838–840, 2015.
  * [18] Gorka Muñoz-Gil, Giovanni Volpe, Miguel Angel Garcia-March, Erez Aghion, Aykut Argun, Chang Beom Hong, Tom Bland, Stefano Bo, J Alberto Conejero, Nicolás Firbas, et al.  Objective comparison of methods to decode anomalous diffusion.  Nature communications, 12(1):1–16, 2021.
  * [19] Hippolyte Verdier, François Laurent, Alhassan Cassé, Christian L Vestergaard, and Jean-Baptiste Masson.  Variational inference of fractional brownian motion with linear computational complexity.  Physical Review E, 106(5):055311, 2022.
  * [20] Jesús Pineda, Benjamin Midtvedt, Harshith Bachimanchi, Sergio Noé, Daniel Midtvedt, Giovanni Volpe, and Carlo Manzo.  Geometric deep learning reveals the spatiotemporal features of microscopic motion.  Nature Machine Intelligence, pages 1–12, 2023.
  * [21] Ilya Sutskever, Oriol Vinyals, and Quoc V Le.  Sequence to sequence learning with neural networks.  In Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 27. Curran Associates, Inc., 2014.
  * [22] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al.  Language models are few-shot learners.  Advances in neural information processing systems, 33:1877–1901, 2020.
  * [23] Scott Reed, Konrad Zolna, Emilio Parisotto, Sergio Gomez Colmenarejo, Alexander Novikov, Gabriel Barth-Maron, Mai Gimenez, Yury Sulsky, Jackie Kay, Jost Tobias Springenberg, et al.  A generalist agent.  arXiv preprint arXiv:2205.06175, 2022.
  * [24] Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poulton, Viktor Kerkez, and Robert Stojnic.  Galactica: A large language model for science.  arXiv preprint arXiv:2211.09085, 2022.
  * [25] Yann LeCun, Bernhard Boser, John S Denker, Donnie Henderson, Richard E Howard, Wayne Hubbard, and Lawrence D Jackel.  Backpropagation applied to handwritten zip code recognition.  Neural computation, 1(4):541–551, 1989.
  * [26] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  Advances in neural information processing systems, 30, 2017.
  * [27] Pakorn Kanchanawong and David A Calderwood.  Organization, dynamics and mechanoregulation of integrin-mediated cell–ecm adhesions.  Nature Reviews Molecular Cell Biology, pages 1–20, 2022.
  * [28] Gorka Muñoz-Gil, Miguel Angel Garcia-March, Carlo Manzo, José D Martín-Guerrero, and Maciej Lewenstein.  Single trajectory characterization via machine learning.  New Journal of Physics, 22(1):013010, 2020.
  * [29] Naor Granik, Lucien E Weiss, Elias Nehme, Maayan Levin, Michael Chein, Eran Perlson, Yael Roichman, and Yoav Shechtman.  Single-particle diffusion characterization by deep learning.  Biophysical journal, 117(2):185–192, 2019.
  * [30] Stefano Bo, Falko Schmidt, Ralf Eichhorn, and Giovanni Volpe.  Measurement of anomalous diffusion using recurrent neural networks.  Phys. Rev. E, 100:010102, Jul 2019.
  * [31] Patrycja Kowalek, Hanna Loch-Olszewska, and Janusz Szwabiński.  Classification of diffusion modes in single-particle tracking data: Feature-based versus deep-learning approach.  Physical Review E, 100(3):032410, 2019.
  * [32] Henrik Seckler and Ralf Metzler.  Bayesian deep learning for error estimation in the analysis of anomalous diffusion.  Nature Communications, 13(1):6717, 2022.
  * [33] Vida Jamali, Cory Hargus, Assaf Ben-Moshe, Amirali Aghazadeh, Hyun Dong Ha, Kranthi K Mandadapu, and A Paul Alivisatos.  Anomalous nanoparticle surface diffusion in lctem is revealed by deep learning-assisted analysis.  Proceedings of the National Academy of Sciences, 118(10):e2017616118, 2021.
  * [34] Gorka Muñoz-Gil, Catalina Romero-Aristizabal, Nicolas Mateos, Felix Campelo, Lara I de Llobet Cucalon, Miguel Beato, Maciej Lewenstein, Maria F Garcia-Parajo, and Juan A Torreno-Pina.  Stochastic particle unbinding modulates growth dynamics and size of transcription factor condensates in living cells.  Proceedings of the National Academy of Sciences, 119(31):e2200667119, 2022.
  * [35] Alessia Gentili and Giorgio Volpe.  Characterization of anomalous diffusion classical statistics powered by deep learning (condor).  Journal of Physics A Mathematical General, 54(31):314003, 2021.
  * [36] Aykut Argun, Giovanni Volpe, and Stefano Bo.  Classification, inference and segmentation of anomalous diffusion with recurrent neural networks.  Journal of Physics A: Mathematical and Theoretical, 54(29):294003, 2021.
  * [37] Héctor Buena Maizón and Francisco J Barrantes.  A deep learning-based approach to model anomalous diffusion of membrane proteins: the case of the nicotinic acetylcholine receptor.  Briefings in Bioinformatics, 23(1), 10 2021.  bbab435.
  * [38] Hippolyte Verdier, Maxime Duval, François Laurent, Alhassan Cassé, Christian L. Vestergaard, and Jean-Baptiste Masson.  Learning physical properties of anomalous random walks using graph neural networks.  Journal of Physics A: Mathematical and Theoretical, 54(23):234001, may 2021.
  * [39] Carlo Manzo.  Extreme learning machine for the characterization of anomalous diffusion from single trajectories (andi-elm).  Journal of Physics A: Mathematical and Theoretical, 54(33):334002, 2021.
  * [40] Dezhong Li, Qiujin Yao, and Zihan Huang.  Wavenet-based deep neural networks for the characterization of anomalous diffusion (wadnet).  Journal of Physics A: Mathematical and Theoretical, 54(40):404003, sep 2021.
  * [41] Òscar Garibo-i Orts, Alba Baeza-Bosca, Miguel A Garcia-March, and J Alberto Conejero.  Efficient recurrent neural network methods for anomalously diffusing single particle short and noisy trajectories.  Journal of Physics A: Mathematical and Theoretical, 54(50):504002, 2021.
  * [42] Nicolás Firbas, Òscar Garibo i Orts, Miguel Angel Garcia-March, and J Alberto Conejero.  Characterization of anomalous diffusion through convolutional transformers.  Journal of Physics A: Mathematical and Theoretical, 56:014001, 2023\.
  * [43] Tong He, Zhi Zhang, Hang Zhang, Zhongyue Zhang, Junyuan Xie, and Mu Li.  Bag of tricks for image classification with convolutional neural networks.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 558–567, 2019.
  * [44] A. Celisse, G. Marot, M. Pierre-Jean, and G.J. Rigaill.  New efficient algorithms for multiple change-point detection with reproducing kernels.  Computational Statistics and Data Analysis, 128:200–220, 2018.
  * [45] Sylvain Arlot, Alain Celisse, and Zaid Harchaoui.  A kernel multiple change-point algorithm via model selection.  Journal of Machine Learning Research, 20(162):1–56, 2019.
  * [46] Carlo Manzo, Juan A Torreno-Pina, Ben Joosten, Inge Reinieren-Beeren, Emilio J Gualda, Pablo Loza-Alvarez, Carl G Figdor, Maria F Garcia-Parajo, and Alessandra Cambi.  The neck region of the c-type lectin dc-sign regulates its surface spatiotemporal organization and virus-binding capacity on antigen-presenting cells.  Journal of biological chemistry, 287(46):38946–38955, 2012.
  * [47] P Massignan, C Manzo, JA Torreno-Pina, MF García-Parajo, M Lewenstein, and GJ Lapeyre Jr.  Nonergodic subdiffusion from brownian motion in an inhomogeneous medium.  Physical review letters, 112(15):150603, 2014.
  * [48] D Montiel, H Cang, and H Yang.  Quantitative characterization of changes in dynamical behavior for single-particle tracking studies.  The Journal of Physical Chemistry B, 110(40):19763–19770, 2006\.
  * [49] Taka A Tsunoyama, Yusuke Watanabe, Junri Goto, Kazuma Naito, Rinshi S Kasai, Kenichi GN Suzuki, Takahiro K Fujiwara, and Akihiro Kusumi.  Super-long single-molecule tracking reveals dynamic-anchorage-induced integrin function.  Nature chemical biology, 14(5):497–506, 2018.
  * [50] Olivier Rossier, Vivien Octeau, Jean-Baptiste Sibarita, Cecile Leduc, Béatrice Tessier, Deepak Nair, Volker Gatterdam, Olivier Destaing, Corinne Albiges-Rizo, Robert Tampé, et al.  Integrins $\beta$1 and $\beta$3 exhibit distinct dynamic nanoscale organizations inside focal adhesions.  Nature cell biology, 14(10):1057–1067, 2012.
  * [51] Ville Satopaa, Jeannie Albrecht, David Irwin, and Barath Raghavan.  Finding a" kneedle" in a haystack: Detecting knee points in system behavior.  In 2011 31st international conference on distributed computing systems workshops, pages 166–171. IEEE, 2011.
  * [52] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei.  Language models are few-shot learners.  In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 1877–1901. Curran Associates, Inc., 2020.
  * [53] Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katie Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei, Marianne Monteiro, Jacob Menick, Sebastian Borgeaud, Andrew Brock, Aida Nematzadeh, Sahand Sharifzadeh, Mikolaj Binkowski, Ricardo Barreira, Oriol Vinyals, Andrew Zisserman, and Karen Simonyan.  Flamingo: a visual language model for few-shot learning, 2022.
  * [54] Scott Reed, Konrad Zolna, Emilio Parisotto, Sergio Gomez Colmenarejo, Alexander Novikov, Gabriel Barth-Maron, Mai Gimenez, Yury Sulsky, Jackie Kay, Jost Tobias Springenberg, Tom Eccles, Jake Bruce, Ali Razavi, Ashley Edwards, Nicolas Heess, Yutian Chen, Raia Hadsell, Oriol Vinyals, Mahyar Bordbar, and Nando de Freitas.  A generalist agent, 2022.
  * [55] Hélène Kabbech and Ihor Smal.  Identification of diffusive states in tracking applications using unsupervised deep learning methods.  In 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI), pages 1–4. IEEE, 2022.
  * [56] Borja Requena and Gorkal Muñoz-Gil.  Borjarequena/step, December 2022.
  * [57] Khuloud Jaqaman, Dinah Loerke, Marcel Mettlen, Hirotaka Kuwata, Sergio Grinstein, Sandra L Schmid, and Gaudenz Danuser.  Robust single-particle tracking in live-cell time-lapse sequences.  Nature methods, 5(8):695–702, 2008.
  * [58] Melissa Linkert, Curtis T Rueden, Chris Allan, Jean-Marie Burel, Will Moore, Andrew Patterson, Brian Loranger, Josh Moore, Carlos Neves, Donald MacDonald, et al.  Metadata matters: access to image data in the real world.  Journal of Cell Biology, 189(5):777–782, 2010.
  * [59] Ralf Metzler, Jae-Hyung Jeon, Andrey G Cherstvy, and Eli Barkai.  Anomalous diffusion models and their properties: non-stationarity, non-ergodicity, and ageing at the centenary of single particle tracking.  Physical Chemistry Chemical Physics, 16(44):24128–24164, 2014.
  * [60] Joseph Klafter and Igor M Sokolov.  First steps in random walks: from tools to applications.  OUP Oxford, 2011.
  * [61] Harvey Scher and Elliott W. Montroll.  Anomalous transit-time dispersion in amorphous solids.  Phys. Rev. B, 12:2455–2477, Sep 1975.
  * [62] Benoit B. Mandelbrot and John W. Van Ness.  Fractional brownian motions, fractional noises and applications.  SIAM Review, 10(4):422–437, 1968.
  * [63] J. Klafter and G. Zumofen.  Lévy statistics in a hamiltonian system.  Phys. Rev. E, 49:4873–4877, Jun 1994.
  * [64] S. C. Lim and S. V. Muniandy.  Self-similar gaussian processes for modeling anomalous diffusion.  Phys. Rev. E, 66:021114, Aug 2002.
  * [65] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala.  Pytorch: An imperative style, high-performance deep learning library.  In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
  * [66] Jeremy Howard and Sylvain Gugger.  Fastai: A layered api for deep learning.  Information, 11(2), 2020.
  * [67] Charles Truong, Laurent Oudre, and Nicolas Vayatis.  Selective review of offline change point detection methods.  Signal Processing, 167:107299, 2020.
  * [68] Diederik P. Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  In ICLR (Poster), 2015.
  * [69] Leslie N. Smith and Nicholay Topin.  Super-convergence: very fast training of neural networks using large learning rates.  In Tien Pham, editor, Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications, volume 11006, pages 369 – 386. International Society for Optics and Photonics, SPIE, 2019.
  * [70] Geoffrey E. Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R. Salakhutdinov.  Improving neural networks by preventing co-adaptation of feature detectors, 2012.
  * [71] Stephen Hanson and Lorien Pratt.  Comparing biases for minimal network construction with back-propagation.  In D. Touretzky, editor, Advances in Neural Information Processing Systems, volume 1. Morgan-Kaufmann, 1988.
  * [72] Anders Krogh and John Hertz.  A simple weight decay can improve generalization.  In J. Moody, S. Hanson, and R.P. Lippmann, editors, Advances in Neural Information Processing Systems, volume 4. Morgan-Kaufmann, 1991.
  * [73] Gorka Muñoz-Gil, Borja Requena, Giovanni Volpe, Miguel Angel Garcia-March, and Carlo Manzo.  AnDiChallenge/ANDI_datasets: Challenge 2020 release, May 2021.

## Acknowledgments

The authors acknowledge Montserrat Masoliver-Prieto and Marta Cullell-Dalmau
for their valuable help with the experimental procedures. GMG acknowledges
support from the European Union (ERC, QuantAI, Project No. 10105529) and the
Austrian Science Fund (FWF) through the SFB BeyondC F7102. CM acknowledges
support through grant RYC-2015-17896 funded by MCIN/AEI/10.13039/501100011033
and “ESF Investing in your future”, grants BFU2017-85693-R and
PID2021-125386NB-I00 funded by MCIN/AEI/10.13039/501100011033/ and FEDER “ERDF
A way of making Europe”, and grant AGAUR 2017SGR940 funded by the Generalitat
de Catalunya. BR and ML acknowledge support from: ERC AdG NOQIA; Ministerio de
Ciencia y Innovación Agencia Estatal de Investigaciones
(PGC2018-097027-B-I00/10.13039/501100011033,
CEX2019-000910-S/10.13039/501100011033, Plan Nacional FIDEUA
PID2019-106901GB-I00, FPI, QUANTERA MAQS PCI2019-111828-2, QUANTERA DYNAMITE
PCI2022-132919, Proyectos de I+D+I “Retos Colaboración” QUSPIN
RTC2019-007196-7); MICIIN with funding from European Union
NextGenerationEU(PRTR-C17.I1) and by Generalitat de Catalunya; Fundació
Cellex; Fundació Mir-Puig; Generalitat de Catalunya (European Social Fund
FEDER and CERCA program, AGAUR Grant No. 2021 SGR 01452, QuantumCAT \
U16-011424, co-funded by ERDF Operational Program of Catalonia 2014-2020);
Barcelona Supercomputing Center MareNostrum (FI-2022-1-0042); EU Horizon 2020
FET-OPEN OPTOlogic (Grant No 899794); EU Horizon Europe Program (Grant
Agreement 101080086 — NeQST), National Science Centre, Poland (Symfonia Grant
No. 2016/20/W/ST4/00314); ICFO Internal “QuantumGaudi” project; European
Union’s Horizon 2020 research and innovation program under the Marie-
Skłodowska-Curie grant agreement No 101029393 (STREDCH) and No 847648 (“La
Caixa” Junior Leaders fellowships ID100010434: LCF/BQ/PI19/11690013,
LCF/BQ/PI20/11760031, LCF/BQ/PR20/11770012, LCF/BQ/PR21/11840013). Views and
opinions expressed in this work are, however, those of the author(s) only and
do not necessarily reflect those of the European Union, European Climate,
Infrastructure and Environment Executive Agency (CINEA), nor any other
granting authority. Neither the European Union nor any granting authority can
be held responsible for them

##  Appendix A Diffusion properties

In this section, we briefly highlight some of the main characteristics of
normal and anomalous diffusion. We refer the reader to Refs. [59, 60] for a
nice and thorough introduction to the field.

Diffusion trajectories are often described by means of their mean squared
displacement (MSD) which, in the case of Brownian motion, shows a linear
scaling with time, i.e. MSD $\propto Dt$, where $D$ is the diffusion
coefficient. However, there can be deviations from such linear scaling,
resulting in a power-law relation between the MSD and time, i.e. MSD $\propto
K_{\alpha}t^{\alpha}$, where $\alpha$ is defined as the anomalous diffusion
exponent and $K_{\alpha}$ is an effective diffusion coefficient. The former
allows us to distinguish between normal (or Brownian) diffusion ($\alpha=1$)
and anomalous diffusion ($\alpha\neq 1$).

The appearance of anomalous diffusion can be associated with very different
phenomena, from the arising of correlations in the motion of the diffusing
particle to the presence of spatiotemporal heterogeneity. To account for most
of these phenomena, we follow Ref. [18] and consider five anomalous diffusion
models with specific ranges for the anomalous diffusion exponent: annealed
transient time model (ATTM) [47] with $\alpha\in[0.05,1]$, continuous-time
random walk (CTRW) [61] with $\alpha\in[0.05,1]$, fractional Brownian motion
(FBM) [62] with $\alpha\in[0.05,1.95]$, Lévy walk (LW) [63] with
$\alpha\in[1.05,2]$, and scaled Brownian motion (SBM) [64] with
$\alpha\in[0.05,2]$.

##  Appendix B Machine learning pipeline

Here, we provide a detailed explanation of the machine learning approach
followed to obtain the results described throughout this work.

As we briefly mention in the main text, we train two different models: one for
the diffusion coefficient task, and one for the anomalous diffusion exponent.
We report the results regarding the prediction of the diffusion coefficient
and anomalous diffusion exponent in Section II. We implement both models
following the same principles with very minor differences. In this section, we
describe the architecture and the training process that we follow and, when
needed, highlight the differences between models.

We provide the source code with extended explanations on how to reproduce the
results in [56]. We make extensive use of the _PyTorch_ [65] and _fastai_ [66]
libraries to implement the architecture and the training procedure. The kernel
changepoint detection method (KCPD) was implemented using the _ruptures_
Python library [67].

###  B.1 Architecture details

We propose to use a model that takes a trajectory $\mathbf{x}$ as input and
outputs the target diffusion properties at each time step. The input
trajectory is a $d$-dimensional vector of arbitrary length $T$, whose
elements, $x_{t}$, correspond to the particle position at every time step $t$.
Then, the output is a one-dimensional vector of length $T$, whose elements
correspond to the diffusion property of interest at every time step, e.g.,
$D_{t}$ in the case of the diffusion coefficient. See Fig. 6 for further
details about the dimensions. Throughout this work, we mainly consider
trajectories of dimension $d=2$.

The model we propose consists of three main modules: an initial convolutional
part that processes the input trajectory; a self-attention-based part that
feeds on the features extracted by the previous one; a shallow pointwise fully
connected feedforward module that provides the desired output dimensions. The
entire architecture is length independent, which allows us to process
trajectories of arbitrary lengths.

Convolutional module – The first main convolutional module allows us to expand
the trajectory dimension with several convolutional filters. This provides the
following layers with a richer embedding based on short-range correlations.

We build it following the XResNet [43] architecture. As we show in Fig. 6, it
consists of an initial convolutional layer, commonly referred to as the stem,
followed by a series of residual blocks that feature a convolutional layer
with a skip connection. We use one-dimensional convolutions with a kernel size
of three and stride one to preserve the trajectory size. However, we use a
kernel size of one in the skip connections, which can act as the identity or a
scaling factor whose main purpose is to match the tensor shapes on both paths
of the residual blocks, as we explain below. This module can take a batch of
input trajectories of size $\left[\text{{batch\\_size}}\times T\times
d\right]$ and output a batch of features of size
$\left[\text{{batch\\_size}}\times T\times\text{{embedding\\_size}}\right]$.

Throughout the architecture, we add a batch normalization layer directly after
every convolutional layer, and we use the rectified linear unit (ReLU)
activation function by default, except in the last output layer.

To produce the results in Section II, we use a single convolutional layer and
a ReLU activation in the stem. We use 64 filters to predict the diffusion
coefficient and 32 filters for the anomalous diffusion exponent. Then, we have
added three residual blocks with 128, 256 and 512 filters, respectively.
Hence, $\text{{embedding\\_size}}=512$. In these blocks, the convolutional
paths have two convolutional layers: the first one increases the embedding
size and the second one preserves the dimensions. In the skip connection, we
only have one convolutional layer that increases the embedding size to match
the dimensions of the convolutional path. We implement the ReLU activation at
the end of the block, after we add the outcome of both paths.

Self-attention module – We process the features extracted by the convolutional
module with a self-attention mechanism that allows the model to capture long-
range correlations.

More precisely, we implement a transformer encoder, as it was introduced in
Ref. [26]. As we illustrate in Fig. 6, the encoder block has two main parts,
both featuring a skip connection followed by a layer normalization after the
sum of both paths. In the first one, we have a multi-head attention layer that
feeds on the input and, in the second one, we have a couple of pointwise
feedforward layers, which are equally applied to each element in the incoming
tensor. Furthermore, we can add a positional encoding before the first encoder
block, which provides information about the relative position of each element
in the trajectory. This module can process a batch of embeddings preserving
its dimensions. Hence, the input and the output both have size
$\left[\text{{batch\\_size}}\times T\times\text{{embedding\\_size}}\right]$.

To produce the results in Section II, we use four transformer encoder blocks
with eight heads in the multi-head attention layers. The pointwise feedforward
part adds two fully-connected layers with embedding_size neurons each, i.e,
512 in this case. Interestingly, we have found that, after the convolutions,
the positional encoding has very little impact on the results. Therefore, in
the interest of simplicity, we have not used it to obtain the results reported
in this work.

Feedforward module – The last main part is a shallow feedforward fully-
connected network that acts element-wise on the features extracted by the
previous module. We tailor this part to the specific task at hand to achieve
the desired output with the proper dimensions.

For instance, in a regression task, the output dimension is one and we use a
scaled sigmoid activation function at the end to define the output range with
some margin, e.g., $\log D\in(-3.1,3.1)$, $\alpha\in(0,2.05)$. This margin
allows the sigmoid to reach the desired values before it saturates. In a
hypothetical case of classification task (as e.g. classifying between
diffusion models as done in Ref. [18]), the final dimension is the number of
classes and we use a softmax activation function. Then, we obtain the
predictions by choosing the class with the maximum activation value. Hence, we
can process a feature batch of size $\left[\text{{batch\\_size}}\times
T\times\text{{embedding\\_size}}\right]$ and output their predictions with
size $\left[\text{{batch\\_size}}\times T\times\text{{num\\_class}}\right]$.
In case that $\text{{num\\_class}}>1$, as in a classification task, we perform
an additional post-processing step to obtain an output of size
$\left[\text{{batch\\_size}}\times T\times 1\right]$ with the corresponding
predictions at each time step.

Figure 6: Machine learning architecture representation

###  B.2 Training procedure

We follow a standard gradient-based training procedure for both of our models.
The only differences between them arise from the training data and how we
process it.

The main training loop consists on:

  1. 1.

Predict the values over a batch of training data.

  2. 2.

Compute the loss function with respect to the true values.

  3. 3.

Update the model parameters based on the loss’ gradient.

We use batches containing 128 trajectories and the L1 loss function, which
corresponds to the mean absolute error. Formally,

$\mathcal{L}_{\text{MAE}}(\mathbf{x})=\frac{1}{n}\sum_{i=1}^{n}\left|y_{i}-f(\mathbf{x}_{i})\right|\,,$
(1)

where $f(\mathbf{x}_{i})$ is the prediction of the $i$-th trajectory in a
batch of $n$ samples.

To perform the parameter update, we use an Adam [68] optimizer. We use the
fastai [66] library to choose the learning rate with the learning rate finder
tool, typically of the order of $10^{-4}$. Then, we implement a schedule over
the training batches both in the learning rate and its momentum, following the
one-cycle policy introduced in Ref. [69]. We train our models until the
performance in the validation set stabilizes, typically between ten to twenty
epochs.

To further prevent overfitting and enhance the model generalization
capabilities, we use dropout [70] and weight decay [71, 72]. Additionally, we
add Gaussian localization noise at different intensities to the trajectories
as a form of data augmentation.

###  B.3 Training, validating, and testing our models

In order to properly evaluate our models, we generate several independent data
sets. We use one to train and validate our models, and we use the others to
test them on unseen scenarios. All the results that we report throughout this
work are obtained using the test sets, which we design to evaluate different
aspects of our models.

In Table 1, we provide the details about the data sets that we use to train,
validate and test our models. These data sets contain simulated trajectories
with their corresponding labels at each time step. We have two main approaches
to simulate the trajectories depending on whether we deal with normal or
anomalous diffusion. Below, we explain how we generate the data for both
cases.

Task | Models | $D$ | $\alpha$ | $\sigma_{\text{noise}}$ | Traj. length | Segments | Seg. length | Size
---|---|---|---|---|---|---|---|---
Train BM | Brownian motion | $[-3,3]$ | 1 | $[-6,2]$ | 200 | $[2,5]$ | $[10,190]$ | 100,000
Train AnDi | all anomalous | 1 | $[0.05,2]$ | 0.1 | 200 | $[2,5]$ | $[10,190]$ | 100,064
Fig. 2A & B | Brownian motion | $[-3,3]$ | 1 | 0 | 200 | $[2,5]$ | $[10,190]$ | 48,000
Fig. 2C, Fig. 9 | all anomalous | 1 | $[0.05,2]$ | $[-5,2]$ | 200 | $[2,5]$ | $[10,190]$ | 49,994
Fig. 8C & D
Fig. 2D, Fig. 8A & B | all anomalous | 1 | $[0.05,2]$ | $\\{0,0.1\\}$ | 200 | $[2,5]$ | $[10,190]$ | 50,000
Fig. 2E & F | Brownian motion | $[-3,3]$ | 1 | 0 | 200 | 2 | $[10,190]$ | 50,000
Fig. 2G & H | FBM | 1 | $[0.05,1.95]$ | 0 | 200 | 2 | $[10,190]$ | 40,000
Fig. 3 | SBM | 1 | $\\{0.1,0.5\\}$ | 0 | 200 | 1 | 200 | 6,000
Fig. 4B, C & D | ATTM | (-6.7, 0) | 0.75 | 0 | 200 | $[1,51]$ | $[1,200]$ | 10,000
Fig. 4E, F & G | (experiment) |  |  |  | $[200,2000]$ |  |  | 755
Fig. 5 | (experiment) |  |  |  | $[20,500]$ |  |  | 4734
Fig. 7A | Brownian motion | $[-3,3]$ | 1 | 0 | $[20,660]$ | $[1,11]$ | $\\{20,40,60\\}$ | 22,000
Fig. 7B | Brownian motion | $[-3,3]$ | 1 | 0 | $[40,660]$ | $[2,11]$ | $\\{20,40,60\\}$ | 20,000
Fig. 7C | Brownian motion | $[-3,3]$ | 1 | $[-6,0]$ | 200 | $[2,5]$ | $[10,190]$ | 48,000
Table 1: Data set details for all the results reported throughout this paper.
The ranges for $D$ and $\sigma_{\text{noise}}$ are in $\log_{10}$ scale, and
we take $\alpha$ intervals of 0.05 within the denoted ranges. We use $20\%$ of
the training data for validation. We take two independent sub-samples of a
test set with 199,976 trajectories: one for Fig. 2C, Fig. 8C & D, Fig. 9 and
the other for Fig. 2D, Fig. 8A & B. In Fig. 2D, we consider noiseless
trajectories, although we add noise with $\sigma_{\text{noise}}=0.1$ to flat
CTRW segments that would result in numerical instabilities for the TA-MSD
method. To generate the noisy trajectories for Fig. 2C, Fig. 8C & D and Fig.
9, we add 128 random levels of localization noise to each trajectory in the
data set, effectively making about $6.4\times 10^{6}$ trajectories. Fig. 2C is
contained in Fig. 9 and, thus, uses the same data. In Fig. 3, we evenly split
the two values of $\alpha$ among all trajectories. In Fig. 4B, C & D we use
ATTM trajectories with $\sigma=0.3$ and $\gamma=0.4$ (see [47, 10], do not
confuse with $\sigma_{\text{noise}}$). We simulate them randomly sampling $D$
and the segment lengths accordingly, and the values here come from analysing
the resulting trajectories, which have 18 different segments on average. For
Fig. 4E, F & G we consider 755 experimental trajectories of the pathogen-
recognition receptor DC-SIGN containing from 200 to 2000 frames sampled at
$60\,$Hz. We obtain all the results in Fig. 5 from 4734 trajectories of the
integrin $\alpha 5\beta 1$ containing from 20 to 500 frames sampled at
$33\,$Hz. The data set for Fig. 7B is a sub-set of the one from Fig. 7A. In
Fig. 7C, we use the same trajectories from Fig. 2A & B and add 128 random
levels of localization noise to each of them, effectively making $6.144\times
10^{6}$ noisy trajectories.

While there are some differences between how we simulate and label our
trajectories for normal and anomalous diffusion, there are several common
factors that hold for all of them. For instance, all segments have constant
diffusion properties and they are, at least, ten time steps long.

Brownian motion – We simulate Brownian motion trajectories by taking
uncorrelated Gaussian noise as the trajectory displacements. We control the
diffusion coefficient at each time step with the standard deviation of the
Gaussian noise, which corresponds to $\sqrt{2D}$. This way, we can easily
generate segments of arbitrary lengths with a constant diffusion coefficient,
$D$, along the trajectories. Finally, we perform the cumulative sum of the
displacements to obtain the trajectory coordinates and we subtract the initial
position such that they start at the origin.

We consider diffusion coefficients across six orders of magnitude
$D\in[10^{-3},10^{3}]$. However, we take its logarithm as labels for the
regression task, such that $y_{i}\in[-3,3]$ at every time step. This greatly
simplifies the problem and allows us to keep a consistent performance across
all orders of magnitude.

Additionally, we can simulate experimental localization noise by adding
Gaussian noise with standard deviation $\sigma_{\text{noise}}$. We use this as
a form of data augmentation during training and to study the model’s
resilience to noise. See Table 1 for further details.

Anomalous diffusion – To simulate anomalous diffusion trajectories, we
consider the five diffusion models introduced in Appendix A with their
respective anomalous diffusion exponent ranges. We generate full trajectories
for each model following the same procedure detailed in the Supplementary
Material from Ref. [18] and using the library provided by the authors [73].
Then, in order to obtain heterogeneous trajectories, we split them into
segments and combine them together. We impose the condition that two
consecutive segments must differ, at least, either in the diffusion model or
the anomalous diffusion exponent. Finally, we add Gaussian localization noise,
with standard deviation $\sigma_{\text{noise}}$. Then, we normalize the
resulting displacements by their standard deviation and subtract the initial
position to ensure that the trajectory starts at the origin.

Therefore, we have two labels at each time step: the anomalous diffusion
exponent and the diffusion model with which the corresponding segment was
generated. This allows us to use the same data for both a regression task in
the anomalous diffusion exponent and a classification task in the diffusion
model. However, in this work, we have mainly focused on the first one.
Furthermore, we balance all the data sets such that there is an even
representation of both the anomalous diffusion exponents and diffusion models
throughout all the time steps.

##  Appendix C Diffusion coefficient prediction

In Section II, we study the capability of STEP to properly infer the diffusion
coefficient at every time step and detect changes in diffusive behavior. Here,
we complement the analysis presented in the main text by considering
additional factors that impact the performance, such as the number of segments
in the trajectories and the localization noise, typical of experimental
setups. We show the results in Fig. 7.

Figure 7: STEP performance predicting the diffusion coefficient at every time
step. (A) Relative error of STEP as a function of the number of segments at
three different segment lengths. (B) Prediction accuracy of STEP as a function
of the number of segments at three different segment lengths. (C) Relative
error of STEP (blue), STEP with known segments (red), and TA-MSD with known
segments (yellow) as a function of the ratio between the localization noise’s
standard deviation $\sigma$ and the diffusion coefficient $D$.

We investigate the effect of the number of segments on the characterization of
the trajectories. To test it, we fix the segment length and generate
trajectories with one to eleven segments (zero to ten changepoints), resulting
in trajectories with very different lengths (see Section B.3 for details). In
Fig. 7A, we see a slight increase of the relative error with the number of
segments, although it has a much lesser impact than the segment length, e.g.,
it is harder to characterize a single segment of twenty points than eight
consecutive segments of 40 time steps. Importantly, even in the presence of 10
changepoints, STEP still heavily outperforms the TA-MSD approach applied to
segments (no changes) of the same size, e.g., the whole curve for a segment
length of 20 in Fig. 7A is well below the TA-MSD point for a segment length of
20 in Fig. 2C.

In Fig. 2E and F, we show how to combine STEP with a KCPD method to detect
diffusion changes. In Fig. 7B, we show the performance as a function of the
number of segments. We see that the shortest segments are the hardest to
characterize. However, segment length becomes less important for sufficiently
long ones, as the curves for lengths 40 and 60 behave fairly similarly. We see
that STEP achieves a better score for shorter segments when the trajectories
are very long (11 segments). This suggests that every additional changepoint
in the trajectory adds a similar amount of error sources which are eventually
outweighed by the accumulated errors along the trajectory as it gets longer.
Nonetheless, even in the most challenging cases with 11 segments, STEP
correctly detect the vast majority of the points.

Finally, we study the resilience of our method to noise, an important
characteristic of experimental trajectories. In practical scenarios,
trajectories are affected by localization noise, which is usually modeled as
Gaussian noise of variance $\sigma_{\text{noise}}^{2}$ added to the
trajectories. Since we consider diffusion coefficients at very different
scales along the trajectories, in Fig. 7C, we plot the error as a function of
the ratio between the noise’s standard deviation and the diffusion
coefficient. We see that STEP strongly outperforms the TA-MSD approach with
known segments even well beyond the noise levels present in relevant
experimental scenarios (usually $\sigma_{\text{noise}}/D<10^{-1}$).
Surprisingly, STEP can correctly extract the diffusion coefficient of constant
segments (red line) even for enormous errors ($\sigma_{\text{noise}}/D>10$),
showing impressive noise resilience.

##  Appendix D Anomalous diffusion exponent prediction for various diffusion
models

In Section II, we briefly show how to use STEP to study particles that
randomly switch between anomalous diffusing states. Here, we thoroughly
characterize the suitability of the method for such task. We use STEP to
predict the anomalous diffusion exponent $\alpha$ at every time step of
trajectories composed of segments with constant anomalous diffusion exponent
and diffusion model, as we detail in Section B.3.

In the main text, we have already studied how the mean absolute error (MAE) in
the $\alpha$ prediction depends on the segment length, and we have compared
STEP to two reference methods. As an additional reference, we obtain an MAE
over all trajectories and time steps of 0.271 with STEP, 0.275 with STEP and
known segments, 0.368 with TA-MSD and known segments, and CONDOR achieved an
MAE of 0.237 for trajectories of the same lengths in the AnDi Challenge [18].

Figure 8: STEP performance to predict $\alpha$ in terms of the mean absolute
error (MAE). We consider the anomalous diffusion models: ATTM, CTRW, FBM, LW
and SBM (see Section B.3). (A) Prediction MAE by diffusion model. (B) MAE as
function of $\alpha$ for SBM segments. (C) MAE as a function of the
localization noise for STEP, TA-MSD, and CONDOR, the latter two with known
segments. (D) MAE as a function of the localization noise for STEP separated
by the diffusion model. The blue line in (C) hence corresponds to the average
of the lines presented in this panel.

Furthermore, we can look at the performance segregated by the diffusion model.
We report the MAE over all segments belonging to each anomalous diffusion
model in Fig. 8A. We observe clear differences for some of the models with
CTRW, FBM, and LW segments holding the lowest errors. Additionally, we provide
a histogram of predicted and true $\alpha$ for all diffusion models in Fig. 9.

In particular, the MAE in scaled Brownian motion (SBM) segments is
significantly larger than in the other models. This has already been observed
in previous works (see for instance Fig. 2d of Ref. [18]), although the
differences here are larger. A detailed inspection shows that the biggest
errors come from shorter segments, in agreement with the results from Fig. 2D.
This is reasonable since the aging in SBM is the source of the anomalous
diffusion [59] (see Section II) and therefore it requires longer segments to
be correctly characterized. It is also reasonable to expect the largest errors
to happen whenever $\alpha\in(0,2)$ is close to its range limits and the
predictions are in the opposite side. We see this in in Fig. 8B, where the MAE
for small and large $\alpha$ are clearly the major contributors to the overall
MAE presented in Fig. 8A.

STEP displays a clear tendency to predict $\alpha\sim 0.8$ for SBM segments,
as we can see in the right-most column of Fig. 9, which matches the results
from Fig. 8B. This behavior is enhanced by the presence of noise, suggesting
that the model struggles to identify any clear behavior in short segments,
which also happen to be the most common.

Interestingly, we find a similar trend in CTRW segments, where the model has a
tendency to predict $\alpha\simeq 0.25$, corresponding to nearly immobile
particles. CTRW trajectories are characterized by jumps at random times,
resulting in segments in which the particle does not move, usually referred to
as waiting times. Hence, many CTRW segments in our heterogeneous trajectories
do not display any movement due to their short lengths, corresponding to a
waiting time section. Therefore, it is impossible for the model to correctly
predict $\alpha$, as it does not have any information to work with.

To a lesser extent, we also find that the model predicts $\alpha\sim 1$ for
low anomalous diffusion exponents in ATTM segments. In ATTM trajectories with
small $\alpha$, we encounter very long segments with low diffusion
coefficients. Similar to the CTRW case, we encounter parts of these long
segments in our heterogeneous trajectories containing a unique diffusion
coefficient, thus behaving like Brownian motion along the observed time
window. Hence, the predictions $\alpha\sim 1$ are correct in these cases.

Finally, we study the resilience of the methods to localization noise, as we
do in Appendix C. We present the MAE as a function of the noise standard
deviation $\sigma_{\text{noise}}$ in Fig. 8C. We observe a consistent
performance of all the methods until reaching considerable levels of noise.
Again, STEP is comparable to CONDOR despite the latter having the advantage of
knowing the segments beforehand.

As we have seen throughout this section, characterizing some diffusion models
is harder than others and the localization noise has a different impact on
them, as we show in Fig. 8D and Fig. 9. While increasing the noise level has
an overall negative effect, we see that the performance on LW segments suffers
the most, while the performance on CTRW segments is barely affected. Overall,
the errors start to increase significantly beyond $\sigma_{\text{noise}}\sim
2\times 10^{-1}$, which would correspond to harsh experimental conditions.
Interestingly, ATTM segments see a drop in MAE with increasing noise for a
limited range.

Figure 9: Predicted vs true anomalous diffusion exponent. 2D histograms
showing the true and predicted anomalous diffusion exponents for the different
diffusion models. Each column contains the histograms belonging to a different
diffusion model for three different localization noise levels:
$\sigma_{\text{noise}}\in[-5,-4)$ (top), $\sigma_{\text{noise}}\in[-2,-1)$
(middle), and $\sigma_{\text{noise}}\in[-1,0)$ (bottom). The low-noise FBM
histogram (first row, center column) is the same as Fig. 2C.