# Preparing for Black Swans :
The Antifragility Imperative for Machine Learning

Ming Jin This work was supported in part by the NSF Safe Learning-Enabled
Systems Program (NSF # 2331775).

###### Abstract

Operating safely and reliably despite continual distribution shifts is vital
for high-stakes machine learning applications. This paper builds upon the
transformative concept of “antifragility” introduced by (Taleb, 2014) as a
constructive design paradigm to not just withstand but benefit from
volatility. We formally define antifragility in the context of online decision
making as dynamic regret’s strictly concave response to environmental
variability, revealing limitations of current approaches focused on resisting
rather than benefiting from nonstationarity. Our contribution lies in
proposing potential computational pathways for engineering antifragility,
grounding the concept in online learning theory and drawing connections to
recent advancements in areas such as meta-learning, safe exploration,
continual learning, multi-objective/quality-diversity optimization, and
foundation models. By identifying promising mechanisms and future research
directions, we aim to put antifragility on a rigorous theoretical foundation
in machine learning. We further emphasize the need for clear guidelines, risk
assessment frameworks, and interdisciplinary collaboration to ensure
responsible application.

##  1 Introduction

As we integrate machine learning (ML) into mission-critical systems in
healthcare, finance, transportation, and social-scale infrastructure like
power grids, a vital question arises about ensuring safety, security, and
reliability despite myriad stressors (Hendrycks et al., 2021b). These manifest
as natural or adversarial perturbations, aleatoric/epistemic uncertainties,
distribution shifts (including domain shift, concept drift, nonstionarity, and
out-of-distribution events).

The prevailing paradigm in ML pursues robustness—shielding systems against
these stressors. However, as S. Sagan cautions, “Things that have never
happened before happen all the time.” Novel outliers routinely breach reactive
defenses, raising concerns about their limitations against rare but impactful
“black swan” events (Taleb, 2010). (Nair et al., 2022) present statistical
arguments about why such long-tailed phenomena prove unexpectedly ubiquitous.

> Rather than merely withstand stressors, can systems be designed to thrive
> because of them?

This inverted perspective of embracing rather than barricading against
disorder constitutes the radical notion of “antifragility.” As originally
defined by (Taleb, 2014):

> “Some things benefit from shocks; they thrive and grow when exposed to
> volatility, randomness, disorder, and stressors and love adventure, risk,
> and uncertainty. Yet, in spite of the ubiquity of the phenomenon, there is
> no word for the exact opposite of fragile. Let us call it antifragile.
> Antifragility is beyond resilience or robustness. The resilient resists
> shocks and stays the same; the antifragile gets better. ”

Despite inspiring examples across various domains, formal frameworks in ML
leveraging antifragility remain scarce. This manuscript proposes a rigorous
definition of antifragility based on the online decision-making (ODM)
framework and identifies potential computational pathways to realize this
vision.

###  1.1 Perspective

We see antifragility as both an attainable goal and a constructive design
solution for ML systems.

Antifragile systems embody an open-world perspective, actively leveraging
continual stressors such as distributional shifts to systematically transform
them into opportunities for enhancement. These systems are capable not just of
adaptation, but of _rapid_ adaptation with improvisation when faced with
unforeseen disruptions. They exhibit _unusual_ robustness that goes beyond
typical stressors to confront potentially unforeseeable, high-impact
challenges. This is achieved through proactive preparation for such (unknown)
unknowns by exposure to and learning from a _diverse_ set of prior
disturbances. By investigating computational mechanisms that enable systems to
benefit from stressors and grounding the concept in rigorous theoretical
frameworks, we can collectively work towards computational antifragility for
ML systems that thrive amidst volatility.

###  1.2 Rejoinders

Antifragility, while a transformative concept, understandably faces
skepticism—reliability and security demand assurance while random tinkering
seems reckless. Volatility may productively “vaccinate,” but its dangers
cannot be casually dismissed, especially in domains where errors carry
irreversible consequences.111In this note, “volatility” broadly include 2nd or
higher order moments of a random variable, _variability_ within time series,
and distributional shifts (nonstationarity).

We do not argue for fully embracing disorder or eliminating all conservative
controls, which could decrease safety and oversight. Instead, we advocate
intelligent, measured openness through safe environment interaction,
selectively harnessing volatility’s benefits under safety constraints. This
purposeful experimentation permits recoverable failures that can be safely
remediated (see Sec. 6.3 for some methods). In critical domains like
healthcare and cyber-physical infrastructures, methods that simulate and
small-scale test disturbances provide safe avenues. Further, natural
perturbations may already provide ample opportunities for learning (even more
than engineered ones), reducing risks of intentional volatility. Hence,
antifragility should not be misconstrued as compromising safety; rather, it
enlarges its boundaries through interactively testing and evolving risk
tolerance. Appendix C discusses guidelines and risk assessment frameworks for
responsible and ethically-informed application of antifragility principles.

Antifragility does not oppose but complements robustness, benefiting as
systems dynamically navigate and strengthen through smaller challenges.
Robustness, whether through simple redundancy or safe controls, provides a
safe net enabling guided volatility. However, antifragility extends beyond
immediate reactions to stressors; it operates on longer timespans under larger
magnitudes, learning and evolving especially when robustness falters against
surprises. Hence, it develops a viable path towards _unusually_ robust systems
by dispersing risks across smaller, manageable events to preempt catastrophic
failures during rare but inevitable black swan events. Robustness sustains
today’s operations; antifragility invests in tomorrow’s upside. App. B.3
further discusses connections to related paradigms such as robustness,
lifelong learning, and cognitive science.

###  1.3 Evidence

The viability of antifragility can be observed in various natural and
engineered systems. While the internal mechanisms enabling antifragility can
be complex and may result from long-term evolution, we can identify
antifragile properties by examining a system’s response to stressors over
time. Several examples, spanning biological, economic, social, and
technological domains, are briefly described below and further detailed in
App. A.

###  1.4 Overview

The conceptual scope of this manuscript is framed through online learning, an
area with long history and contemporary relevance at the core of ODM that has
seen much cross-pollination with related fields such as optimization and
reinforcement learning (RL). We believe this provides a solid foundation for
establishing antifragility given its matching characteristic of
nonstationarity. Specifically, we leverage the well-defined dynamic regret to
provide a rigorous definition of antifragility (Sec. 3) and characterize some
fundamental impediments and open problems (Sec. 4). Through connections to RL
and its variants, representing typical interactive settings, as well as areas
like meta-learning, continual learning, and foundation models among others, we
aim to spark constructive discussions (Sec. 6), rally momentum to address gaps
(Sec. 7), and spur work towards computational antifragility.

##  2 Preliminaries

Online learning tackles sequential decisions based on live data streams,
gaining increasing relevance amid real-time needs (Hoi et al., 2021).
Foundational introductions are provided in (Shalev-Shwartz et al., 2012; Hazan
et al., 2016; Orabona, 2019). The setup involves an action set $\mathcal{A}$,
a time horizon $T$, and a sequence of cost functions
$\ell_{1},\ldots,\ell_{T}$. For any comparator $u\in\mathcal{A}$, the static
regret is:

$R_{T}(u)\triangleq\sum_{t=1}^{T}\ell_{t}(a_{t})-\sum_{t=1}^{T}\ell_{t}(u),$
(1)

where $a_{t}$ is the algorithm’s output at time $t$. Unlike static regret, the
dynamic regret is defined against a sequence of comparators
$u_{1:T}=(u_{1},\ldots,u_{T})$ as:

$R_{T}(u_{1:T})\triangleq\sum_{t=1}^{T}\ell_{t}(a_{t})-\sum_{t=1}^{T}\ell_{t}(u_{t}).$
(2)

Dynamic regret is particularly relevant in nonstationary environments. Two
common measures for modeling the environment’s stationarity include: 1) Path-
Length: Introduced by (Zinkevich, 2003), the path-length of the sequence
$u_{1:T}$ is defined as:
$V^{u}_{T}(u_{1:T})\triangleq\sum_{t=2}^{T}\rho(u_{t},u_{t-1}),$ where $\rho$
is a metric that measures the distance between $u_{t}$ and $u_{t-1}$, e.g.,
(squared) Euclidean distance. 2) Temporal Variability: Defined as the
variation in loss functions over time:
$V^{f}_{T}(\ell_{1:T})\triangleq\sum_{t=2}^{T}\max_{a\in\mathcal{V}}|\ell_{t}(a)-\ell_{t-1}(a)|$
and
$V^{g}_{T}(\ell_{1:T})\triangleq\sum_{t=2}^{T}\max_{a\in\mathcal{V}}\|\nabla\ell_{t}(a)-\nabla\ell_{t-1}(a)\|_{2}^{2}.$
Essentially, path length measures the cumulative change in the (optimal)
decision over time, and temporal variability quantifies the maximum
fluctuation in loss between consecutive time steps for any decision. A high
path length indicates the optimal decisions vary significantly over time,
while a high temporal variability reflects the extent to which the
environment’s feedback changes from one moment to the next, both of which
signal a more volatile environment.222For simplicity, we do not discuss other
performance measure variants (see (Rakhlin et al., 2011) for an excellent
exposition) or other nonstationarity measures.

Online gradient descent (OGD) is arguably the most fundamental online learning
algorithm. For introduction on this subject, we refer to (Shalev-Shwartz et
al., 2012; Hazan et al., 2016); (Orabona, 2019) provides a unified view of
many online learning algorithms as instantiations of Online Mirror Descent or
Follow-The-Regularized-Leader and their variants (Orabona, 2019). Existing
approaches reveal strengths handling real-time data and certain environment
changes (Hoi et al., 2021). However, open problems exist dealing with
nonstationarity and distribution shifts, as discussed in Sec. 4.

##  3 Regret-Based Definition of Antifragility

Antifragility, a concept going beyond resilience and robustness, refers to
systems that don’t merely withstand disorder but benefit from it. This concept
is particularly pertinent in the realm of online learning, where algorithms
continually encounter shifting data streams and evolving decision contexts.
Antifragility in this setting implies an algorithm’s ability not just to cope
with these changes but to use them as a springboard for enhanced performance
and adaptability.

Thus, in environments characterized by rapid and unpredictable changes,
minimizing static regret is insufficient as it fails to account for the
dynamic nature of the environment. Dynamic regret becomes vital as it compares
system’s performance relative to the best possible action improves over time,
despite frequent changes.

Let $U^{K}_{T}$ denote the set of comparator sequences $u_{1:T}$ where each
comparator $u_{t}$ remains constant for every block of $K$ time steps and may
change at the beginning of each subsequent block. Formally,

$\displaystyle U^{K}_{T}=\big{\\{}$ $\displaystyle u_{1:T}\mid
u_{t}=\tilde{u}_{(i)},\;\quad\text{for}\;(i-1)K<t\leq
iK,\;i\in[\mathrm{ceil}({T}/{K})]\big{\\}},$

where $\tilde{u}_{(i)}$ represents the comparator used for the $i$-th block of
$K$ periods. For instance, if $K=2$, the sequences in $U^{K}_{T}$ will be in
the form of
$\tilde{u}_{(1)},\tilde{u}_{(1)},\tilde{u}_{(2)},\tilde{u}_{(2)},\ldots,\tilde{u}_{({T/2})},\tilde{u}_{({T/2})}$.
In sequences from $U^{K}_{T}$, the volatility of the environment, as measured
by path-length or temporal variability, is $\Omega(T/K)$, i.e., the system
keeps changing. In this context, we provide a regret-based definition of
antifragility of order $K$.

######  Definition 1 (Order-$K$ Antifragility).

A system is defined as _order- $K$ antifragile_ if it achieves sublinear
dynamic regret for any sequence in $U^{K}_{T}$, i.e.,

$R_{T}^{d}(u_{1:T})=h(V_{T}),\quad\forall\;u_{1:T}\in U^{K}_{T},$ (3)

where $K$ is the _order_ and $h$ is a non-negative _strictly concave_ function
uniformly dominated by the identity function. We refer to $h$ as the
variability-response function.

######  Remark 3.1.

The set inclusion relationship $U^{K}_{T}\subseteq U^{K^{\prime}}_{T}$ for any
$K^{\prime}\leq K$ indicates that a system antifragile at a lower order
(smaller $K$) inherently possesses antifragility at higher orders (larger
$K$). Consequently, achieving order-1 antifragility against $U^{1}_{T}$ where
the comparator changes every time step is the most challenging. This scenario
is not just about mere adaptation; it transcends to a level of understanding
the system’s evolution, implying a capacity for perfect prediction. Order-1
antifragility suggests a system’s ability to anticipate and respond to the
utmost level of environmental volatility. Thus, a system that achieves
antifragility at order 0 sets a high benchmark.

######  Remark 3.2.

A potentially confusing point in various personal discussions is that the
mention of _sublinear_ dynamic regret in existing literature is often under
the condition that the variation budget is bounded or sublinear in $T$. For
instance, if $V_{T}=O(T)$, then existing bounds in the form of
$O(V_{T}^{\alpha}T^{1-\alpha})$ would be $O(T)$, which does not satisfy the
antifragility condition (since the corresponding variability response function
is linear). A few exceptions exist. For instance, (Cheng et al., 2020) can be
viewed as a reduction to static regret using smoothness constraint on dynamics
(see also Sec. 6.2 for a general framework).

######  Remark 3.3.

There are some special cases where even though the variation is linear under
one notion of nonstationarity, the system may have sublinear dynamic regret by
taking advantage of some “best of different worlds” bounds since the variation
is constant in a different notion (see the example from (Jadbabaie et al.,
2015)). We exclude these cases because we consider these cases as
approximately stationary environment.

###  3.1 Relation to (Taleb & Douady, 2013)

(Taleb & Douady, 2013) quantifies antifragility in terms of structural
properties of a stochastic payoff distribution—specifically, the sensitivity
of positive tail integrals to volatility changes—formalizing a distribution-
centric characterization (see also (Taleb & West, 2023)). This definition
focuses on a single random variable. Our proposed dynamic regret-based
antifragility definition adopts a learning-theoretic perspective applicable to
nonstationary decision-making systems. The two formalizations offer
complementary lenses—probabilistic versus adaptive. Taleb’s notion also
distinguishes downside robustness from upside gain, imposing theoretical
limits on excessive volatility that causes irrecoverable failure. Additional
aspects therein could help refine our definition further.

Our regret-based definition examines adaptive risks under volatility, aligned
with Taleb’s perspective on distributional effects of uncertainty. Recall that
well-known online-to-batch techniques relate regret bounds to expected excess
risks $\mathbb{E}_{\ell_{t}\sim\mathcal{P}}[\frac{1}{T}\ell_{t}(a)]$ w.r.t.
loss distribution $\mathcal{P}$ (e.g. (Cesa-Bianchi et al., 2004), Shalev-
Shwartz et al. (2012, Thm. 5.1), and recently (Lugosi & Neu, 2023)), as well
as high probability bounds (e.g., (Rakhlin & Sridharan, 2017; Orabona & Jun,
2023; van der Hoeven et al., 2023)). See also recent extensions to non-IID
batch settings such as stationary mixing processes (e.g. Markov chains)
(Agarwal & Duchi, 2012) and nonstationary stochastic processes (Tao & Lu,
2020). These bound the path-dependent error
$\mathbb{E}_{\ell_{T+s}}[\ell_{T+s}(a)|\\{\ell_{t}\\}_{t=1}^{T}]$ over horizon
$T+s$ suitable for forecasting metrics (Kuznetsov & Mohri, 2017).

One intriguing question is how to directly relate antifragility’s dynamic
regret to batch learning notions. To avoid trivial bounds (e.g., arising from
the greedy optimization strategy), we have to exploit the strictly concave
property of the variability-response function. One viable path is considering
improper batch learning with latent contextual information, corresponding to
an online predictor that understands latent sequence variables; this is akin
to the extension of the batch settings from Markov Decision Processes (MDPs)
(Kuznetsov & Mohri, 2017) to Partially Observable MDPs (POMDPs) (Sec. 6.2).

###  3.2 Further discussion

The most compelling theoretical aspect of antifragility lies in achieving
_strictly concave response to variability_. Current views in online learning
regards this as implausible per existing lower bounds (Sec. 4). However,
observable antifragility in natural and engineered systems necessitates
reevaluating underlying assumptions, especially concerning nonstationary
environments. This reshifts focus onto theories and algorithms capable of
adapting to and benefiting from real-world complexities.

See App. B.1 for a potential framework to quantify antifragility.

##  4 Insights from Lower Bound Analysis

Given antifragility’s assertion on strictly concave response to variability
(Definition 3), we tackle a fundamental criticism that it is theoretically
unachievable due to existing lower bounds. Hence, we provide more discussions
on lower bound analyses to reveal core obstacles—unpredictable shifts under
adversarial constructions and information-theoretic limits posed by
unstructured function classes. Rather than attempting a comprehensive review,
our focus is on reevaluating assumptions to uncover potential inductive biases
that could exceed established mathematical worst-case bounds.

Unpredictable and even adversarial shifts pose a fundamental obstacle for
antifragility. (Besbes et al., 2015) construct cost sequences with randomized,
unpredictable optima ensures no policy can reliably predict shifting minima.
Leveraging the minimax static regret bound from (Abernethy et al., 2008),
(Zhang et al., 2018) construct a piecewise constant comparator sequence with
$V_{T}^{u}$ adversarially chosen segments to yield a
$\Omega(\sqrt{T(1+V^{u}_{T})})$ lower bound. The reduction from general
sequences to piecewise constants with post-hoc adversarial minimization drives
the result (c.f., Zhang et al. (2018, Eq. 19)).

Information-theoretic factors also constrain achievable regret. (Campolongo &
Orabona, 2021) use extreme value distributions exhibit a
$\Omega(V_{T}^{\gamma})$ lower bound on dynamic regret for any
$\gamma\in(0,1)$ . This aligns with results showing sublinear dynamic regret
necessitates variation vanishing relative to time horizon Besbes et al. (2015,
Prop. 1).

(Baby & Wang, 2021) derive an information-theoretic lower bound
$\Omega(V_{T}^{u\;2/3}T^{1/3})$ matching their upper regret bound using
wavelet-based function estimation theory for Triebel and Besov classes (Donoho
& Johnstone, 1998). Such flexible function spaces with considerable spatial
inhomogeneity can model spiky bounded variation comparators. However, certain
assumptions may be overly pessimistic:

  * •

Independent wavelet coefficients (as in Besov space) may overstate
unpredictability. Real functions exhibit structured smoothness coupling among
wavelet coefficients beneficial for forecasting.

  * •

The minimax argument uses the least favorable priors concentrated on functions
with isolated sparse spikes—the hardest to estimate adaptively. Additional
restrictions such as smoothness, periodicity, or curvature can reduce the
complexity of the problem.

While no algorithm exceeds constructed adversarial unpredictability or
estimation limits mathematically, practical settings with appropriate
inductive biases could plausibly break theoretical barriers. Even adversarial
settings may have unmodeled weaknesses to exploit. For instance:

  * •

POMDP setting and its variants: Incorporating latent, unobserved states can
provide predictability to account for apparent variability. Policies could
potentially leverage these hidden modes to anticipate regime shifts (see Sec.
6.2).

  * •

Smoothness, stability, or contraction constraints: Rather than entirely
arbitrary loss functions, incorporating properties of the underlying dynamical
systems, e.g., smoothness (Cheng et al., 2020) and contraction (Lin et al.,
2023), may expose exploitable patterns within variability.

  * •

Explicitly incorporating predictability: Directly modeling predictable regime
changes could bypass limitations of adversarial assumptions (e.g., (Rakhlin &
Sridharan, 2013; Hall & Willett, 2013)).

  * •

Meta-learning settings: Providing policies with a few steps of gradient
feedback after losses are revealed can reasonably measure adaptation (see Sec.
6.5).

However, care is needed so constraints do not exclude responding to black swan
events. Smoothness and predictability may preclude “surprise” events in the
modeling stage. Rather than the raw input space, it could be more prudent to
impose structure in latent spaces (such as wavelet space (Donoho & Johnstone,
1998), which naturally decomposes functions into global and local structures).
Predictability may be lacking for heavy-tailed distributions, so the meta-
learning settings that emphasize adaptation, improvision, and recovery can
provide complementary benefits.

We believe online learning frameworks remain a core paradigm, but retaining
these formulations with use-inspired augmentations can further antifragility
goals. For instance, inherent structure may have already manifested in
physical systems due to natural laws. As an analogy, certain quadratic
constrained quadratic programming problems known to be NP-hard in general
become polynomial-time solvable when the underlying graph structure or
optimization has exploitable chordal sparsity (Lavaei & Low, 2011) or benign
landscape (Ge et al., 2016). Predictability may also be addressed through
changes outside purely algorithmic scope as part of the broader decision
stack, such as increasing sensor observation frequency in hardware components.
We advocate expanding modeling to system-level factors whenever suitable,
whether stemming from problem geometry, physics, or hardware capacities.

##  5 Significance and Impact

Many biological organisms exhibit _intrinsic_ antifragility, gaining strength
from stressors such as hypoxic conditioning or bioenergetic regeneration
(Taleb & West, 2023). Incorporating antifragility principles into clinical
practice can boost effectiveness in areas such as cancer treatment, nutrition
cycles, and exercise regimens exercise regimens (see Taleb & West (2023, Table
A1)).

However, sociotechnical infrastructure still lacks flexible adaptation to
mounting volatility and looming black swans. Modern power systems face
increasing supply and demand uncertainties, amplifying extreme stress events
risks. But legacy robustness cannot manage cascading outage threats. Despite
smart grid enhancements, fragility persists with 60% of assessment criteria
indicating high likelihood of performance degradation (Johnson & Gheorghe,
2013). Antifragility can potentially be used to improve cybersecurity, e.g.,
to detect zero-day attacks (Manzoor et al., 2024), or power system resilience
by improving fast response rate to critical load restoration (ul Abdeen et
al., 2024).

Similar reliability challenges under long tails emerge in areas like epidemic
control, supply chains, and agricultural ecosystems. Even human collectives
struggle with rare yet impactful events, as the 2008 crisis and COVID-19
pandemic. This blueprint is to pioneer ML tools to computationally induce
antifragility where intrinsically lacking.

##  6 Towards Computational Antifragility

This section identifies ML subareas that hold potential for cross-pollination
of ideas towards engineering antifragility in ML systems. The goal is sparking
discussion by exemplifying useful mechanisms and surfacing gaps through a non-
comprehensive but targeted sampling of domains.

###  6.1 Nonstationary online learning: Forecasting shifts

Nonstationary online learning is a foundational setup for studying
antifragility. As discussed in Sec. 4, reactive mechanisms (e.g., sliding
window) without consolidating experience in generalizable knowledge (e.g.,
restarting mechanism) may face fundamental limits.

A promising direction is to adopt a proactive approach by synergizing with the
field of forecasting, which has abundant important results dealing with
nonstationarity (e.g., (Kuznetsov & Mohri, 2015; Ansari et al., 2023)). Prior
works such as (Rakhlin & Sridharan, 2013) provide a viable reduction approach
that can potentially achieve sublinear dynamic regret if forecasting models
demonstrate antifragility. For nonstationary settings, recent works explore
forecasting value functions (Chandak et al., 2020), perturbations (Yu et al.,
2020a), and models (Lee et al., 2023a), showing promising
empirical/theoretical results. Another direction is multimodal approach (see
(Liang & Morency, 2023) for a recent survey), as certain modes (e.g.,
language) may exhibit more stationarity or support large-scale diversified
data for better forecasting.

###  6.2 POMDP and variants: Modeling and identifying nonstationarity

POMDPs provide a mathematical framework for sequential decision making under
uncertainty and partial observability with diverse real-world applications
(Kurniawati, 2022). POMDPs can capture nonstationarity as hidden state
evolution, epistemic uncertainty as state ambiguity, and aleatoric uncertainty
as inherent stochastic transitions/rewards. The potential reduction of dynamic
regret to a static regret with respect to an optimal policy that plans with
oracle access to hidden state means algorithms in this setting may potentially
handle order-$K$ antifragility (e.g., $K$ is the interactions needed for
inferring latent state as in decodable POMDP (Efroni et al., 2022)).

POMDPs are NP-hard in general (Papadimitriou & Tsitsiklis, 1987; Vlassis et
al., 2012), since they involve reasoning about all possible belief states of
the environment. Thus, recent works have focused on more structured,
simplified yet realistic settings. The Latent MDP (LMDP) (Kwon et al., 2021)
assumes an unrevealed latent factor identifying the MDP, closely relating to
predictive state representations (PSR) (Singh et al., 2004) that directly
model future observation probabilities conditional on past observations. (Lee
et al., 2023b) considers a similar setting called Hindsight Observable Markov
Decision Process (HOMDP) where the latent states are revealed to the learner
in hindsight and provides regret bounds $O(X)$ where $X$ is the number of
latent states. Block MDPs (Krishnamurthy et al., 2016; Du et al., 2019) and
decodable POMDP (Efroni et al., 2022) are special classes of POMDPs that
enable current (or recent) observations to exactly decode current latent
states with some hidden decoding function. Other related formulations include
hidden-model MDPs (Chades et al., 2012), multitask RL (Brunskill & Li, 2013),
multi-model MDPs (Steimle et al., 2021), contextual MDPs (Hallak et al.,
2015), factored MDPs (Huang et al., 2021; Feng et al., 2022), and Bayes-
adaptive MDPs (Zintgraf et al., 2019); strategies include clustering
(Brunskill & Li, 2013; Kwon et al., 2021), variational inference (Zintgraf et
al., 2019; Yang et al., 2019a; Ghavamzadeh et al., 2015; Huang et al., 2021),
and causal inference (Feng et al., 2022).

A key insight is the necessity of memory-based policies that leverage
histories to reason about latent states at test time. Existing methods advance
towards antifragility in the sense that if the number of latent states is
finite or sublinear, then the dynamic regret (when viewd as the static regret
with respect to optimal POMDP policy) is sublinear regardless of switching
variation budgets (Kwon et al., 2021; Efroni et al., 2022); however, they fall
short in lifelong learning under continuous perturbation that introduces new
states. Additionally, while some test-time adaptation is enabled through
training diversity, assumptions of a closed-world post-training limit
adaptation capabilities. While some broader settings have been examined (c.f.,
(Zhong et al., 2022) that include POMDPs and PSRs and provide conditions for
sample efficient learning using generic posterior sampling algorithms),
identifying realistic conditions motivated by use cases that still allow
computationally and statistically tractable learning for antifragility remains
an open question. Finally, causal RL that identifies the latent change factors
represents a promising direction for fast adaptation and interpretability
(Zeng et al., 2023).

###  6.3 Safe/Robust learning: Expanding boundaries

Safe learning aims to provide guarnatees that policies uphold constraints on
avoiding unsafe regions or performance thresholds (Gu et al., 2022b; Brunke et
al., 2022). Antifragility necessitates cautiously expanding safety boundaries,
which can be achieved via:

  * •

Safe exploration that predicts safety risks (Thomas et al., 2021; Wachi et
al., 2023), under guarantees of stability (Berkenkamp et al., 2017) or
returnability (Eysenbach et al., 2018).

  * •

Transferrable safety, like safety critics (Srinivasan et al., 2020; Bharadhwaj
et al., 2020) or safe meta-policy adaptation (Khattar et al., 2023), to
substantially reduce incidents through past experience.

  * •

Learning-based control-theoretic methods such as reachability analysis for
returnability (Bansal et al., 2017), robust control such as integral-quadratic
constraints (Megretski & Rantzer, 1997; Gu et al., 2022a), model-predictive
control (Hewing et al., 2020; Brunke et al., 2022), control-barrier functions
(Ames et al., 2019; Dawson et al., 2023), and neural Lyapunov methods (Dawson
et al., 2023).

###  6.4 Continual learning: Balancing plasticity & stability

Continual learning (also called incremental/endless/lifelong learning and
closely related to nonstationary learning) refers to the ability of acquiring,
updating, accumulating, and exploiting knowledge throughout a system’s
lifetime (Lesort et al., 2020; De Lange et al., 2021; Khetarpal et al., 2022).
The key challenge is catastrophic forgetting—adapting to new distributions
reduces prior capabilities. (Kim et al., 2022) decomposes continual learning
into within-task prediction and task-id prediction (related to OOD detection)
as necessary and sufficient conditions, providing a constructive reduction
approach. Though foundation models may suffer less forgetting (Ramasesh et
al., 2021), antifragility benefits from techniques balancing plasticity and
stability (e.g., via regularization and experience replay), especially when
updating models as novelty inevitably arises. Also, the field has recently
focused on generalization (Wang et al., 2023), a crucial element for
antifragility to enable improvisation.

###  6.5 Meta learning: Fast adaptation

Meta-learning (also called learning-to-learn) focuses on rapidly adapting to
new tasks by acquiring reusable knowledge about the learning process itself
(Hospedales et al., 2021; Beck et al., 2023). Approaches can be generally
viewed as a primary (outer level) system progressively improving the learning
of a secondary (inner level) system, where the inner system performs
sequentially arriving tasks while the outer system does adaptation using
algorithms like RL (Duan et al., 2016; Wang et al., 2016), online learning
(Khodak et al., 2019), Blackwell approachability strategies (Niazadeh et al.,
2021), and evolution strategies (Houthooft et al., 2018; Lu et al., 2022;
Khattar & Jin, 2023). Memory-based meta-learning offers computational
shortcuts to difficult probabilistic inferences for acceleration (Ortega et
al., 2019) (c.f., Sec. 6.2). In-context learning (ICL) (Brown et al., 2020)
represents a recent paradigm that does not require parameter updates at test
time (see Sec. 6.6).

Black swan events often involve solving hard optimization problems due to the
high-dimensional search space and nonconvex landscape; learning a
representation such as a latent low-dimensional manifold (Sel et al., 2023) or
latent variable identifier (Yang et al., 2019a) can facilitate efficient
search and even single attempt success. Furthermore, while recent works such
as (Khattar et al., 2023) exhibit certain properties of $K$-order
antifragility by embedding a sequence of safe RL problems within an online
learning framework (where $K$ is the number of within-task steps) in an
adversarial setting, further considering the predictability of the environment
and leveraging memory in the primary learning system can help break the lower
bound for the reason outlined in Sec. 4. An important goal is to perform well
in a single attempt,

###  6.6 Foundation models: Pretraining and memory

Foundation models have shown impressive capabilities in various problems
(Bommasani et al., 2021), which crucially depend on pretraining on vast and
diverse tasks to gain on-the-fly adaptation, i.e., in-context learning (ICL).
Recent works extend this paradigm to online decision making such as adaptive
agent (AdA) (Bauer et al., 2023), algorithm distillation (Laskin et al.,
2022), Decision-Pretrained Transformer (DPT) (Lee et al., 2023c), and Gato
(Reed et al., 2022). Notably, the training resembles next-token prediction and
is often as simple as supervised learning with autoregressive loss (Laskin et
al., 2022; Reed et al., 2022; Lee et al., 2023c) (though the history-dependent
policy can be viewed as a computational posterior sampling for POMDPs (Osband
et al., 2013; Lee et al., 2023c)).

For test-time adaptation, memory architectures such as Neural Turing Machines
(Graves et al., 2014) offer a direct approach, with learning involving
discovery and encoding of abstract rules. Retrieval-augmented generation (as
surveyed in (Gao et al., 2023; Zhao et al., 2023)) offers prospects for
rapidly selecting and improvising from existing skill repertoires.

###  6.7 QD and MO learning: Adaptive repertoire

A key imperative for managing OOD events is rapidly searching for compensatory
behaviors in unfamiliar scenarios, such as a robot adopting alternatives when
damaged (Cully et al., 2015). Antifragile systems thus have to judiciously
leverage natural perturbations to gather diverse response repertoires.

Quality-diversity (QD) formalizes such expansion—optimizing solution archives
over behavior descriptors to maintain diversity (Chatzilygeroudis et al.,
2021). Mathematically, given a solution space $\Pi$, QD considers an objective
$f_{0}(\pi)$ for $\pi\in\Pi$ and behavior descriptors $f_{i}(\pi)$, $\forall
i\in[k]$ capturing solution properties, e.g., reversibility, information
gathering, robustness, safety compliance, and simplicity. Recent methods move
towards more open-ended domains with automatic descriptor discovery using low-
dimensional embeddings (Grillotti & Cully, 2022), with feedback provided by
human (Ding et al., 2023) or AI (Bradley et al., 2023). For antifragility,
online QD by synergizing with continual/meta/online learnings can dynamically
maintain a diverse repertoire to retrieve or improvise appropriate strategies,
e.g., substituting alternative behaviors when physical capacities degrade;
switching to safety-preserving actions on constraint changes; activating
uncertainty-robust policies amid chaos; and recruiting streamlined approaches
when data is scarce post-disruptions.

Multiobjective (MO) learning provides a set of tools that can complement QD to
balance potentially conflicting behavioral descriptors (Yu et al., 2020b; Liu
et al., 2021; Gu et al., 2024), directly align objectives with preferences
(Mahapatra & Rajan, 2020; Chen & Kwok, 2022), and maintain a set of Pareto
optimal policy repertoire for test-time adaptation (Yang et al., 2019b; Navon
et al., 2020; Lin et al., 2022; Kyriakis & Deshmukh, 2022; Dimitriadis et al.,
2023).

###  6.8 Adversarial ML: Vaccination by attack

Adversarial ML involves exposing systems to deliberately designed inputs to
induce failures and understand robustness/fragility (Hendrycks et al., 2021b).
In addition to existing robustness benchmarks (Hendrycks & Dietterich, 2019;
Hendrycks et al., 2021a; Koh et al., 2021), researchers could create more
environments to stress-test systems (e.g., autonomous driving (Tang et al.,
2023)) and estimate the antifragility regret. For instance, competitions like
the AI Village at DEFCON conferences enable wide discovery of harms from
public participation to address them (The White House, 2024). Using such
benchmarks and environments, researchers could improve ML systems to withstand
black swans, long tails, and structurally novel events. Another interesting
direction could be antifragility training, where exposing systems to a variety
of attacks during training may lead to a better understanding of potential new
attacks, as explored by Manzoor et al. (2024) for zero-day attack detection.

##  7 Call for Actions and Future Directions

To operate reliably in open-world high-stakes environments, ML systems need to
endure unforeseen events and tail risks. While there has been significant
progress in ODM under nonstationarity, vast opportunities remain untapped for
channeling volatility into intrinsic asset rather than solely a fragility
risk. This necessitates an open-system perspective that sustains safety
expansions under distributional shifts. Antifragility constitutes pioneering
conceptual and computational upgrades to mitigate long-term risks, focusing on
_unusual robustness_ against black swan events.

Significant open challenges remain in architecting decision systems to
reliably _improve_ under distributional shifts. Core future pursuits involve:

  * •

Developing practical algorithms actualizing key antifragility principles, such
as leveraging short-term predictability and embracing small, controlled
failures/risks hedge against catastrophic failures, accumulating options and
strategic flexibility, rapidly adapting and agile improvision, distilling
invariant structures and generalizable patterns from diverse experience, and
proactive inoculation against outliers with an emergence of generalization
capabilities.

  * •

Formalizing foundations for antifragility guarantees grounded in learning
theories so systems demonstrably gain from volatility through principled
mechanisms, while ensuring safe exploration and adaptation. This requires
characterizing the “value of volatility” (c.f., the psychology counterpart of
challenge-hindrance framework (O’Brien & Beehr, 2019)) for selective exposures
under controlled experimentations.

  * •

Architecting entire decision-making stack (spanning ML components, physical
computing infrastructure, and downstream applications) with end-to-end
antifragility. Priorities involve composing modular system architectures that
sustain diversity of observational sensing, behavioral responses, and safety
constraints. This prevents isolated fragilities from cascading across layers
and retains operation despite disruptions. Foundation models have potential
for transfer and adaptation but require expanded competencies and
consolidation mechanisms to mitigate negative interference and catastrophic
forgetting.

  * •

Developing benchmarks and simulation environments that calibrate controllable
uncertainty injections for reliably quantifying volatility-related performance
improvements, especially as measured by long-tail high-impact events.

Progress demands constructive engagement across computational subfields of ML,
optimization, and control communities, deliberate involvement of
interdisciplinary researchers, and proactive participation with stakeholders.

## References

  * Abernethy et al. (2008) Abernethy, J., Bartlett, P., Rakhlin, A., and Tewari, A.  Optimal strategies and minimax lower bounds for online convex games.  2008\.
  * Agarwal & Duchi (2012) Agarwal, A. and Duchi, J. C.  The generalization ability of online algorithms for dependent data.  _IEEE Transactions on Information Theory_ , 59(1):573–587, 2012.
  * Ames et al. (2019) Ames, A. D., Coogan, S., Egerstedt, M., Notomista, G., Sreenath, K., and Tabuada, P.  Control barrier functions: Theory and applications.  In _2019 18th European control conference (ECC)_ , pp. 3420–3431. IEEE, 2019.
  * Ansari et al. (2023) Ansari, A. F., Heng, A., Lim, A., and Soh, H.  Neural continuous-discrete state space models for irregularly-sampled time series.  _arXiv preprint arXiv:2301.11308_ , 2023.
  * Baby & Wang (2021) Baby, D. and Wang, Y.-X.  Optimal dynamic regret in exp-concave online learning.  In _Conference on Learning Theory_ , pp. 359–409. PMLR, 2021.
  * Bansal et al. (2017) Bansal, S., Chen, M., Herbert, S., and Tomlin, C. J.  Hamilton-jacobi reachability: A brief overview and recent advances.  In _2017 IEEE 56th Annual Conference on Decision and Control (CDC)_ , pp. 2242–2253. IEEE, 2017.
  * Bauer et al. (2023) Bauer, J., Baumli, K., Behbahani, F., Bhoopchand, A., Bradley-Schmieg, N., Chang, M., Clay, N., Collister, A., Dasagi, V., Gonzalez, L., et al.  Human-timescale adaptation in an open-ended task space.  In _International Conference on Machine Learning_ , pp. 1887–1935. PMLR, 2023.
  * Beck et al. (2023) Beck, J., Vuorio, R., Liu, E. Z., Xiong, Z., Zintgraf, L., Finn, C., and Whiteson, S.  A survey of meta-reinforcement learning.  _arXiv preprint arXiv:2301.08028_ , 2023.
  * Berkenkamp et al. (2017) Berkenkamp, F., Turchetta, M., Schoellig, A., and Krause, A.  Safe model-based reinforcement learning with stability guarantees.  _Advances in neural information processing systems_ , 30, 2017.
  * Besbes et al. (2015) Besbes, O., Gur, Y., and Zeevi, A.  Non-stationary stochastic optimization.  _Operations research_ , 63(5):1227–1244, 2015.
  * Bharadhwaj et al. (2020) Bharadhwaj, H., Kumar, A., Rhinehart, N., Levine, S., Shkurti, F., and Garg, A.  Conservative safety critics for exploration.  In _International Conference on Learning Representations_ , 2020.
  * Bommasani et al. (2021) Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al.  On the opportunities and risks of foundation models.  _arXiv preprint arXiv:2108.07258_ , 2021.
  * Bradley et al. (2023) Bradley, H., Dai, A., Teufel, H., Zhang, J., Oostermeijer, K., Bellagente, M., Clune, J., Stanley, K., Schott, G., and Lehman, J.  Quality-diversity through ai feedback.  _arXiv preprint arXiv:2310.13032_ , 2023.
  * Brown et al. (2020) Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al.  Language models are few-shot learners.  _Advances in neural information processing systems_ , 33:1877–1901, 2020.
  * Brunke et al. (2022) Brunke, L., Greeff, M., Hall, A. W., Yuan, Z., Zhou, S., Panerati, J., and Schoellig, A. P.  Safe learning in robotics: From learning-based control to safe reinforcement learning.  _Annual Review of Control, Robotics, and Autonomous Systems_ , 5:411–444, 2022.
  * Brunskill & Li (2013) Brunskill, E. and Li, L.  Sample complexity of multi-task reinforcement learning.  _arXiv preprint arXiv:1309.6821_ , 2013.
  * Campolongo & Orabona (2021) Campolongo, N. and Orabona, F.  A closer look at temporal variability in dynamic online learning.  _arXiv preprint arXiv:2102.07666_ , 2021.
  * Cesa-Bianchi et al. (2004) Cesa-Bianchi, N., Conconi, A., and Gentile, C.  On the generalization ability of on-line learning algorithms.  _IEEE Transactions on Information Theory_ , 50(9):2050–2057, 2004.
  * Chades et al. (2012) Chades, I., Carwardine, J., Martin, T., Nicol, S., Sabbadin, R., and Buffet, O.  Momdps: a solution for modelling adaptive management problems.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 26, pp. 267–273, 2012.
  * Chandak et al. (2020) Chandak, Y., Theocharous, G., Shankar, S., White, M., Mahadevan, S., and Thomas, P.  Optimizing for the future in non-stationary mdps.  In _International Conference on Machine Learning_ , pp. 1414–1425. PMLR, 2020.
  * Chatzilygeroudis et al. (2021) Chatzilygeroudis, K., Cully, A., Vassiliades, V., and Mouret, J.-B.  Quality-diversity optimization: a novel branch of stochastic optimization.  In _Black Box Optimization, Machine Learning, and No-Free Lunch Theorems_ , pp. 109–135. Springer, 2021.
  * Chen & Kwok (2022) Chen, W. and Kwok, J.  Multi-objective deep learning with adaptive reference vectors.  _Advances in Neural Information Processing Systems_ , 35:32723–32735, 2022.
  * Cheng et al. (2020) Cheng, C.-A., Lee, J., Goldberg, K., and Boots, B.  Online learning with continuous variations: Dynamic regret and reductions.  In _International Conference on Artificial Intelligence and Statistics_ , pp. 2218–2228. PMLR, 2020.
  * Cully et al. (2015) Cully, A., Clune, J., Tarapore, D., and Mouret, J.-B.  Robots that can adapt like animals.  _Nature_ , 521(7553):503–507, 2015.
  * Dawson et al. (2023) Dawson, C., Gao, S., and Fan, C.  Safe control with learned certificates: A survey of neural lyapunov, barrier, and contraction methods for robotics and control.  _IEEE Transactions on Robotics_ , 2023.
  * De Lange et al. (2021) De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., and Tuytelaars, T.  A continual learning survey: Defying forgetting in classification tasks.  _IEEE transactions on pattern analysis and machine intelligence_ , 44(7):3366–3385, 2021.
  * Dimitriadis et al. (2023) Dimitriadis, N., Frossard, P., and Fleuret, F.  Pareto manifold learning: Tackling multiple tasks via ensembles of single-task models.  In _International Conference on Machine Learning_ , pp. 8015–8052. PMLR, 2023.
  * Ding et al. (2023) Ding, L., Zhang, J., Clune, J., Spector, L., and Lehman, J.  Quality diversity through human feedback.  _arXiv preprint arXiv:2310.12103_ , 2023.
  * Donoho & Johnstone (1998) Donoho, D. L. and Johnstone, I. M.  Minimax estimation via wavelet shrinkage.  _The annals of Statistics_ , 26(3):879–921, 1998.
  * Du et al. (2019) Du, S., Krishnamurthy, A., Jiang, N., Agarwal, A., Dudik, M., and Langford, J.  Provably efficient rl with rich observations via latent state decoding.  In _International Conference on Machine Learning_ , pp. 1665–1674. PMLR, 2019.
  * Duan et al. (2016) Duan, Y., Schulman, J., Chen, X., Bartlett, P. L., Sutskever, I., and Abbeel, P.  Rl2: Fast reinforcement learning via slow reinforcement learning.  _arXiv preprint arXiv:1611.02779_ , 2016.
  * Efroni et al. (2022) Efroni, Y., Jin, C., Krishnamurthy, A., and Miryoosefi, S.  Provable reinforcement learning with a short-term memory.  In _International Conference on Machine Learning_ , pp. 5832–5850. PMLR, 2022.
  * Eysenbach et al. (2018) Eysenbach, B., Gu, S., Ibarz, J., and Levine, S.  Leave no trace: Learning to reset for safe and autonomous reinforcement learning.  In _International Conference on Learning Representations_ , 2018.
  * Feng et al. (2022) Feng, F., Huang, B., Zhang, K., and Magliacane, S.  Factored adaptation for non-stationary reinforcement learning.  _Advances in Neural Information Processing Systems_ , 35:31957–31971, 2022.
  * Folke et al. (2004) Folke, C., Carpenter, S., Walker, B., Scheffer, M., Elmqvist, T., Gunderson, L., and Holling, C. S.  Regime shifts, resilience, and biodiversity in ecosystem management.  _Annu. Rev. Ecol. Evol. Syst._ , 35:557–581, 2004.
  * Gammaitoni et al. (1998) Gammaitoni, L., Hänggi, P., Jung, P., and Marchesoni, F.  Stochastic resonance.  _Reviews of modern physics_ , 70(1):223, 1998.
  * Gao et al. (2023) Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., and Wang, H.  Retrieval-augmented generation for large language models: A survey.  _arXiv preprint arXiv:2312.10997_ , 2023.
  * Ge et al. (2016) Ge, R., Lee, J. D., and Ma, T.  Matrix completion has no spurious local minimum.  _Advances in neural information processing systems_ , 29, 2016.
  * Ghavamzadeh et al. (2015) Ghavamzadeh, M., Mannor, S., Pineau, J., Tamar, A., et al.  Bayesian reinforcement learning: A survey.  _Foundations and Trends® in Machine Learning_ , 8(5-6):359–483, 2015.
  * Graves et al. (2014) Graves, A., Wayne, G., and Danihelka, I.  Neural turing machines.  _arXiv preprint arXiv:1410.5401_ , 2014.
  * Grillotti & Cully (2022) Grillotti, L. and Cully, A.  Unsupervised behavior discovery with quality-diversity optimization.  _IEEE Transactions on Evolutionary Computation_ , 26(6):1539–1552, 2022.
  * Gu et al. (2022a) Gu, F., Yin, H., El Ghaoui, L., Arcak, M., Seiler, P., and Jin, M.  Recurrent neural network controllers synthesis with stability guarantees for partially observed systems.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 36, pp. 5385–5394, 2022a.
  * Gu et al. (2022b) Gu, S., Yang, L., Du, Y., Chen, G., Walter, F., Wang, J., Yang, Y., and Knoll, A.  A review of safe reinforcement learning: Methods, theory and applications.  _arXiv preprint arXiv:2205.10330_ , 2022b.
  * Gu et al. (2024) Gu, S., Sel, B., Ding, Y., Wang, L., Lin, Q., Jin, M., and Knoll, A.  Balance reward and safety optimization for safe reinforcement learning: A perspective of gradient manipulation.  In _AAAI Conference on Artificial Intelligence (AAAI)_ , 2024.
  * Hall & Willett (2013) Hall, E. and Willett, R.  Dynamical models and tracking regret in online convex programming.  In _International Conference on Machine Learning_ , pp. 579–587. PMLR, 2013.
  * Hallak et al. (2015) Hallak, A., Di Castro, D., and Mannor, S.  Contextual markov decision processes.  _arXiv preprint arXiv:1502.02259_ , 2015.
  * Hayek (2013) Hayek, F. A.  The use of knowledge in society.  In _Modern Understandings of Liberty and Property_ , pp. 27–38. Routledge, 2013.
  * Hazan et al. (2016) Hazan, E. et al.  Introduction to online convex optimization.  _Foundations and Trends® in Optimization_ , 2(3-4):157–325, 2016.
  * Hendrycks & Dietterich (2019) Hendrycks, D. and Dietterich, T.  Benchmarking neural network robustness to common corruptions and perturbations.  _arXiv preprint arXiv:1903.12261_ , 2019.
  * Hendrycks et al. (2021a) Hendrycks, D., Basart, S., Mu, N., Kadavath, S., Wang, F., Dorundo, E., Desai, R., Zhu, T., Parajuli, S., Guo, M., et al.  The many faces of robustness: A critical analysis of out-of-distribution generalization.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pp. 8340–8349, 2021a.
  * Hendrycks et al. (2021b) Hendrycks, D., Carlini, N., Schulman, J., and Steinhardt, J.  Unsolved problems in ml safety.  _arXiv preprint arXiv:2109.13916_ , 2021b.
  * Hewing et al. (2020) Hewing, L., Wabersich, K. P., Menner, M., and Zeilinger, M. N.  Learning-based model predictive control: Toward safe learning in control.  _Annual Review of Control, Robotics, and Autonomous Systems_ , 3:269–296, 2020.
  * Hoi et al. (2021) Hoi, S. C., Sahoo, D., Lu, J., and Zhao, P.  Online learning: A comprehensive survey.  _Neurocomputing_ , 459:249–289, 2021.
  * Hospedales et al. (2021) Hospedales, T., Antoniou, A., Micaelli, P., and Storkey, A.  Meta-learning in neural networks: A survey.  _IEEE transactions on pattern analysis and machine intelligence_ , 44(9):5149–5169, 2021.
  * Houthooft et al. (2018) Houthooft, R., Chen, Y., Isola, P., Stadie, B., Wolski, F., Jonathan Ho, O., and Abbeel, P.  Evolved policy gradients.  _Advances in Neural Information Processing Systems_ , 31, 2018.
  * Huang et al. (2021) Huang, B., Feng, F., Lu, C., Magliacane, S., and Zhang, K.  Adarl: What, where, and how to adapt in transfer reinforcement learning.  In _International Conference on Learning Representations_ , 2021.
  * Hughes et al. (2003) Hughes, T. P., Baird, A. H., Bellwood, D. R., Card, M., Connolly, S. R., Folke, C., Grosberg, R., Hoegh-Guldberg, O., Jackson, J. B., Kleypas, J., et al.  Climate change, human impacts, and the resilience of coral reefs.  _science_ , 301(5635):929–933, 2003.
  * Jadbabaie et al. (2015) Jadbabaie, A., Rakhlin, A., Shahrampour, S., and Sridharan, K.  Online optimization: Competing with dynamic comparators.  In _Artificial Intelligence and Statistics_ , pp. 398–406. PMLR, 2015.
  * Johnson & Gheorghe (2013) Johnson, J. and Gheorghe, A. V.  Antifragility analysis and measurement framework for systems of systems.  _International journal of disaster risk science_ , 4:159–168, 2013.
  * Khattar & Jin (2023) Khattar, V. and Jin, M.  Winning the citylearn challenge: adaptive optimization with evolutionary search under trajectory-based guidance.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 37, pp. 14286–14294, 2023.
  * Khattar et al. (2023) Khattar, V., Ding, Y., Sel, B., Lavaei, J., and Jin, M.  A cmdp-within-online framework for meta-safe reinforcement learning.  In _The Eleventh International Conference on Learning Representations_ , 2023.
  * Khetarpal et al. (2022) Khetarpal, K., Riemer, M., Rish, I., and Precup, D.  Towards continual reinforcement learning: A review and perspectives.  _Journal of Artificial Intelligence Research_ , 75:1401–1476, 2022.
  * Khodak et al. (2019) Khodak, M., Balcan, M.-F. F., and Talwalkar, A. S.  Adaptive gradient-based meta-learning methods.  _Advances in Neural Information Processing Systems_ , 32, 2019.
  * Kim et al. (2022) Kim, G., Xiao, C., Konishi, T., Ke, Z., and Liu, B.  A theoretical study on solving continual learning.  _Advances in Neural Information Processing Systems_ , 35:5065–5079, 2022.
  * Koh et al. (2021) Koh, P. W., Sagawa, S., Marklund, H., Xie, S. M., Zhang, M., Balsubramani, A., Hu, W., Yasunaga, M., Phillips, R. L., Gao, I., et al.  Wilds: A benchmark of in-the-wild distribution shifts.  In _International Conference on Machine Learning_ , pp. 5637–5664. PMLR, 2021.
  * Krishnamurthy et al. (2016) Krishnamurthy, A., Agarwal, A., and Langford, J.  Pac reinforcement learning with rich observations.  _Advances in Neural Information Processing Systems_ , 29, 2016.
  * Kurniawati (2022) Kurniawati, H.  Partially observable markov decision processes and robotics.  _Annual Review of Control, Robotics, and Autonomous Systems_ , 5:253–277, 2022.
  * Kuznetsov & Mohri (2015) Kuznetsov, V. and Mohri, M.  Learning theory and algorithms for forecasting non-stationary time series.  _Advances in neural information processing systems_ , 28, 2015.
  * Kuznetsov & Mohri (2017) Kuznetsov, V. and Mohri, M.  Generalization bounds for non-stationary mixing processes.  _Machine Learning_ , 106(1):93–117, 2017.
  * Kwon et al. (2021) Kwon, J., Efroni, Y., Caramanis, C., and Mannor, S.  Rl for latent mdps: Regret guarantees and a lower bound.  _Advances in Neural Information Processing Systems_ , 34:24523–24534, 2021.
  * Kyriakis & Deshmukh (2022) Kyriakis, P. and Deshmukh, J.  Pareto policy adaptation.  In _International Conference on Learning Representations_ , volume 2022, 2022.
  * Laskin et al. (2022) Laskin, M., Wang, L., Oh, J., Parisotto, E., Spencer, S., Steigerwald, R., Strouse, D., Hansen, S. S., Filos, A., Brooks, E., et al.  In-context reinforcement learning with algorithm distillation.  In _The Eleventh International Conference on Learning Representations_ , 2022.
  * Lavaei & Low (2011) Lavaei, J. and Low, S. H.  Zero duality gap in optimal power flow problem.  _IEEE Transactions on Power systems_ , 27(1):92–107, 2011.
  * Lee et al. (2023a) Lee, H., Ding, Y., Lee, J., Jin, M., Lavaei, J., and Sojoudi, S.  Tempo adaptation in non-stationary reinforcement learning.  In _Thirty-seventh Conference on Neural Information Processing Systems_ , 2023a.
  * Lee et al. (2023b) Lee, J., Agarwal, A., Dann, C., and Zhang, T.  Learning in pomdps is sample-efficient with hindsight observability.  In _International Conference on Machine Learning_ , pp. 18733–18773. PMLR, 2023b.
  * Lee et al. (2023c) Lee, J. N., Xie, A., Pacchiano, A., Chandak, Y., Finn, C., Nachum, O., and Brunskill, E.  Supervised pretraining can learn in-context reinforcement learning.  _arXiv preprint arXiv:2306.14892_ , 2023c.
  * Lesort et al. (2020) Lesort, T., Lomonaco, V., Stoian, A., Maltoni, D., Filliat, D., and Díaz-Rodríguez, N.  Continual learning for robotics: Definition, framework, learning strategies, opportunities and challenges.  _Information fusion_ , 58:52–68, 2020.
  * Liang & Morency (2023) Liang, P. P. and Morency, L.-P.  Tutorial on multimodal machine learning: Principles, challenges, and open questions.  In _Companion Publication of the 25th International Conference on Multimodal Interaction_ , pp. 101–104, 2023.
  * Lin et al. (2022) Lin, X., Yang, Z., Zhang, X., and Zhang, Q.  Pareto set learning for expensive multi-objective optimization.  _Advances in Neural Information Processing Systems_ , 35:19231–19247, 2022.
  * Lin et al. (2023) Lin, Y., Preiss, J. A., Anand, E. T., Li, Y., Yue, Y., and Wierman, A.  Online adaptive policy selection in time-varying systems: No-regret via contractive perturbations.  In _Thirty-seventh Conference on Neural Information Processing Systems_ , 2023.
  * Liu et al. (2021) Liu, B., Liu, X., Jin, X., Stone, P., and Liu, Q.  Conflict-averse gradient descent for multi-task learning.  _Advances in Neural Information Processing Systems_ , 34:18878–18890, 2021.
  * Lu et al. (2022) Lu, C., Kuba, J., Letcher, A., Metz, L., Schroeder de Witt, C., and Foerster, J.  Discovered policy optimisation.  _Advances in Neural Information Processing Systems_ , 35:16455–16468, 2022.
  * Lugosi & Neu (2023) Lugosi, G. and Neu, G.  Online-to-pac conversions: Generalization bounds via regret analysis.  _arXiv preprint arXiv:2305.19674_ , 2023.
  * Mahapatra & Rajan (2020) Mahapatra, D. and Rajan, V.  Multi-task learning with user preferences: Gradient descent with controlled ascent in pareto optimization.  In _International Conference on Machine Learning_ , pp. 6597–6607. PMLR, 2020.
  * Manifesto (2001) Manifesto, A.  Manifesto for agile software development.  2001\.
  * Manzoor et al. (2024) Manzoor, F., Khattar, V., Liu, C.-C., and Jin, M.  Zero-day attack detection in digital substations using in-context learning.  Technical report, 2024.  URL http://www.jinming.tech/papers/ZDA-detect-2024.pdf.
  * Mattson (2008) Mattson, M. P.  Hormesis defined.  _Ageing research reviews_ , 7(1):1–7, 2008.
  * McGrath (1999) McGrath, R. G.  Falling forward: Real options reasoning and entrepreneurial failure.  _Academy of Management review_ , 24(1):13–30, 1999.
  * Megretski & Rantzer (1997) Megretski, A. and Rantzer, A.  System analysis via integral quadratic constraints.  _IEEE transactions on automatic control_ , 42(6):819–830, 1997.
  * Nair et al. (2022) Nair, J., Wierman, A., and Zwart, B.  _The fundamentals of heavy tails: Properties, emergence, and estimation_ , volume 53.  Cambridge University Press, 2022.
  * Navon et al. (2020) Navon, A., Shamsian, A., Fetaya, E., and Chechik, G.  Learning the pareto front with hypernetworks.  In _International Conference on Learning Representations_ , 2020.
  * Niazadeh et al. (2021) Niazadeh, R., Golrezaei, N., Wang, J. R., Susan, F., and Badanidiyuru, A.  Online learning via offline greedy algorithms: Applications in market design and optimization.  In _Proceedings of the 22nd ACM Conference on Economics and Computation_ , pp. 737–738, 2021.
  * Norris et al. (2008) Norris, F. H., Stevens, S. P., Pfefferbaum, B., Wyche, K. F., and Pfefferbaum, R. L.  Community resilience as a metaphor, theory, set of capacities, and strategy for disaster readiness.  _American journal of community psychology_ , 41:127–150, 2008.
  * O’Brien & Beehr (2019) O’Brien, K. E. and Beehr, T. A.  So far, so good: Up to now, the challenge–hindrance framework describes a practical and accurate distinction.  _Journal of Organizational Behavior_ , 40(8):962–972, 2019.
  * Orabona (2019) Orabona, F.  A modern introduction to online learning.  _arXiv preprint arXiv:1912.13213_ , 2019.
  * Orabona & Jun (2023) Orabona, F. and Jun, K.-S.  Tight concentrations and confidence sequences from the regret of universal portfolio.  _IEEE Transactions on Information Theory_ , 2023.
  * Ortega et al. (2019) Ortega, P. A., Wang, J. X., Rowland, M., Genewein, T., Kurth-Nelson, Z., Pascanu, R., Heess, N., Veness, J., Pritzel, A., Sprechmann, P., et al.  Meta-learning of sequential strategies.  _arXiv preprint arXiv:1905.03030_ , 2019.
  * Osband et al. (2013) Osband, I., Russo, D., and Van Roy, B.  (more) efficient reinforcement learning via posterior sampling.  _Advances in Neural Information Processing Systems_ , 26, 2013.
  * Ott et al. (1990) Ott, E., Grebogi, C., and Yorke, J. A.  Controlling chaos.  _Physical review letters_ , 64(11):1196, 1990.
  * Page (2008) Page, S.  _The difference: How the power of diversity creates better groups, firms, schools, and societies-new edition_.  Princeton University Press, 2008.
  * Papadimitriou & Tsitsiklis (1987) Papadimitriou, C. H. and Tsitsiklis, J. N.  The complexity of markov decision processes.  _Mathematics of operations research_ , 12(3):441–450, 1987.
  * Plotkin (2005) Plotkin, S. A.  Vaccines: past, present and future.  _Nature medicine_ , 11(Suppl 4):S5–S11, 2005.
  * Rakhlin & Sridharan (2013) Rakhlin, A. and Sridharan, K.  Online learning with predictable sequences.  In _Conference on Learning Theory_ , pp. 993–1019. PMLR, 2013.
  * Rakhlin & Sridharan (2017) Rakhlin, A. and Sridharan, K.  On equivalence of martingale tail bounds and deterministic regret inequalities.  In _Conference on Learning Theory_ , pp. 1704–1722. PMLR, 2017.
  * Rakhlin et al. (2011) Rakhlin, A., Sridharan, K., and Tewari, A.  Online learning: Beyond regret.  In _Proceedings of the 24th Annual Conference on Learning Theory_ , pp. 559–594. JMLR Workshop and Conference Proceedings, 2011.
  * Ramasesh et al. (2021) Ramasesh, V. V., Lewkowycz, A., and Dyer, E.  Effect of scale on catastrophic forgetting in neural networks.  In _International Conference on Learning Representations_ , 2021.
  * Reed et al. (2022) Reed, S., Zolna, K., Parisotto, E., Colmenarejo, S. G., Novikov, A., Barth-maron, G., Giménez, M., Sulsky, Y., Kay, J., Springenberg, J. T., et al.  A generalist agent.  _Transactions on Machine Learning Research_ , 2022.
  * Roese & Epstude (2017) Roese, N. J. and Epstude, K.  The functional theory of counterfactual thinking: New evidence, new challenges, new insights.  In _Advances in experimental social psychology_ , volume 56, pp. 1–79. Elsevier, 2017.
  * Ryan et al. (2013) Ryan, K. C., Knapp, E. E., and Varner, J. M.  Prescribed fire in north american forests and woodlands: history, current practice, and challenges.  _Frontiers in Ecology and the Environment_ , 11(s1):e15–e24, 2013.
  * Schoenfeld (2010) Schoenfeld, B. J.  The mechanisms of muscle hypertrophy and their application to resistance training.  _The Journal of Strength & Conditioning Research_, 24(10):2857–2872, 2010.
  * Schöll & Schuster (2008) Schöll, E. and Schuster, H. G.  Handbook of chaos control.  2008\.
  * Sel et al. (2023) Sel, B., Tawaha, A., Ding, Y., Jia, R., Ji, B., Lavaei, J., and Jin, M.  Learning-to-learn to guide random search: Derivative-free meta blackbox optimization on manifold.  In _Learning for Dynamics and Control Conference_ , pp. 38–50. PMLR, 2023.
  * Shalev-Shwartz et al. (2012) Shalev-Shwartz, S. et al.  Online learning and online convex optimization.  _Foundations and Trends® in Machine Learning_ , 4(2):107–194, 2012.
  * Singh et al. (2004) Singh, S., James, M. R., and Rudary, M. R.  Predictive state representations: a new theory for modeling dynamical systems.  In _Proceedings of the 20th conference on Uncertainty in artificial intelligence_ , pp. 512–519, 2004.
  * Srinivasan et al. (2020) Srinivasan, K., Eysenbach, B., Ha, S., Tan, J., and Finn, C.  Learning to be safe: Deep rl with a safety critic.  _arXiv preprint arXiv:2010.14603_ , 2020.
  * Steimle et al. (2021) Steimle, L. N., Kaufman, D. L., and Denton, B. T.  Multi-model markov decision processes.  _IISE Transactions_ , 53(10):1124–1139, 2021.
  * Taleb (2010) Taleb, N. N.  _The Black Swan:: The Impact of the Highly Improbable: With a new section:” On Robustness and Fragility”_ , volume 2.  Random house trade paperbacks, 2010.
  * Taleb (2014) Taleb, N. N.  _Antifragile: Things that gain from disorder_ , volume 3.  Random House Trade Paperbacks, 2014.
  * Taleb (2018) Taleb, N. N.  (anti) fragility and convex responses in medicine.  In _Unifying Themes in Complex Systems IX: Proceedings of the Ninth International Conference on Complex Systems 9_ , pp. 299–325. Springer, 2018.
  * Taleb & Douady (2013) Taleb, N. N. and Douady, R.  Mathematical definition, mapping, and detection of (anti) fragility.  _Quantitative Finance_ , 13(11):1677–1689, 2013.
  * Taleb & West (2023) Taleb, N. N. and West, J.  Working with convex responses: Antifragility from finance to oncology.  _Entropy_ , 25(2):343, 2023.
  * Tang et al. (2023) Tang, S., Zhang, Z., Zhang, Y., Zhou, J., Guo, Y., Liu, S., Guo, S., Li, Y.-F., Ma, L., Xue, Y., et al.  A survey on automated driving system testing: Landscapes and trends.  _ACM Transactions on Software Engineering and Methodology_ , 2023.
  * Tao & Lu (2020) Tao, Y. and Lu, S.  From online to non-iid batch learning.  In _Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_, pp. 328–337, 2020.
  * Tedeschi & Calhoun (2004) Tedeschi, R. G. and Calhoun, L. G.  ” posttraumatic growth: conceptual foundations and empirical evidence”.  _Psychological inquiry_ , 15(1):1–18, 2004.
  * The White House (2024) The White House.  Red-teaming large language models to identify novel ai risks, 2024.  URL https://www.whitehouse.gov/ostp/news-updates/2023/08/29/red-teaming-large-language-models-to-identify-novel-ai-risks/.  Accessed: 1/24/2024.
  * Thomas et al. (2021) Thomas, G., Luo, Y., and Ma, T.  Safe reinforcement learning by imagining the near future.  _Advances in Neural Information Processing Systems_ , 34:13859–13869, 2021.
  * ul Abdeen et al. (2024) ul Abdeen, Z., Zhang, X., Gill, W., and Jin, M.  Enhancing distribution system resilience: A first-order meta-rl algorithm for critical load restoration.  Technical report, 2024.  URL http://www.jinming.tech/papers/MetaCLR-2024.pdf.
  * van der Hoeven et al. (2023) van der Hoeven, D., Zhivotovskiy, N., and Cesa-Bianchi, N.  High-probability risk bounds via sequential predictors.  _arXiv preprint arXiv:2308.07588_ , 2023.
  * Vlassis et al. (2012) Vlassis, N., Littman, M. L., and Barber, D.  On the computational complexity of stochastic controller optimization in pomdps.  _ACM Transactions on Computation Theory (TOCT)_ , 4(4):1–8, 2012.
  * Wachi et al. (2023) Wachi, A., Hashimoto, W., Shen, X., and Hashimoto, K.  Safe exploration in reinforcement learning: A generalized formulation and algorithms.  In _Thirty-seventh Conference on Neural Information Processing Systems_ , 2023.
  * Wang et al. (2016) Wang, J. X., Kurth-Nelson, Z., Tirumala, D., Soyer, H., Leibo, J. Z., Munos, R., Blundell, C., Kumaran, D., and Botvinick, M.  Learning to reinforcement learn.  _arXiv preprint arXiv:1611.05763_ , 2016.
  * Wang et al. (2023) Wang, L., Zhang, X., Su, H., and Zhu, J.  A comprehensive survey of continual learning: Theory, method and application.  _arXiv preprint arXiv:2302.00487_ , 2023.
  * Yang et al. (2019a) Yang, J., Petersen, B., Zha, H., and Faissol, D.  Single episode policy transfer in reinforcement learning.  In _International Conference on Learning Representations_ , 2019a.
  * Yang et al. (2019b) Yang, R., Sun, X., and Narasimhan, K.  A generalized algorithm for multi-objective reinforcement learning and policy adaptation.  _Advances in neural information processing systems_ , 32, 2019b.
  * Yu et al. (2020a) Yu, C., Shi, G., Chung, S.-J., Yue, Y., and Wierman, A.  The power of predictions in online control.  _Advances in Neural Information Processing Systems_ , 33:1994–2004, 2020a.
  * Yu et al. (2020b) Yu, T., Kumar, S., Gupta, A., Levine, S., Hausman, K., and Finn, C.  Gradient surgery for multi-task learning.  _Advances in Neural Information Processing Systems_ , 33:5824–5836, 2020b.
  * Zeng et al. (2023) Zeng, Y., Cai, R., Sun, F., Huang, L., and Hao, Z.  A survey on causal reinforcement learning.  _arXiv preprint arXiv:2302.05209_ , 2023.
  * Zhang et al. (2018) Zhang, L., Lu, S., and Zhou, Z.-H.  Adaptive online learning in dynamic environments.  _Advances in neural information processing systems_ , 31, 2018.
  * Zhao et al. (2023) Zhao, R., Chen, H., Wang, W., Jiao, F., Do, X. L., Qin, C., Ding, B., Guo, X., Li, M., Li, X., et al.  Retrieving multimodal information for augmented generation: A survey.  _arXiv preprint arXiv:2303.10868_ , 2023.
  * Zhong et al. (2022) Zhong, H., Xiong, W., Zheng, S., Wang, L., Wang, Z., Yang, Z., and Zhang, T.  A posterior sampling framework for interactive decision making.  _arXiv preprint arXiv:2211.01962_ , 2022.
  * Zinkevich (2003) Zinkevich, M.  Online convex programming and generalized infinitesimal gradient ascent.  In _Proceedings of the 20th international conference on machine learning (ICML-03)_ , pp. 928–936, 2003.
  * Zintgraf et al. (2019) Zintgraf, L., Shiarlis, K., Igl, M., Schulze, S., Gal, Y., Hofmann, K., and Whiteson, S.  Varibad: A very good method for bayes-adaptive deep rl via meta-learning.  In _International Conference on Learning Representations_ , 2019.

##  Appendix A Concrete examples and empirical evidence

_Biological systems:_ Evolution is inherently antifragile—pressures prompt
adaptations enabling greater species resilience over generations, like
bacteria developing stronger resistance to antibiotics designed to eliminate
them. Tropism in plants allow dynamically bending towards beneficial stimuli
like light, enhancing robustness despite variability. Even mild climate
changes may elicit adaptive responses in coral resilience (Hughes et al.,
2003). Ecosystems with higher biodiversity demonstrate greater adaptability to
environmental changes (Folke et al., 2004), and can be actively leveraged to
improve resilience, e.g., prescribed fire (Ryan et al., 2013). The immune
system strengthens from exposure to pathogens (e.g., through vaccination
(Plotkin, 2005)), and skeletal muscles grow in response to the moderate stress
of exercise (Schoenfeld, 2010).

_Economic systems:_ Decentralized markets, characterized by price fluctuations
and competition, drive innovation and efficiency (Hayek, 2013).
Entrepreneurship often benefits from failure and adversity, leading to future
success (McGrath, 1999). Some investment strategies, such as antifragile
portfolios like “long vega” and “long gamma” financial derivatives, are
designed to profit from market volatility (Taleb & Douady, 2013).

_Social systems:_ Collective intelligence, which relies on the diversity of
opinions and experiences (viewed as internal opinion “stress testing”),
enhances problem-solving and decision-making capabilities (Page, 2008).
Resilient communities adapt and thrive under challenge (Norris et al., 2008).

_Technological systems:_ Agile development allows rapid response to changing
requirements (Manifesto, 2001).

_Psychology:_ Adversity can lead to post-traumatic growth (Tedeschi & Calhoun,
2004). Hormesis (Mattson, 2008), where low doses of toxins trigger beneficial
responses, is another example.

_Engineering systems._ Early steam engines advanced from fragile explosiveness
to reliable operations due to an engineering discovery—intentionally
introducing randomness (dithering) stabilized operations by overcoming
mechanical stiction. Chaos Control theory explores how the principles of chaos
theory can be applied to engineering systems to achieve faster control and
stability (Ott et al., 1990). The concept of using noise for stabilization in
early steam engine design, known as “stochastic resonance” or “noise-induced
stability,” is related to the principles of Chaos Control theory (Gammaitoni
et al., 1998). By understanding and leveraging the properties of chaotic
systems, engineers can design more efficient and responsive control systems
that are agile, adaptive, and capable of rapidly responding to changes in
their environment (Schöll & Schuster, 2008).

While some sources may use terms like “resilience,” these examples go beyond
mere recovery. Under stress, a resilient system bounces back; an antifragile
system bounces forward, returning to a state stronger than before. This
distinction highlights the potential benefits of designing with antifragility
in mind.

##  Appendix B FAQs

###  B.1 How to quantify and measure antifragility?

While antifragility can be rigorously checked in theoretical settings (based
on the proposed antifragility definition), assessing it in practice requires a
comprehensive approach. To establish antifragility as a system property, it is
insufficient to observe the system under a single trajectory. Instead, we need
to systematically expose the system to different levels of stress (controlled
experiments) and observe its performance across various settings and case
studies

  * •

Use high-fidelity simulations when real-world experiments are limited.

  * •

Analyze system responses to past stress in settings like hindsight POMDPs or
continual learning with immediate feedback. Define stress as diversity and
coverage of high-impact novel examples. There are also existing works in
medical research and quantifative finance by (Taleb & Douady, 2013; Taleb,
2018; Taleb & West, 2023) and co-authors that attempt to quantify
antifragility using various case studies.

  * •

Collaborate to define benchmarks, design experiments, and build simulations.

  * •

Leverage ML to learn variability-performance relationships or identify
patterns under different stress levels, uncovering concave responses from
historical or simulated data.

###  B.2 What are the trade-offs in applying antifragility for decision
systems?

Antifragility is a long-term property focusing on thriving after experiencing
stressful conditions, while efficiency, interpretability, and fairness are
often considered under normal conditions. These properties focus on different
system aspects:

  * •

Antifragile systems may trade efficiency for redundancy, diversity, and
adaptability. However, antifragility can complement normal-condition
efficiency with “context-dependent response mechanisms” for fast mode
switching, resulting in less pronounced trade-offs. Selectively applying
antifragility to critical parts can help.

  * •

Complex antifragile mechanisms can be difficult to explain, but
interpretability can be potentially incorporated by design using XAI, rule-
based systems, or human-in-the-loop approaches.

  * •

Antifragile systems learning under stress might inadvertently amplify biases
or discriminatory patterns. Techniques to mitigate bias, ensure data
diversity, maintain balanced risks across regions, and regular audits are
important.

###  B.3 How is antifragility related to paradigms such as robustness,
lifelong learning, and cognitive science?

Antifragility emphasizes _unusual_ robustness, with systems proactively
seeking out and leveraging (virtual) stressors for open-ended learning and
continuous improvement. This approach enables a potentially more effective
response to perturbations outside the typical range of expected scenarios
(i.e., black swans), which may correspond to unknown attacks in adversarial
ML. Designing antifragile learning strategies requires understanding the
nature of stressors, system dynamics, and desired outcomes. It may also hint
at a solution for breaking the defense-attack arms race by potentially
enabling the emergence of defense capabilities through antifragility training
(e.g., Manzoor et al. (2024)).

Lifelong and open-world learning approaches share similarities with
antifragility but can benefit from incorporating its principles to enhance
their ability to handle unexpected events and benefit from stressors.
Antifragile systems, such as the immune system (through vaccination (Plotkin,
2005)) and ecosystems (through prescribed burns (Ryan et al., 2013)),
demonstrate the value of controlled perturbations for increased resilience.

Antifragility focuses on decision-making scenarios where the magnitude of
wrong decisions is important and on unusual robustness against extreme
perturbations, an aspect not fully addressed in existing robustness or open-
ended/continual learning.

Cognitive science paradigms like counterfactual simulation (Roese & Epstude,
2017) share common ground with antifragility, but the latter puts these ideas
on a more theoretical foundation and addresses the computational aspects of
designing systems that benefit from uncertainty.

##  Appendix C Ethical Considerations

###  C.1 Addressing potential harms of antifragility training for LLMs

When applying antifragility principles to the training of ML models, such as
large language models (LLMs), it is crucial to prioritize ethical
considerations to prevent harm and ensure socially responsible development.
Key ethical considerations include:

  * •

Ensuring fair treatment, appropriate compensation, and protection of human
workers involved in providing feedback or data for model training.

  * •

Carefully curating and vetting training data and inputs to avoid exposing
models or human workers to harmful, offensive, or biased content.

  * •

Maintaining transparency in model development processes, regularly auditing
models for potential biases or harmful outputs, and establishing clear
accountability measures.

  * •

Engaging with diverse stakeholders, including ethicists, policymakers, and
community representatives, to identify potential risks, unintended
consequences, and align model development with societal values.

  * •

Continuously monitoring model performance and impacts on users and society,
promptly addressing any signs of harmful or biased outputs, and refining the
model and training processes accordingly.

###  C.2 Challenges and mitigation strategies for safety-critical domains

Antifragile approaches may introduce unacceptable risks in safety-critical
domains. Key challenges include:

  * •

Strict regulations and standards that may be violated by intentional stress
exposure.

  * •

Difficulty determining safe stress levels. Too much stress is risky; too
little may not provide sufficient opportunities for improvement.

  * •

Controlled experiments key to antifragility may be impossible due to safety
concerns.

  * •

Safety-critical systems are interconnected, with multiple subsystems
interacting in intricate ways. Introducing antifragile mechanisms in one part
may have unintended consequences or cascading effects on other parts.

  * •

Stakeholder resistance to intentionally stressing systems.

To harness antifragility benefits while ensuring safety:

  * •

Leverage simulations and digital twins for safe exploration.

  * •

Thoroughly assess risks and have strong mitigation plans. This includes
identifying potential failure modes, defining safety boundaries, and
establishing contingency plans to prevent or limit the consequences of
failures.

  * •

Introduce antifragility gradually, starting with low-risk parts and monitoring
closely.

  * •

Capitalize on inherent stress from historical failures and natural
disruptions, using causal reasoning to maximize data value.

  * •

Continually adapt and adjust mechanisms based on feedback.

  * •

Last but not least, build trust with stakeholders by clearly explaining how
the system adapts. This requires close collaboration among researchers,
practitioners, regulators, and stakeholders to ensure that antifragile systems
are designed and implemented in a responsible and context-appropriate manner.