# AIFS - ECMWF’s data-driven forecasting system

Simon Lang Mihai Alexe11footnotemark: 1 Matthew Chantry Jesper Dramsch Florian
Pinault Baudouin Raoult Mariana C. A. Clare Christian Lessig Michael Maier-
Gerber Linus Magnusson Zied Ben Bouallègue Ana Prieto Nemesio Peter D. Dueben
Andrew Brown Florian Pappenberger Florence Rabier equal contribution

(May 2024)

###### Abstract

Machine learning-based weather forecasting models have quickly emerged as a
promising methodology for accurate medium-range global weather forecasting.
Here, we introduce the Artificial Intelligence Forecasting System (AIFS), a
data driven forecast model developed by the European Centre for Medium-Range
Weather Forecasts (ECMWF). AIFS is based on a graph neural network (GNN)
encoder and decoder, and a sliding window transformer processor, and is
trained on ECMWF’s ERA5 re-analysis and ECMWF’s operational numerical weather
prediction (NWP) analyses. It has a flexible and modular design and supports
several levels of parallelism to enable training on high-resolution input
data. AIFS forecast skill is assessed by comparing its forecasts to NWP
analyses and direct observational data. We show that AIFS produces highly
skilled forecasts for upper-air variables, surface weather parameters and
tropical cyclone tracks. AIFS is run four times daily alongside ECMWF’s
physics-based NWP model and forecasts are available to the public under
ECMWF’s open data policy.

##  1 Introduction

Accurate weather forecasts are of great importance for society, helping to
save lives and protect property. The last years have seen substantial progress
in the realm of data-driven weather forecasting. Following exploratory work
(Dueben and Bauer (2018)), researchers and technological companies like
Google, Huawei and NVIDIA have built data-driven weather forecasting models
that outperform leading physics-based global numerical weather prediction
(NWP) models in many of the standard forecast scores (e.g. Pathak et al.
(2022); Keisler (2022); Lam et al. (2023); Chen et al. (2023); Bi et al.
(2023); Nguyen et al. (2023), see also Ben Bouallègue et al. (2024) for
verification results), like geopotential at 500 hPa root mean squared error
(RMSE) and anomaly correlation coefficient (ACC). At the same time, forecasts
can be produced within minutes on a single GPU (graphics processing unit),
compared to the O(1000) CPUs required for physics-based NWP forecasts.

The data-driven models are trained on historical weather data, usually a
subset of the reanalysis dataset ERA5 (Hersbach et al. (2020)) produced by the
European Centre for Medium-Range Weather Forecasts (ECMWF) in the context of
the EU Copernicus programme. When producing real-time forecasts, they
currently rely on operational NWP initial conditions, such as ECMWF’s 4D-Var
(Rabier et al. (2000)) analysis.

The decision was made at ECMWF to implement an experimental data-driven
forecasting system ready for operational use. This involved the implementation
of the data-driven forecast model, together with an end-to-end pipeline
covering training and inference, including dataset generation, reproducible
training, operational inference, the use of operational verification tools,
product generation and dissemination of forecast data.

ECMWF’s data-driven forecast model is called AIFS – Artificial Intelligence
Forecasting System, a homage to ECMWF’s IFS, the Integrated Forecasting
System. AIFS is implemented in Python and utilises the PyTorch (Paszke et al.
(2019)) and PyTorch Geometric (Fey and Lenssen (2019)) machine learning
frameworks. Together with AIFS we develop the Anemoi (Greek: ’Winds’)
framework, a toolbox that aims to provide building blocks for data driven
weather forecast models.

##  2 Model

A diverse set of deep learning architectures is being employed in the context
of data-driven weather forecasts, e.g., vision transformers (Bi et al. (2023);
Nguyen et al. (2023)), neural operators (Pathak et al. (2022)) and graph
neural networks (GNNs, Keisler (2022); Lam et al. (2023)).

The first version of AIFS was introduced in experimental operational mode in
October 2023 (Lang et al. (2024)). Its architecture was inspired by Keisler
(2022) and Lam et al. (2023). Specifically, it consisted of an encoder-
processor-decoder design similar to that described in Battaglia et al. (2016).
The encoder aggregates information from the input data grid onto the lower
resolution processor grid (Fig. 1). Here, a cut-off radius is employed with a
certain distance around each processor grid point. The decoder projects the
latent state back onto the input data grid. In the decoder, each grid point of
the input data grid is connected to its three closest processor grid points.
The edge features of the encoder and decoder graph are edge length and
direction.

As in Lam et al. (2023), the hidden processor grid was based on an icosphere,
though at a lower refinement level with approximately 10,000 nodes and 80,000
edges. The spatial resolution of the first AIFS version was approximately 1°.
AIFS underwent an update in February 2024 to a spatial resolution of
approximately 0.25°. The GNN flavour used in the encoder and decoder changed
to an attention / transformer based variant following Shi et al. (2021). The
processor is now based on a pre-norm transformer with shifted window attention
(Child et al. (2019), Beltagy et al. (2020), Dao (2023), Jiang et al. (2023))
and GELU activation function (Hendrycks and Gimpel (2023)). Consequently, it
does not take explicit edge information into account but attention is instead
computed along latitude bands (see Fig. 2). Attention windows are chosen in
such a way that a complete grid neighbourhood is always included. The
processor grid is now based on a O96 octahedral reduced Gaussian grid (Wedi
(2014)), with 40320 grid points (approximately 1 degree spatial resolution,
which is roughly the same as the processor resolution of Lam et al. (2023)).
The AIFS has 16 processor layers in total. Schematics of AIFS encoder-decoder
and processor blocks are shown in Figure 3.

Figure 1: Example visualisation of an AIFS encoder (left) and decoder (right)
graph. ERA5 grid points are shown as small black circles, processor grid
points as larger grey circles. Edges are shown as black lines. Only a small
region of the globe is plotted for visibility. Figure 2: Attention windows
(grid points highlighted in blue) for different grid points (red). The grey
grid points show an example of how far information can travel within 6 layers
of the processor (AIFS has 16 processor layers in total). For illustration
purposes, a lower-resolution grid is shown than that used in the AIFS.

The encoder and decoder operate on data at the native ERA5 spatial grid
resolution, the N320 reduced Gaussian grid (approximately 31 km). This is in
contrast to other data-driven weather models. One advantage of reduced
Gaussian grids is that they have a much more uniform resolution around the
globe, without grid spacing collapsing close to the poles. As a consequence,
the total number of grid points and edges in the encoder and decoder is
reduced compared to a regular longitude-latitude grid of similar resolution.
The N320 reduced Gaussian grid has 542,080 grid points, while a
0.25°longitude-latitude grid has 1,038,240 grid points.

Input data and edge features are embedded via a linear layer before the
encoder maps the input data from the ERA5 grid to the processor grid. The
decoder maps the latent space back to the ERA5 input grid, before a linear
layer projects it back to physical space.

Another feature of AIFS is that we add eight learnable parameters (features)
to each node on the input and the processor grids, as well as to the edges of
the encoder and decoder graphs. This allows the network to learn additional
useful features that are not directly present in the input data or pre-defined
as either node or edge features.

To reduce the GPU memory footprint, AIFS makes extensive use of activation
checkpointing during the forward model pass: instead of saving intermediate
activations in GPU memory, these are re-computed during the backward pass
(Chen et al. (2016)).

Figure 3: AIFS encoder / decoder and processor block schematics: GNN block
(left), processor block (right). The GNN block uses a multi-head graph
transformer convolution operation to update the nodes and the edges of the
processor, whereas the pre-norm transformer block relies on multi-head self-
attention (Vaswani et al. (2017)).

In addition to data parallelism, where a batch is divided into smaller sub-
batches and these are processed simultaneously across multiple GPUs, AIFS
implements sequence / tensor parallelism. Here, one instance of the neural
network is split across multiple GPUs (e.g. Kurth et al. (2022)). This enables
the training on high resolution data and over multiple auto-regressive steps.
We support two modes of sequence parallelism. In the GNN attention encoder,
decoder and in the transformer processor, attention heads are sharded across
GPUs, with synchronisation relying on efficient all-to-all communication
primitives, as proposed by Jacobs et al. (2023) . As an alternative option for
the GNN encoder and decoder, we also implement node and edge sharding
according to 1-hop graph neighbourhoods. Here, device-to-device communication
is minimised through a judicious re-ordering of the graph nodes, such that
only a single gather and reduce operation is required in the forward and
backward pass of the model, respectively.

The benefits of tensor parallelism are twofold: the computational load can be
distributed between GPUs to reduce run-time, and activations are split between
GPUs, increasing the amount of GPU memory that is available to one model
instance. The increased amount of available GPU memory makes training on high-
resolution data possible. Weak parallel scaling tests have demonstrated that
AIFS can scale quasi-linearly up to at least 2048 GPUs.

ECMWF’s goal is to develop a probabilistic forecasting system. To this end,
AIFS also implements a device-to-device communication pattern that allows for
ensemble-based training towards probabilistic scores (e.g. Gneiting and
Raftery (2007)), such as fair scores (Ferro (2013)). Complementary to the
sequence parallelism described above, ensemble members can be distributed
across several GPUs. This allows a direct probabilistic score-based
optimisation of larger ensembles during training.

##  3 Training

AIFS is trained to produce 6-hour forecasts. It receives as input a
representation of the atmospheric states (ERA5 or ECMWF’s operational
analysis) at $t_{-6h}$, $t_{0}$, and then forecasts the state at time
$t_{+6h}$. The full list of input and output fields of AIFS is shown in Table
1.

Field |  Level type |  Input/Output
---|---|---
Geopotential, horizontal and vertical wind components, specific humidity, temperature |  Pressure level: 50, 100, 150, 200, 250, 300, 400, 500, 600, 700, 850, 925, 1000 |  Both
Surface pressure, mean sea-level pressure, sea-surface temperature, skin temperature, 2 m temperature, 2 m dewpoint temperature, 10 m horizontal wind components, total column water |  Surface |  Both
Total precipitation, convective precipitation |  Surface |  Output
Land-sea mask, orography, standard deviation of sub-grid orography, slope of sub-scale orography, insolation, latitude/longitude, time of day/day of year |  Surface |  Input
Table 1: Input and output variables of AIFS.

Forecasts for longer lead-times are calculated in an auto-regressive fashion
by initialising the model from its own prediction - so called rollout. In
order to improve forecast scores, rollout is also used during training
(Keisler (2022)). Here, AIFS follows the training principles established by
Lam et al. (2023): a pre-training phase, during which the model is given the
task to forecast 6 h ahead is followed by a second phase, where the model
produces a forecast up to 72 h. Gradients flow through the entire forecast
chain during backpropagation.

Pre-training was performed on ERA5 for the years 1979 to 2020 with a cosine
learning rate (LR) schedule and a total of 260,000 steps. The LR is increased
from 0 to $10^{-4}$ during the first 1000 steps, then it is annealed to a
minimum of $3\times 10^{-7}$. The pre-training is then followed by rollout on
ERA5 for the years 1979 to 2018, this time with a LR of $6\times 10^{-7}$. As
in Lam et al. (2023) we increase the rollout every 1000 training steps up to a
maximum of 72 h (12 auto-regressive steps). Finally, to further improve
forecast performance, we fine-tune the model on operational real-time IFS NWP
analyses. This is done via another round of rollout training, this time using
IFS operational analysis data from 2019 and 2020. For fine-tuning and
initialisation of the model during inference, IFS fields are interpolated from
their native O1280 resolution (approximately 0.1°) down to N320 (approximately
0.25°). As an optimizer, we use AdamW (Loshchilov and Hutter (2019)) with the
$\beta$-coefficients set to 0.9 and 0.95.

Input and output states are normalised to unit variance and zero mean for each
level. Some of the forcing variables, like orography, are min-max normalised.
The loss (objective) function is an area-weighted mean squared error (MSE)
between the target atmospheric state and prediction. A loss scaling is applied
for each output variable. The scaling was chosen empirically such that all
prognostic variables have roughly equal contributions to the loss, with the
exception of the vertical velocities, for which the weight was reduced. The
loss weights also decrease linearly with height, which means that levels in
the upper atmosphere (e.g., 50 hPa) contribute relatively little to the total
loss value.

Data parallelism is used for training, with a batch size of 16. One model
instance is split across four 40GB A100 GPUs within one node. Training is done
using mixed precision (Micikevicius et al. (2018)), and the entire process
takes about one week, with 64 GPUs in total. Running a 10 day forecast takes
approximately 2 minutes 30 seconds on a single A100 GPU, including the input
and output of forecast data.

##  4 Results

In general, AIFS produces highly skilful forecasts as can be seen from the ACC
for the Northern Hemisphere compared to IFS and ERA5 forecasts (Fig. 4),
amounting to a forecast advantage of more than 12 h for longer lead-times. The
time-series shows how the forecast skill of IFS evolved over the years. Here,
improvements in the forecast model, data-assimilation system, observation
usage, and the inherent variability of atmospheric predictability drive
changes in skill. In comparison, ERA5 is a frozen system, and forecast skill
changes are mostly related to atmospheric variability. It is apparent that
AIFS represents a step change in forecast skill compared to IFS.

Figure 4: Northern Hemisphere ACC (anomaly correlation) of geopotential at 500
hPa for AIFS (blue) and IFS (red) and ERA5 (black) forecasts for different
years: 2 day forecasts (top left), 6 day forecasts (top right) and 10 day
forecasts (bottom). Forecasts are initialised on 00 UTC each day and shown is
a 30 day running mean. Insets show a zoomed-in view.

Figure 5 shows a scorecard that compares different forecast skill measures of
AIFS to those of IFS across a range of variables. For verification, each
system is compared against the operational ECMWF analysis from which the
forecasts are initialised. In addition, the forecasts are compared against
radiosonde observations of geopotential, temperature and windspeed, and SYNOP
observations of 2 m temperature, 10 m wind and 24 h total precipitation. The
definition of the metrics, such as ACC (ccaf), RMSE (rmsef) and forecast
activity (standard deviation of forecast anomaly, sdaf) can be found in e.g
Ben Bouallègue et al. (2024).

AIFS shows better scores throughout the troposphere, up to 100 hPa. Forecast
improvements are substantial, of the order of $~{}10\%$. IFS shows better
scores at 50 hPa, which is in line with the reduced weight given to these
pressure levels in training. In general, AIFS exhibits substantially lower
forecast errors than IFS after day 1 of the forecast. Forecast scores are
worse for AIFS compared to IFS at day 1 when forecast data is verified against
NWP analyses, e.g. for 850 hPa wind speed in the northern and southern extra-
tropics. However, this degradation is not present in verification against
radiosonde observations. IFS analyses and forecasts can exhibit correlations
that can extend some time into the forecast, and this might be an explanation
for the apparent larger errors of AIFS forecasts at day 1.

Similarly, while AIFS’s 100 hPa scores also seem worse than IFS for
geopotential ($z$) and temperature, when verified against analyses, this is
not necessarily the case when compared against radiosondes. For example, for
Northern Hemisphere extra-tropics temperature at 100 hPa, RMSE and ACC are
degraded for AIFS compared to IFS in the analysis based verification, while
they appear improved when AIFS and IFS are verified against radiosonde
observations.

In line with upper-air scores, AIFS surface forecasts score consistently
better than IFS when verified against analyses and observations, e.g., 2m
temperature and 10m wind as shown in Figure 6. For reference, we also show the
initial AIFS version, with approximately 1°spatial resolution. While the
initial lower resolution AIFS version was actually quite close to the current
AIFS in terms of upper-air scores (not shown), large differences become
apparent when comparing surface scores against observations. For surface
fields, spatial resolution is of great importance.

Figure 5: Scorecard comparing forecast scores of AIFS versus IFS (2022).
Forecasts are initialised on 00 and 12 UTC. Shown are relative score changes
as function of lead time (day 1 to 10) for northern extra-tropics (n.hem),
southern extra-tropics (s.hem), tropics and Europe. Blue colours mark score
improvements and red colours score degradations. Purple colours indicate an
increased in standard deviation of forecast anomaly, while green colours
indicate a reduction. Framed rectangles indicate 95$\%$ significance level.
Variables are geopotential (z), temperature (t), wind speed (ff), mean sea
level pressure (msl), 2 m temperature (2t), 10 m wind speed (10ff) and 24 hr
total precipitation (tp). Numbers behind variable abbreviations indicate
variables on pressure levels (e.g., 500 hPa), and suffix indicates
verification against IFS NWP analyses (an) or radiosonde and SYNOP
observations (ob). Scores shown are anomaly correlation (ccaf), SEEPS (seeps,
for precipitation), RMSE (rmsef) and standard deviation of forecast anomaly
(sdaf, see text for more explanation).

Total precipitation results are more mixed. In the extra-tropics, AIFS is less
skilful than IFS in terms of SEEPS (Rodwell et al. (2010)) at shorter forecast
ranges. At longer forecast ranges, AIFS is more skilfull than IFS. In the
tropics, total precipitation forecast skill is in general improved compared to
IFS.

Figure 6: RMSE of 2m temperature (left) and 10m wind speed (right) for AIFS
(blue, dashed), IFS (red, solid) and previous lower resolution AIFS version
(cyan, solid). Forecasts for 2022, Northern Hemisphere. Forecasts are
initialised on 00 and 12 UTC each day.

The scorecard also shows that overall AIFS’s forecast activity is somewhat
reduced compared to IFS. The reduction of forecast activity is accompanied by
a smoothing, or ”blurring”, of the forecast fields with lead-time, a behaviour
described by, e.g., Pathak et al. (2022), Keisler (2022) and Lam et al.
(2023), which is also visible in global spectra (not shown). While short-range
forecasts still contain quite a lot of small scale structures, these are
washed out in longer range forecasts (Fig. 7).

Figure 7: Meridional wind at 850 hPa: IFS (left) and AIFS (right) for 1
January 2023 00 UTC date, 12 h (top), 120 h (middle) and 240 h (bottom)
forecasts. For plotting, IFS and AIFS forecast fields were interpolated to a
0.25$\degree$ regular latitude-longitude grid. The MSE objective used in AIFS
training leads to more smoothing at longer lead times.

Nevertheless, the forecast activity is not as strongly reduced as in the case
of the ensemble mean (compare the blue, red and grey curves in Fig. 8) of IFS-
ENS (ECMWF’s NWP ensemble forecasts, Molteni et al. (1996); see Lang et al.
(2021), Lang et al. (2023) for latest configuration), which filters out
unpredictable features with lead time.

For reference, we also include the activity of GraphCast forecasts (Lam et al.
(2023)), which is quite similar to AIFS. The GraphCast version shown here has
been fine-tuned by Lam et al. (2023) to be initialised from operational IFS
analyses and is run daily at ECMWF. When looking at forecast performance, it
is also apparent that the difference between AIFS and GraphCast is much
smaller than between AIFS and IFS. This is in line with both being trained on
the same data (ERA5) and IFS analyses, and similarities of their respective
training regimes (rollout training, etc.). Interestingly, there are longer
periods during the year, where one model seems to perform better than the
other, i.e. one curve is below the other for several weeks. This may either be
caused by intrinsic variability or by differences in model architectures or
training and fine-tuning strategies. It also highlights the need for long
verification statistics.

Figure 8: AIFS, IFS, Graphcast and IFS ENS mean comparison of 30 day running
means of 7 day forecasts. Shown is geopotential at 500 hPa, Northern extra-
tropics, 2022, top: forecast activity (standard deviation of forecast
anomaly), bottom: ACC (anomaly correlation); AIFS (blue), IFS (red), GraphCast
(orange) and IFS-ENS (grey).

In line with results reported in the literature for data-driven models (see
e.g., Bi et al. (2023)), Lam et al. (2023)), AIFS exhibits substantially lower
tropical cyclone (TC) position forecast errors than IFS (see Fig. 9). To a
large extent, this can be explained by a reduced slow bias in the TC
propagation speed, which leads to lower along-track TC forecast errors. On the
other hand, TC intensity errors are larger than the ones from IFS. This is the
case for central pressure, as well as maximum wind speed (not shown). Here,
AIFS produces less intense TCs than IFS on average, which manifests in a
larger TC intensity bias. This might be explained by the on average too low
intensity of TCs in ERA5 and IFS analyses, as well as the tendency of AIFS to
smooth forecast fields during the forecast. There is also an indication that,
on average, AIFS generates fewer TCs than IFS (not shown).

Figure 9: Comparison of IFS and AIFS Mean tropical cyclone position error
(upper-left), mean tropical cyclone propagation speed (upper-right), tropical
cyclone mean central pressure error (bottom-left) and tropical cyclone mean
central pressure (bottom right) of AIFS (light blue) and IFS (dark blue),
January 2022 to December 2023.

##  5 Discussion and conclusions

AIFS demonstrates highly competitive forecast performance for both, upper-air
and surface variables. These results hold when verified against NWP analyses
as well as radiosonde and SYNOP observations. In addition, AIFS produces e.g.
accurate tropical cyclone track forecasts.

The current implementation of AIFS shares certain limitations with some of the
other data-driven weather forecast models that are trained with a weighted MSE
loss, such as blurring of the forecast fields at longer lead times. The
blurring can be traced back to the MSE training objective, where the model
learns an implicit smoothing in order to avoid the double penalty (e.g.
Hoffman et al. (1995); Ebert et al. (2013)) of misplaced structures in the
forecast. The smoothing depends on the length of the optimisation window used
during training. In the extreme, forecasts produced by a model optimised out
to day 10 would be similar to an ensemble mean, with strongly reduced activity
and detail (see Bouallègue et al. (2024)). Preliminary results show that
training towards a probabilistic objective is producing sharp forecast fields
throughout the forecast, consistent with the results reported by Price et al.
(2023).

Further limitations include the relatively ad-hoc approach to fine-tuning on
the operational IFS analysis. A more in-depth exploration of different fine-
tuning strategies could be beneficial.

AIFS currently uses ”full” normalised states in its training loss. This stands
in contrast to predicting normalised forecast tendencies as done by, e.g.,
Keisler (2022) or Lam et al. (2023). The potential downside here is that the
output of the last linear layer of AIFS that maps latent space variables to
physical space may not be adequately normalised. This will likely be revised
in a future AIFS upgrade. This revision will potentially also facilitate
improved strategies for the loss scaling that is currently done empirically on
a per variable basis.

Currently, AIFS exhibits reduced forecast skill in the stratosphere forecast
owing to the linear loss scaling with height (e.g. Lam et al. (2023)), i.e.,
only little emphasis is put on variables in the stratosphere. Optimising loss
scaling with height is expected to improve stratospheric forecast skill.

A further area for potential improvements is the currently reduced intensity
of some high-impact systems such as tropical cyclones. Here, improved input
data, such as high resolution re-analyses could potentially help. As stated
above, the smoothing with forecast lead time could be alleviated by refining
the training loss or by training probabilistic models.

The strength and weaknesses of data-driven forecast models have been
investigated in Ben Bouallègue et al. (2024) and Charlton-Perez et al. (2024),
and in an idealised context in Hakim and Masanam (2023). However, further work
is required to assess how the models perform in practice and how they compare
to physics-based NWP models.

The modular architecture of AIFS makes it highly flexible, scalable and
extendable. Ongoing research work focuses on the extension of AIFS to
probabilistic ensemble forecasting, either via CRPS-based training (e.g.
Pacchiardi et al. (2024), Kochkov et al. (2024)) or diffusion-based generative
models (Sohl-Dickstein et al. (2015); Ho et al. (2020); Karras et al. (2022);
Price et al. (2023)). Because of the comparatively small cost associated with
a single data-driven forecast, it will be possible to run ensemble forecasts
consisting of a large number of members.

Other important areas of research are data-driven long-range forecasts, e.g.
monthly and seasonal forecasts, improving the representation of precipitation
fields (Chantry et al. (2024)) and data-driven regional modelling (Nipen et
al. (2024)). A further area of exploration is the augmentation of AIFS
training and initialisation with observations, e.g. SYNOP and satellite
observations, to correct for analysis errors and allow the model to produced
improved point forecasts. These developments will also make it possible to
generate fully observation-driven (McNally et al. (2024)) forecasts.

We plan to release the AIFS source code and weights under an open source
license later this year, together with the Anemoi framework. AIFS forecasts
are run four times daily at ECMWF, and are available to the public under
ECMWF’s open data policy.

## References

  * Battaglia et al. [2016] Peter W. Battaglia, Razvan Pascanu, Matthew Lai, Danilo Rezende, and Koray Kavukcuoglu.  Interaction networks for learning about objects, relations and physics.  _arXiv preprint arXiv:1612.00222_ , 2016.
  * Beltagy et al. [2020] Iz Beltagy, Matthew E. Peters, and Arman Cohan.  Longformer: The long-document transformer.  _arXiv preprint arXiv:2004.05150_ , 2020.
  * Ben Bouallègue et al. [2024] Zied Ben Bouallègue, Mariana C A Clare, Linus Magnusson, Estibaliz Gascón, Michael Maier-Gerber, Martin Janoušek, Mark Rodwell, Florian Pinault, Jesper S Dramsch, Simon T K Lang, Baudouin Raoult, Florence Rabier, Matthieu Chevallier, Irina Sandu, Peter Dueben, Matthew Chantry, and Florian Pappenberger.  The rise of data-driven weather forecasting: A first statistical assessment of machine learning-based weather forecasts in an operational-like context.  _Bulletin of the American Meteorological Society_ , 2024.  ISSN 1520-0477.  doi: doi.org/10.1175/BAMS-D-23-0162.1.  URL http://dx.doi.org/10.1175/BAMS-D-23-0162.1.
  * Bi et al. [2023] K. Bi, L. Xie, H. Zhang, et al.  Accurate medium-range global weather forecasting with 3D neural networks.  _Nature_ , 619:533–538, 2023.  doi: 10.1038/s41586-023-06185-3.
  * Bouallègue et al. [2024] Zied Ben Bouallègue, Rilwan Adewoyin, Mihai Alexe, Matthew Chantry, Mariana Clare, Jesper Dramsch, Sara Hahner, Simon Lang, Christian Lessig, Linus Magnusson, Michael Maier-Gerber, Gert Mertes, Gabriel Moldovan, Ana Prieto Nemesio, Cathal O’Brien, Florian Pinault, Baudouin Raoult, Mario Santa Cruz, Helen Theissen, and Steffen Tietsche.  A new ml model in the ecmwf web charts, 2024.
  * Chantry et al. [2024] M. Chantry, Rilwan Adewoyin, Mihai Alexe, Zied Ben Bouallègue, Mariana Clare, Jesper Dramsch, Sara Hahner, Simon Lang, Christian Lessig, Linus Magnusson, Michael Maier-Gerber, Gert Mertes, Gabriel Moldovan, Ana Prieto Nemesio, Cathal O’Brien, Florian Pinault, Baudouin Raoult, Mario Santa Cruz, Helen Theissen, and Steffen Tietsche.  It’s rain(ing) data.  https://www.ecmwf.int/en/about/media-centre/aifs-blog/2024/its-raining-data, 2024.  Accessed: 2024-05-17.
  * Charlton-Perez et al. [2024] A.J. Charlton-Perez, H.F. Dacre, S. Driscoll, et al.  Do ai models produce better weather forecasts than physics-based models? a quantitative evaluation case study of storm ciarán.  _npj Clim Atmos Sci_ , 7:93, 2024.  doi: 10.1038/s41612-024-00638-w.  URL https://doi.org/10.1038/s41612-024-00638-w.
  * Chen et al. [2023] Lei Chen, Xiaohui Zhong, Feng Zhang, Yuan Cheng, Yinghui Xu, Yuan Qi, and Hao Li.  Fuxi: a cascade machine learning forecasting system for 15-day global weather forecast.  _npj Climate and Atmospheric Science_ , 6(1), November 2023.  ISSN 2397-3722.  doi: 10.1038/s41612-023-00512-1.  URL http://dx.doi.org/10.1038/s41612-023-00512-1.
  * Chen et al. [2016] T. Chen, B. Xu, C. Zhang, and C. Guestrin.  Training deep nets with sublinear memory cost.  _arXiv preprint arXiv:1604.06174_ , Apr 22 2016.
  * Child et al. [2019] Rewon Child, Scott Gray, Alec Radford, and Ilya Sutskever.  Generating long sequences with sparse transformers.  _arXiv preprint arXiv:1904.10509_ , 2019.
  * Dao [2023] Tri Dao.  FlashAttention-2: Faster attention with better parallelism and work partitioning.  _arXiv preprint arXiv:2307.08691_ , 2023.
  * Dueben and Bauer [2018] Peter D. Dueben and Peter Bauer.  Challenges and design choices for global weather and climate models based on machine learning.  June 2018.  doi: 10.5194/gmd-2018-148.  URL http://dx.doi.org/10.5194/gmd-2018-148.
  * Ebert et al. [2013] E. Ebert, L. Wilson, A. Weigel, M. Mittermaier, P. Nurmi, P. Gill, M. Göber, S. Joslyn, B. Brown, T. Fowler, and A. Watkins.  Progress and challenges in forecast verification.  _Meteorological Applications_ , 20(2):130–139, 2013.  doi: https://doi.org/10.1002/met.1392.
  * Ferro [2013] C. A. T. Ferro.  Fair scores for ensemble forecasts.  _Quarterly Journal of the Royal Meteorological Society_ , 140(683):1917–1923, December 2013.  ISSN 0035-9009.  doi: 10.1002/qj.2270.  URL http://dx.doi.org/10.1002/qj.2270.
  * Fey and Lenssen [2019] M. Fey and J. E. Lenssen.  Fast graph representation learning with PyTorch Geometric.  _arXiv preprint arXiv:1903.02428_ , Apr 25 2019.
  * Gneiting and Raftery [2007] Tilmann Gneiting and Adrian E Raftery.  Strictly proper scoring rules, prediction, and estimation.  _Journal of the American Statistical Association_ , 102(477):359–378, 2007.  doi: 10.1198/016214506000001437.
  * Hakim and Masanam [2023] Gregory J. Hakim and Sanjit Masanam.  Dynamical tests of a deep-learning weather prediction model.  _arXiv preprint arXiv:2309.10867_ , 2023.
  * Hendrycks and Gimpel [2023] Dan Hendrycks and Kevin Gimpel.  Gaussian Error Linear Units (GELUs).  _arXiv preprint arXiv:1606.08415_ , 2023.
  * Hersbach et al. [2020] H. Hersbach, B. Bell, P. Berrisford, et al.  The ERA5 global reanalysis.  _QJ R Meteorol Soc_ , 146:1999–2049, 2020.  doi: 10.1002/qj.3803.
  * Ho et al. [2020] Jonathan Ho, Ajay Jain, and Pieter Abbeel.  Denoising diffusion probabilistic models.  _arXiv preprint arXiv:2006.11239_ , 2020.
  * Hoffman et al. [1995] Ross N. Hoffman, Zheng Liu, Jean-Francois Louis, and Christopher Grassoti.  Distortion representation of forecast errors.  _Monthly Weather Review_ , 123(9):2758–2770, September 1995.  ISSN 1520-0493.  doi: 10.1175/1520-0493(1995)123¡2758:drofe¿2.0.co;2.  URL http://dx.doi.org/10.1175/1520-0493(1995)123<2758:DROFE>2.0.CO;2.
  * Jacobs et al. [2023] Sam Ade Jacobs, Masahiro Tanaka, Chengming Zhang, Minjia Zhang, Shuaiwen Leon Song, Samyam Rajbhandari, and Yuxiong He.  Deepspeed Ulysses: System optimizations for enabling training of extreme long sequence transformer models.  _arXiv preprint arXiv:2309.14509_ , 2023.
  * Jiang et al. [2023] Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed.  Mistral 7B.  _arXiv preprint arXiv:2310.06825_ , 2023.
  * Karras et al. [2022] Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine.  Elucidating the design space of diffusion-based generative models.  _arXiv preprint arXiv:2206.00364_ , 2022.
  * Keisler [2022] R. Keisler.  Forecasting global weather with graph neural networks.  _arXiv preprint arXiv:2202.07575_ , Feb 15 2022.
  * Kochkov et al. [2024] Dmitrii Kochkov, Janni Yuval, Ian Langmore, Peter Norgaard, Jamie Smith, Griffin Mooers, Milan Klöwer, James Lottes, Stephan Rasp, Peter Düben, Sam Hatfield, Peter Battaglia, Alvaro Sanchez-Gonzalez, Matthew Willson, Michael P. Brenner, and Stephan Hoyer.  Neural general circulation models for weather and climate.  _arXiv preprint arXiv:2311.07222_ , 2024.
  * Kurth et al. [2022] T. Kurth, S. Subramanian, P. Harrington, J. Pathak, M. Mardani, D. Hall, A. Miele, K. Kashinath, and A. Anandkumar.  FourCastNet: Accelerating global high-resolution weather forecasting using adaptive fourier neural operators.  _arXiv preprint arXiv:2208.05419_ , Aug 8 2022.
  * Lam et al. [2023] Remi Lam, Alvaro Sanchez-Gonzalez, Matthew Willson, Peter Wirnsberger, Meire Fortunato, Ferran Alet, Suman Ravuri, Timo Ewalds, Zach Eaton-Rosen, Weihua Hu, Alexander Merose, Stephan Hoyer, George Holland, Oriol Vinyals, Jacklynn Stott, Alexander Pritzel, Shakir Mohamed, and Peter Battaglia.  Learning skillful medium-range global weather forecasting.  _Science_ , 382(6677):1416–1421, December 2023.  ISSN 1095-9203.  doi: 10.1126/science.adi2336.  URL http://dx.doi.org/10.1126/science.adi2336.
  * Lang et al. [2024] S. Lang, M. Alexe, M. Chantry, J. Dramsch, F. Pinault, B. Raoult, Z. Ben Bouallegue, M. Clare, C. Lessig, L. Magnusson, and A. Prieto Nemesio.  AIFS: A new ECMWF forecasting system.  _ECMWF Newsletter_ , (178), 2024.
  * Lang et al. [2023] Simon Lang, Mark Rodwell, and Dinand Schepers.  IFS upgrade brings many improvements and unifies medium-range resolutions.  _ECMWF Newsletter 176_ , pages 21–28, 2023.  doi: 10.21957/slk503fs2i.
  * Lang et al. [2021] S.T.K. Lang, A. Dawson, M. Diamantakis, P. Dueben, S. Hatfield, M. Leutbecher, et al.  More accuracy with less precision.  _Q.J.R. Meteorol. Soc._ , 147(741):4358–4370, 2021.  doi: 10.1002/qj.4181.
  * Loshchilov and Hutter [2019] Ilya Loshchilov and Frank Hutter.  Decoupled weight decay regularization.  In _International Conference on Learning Representations_ , 2019.  URL https://openreview.net/forum?id=Bkg6RiCqY7.
  * McNally et al. [2024] Tony McNally, Christian Lessig, Peter Lean, Matthew Chantry, Mihai Alexe, and Simon Lang.  Red sky at night… Producing weather forecasts directly from observations.  _ECMWF Newsletter No. 178_ , 2024.  doi: 10.21957/1a8466ec2f.
  * Micikevicius et al. [2018] Paulius Micikevicius, Sharan Narang, Jonah Alben, Gregory Diamos, Erich Elsen, David Garcia, Boris Ginsburg, Michael Houston, Oleksii Kuchaiev, Ganesh Venkatesh, and Hao Wu.  Mixed precision training.  _arXiv preprint arXiv:1710.03740_ , 2018.
  * Molteni et al. [1996] F. Molteni, R. Buizza, T. N. Palmer, and T. Petroliagis.  The ECMWF ensemble prediction system: Methodology and validation.  _Quarterly Journal of the Royal Meteorological Society_ , 122(529):73–119, 1996.  doi: https://doi.org/10.1002/qj.49712252905.  URL https://rmets.onlinelibrary.wiley.com/doi/abs/10.1002/qj.49712252905.
  * Nguyen et al. [2023] Tung Nguyen, Rohan Shah, Hritik Bansal, Troy Arcomano, Sandeep Madireddy, Romit Maulik, Veerabhadra Kotamarthi, Ian Foster, and Aditya Grover.  Scaling transformer neural networks for skillful and reliable medium-range weather forecasting.  _arXiv preprint arXiv:2312.03876_ , 2023.
  * Nipen et al. [2024] T. Nipen, M. Chantry, and et al.  Data-driven regional modelling.  https://www.ecmwf.int/en/about/media-centre/aifs-blog/2024/data-driven-regional-modelling, 2024.  Accessed: 2024-05-17.
  * Pacchiardi et al. [2024] Lorenzo Pacchiardi, Rilwan Adewoyin, Peter Dueben, and Ritabrata Dutta.  Probabilistic forecasting with generative networks via scoring rule minimization.  _arXiv preprint arXiv:2112.08217_ , 2024.
  * Paszke et al. [2019] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala.  PyTorch: An imperative style, high-performance deep learning library.  _arXiv preprint arXiv:1912.01703_ , 2019.
  * Pathak et al. [2022] J. Pathak, S. Subramanian, P. Harrington, S. Raja, A. Chattopadhyay, M. Mardani, T. Kurth, D. Hall, Z. Li, K. Azizzadenesheli, and P. Hassanzadeh.  FourCastNet: A global data-driven high-resolution weather model using adaptive fourier neural operators.  _arXiv preprint arXiv:2202.11214_ , Feb 22 2022.
  * Price et al. [2023] Ilan Price, Alvaro Sanchez-Gonzalez, Ferran Alet, Timo Ewalds, Andrew El-Kadi, Jacklynn Stott, Shakir Mohamed, Peter Battaglia, Remi Lam, and Matthew Willson.  GenCast: Diffusion-based ensemble forecasting for medium-range weather.  _arXiv preprint arXiv:2312.15796_ , 2023.
  * Rabier et al. [2000] F. Rabier, H. Järvinen, E. Klinker, J.-F. Mahfouf, and A. Simmons.  The ECMWF operational implementation of four-dimensional variational assimilation. i: Experimental results with simplified physics.  _Quarterly Journal of the Royal Meteorological Society_ , 126(564):1143–1170, 2000.  doi: https://doi.org/10.1002/qj.49712656415.  URL https://rmets.onlinelibrary.wiley.com/doi/abs/10.1002/qj.49712656415.
  * Rodwell et al. [2010] Mark J. Rodwell, David S. Richardson, Tim D. Hewson, and Thomas Haiden.  A new equitable score suitable for verifying precipitation in numerical weather prediction.  _Quarterly Journal of the Royal Meteorological Society_ , 136(650):1344–1363, 2010.  doi: https://doi.org/10.1002/qj.656.  URL https://rmets.onlinelibrary.wiley.com/doi/abs/10.1002/qj.656.
  * Shi et al. [2021] Yunsheng Shi, Zhengjie Huang, Shikun Feng, Hui Zhong, Wenjin Wang, and Yu Sun.  Masked label prediction: Unified message passing model for semi-supervised classification.  _arXiv preprint arXiv:2009.03509_ , 2021.
  * Sohl-Dickstein et al. [2015] Jascha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, and Surya Ganguli.  Deep unsupervised learning using nonequilibrium thermodynamics.  _CoRR_ , abs/1503.03585, 2015.  URL http://arxiv.org/abs/1503.03585.
  * Vaswani et al. [2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  _arXiv preprint arXiv:1706.03762_ , 2017.
  * Wedi [2014] N. P. Wedi.  Increasing the horizontal resolution in numerical weather prediction and climate simulations: illusion or panacea?  _Philosophical Transactions of the Royal Society A_ , 372, 2014.  doi: 10.1098/rsta.2013.0289.


# A two-track tour of Cauchy’s _Cours_

Mikhail G. Katz  M. Katz, Department of Mathematics, Bar Ilan University,
Ramat Gan 5290002 Israel<EMAIL_ADDRESS>
###### Abstract.

Cauchy published his _Cours d’Analyse_ 200 years ago. We analyze Cauchy’s take
on the concepts of _rigor_ , _continuity_ , and _limit_ , and explore a pair
of approaches in the literature to the meaning of his infinitesimal analysis
and his sum theorem on the convergence of series of continuous functions.

###### 2020 Mathematics Subject Classification:

Primary 01A55, Secondary 26E35, 01A85

##  1\. Rigor then and now

Building upon pioneering work by Kepler, Fermat, Cavalieri, Gregory, Wallis,
Barrow, and others, Isaac Newton and Gottfried Wilhelm von Leibniz invented
the calculus in the 17th century. While immediately acquiring an enthusiastic
following, the new methods proved to be controversial in the eyes of some of
their contemporaries, who employed the more traditional methods of their
predecessors. One of the controversial aspects of the new technique was
Leibniz’s distinction between assignable and inassignable quantities
(including infinitesimals and infinite quantities; see [23], [1], [2]). At the
French Academy, the opposition to the new calculus was led by Michel Rolle,
and across the Channel, by George Berkeley. The scientific success of the new
methods ultimately silenced the opposition, but lingering doubts persisted
(fed in part by doctrinal theological issues; see [6]). A new era was ushered
in by Augustin-Louis Cauchy’s textbook _Cours d’Analyse_ , addressed to the
students of the Ecole Polytechnique in Paris.

Cauchy published his _Cours d’Analyse_ (CDA) 200 years ago. The book was of
fundamental importance for the development of both real and complex analysis.
Hans Freudenthal mentioned in his essay on Cauchy (for the _Dictionary of
Scientific Biography_) [15] that Niels Henrik Abel described the CDA as “an
excellent work which should be read by every analyst who loves mathematical
rigor.” But what did _rigor_ mean to Abel and Cauchy?

In the early 19th century context, the term _rigor_ referred to the standard
of mathematical precision set by the geometry of Euclid. This context enables
us to understand Cauchy’s intention when, in the introduction to CDA, he
referred to ‘all the rigor one demands from geometry’, committing himself
further to ‘never rely on arguments taken from the generality of algebra’. We
see that Cauchy’s notion of rigor in CDA is distinct from “what has been
called the nineteenth-century ‘rigorization’ of real analysis” [18, p. 221].

Like the term _rigor_ , Cauchy’s term _generality of algebra_ requires
explanation to be comprehensible to modern readers. It refers to certain
techniques used by his predecessors, particularly Euler and Lagrange, that
today would be considered cavalier, and specifically (1) proofs based on
algebraic manipulation of divergent series, and (2) the idea that algebraic
rules and formulas valid in the real domain remain valid in the complex
domain. Some of the proofs that fall under item (1) have since been justified
using summation techniques developed later; some of the techniques under item
(2), in terms of analytic continuation.

By the standards of the current century, some of Freudenthal’s comments could
be considered controversial. Thus, Freudenthal writes:

  1. Terms like “infinitesimally small” prevail in Cauchy’s limit arguments and epsilontics still looks far away, but there is one exception. His proof … of the well-known theorem

$\text{ If}\lim_{x\to\infty}(f(x+1)-f(x))=\alpha,\text{
then}\lim_{x\to\infty}x^{-1}f(x)=\alpha$

is a paragon, and the first example, of epsilontics – the character
$\varepsilon$ even occurs there. [15, p. 137]

The claim that infinitesimals ‘prevail’ in Cauchyan foundations of analysis,
whereas $\varepsilon$-$\delta$ arguments ‘look far away’ and are limited to a
small number of exceptions, may surprise a reader whose perceptions of
Cauchyan rigor are influenced by Judith Grabiner’s views [16] and publications
that followed them, especially if they tend to identify _rigor_ with the
jettisoning of infinitesimals in favor of $\varepsilon$-$\delta$ arguments
based on an ‘algebra of inequalities’. Some historians today would view both
Freudenthal’s and Grabiner’s perspectives as outdated. But the dual view of
Cauchyan analysis persists in the current literature.

##  2\. Dual tracks

The track-A view holds that Cauchy, ahead of his time, worked primarily with
an Archimedean continuum, and pioneered many of the techniques that would
become known as $\varepsilon$-$\delta$ in the next century.

The track-B view holds that Cauchy, like most of his contemporaries and
colleagues at the Ecole Polytechnique, based his analysis primarily on
variable quantities and infinitesimals; see Laugwitz [19], [20].

What were then the Cauchyan foundations of analysis? While we won’t purport to
provide a definitive answer in this short note, we will let Cauchy speak for
himself (using the translation by Bradley and Sandifer [8]). Cauchy writes in
the introduction to CDA:

  1. In speaking of the continuity of functions, I could not dispense with a treatment of the principal properties of infinitely small quantities, properties which serve as the foundation of the infinitesimal calculus. [8, p. 1]

Track-A advocates read this passage as a concession to the management of the
Ecole, and argue that Cauchy “could not dispense with a treatment of …
infinitely small quantities” because of explicit mandates from the Ecole,
against his better judgement.

Track-B advocates read this passage as a recognition by Cauchy (in a departure
from his pre-1820 approaches) that a convincing and accessible treatment of
continuity necessitates infinitesimals, and note that Cauchy’s favorable
judgment of infinitesimals is corroborated by their use in his research long
after the end of his teaching stint at the Ecole; see [5].

Readers searching for an $\varepsilon$-$\delta$ definition of limit in CDA may
be surprised to find instead the following definition:

  1. We call a quantity _variable_ if it can be considered as able to take on successively many different values. … When the values successively attributed to a particular variable indefinitely approach a fixed value in such a way as to end up by differing from it by as little as we wish, this fixed value is called the _limit_ of all the other values. [8, p. 6]

Here the notion of a variable quantity is taken as primary, and limits are
defined in terms of variable quantities. Variable quantities similarly provide
the basis for the definition of infinitesimals:

  1. When the successive numerical values of such a variable decrease indefinitely, in such a way as to fall below any given number, this variable becomes what we call _infinitesimal_ , or an _infinitely small quantity_. A variable of this kind has zero as its limit. [8, p. 7]

Track-A advocates read this as asserting that an infinitesimal is merely a
null sequence (i.e., a sequence tending to zero), and take the last sentence
to refer to infinitesimals. Track-B advocates point out that Cauchy did not
write that a variable quantity _is_ an infinitesimal, but rather that a
variable quantity _becomes_ an infinitesimal, implying a change in nature
(from being a variable quantity to being an infinitesimal); and take the last
sentence to refer to the variable quantity mentioned at the beginning of the
passage.

Returning to limits, Cauchy writes:

  1. When a variable quantity converges towards a fixed limit, it is often useful to indicate this limit with particular notation. We do this by placing the abbreviation

lim

in front of the variable quantity in question. [8, p. 12]

Bradley and Sandifer note that the 1821 edition of the CDA used the notation
“lim.” (with a period). Cauchy’s description of “lim.” as an abbreviation
suggests that he viewed it as secondary to, or an aspect of, the concept of a
variable quantity.

In Chapter 2, Cauchy returns to the definition of infinitesimals:

  1. We say that a variable quantity becomes _infinitely small_ when its numerical value decreases indefinitely in such a way as to converge towards the limit zero. [8, p. 21]

The relation between the concepts of variable quantity and infinitesimal was
already discussed above, as well as the possible ambiguity of the verb
_becomes_ ; limits again play a secondary role. Cauchy proceeds next to the
properties of infinitesimals:

  1. Infinitely small and infinitely large quantities enjoy several properties that lead to the solution of important questions, which I will explain in a few words. Let $\alpha$ be an infinitely small quantity, that is a variable whose numerical value decreases indefinitely. [8, p. 22]

Track-A advocates point out that here Cauchy states that an infinitesimal is a
variable quantity. Track-B advocates note that here Cauchy is no longer
dealing with detailed definitions, and this particular formulation is merely
shorthand for the more careful definition in terms of _becoming_ elaborated
earlier.

##  3\. Continuity in 1817, 1821, and beyond

Cauchy’s first documented characterisation of continuity is found in a record
of a course summary dating from march 1817 (a month before the earliest
written mention of Bolzano’s _Rein analytischer Beweis_ in an Olms catalogue).
The definition can be described as reasonably precise in the sense of enabling
a straightforward transcription as an impeccable modern definition; see [3].
In modern mathematics, a real function $\varphi$ is continuous at
$c\in{\mathbb{R}}$ if and only if for each sequence $(x_{n})$ converging to
$c$, one has

$\varphi(c)=\varphi\Big{(}\lim_{n\to\infty}x_{n}\Big{)}=\lim_{n\to\infty}\varphi(x_{n}),$
(3.1)

or briefly $\varphi\circ\lim=\lim\circ\,\varphi$ at $c$, expressing the
commutation of $\varphi$ and $\lim$. In 1817, Cauchy wrote:

  1. The limit of a continuous function of several variables is [equal to] the same function of their limit. Consequences of this Theorem with regard to the continuity of composite functions dependent on a single variable.

(Being part of a summary, the second phrase is not a complete sentence.)
Cauchy’s 1817 characterisation of continuity in terms of commutation of
$\varphi$ and $\lim$ as in (3.1) does not use infinitesimals and thus
contrasts with his definitions involving infinitesimals given four years later
in CDA. Surprisingly, it is the 1817 characterisation that is actually used in
CDA; see Section 5.

Here is Cauchy’s first definition of continuity in CDA:

  1. Let $f(x)$ be a function of the variable $x$, and suppose that for each value of $x$ between two given limits, the function always takes a unique finite value. If, beginning with a value of $x$ contained between these limits, we add to the variable $x$ an _infinitely small increment_ $\alpha$, the function itself is incremented by the difference $f(x+\alpha)-f(x)$, which depends both on the new variable $\alpha$ and on the value of $x$. Given this, the function $f(x)$ is a _continuous_ function of $x$ between the assigned limits if, for each value of $x$ between these limits, the numerical value of the difference $f(x+\alpha)-f(x)$ decreases indefinitely with the numerical value of $\alpha$. [8, p. 26]

Note that, while the increment $\alpha$ is described as infinitesimal, the
resulting change $f(x+\alpha)-f(x)$ is not. This 1821 definition can be seen
as intermediary between the march 1817 characterisation in terms of variables
(not mentioning infinitesimals), and his second 1821 definition stated purely
in terms of infinitesimals:

  1. In other words, _the function $f(x)$ is continuous with respect to $x$ between the given limits if, between these limits, an infinitely small increment in the variable always produces an infinitely small increment in the function itself_. (ibid.)

Significantly, Cauchy’s abbreviation “lim.” appeared in none of the
definitions of continuity given in CDA.111Surprisingly, a contrary claim
appears in the recent literature: “Cauchy gave a faultless definition of
continuous function, using the notion of ‘limit’ for the first time. Following
Cauchy’s idea, Weierstrass popularized the $\epsilon$-$\delta$ argument in the
1870’s” Dani–Papadopoulos [13], 2019, p. 283). The notion of limit does not
appear in any of Cauchy’s definitions of continuity in CDA. It is the second
definition purely in terms of infinitesimals that reappears in the following
works by Cauchy:

  * •

_Résumé des Leçons_ (1823); English translation [9, p. 9];

  * •

_Leçons sur le Calcul Différentiel_ (1829) [10, p. 9];

  * •

_Mémoire sur l’Analyse Infinitésimale_ (1844) [11, p. 17];

  * •

the 1853 article [12] on the sum theorem (see Section 6).

Nonetheless, the perception that Cauchy allegedly ‘sought to establish
foundations for real analysis that gave no role to infinitesimals’ has firmly
entered the canonical creation narrative of modern mathematical analysis; see
e.g., [21].

##  4\. A modern digression: infinitesimals without Choice

Cauchy’s definition of continuity in CDA, in its B-track interpretation, is
harder to follow for modern readers more familiar with the
$\varepsilon$-$\delta$ definition of continuity à la Weierstrass–Dini than
with the definition in a modern infinitesimal theory. We therefore provide a
formalisation of Cauchy’s procedures involving continuity in terms of the
theory SPOT (acronym of its axioms) developed in [17]; see also [14]. SPOT has
the advantage of being conservative over the traditional Zermelo–Fraenkel set
theory (ZF) and therefore depends on neither the Axiom of Choice nor the
existence of ultrafilters.

The language of ZF is limited to the two-place membership relation $\in$. The
language of SPOT includes also a predicate ST, where ST$(x)$ reads ‘$x$ is
standard’. Such a distinction between standard and nonstandard entities can be
thought of as formalizing the Leibnizian distinction between assignable and
inassignable quantities; see [2]. The standard ordered field ${\mathbb{R}}$
has both standard and nonstandard elements. An element $\alpha$ is
_infinitesimal_ if $|\alpha|<r$ for each standard $r>0$. Let $x$ be a standard
point in the domain of a real standard function $f$. Then $f$ is continuous at
$x$ (in the traditional sense of the $\varepsilon$-$\delta$ definition) if and
only if

$\text{infinitesimal }\alpha\text{ produce infinitesimal changes
}f(x+\alpha)-f(x),$ (4.1)

whenever $x+\alpha$ is in the domain of $f$. Continuity in an interval, say
$(0,1)$, is equivalent to the satisfaction of condition (4.1) at every
standard point $x\in(0,1)$.

It should be mentioned that, according to most scholars, Cauchy did not
formulate the notion of continuity at a point, but only ‘continuity between
limits’ (i.e., in an interval) or in a neighborhood of a point. Freudenthal
remarks:

  1. It is the weakest point in Cauchy’s reform of calculus that he never grasped the importance of uniform continuity. [15, p. 137]

From an A-track viewpoint, the weakness is that there seems to be no trace in
Cauchy of the idea that a significant issue is whether the allowable error is
independent of the point $x$ or not.

From a B-track viewpoint, the weakness is that Cauchy did not make it clear
whether condition (4.1) is expected to be satisfied only at assignable points
$x$ or at all points of the interval. Note that uniform continuity of $f$ on,
say, $(0,1)$ is equivalent to (4.1) being satisfied at all points of $(0,1)$.
For example, $\frac{1}{x}$ fails to be uniformly continuous because of the
failure of (4.1) at an infinitesimal input $x=\beta>0$: indeed, the change
$\frac{1}{\beta+\alpha}-\frac{1}{\beta}$ is not infinitesimal if, say,
$\alpha=\beta$. See further in Section 6.

By Section 2.3, Cauchy reaches the Theorem described by Freudenthal as a
_paragon_ of $\varepsilon$-$\delta$ arguments. What Cauchy actually shows is
that if $f(x+1)-f(x)$ is between $k-\varepsilon$ and $k+\varepsilon$ then
(assuming monotonicity) $\frac{f(x)}{x}$ is similarly between $k-\varepsilon$
and $k+\varepsilon$. If anything this is a paragon of
$\varepsilon$-$\varepsilon$ arguments, since here the $\delta$ equals
$\varepsilon$! The trademark feature of modern $\varepsilon$-$\delta$
arguments, namely an explicit (nontrivial) dependence of $\delta$ on
$\varepsilon$, does not appear anywhere in Cauchy’s alleged ‘algebra of
inequalities’, lending support to Freudenthal’s sentiment that ‘epsilontics
looks far away’.

Here Cauchy mentions that an infinite limit ‘is larger than any _assignable_
number’ indicating familiarity with this Leibnizian term (which occurs nine
times in CDA).

##  5\. Functional equations and continuity via commutation

In Chapter 5 Cauchy studies functional relations for continuous functions, and
treats the following problem:

  1. Problem I. – To determine the function $\varphi(x)$ in such a manner that it remains continuous between any two real limits of the variable $x$ and so that for all real values of the variables $x$ and $y$, we have $\varphi(x+y)=\varphi(x)+\varphi(y)$. [8, p. 71]

Cauchy arrives at
$\varphi\big{(}\frac{m}{n}\alpha\big{)}=\frac{m}{n}\varphi(\alpha)$ and argues
as follows:

  1. Then, by supposing that the fraction $\frac{m}{n}$ varies in such a way as to converge towards any number $\mu$, and passing to the limit, we find that $\varphi(\mu\alpha)=\mu\varphi(\alpha)$. [8, p. 72]

Here Cauchy exploits the 1817 characterisation of the continuity of $\varphi$
in terms of the commutation of $\varphi$ and $\lim$ as summarized in formula
(3.1), rather than the definitions presented in Chapter 2 of CDA (the 1817
characterisation is also used in the proof of the Intermediate Value Theorem).
Curiously, Cauchy provides no explanatory comment. Possibly, Cauchy wrote the
material in Chapter 5 with definition (3.1) in mind, and introduced the
definitions in Chapter 2 at a later stage in the writing of the book. Cauchy
applies a similar technique to study the functional relation
$\varphi(x+y)=\varphi(x)\varphi(y)$ and other variations.

##  6\. Sum theorem and convergence _always_

Chapter 6 includes Cauchy’s controversial sum theorem. We summarize the
historical facts. The 1821 formulation of the theorem appears to be incorrect
to the modern reader, as it seems to assert that pointwise convergence of a
series of continuous functions implies the continuity of the sum. Already in
1826 Abel pointed out that the theorem ‘suffers exceptions’. Cauchy was
curiously silent on the subject of the sum theorem for several decades. Then
in 1853 he presented a modified statement of the sum theorem, mentioned an
example similar to Abel’s (without mentioning Abel by name), and explained why
the example does not contradict the (modified) theorem. Numerous scholars have
attempted to explain what the modification was (if any) and to interpret
Cauchy’s sum theorem in terms intelligible to modern audiences.

Cauchy considers the series obtained as the sum of the terms of the sequence
$u_{0},u_{1},u_{2},\ldots,u_{n},u_{n+1},\ldots$, denoted (1). In its 1821
formulation, the theorem asserts the following.

  1. When the various terms of series (1) are functions of the same variable $x$, continuous with respect to this variable in the neighborhood of a particular value for which the series converges, the sum $s$ of the series is also a continuous function of $x$ in the neighborhood of this particular value. [8, p. 90]

Here is the 1853 formulation:

  1. If the various terms of the series

$\hfill u_{0},u_{1},u_{2},\ldots,u_{n},u_{n+1},\ldots\hfil$

are functions of a real variable $x$ which are continuous, with respect to
this variable, between the given bounds; and if, furthermore, the sum

$u_{n}+u_{n+1}+\ldots+u_{n^{\prime}-1}$ (3)

_always_ becomes infinitely small for infinitely large values of the whole
numbers $n$ and $n^{\prime}>n$, then the series (1) will converge and the sum
$s$ of the series will be, between the given bounds, a continuous function of
the variable $x$. [12, pp. 456–457] (translation ours)

Note that Cauchy adds the word _toujours_ (always). But what exactly is
supposed to happen always in 1853, and how does this modify the 1821
hypothesis? Cauchy himself provides a hint in his 1853 analysis of the example
$\sum_{n}\frac{\sin nx}{n}$, representing a (discontinuous) sawtooth waveform.
But the hint he provides is itself puzzling. Cauchy evaluates
$u_{n}+u_{n+1}+\ldots+u_{n^{\prime}-1}$ at $x=\frac{1}{n}$ and shows that the
sum does not become arbitrarily small, and in fact can be made “sensibly
equal” to the integral $\int_{1}^{\infty}\frac{\sin x}{x}dx=0.6244\ldots$

Here Cauchy explicitly describes $n$ (and $n^{\prime}$) as infinite; then
$x=\frac{1}{n}$ is infinitesimal.

Many scholars of both A-track and B-track persuasion have argued that Cauchy
meant to add what is known today as the condition of uniform convergence.

A-track advocates may interpret the condition as requiring the sums
$u_{n}+u_{n+1}+\ldots+u_{n^{\prime}-1}$ to be small independently of
$n,n^{\prime}$ and also of the input $x$, a condition that can be stated
formally in terms of an alternating quantifier string of the type

$\forall\varepsilon>0\;\exists\delta>0\;\forall
n<n^{\prime}\in\mathbb{N}\;\forall
x\;\big{(}n>\tfrac{1}{\delta}\to|u_{n}(x)+u_{n+1}(x)+\ldots+u_{n^{\prime}-1}(x)|<\varepsilon\big{)}.$

Something along these (long) lines would have to be attributed to Cauchy, in
inchoate form, in order to interpret the addition of uniform convergence in an
A-track fashion. What is unclear is how the word _always_ manages to allude to
such independence, particularly since Cauchy seems to have overlooked its
significance in the context of continuity (see Freudenthal’s remark quoted in
Section 4 and the ensuing discussion). Moreover, the evaluation at what seems
to be a new type of point, namely $x=\frac{1}{n}$, appearing in Cauchy’s
discussion of $\sum_{n}\frac{\sin nx}{n}$, suggests that the insistence on the
qualifier _always_ indicates an extension of the inputs $x$ to include
additional ones (that were not _always_ included before).

A B-track reading of Cauchy’s 1853 hypothesis interprets the qualifier
_always_ as referring to additional inassignable inputs $x$ (including
infinitesimal values such as $\frac{1}{n}$ for infinite $n$). If one requires
the sum $u_{n}+u_{n+1}+\ldots+u_{n^{\prime}-1}$ to be infinitesimal for all
infinite $n,n^{\prime}$ and all inputs $x$ (standard and nonstandard) then one
indeed obtains a condition equivalent to uniform convergence (Robinson [22,
Theorem 4.6.1, p. 116]), guaranteeing the continuity of the limit function.
For more details see [7], [4].

##  7\. Conclusion

Though we have sketched widely divergent readings of Cauchy’s definitions and
his theorems, we hope to have conveyed to the reader a sense that not only the
concept of _rigor_ but also _continuity_ and _limit_ may have had a different
meaning to Cauchy than they do to us today, underscoring the contingency of
the historical evolution of mathematics. While Cauchy incontestably made
extensive use of _bona fide_ infinitesimals in fields as varied as
differential geometry, elasticity theory, and geometric probability (see [5]),
the nature of his foundational stance remains controversial. But perhaps this
is as it should be: with a genius of Cauchy’s caliber, tidy construals of his
work may necessarily amount to a flattening of his multi-dimensional vision.

## References

  * [1] Bair, J.; Błaszczyk, P.; Ely, R.; Henry, V.; Kanovei, V.; Katz, K.; Katz, M.; Kutateladze, S.; McGaffey, T.; Reeder, P.; Schaps, D.; Sherry, D.; Shnider, S. Interpreting the infinitesimal mathematics of Leibniz and Euler. _Journal for General Philosophy of Science_ 48 (2017), no. 2, 195–238. See http://dx.doi.org/10.1007/s10838-016-9334-z and https://arxiv.org/abs/1605.00455
  * [2] Bair, J.; Błaszczyk, P.; Ely, R.; Katz, M.; Kuhlemann, K. Procedures of Leibnizian infinitesimal calculus: An account in three modern frameworks. _British J. Hist. Math._ 36 (2021), no. 2. https://doi.org/10.1080/26375451.2020.1851120, https://arxiv.org/abs/2011.12628
  * [3] Bair, J.; Błaszczyk, P.; Fuentes Guillén, E.; Heinig, P.; Kanovei, V.; Katz, M. Continuity between Cauchy and Bolzano: Issues of antecedents and priority. _British J. Hist. Math._ 35 (2020), no. 3, 207–224. https://doi.org/10.1080/26375451.2020.1770015, https://arxiv.org/abs/2005.13259
  * [4] Bair, J.; Błaszczyk, P.; Heinig, P.; Kanovei, V.; Katz, M. 19th century real analysis, forward and backward. _Antiquitates Mathematicae_ 13 (2019), 19–49. http://doi.org/10.14708/am.v13i1.6440, https://arxiv.org/abs/1907.07451
  * [5] Bair, J.; Błaszczyk, P.; Heinig, P.; Kanovei, V.; Katz, M. Cauchy’s work on integral geometry, centers of curvature, and other applications of infinitesimals. _Real Analysis Exchange_ 45 (2020), no. 1, 127–150. https://doi.org/10.14321/realanalexch.45.1.0127, https://arxiv.org/abs/2003.00438
  * [6] Bair, J.; Katz, M.; Sherry, D. Fermat’s dilemma: Why did he keep mum on infinitesimals? and the European theological context. _Foundations of Science_ 23 (2018), no. 3, 559–595. http://doi.org/10.1007/s10699-017-9542-y, https://arxiv.org/abs/1801.00427
  * [7] Bascelli, T.; Błaszczyk, P.; Borovik, A.; Kanovei, V.; Katz, K.; Katz, M.; Kutateladze, S.; McGaffey, T.; Schaps, D.; Sherry, D. Cauchy’s infinitesimals, his sum theorem, and foundational paradigms. _Foundations of Science_ 23 (2018), no. 2, 267–296. http://doi.org/10.1007/s10699-017-9534-y, https://arxiv.org/abs/1704.07723
  * [8] Bradley, R.; Sandifer, C. _Cauchy’s Cour d’analyse. An annotated translation_. Sources and Studies in the History of Mathematics and Physical Sciences. Springer, New York, 2009\.
  * [9] Cates, D. _Cauchy’s Calcul infinitésimal. An annotated English translation_. Springer, Cham, 2019.
  * [10] Cauchy, A.-L. _Leçons sur le Calcul Différentiel_. Bure Frères, Paris, 1829.
  * [11] Cauchy, A.-L. _Exercices d’Analyse et de Physique Mathématique. Mémoire sur l’Analyse Infinitésimale_. Bachelier, Paris, 1844.
  * [12] Cauchy, A.-L. Note sur les séries convergentes dont les divers termes sont des fonctions continues d’une variable réelle ou imaginaire, entre des limites données. _Compt. Rend. Acad. Roy. Sci._ 36 (1853), 454–459.
  * [13] Dani, S.; Papadopoulos, A. (Eds.) _Geometry in History_ , Springer, 2019.
  * [14] Fletcher, P.; Hrbacek, K.; Kanovei, V.; Katz, M.; Lobry, C.; Sanders, S. Approaches to analysis with infinitesimals following Robinson, Nelson, and others. _Real Analysis Exchange_ 42 (2017), no. 2, 193–252. https://arxiv.org/abs/1703.00425, http://doi.org/10.14321/realanalexch.42.2.0193
  * [15] Freudenthal, H. Cauchy, Augustin-Louis. In _Dictionary of Scientific Biography, Vol. 3_ , Charles Gillispie (Ed.), Charles Scribner’s Sons, New York, 1981.
  * [16] Grabiner, J. _The origins of Cauchy’s rigorous calculus_. MIT Press, Cambridge, Mass.–London, 1981.
  * [17] Hrbacek, K.; Katz, M. Infinitesimal analysis without the Axiom of Choice. _Ann. Pure Appl. Logic_ 172 (2021), no. 6, 102959. https://doi.org/10.1016/j.apal.2021.102959, https://arxiv.org/abs/2009.04980
  * [18] Kanamori, A. Cantor and Continuity. In _The History of Continua: Philosophical and Mathematical Perspectives_ , 219–254, S. Shapiro, J. Hellman (Eds.), Oxford University Press, Oxford, 2021.
  * [19] Laugwitz, D. Infinitely small quantities in Cauchy’s textbooks. _Historia Mathematica_ 14 (1987), 258–274.
  * [20] Laugwitz, D. Definite values of infinite sums: aspects of the foundations of infinitesimal analysis around 1820. _Arch. Hist. Exact Sci._ 39 (1989), 195–245.
  * [21] Oppy, G.; Hájek, A.; Easwaran, K.; Mancosu, P. Infinity. Stanford Encyclopedia of Philosophy, april 2021. https://plato.stanford.edu/entries/infinity
  * [22] Robinson, A. _Non-standard analysis_. North-Holland, Amsterdam, 1966.
  * [23] Sherry, D.; Katz, M. Infinitesimals, imaginaries, ideals, and fictions. _Studia Leibnitiana_ 44 (2012), no. 2, 166–192. (Article was published in 2014 even though the journal issue lists the year as 2012.) http://www.jstor.org/stable/43695539, https://arxiv.org/abs/1304.2137