update readme for 1.1 (#1)
Browse files- update readme for 1.1 (94fd7768eb7044be6c6d7ebf70a8dbaec7b8b1e9)
Co-authored-by: Ewan Pinnington <[email protected]>
README.md
CHANGED
@@ -8,20 +8,17 @@ language:
|
|
8 |
library_name: anemoi
|
9 |
---
|
10 |
|
11 |
-
# AIFS Single - v1.
|
12 |
|
13 |
<!-- Provide a quick summary of what the model is/does. -->
|
14 |
|
15 |
Here, we introduce the **Artificial Intelligence Forecasting System (AIFS)**, a data driven forecast
|
16 |
model developed by the European Centre for Medium-Range Weather Forecasts (ECMWF).
|
17 |
|
18 |
-
The release of AIFS Single v1.
|
19 |
-
supersedes the existing
|
20 |
-
The new version, 1.
|
21 |
-
|
22 |
-
- Improved performance for upper-level atmospheric variables (AIFS Single still uses 13 pressure-levels, so this improvement mainly refers to 50 and 100 hPa)
|
23 |
-
- Improved skill for total precipitation.
|
24 |
-
- Additional output variables, including 100 meter winds, snow-fall, surface solar-radiation and land variables such as soil-moisture and soil-temperature.
|
25 |
|
26 |
<div style="display: flex; justify-content: center;">
|
27 |
<img src="assets/radiation_cloudcover.gif" alt="AIFS 10 days Forecast" style="width: 50%;"/>
|
@@ -33,7 +30,7 @@ are available to the public under ECMWF’s open data policy (https://www.ecmwf.
|
|
33 |
Note that due to the non-determinism of GPUs, users will be unable to exactly reproduce an official AIFS forecast
|
34 |
when running AIFS Single themselves.
|
35 |
|
36 |
-
For more
|
37 |
|
38 |
## Data Details
|
39 |
|
@@ -102,7 +99,7 @@ There are no changes in resolution compared to previous version AIFS Single v0.2
|
|
102 |
|
103 |
| | Component | Horizontal Resolution [kms] | Vertical Resolution [levels] |
|
104 |
|---|:---:|:---:|:---:|
|
105 |
-
| Atmosphere | AIFS-single v1.
|
106 |
|
107 |
|
108 |
### Model Sources
|
@@ -138,17 +135,17 @@ step-by-step workflow is specified to run the AIFS using the HuggingFace model:
|
|
138 |
🚨 **Note** we train AIFS using `flash_attention` (https://github.com/Dao-AILab/flash-attention).
|
139 |
The use of 'Flash Attention' package also imposes certain requirements in terms of software and hardware. Those can be found under #Installation and Features in https://github.com/Dao-AILab/flash-attention
|
140 |
|
141 |
-
🚨 **Note** the `aifs_single_v1.
|
142 |
That file does not contain any information about the optimizer states, lr-scheduler states, etc.
|
143 |
|
144 |
-
## How to train AIFS Single v1.
|
145 |
|
146 |
To train this model you can use the configuration files included in this repository and the following Anemoi packages:
|
147 |
|
148 |
```
|
149 |
-
anemoi-training==0.
|
150 |
-
anemoi-models==0.
|
151 |
-
anemoi-graphs==0.
|
152 |
```
|
153 |
and run the pretraining stage as follows,
|
154 |
|
@@ -197,7 +194,7 @@ the forcing variables, like orography, are min-max normalised.
|
|
197 |
|
198 |
### Training Procedure
|
199 |
|
200 |
-
Based on the different experiments we have made - the final training recipe for AIFS Single v1.
|
201 |
from the one used for AIFS Single v0.2.1 since we found that we could get a well trained model by skipping the ERA5
|
202 |
rollout and directly doing the rollout on the operational-analysis (extended) dataset. When we say 'extended' we refer
|
203 |
to the fact that for AIFS Single v0.2.1 we used just operational-analysis data from 2019 to 2021, while in this new
|
@@ -252,7 +249,7 @@ variables. For verification, each system is compared against the operational ECM
|
|
252 |
are initialised. In addition, the forecasts are compared against radiosonde observations of geopotential, temperature
|
253 |
and windspeed, and SYNOP observations of 2 m temperature, 10 m wind and 24 h total precipitation. The definition
|
254 |
of the metrics, such as ACC (ccaf), RMSE (rmsef) and forecast activity (standard deviation of forecast anomaly,
|
255 |
-
sdaf) can be found in e.g Ben Bouallegue et al. ` [2024].
|
256 |
|
257 |
### AIFS Single v1.0 vs AIFS Single v0.2.1 (2023)
|
258 |
|
|
|
8 |
library_name: anemoi
|
9 |
---
|
10 |
|
11 |
+
# AIFS Single - v1.1
|
12 |
|
13 |
<!-- Provide a quick summary of what the model is/does. -->
|
14 |
|
15 |
Here, we introduce the **Artificial Intelligence Forecasting System (AIFS)**, a data driven forecast
|
16 |
model developed by the European Centre for Medium-Range Weather Forecasts (ECMWF).
|
17 |
|
18 |
+
The release of AIFS Single v1.1 represents a slight modification to the AIFS model. Version 1.1
|
19 |
+
supersedes the existing operational version, [1.1.0 AIFS-single](https://huggingface.co/ecmwf/aifs-single-1.0).
|
20 |
+
The new version, 1.1, brings minor changes to the v1.0 model. These changes mainly correspond to the removal of
|
21 |
+
spurious rainfall points caused by incorrect soil moisture loss weighting during training of the v1.0 model.
|
|
|
|
|
|
|
22 |
|
23 |
<div style="display: flex; justify-content: center;">
|
24 |
<img src="assets/radiation_cloudcover.gif" alt="AIFS 10 days Forecast" style="width: 50%;"/>
|
|
|
30 |
Note that due to the non-determinism of GPUs, users will be unable to exactly reproduce an official AIFS forecast
|
31 |
when running AIFS Single themselves.
|
32 |
|
33 |
+
For more information on this update please see the [confluence page](https://confluence.ecmwf.int/display/FCST/Implementation+of+AIFS+Single+v1)
|
34 |
|
35 |
## Data Details
|
36 |
|
|
|
99 |
|
100 |
| | Component | Horizontal Resolution [kms] | Vertical Resolution [levels] |
|
101 |
|---|:---:|:---:|:---:|
|
102 |
+
| Atmosphere | AIFS-single v1.1 | ~ 31 | 13 |
|
103 |
|
104 |
|
105 |
### Model Sources
|
|
|
135 |
🚨 **Note** we train AIFS using `flash_attention` (https://github.com/Dao-AILab/flash-attention).
|
136 |
The use of 'Flash Attention' package also imposes certain requirements in terms of software and hardware. Those can be found under #Installation and Features in https://github.com/Dao-AILab/flash-attention
|
137 |
|
138 |
+
🚨 **Note** the `aifs_single_v1.1.ckpt` checkpoint just contains the model’s weights.
|
139 |
That file does not contain any information about the optimizer states, lr-scheduler states, etc.
|
140 |
|
141 |
+
## How to train AIFS Single v1.1
|
142 |
|
143 |
To train this model you can use the configuration files included in this repository and the following Anemoi packages:
|
144 |
|
145 |
```
|
146 |
+
anemoi-training==0.4.0
|
147 |
+
anemoi-models==0.5.0
|
148 |
+
anemoi-graphs==0.5.2
|
149 |
```
|
150 |
and run the pretraining stage as follows,
|
151 |
|
|
|
194 |
|
195 |
### Training Procedure
|
196 |
|
197 |
+
Based on the different experiments we have made - the final training recipe for AIFS Single v1.1 has deviated slightly
|
198 |
from the one used for AIFS Single v0.2.1 since we found that we could get a well trained model by skipping the ERA5
|
199 |
rollout and directly doing the rollout on the operational-analysis (extended) dataset. When we say 'extended' we refer
|
200 |
to the fact that for AIFS Single v0.2.1 we used just operational-analysis data from 2019 to 2021, while in this new
|
|
|
249 |
are initialised. In addition, the forecasts are compared against radiosonde observations of geopotential, temperature
|
250 |
and windspeed, and SYNOP observations of 2 m temperature, 10 m wind and 24 h total precipitation. The definition
|
251 |
of the metrics, such as ACC (ccaf), RMSE (rmsef) and forecast activity (standard deviation of forecast anomaly,
|
252 |
+
sdaf) can be found in e.g Ben Bouallegue et al. ` [2024]. No significant changes in skill wer found in the v1.1 model fix.
|
253 |
|
254 |
### AIFS Single v1.0 vs AIFS Single v0.2.1 (2023)
|
255 |
|