Grp. 1 | 30.56±12.7 | 61.11±21.0 | 50.00±0.0 | 47.22±12.7 | 47.22±9.6 | 50.00±22.0 | 77.78±4.8 | 47.22±33.7 | 44.44±12.7 | 36.11±17.3 | 44.44±9.6
FNR | Grp. 0 | 17.78±7.7 | 17.78±16.8 | 13.33±6.7 | 6.67±6.7 | 4.45±3.9 | 13.33±6.7 | 13.33±0.0 | 17.78±20.4 | 11.11±3.8 | 17.78±13.9 | 17.78±10.2
Grp. 1 | 25.64±4.4 | 7.69±0.0 | 10.25±4.4 | 10.25±4.4 | 23.07±13.3 | 12.82±4.4 | 5.13±4.4 | 23.07±20.4 | 12.82±4.4 | 20.51±8.9 | 20.51±16.0
EqOdd | - | 89.12±6.4 | 86.84±5.0 | 92.28±4.8 | 91.36±4.1 | 80.17±5.4 | 90.78±6.2 | 86.37±1.9 | 81.22±7.0 | 93.59±3.0 | 92.82±0.8 | 85.21±12.2
PAPILA | Age | BCE | Grp. 0 | 0.34±0.04 | 0.27±0.02 | 0.52±0.03 | 0.15±0.13 | 0.20±0.13 | 0.28±0.03 | 0.53±0.02 | 0.16±0.02 | 0.23±0.05 | 0.09±0.08 | 0.72±0.43
Grp. 1 | 0.74±0.05 | 0.82±0.04 | 1.36±0.15 | 1.42±0.65 | 0.86±0.17 | 0.87±0.18 | 0.68±0.01 | 0.67±0.06 | 0.76±0.03 | 4.50±2.60 | 1.25±0.22
ECE | Grp. 0 | 0.15±0.04 | 0.17±0.06 | 0.11±0.00 | 0.08±0.04 | 0.10±0.03 | 0.15±0.01 | 0.41±0.01 | 0.12±0.02 | 0.12±0.02 | 0.04±0.03 | 0.11±0.00
Grp. 1 | 0.25±0.02 | 0.26±0.01 | 0.31±0.02 | 0.32±0.10 | 0.28±0.06 | 0.30±0.08 | 0.24±0.03 | 0.25±0.05 | 0.26±0.03 | 0.35±0.03 | 0.31±0.05
TPR@80 TNR | Grp. 0 | 24.59±8.6 | 20.00±0.0 | 32.10±19.2 | 32.44±21.6 | 28.57±14.8 | 24.67±22.8 | 20.00±0.0 | 20.00±0.0 | 20.00±0.0 | 20.00±0.0 | 28.00±13.9
Grp. 1 | 23.27±4.7 | 21.90±3.3 | 17.02±3.6 | 16.67±7.9 | 19.21±5.1 | 30.48±16.2 | 15.24±5.9 | 16.49±11.0 | 25.08±8.3 | 25.08±3.1 | 19.81±2.9
FPR | Grp. 0 | 6.25±6.2 | 0.00±0.0 | 6.25±6.2 | 2.08±3.6 | 2.08±3.6 | 18.75±27.2 | 2.08±3.6 | 14.58±13.0 | 0.00±0.0 | 4.17±7.2 | 6.25±6.2
Grp. 1 | 56.86±3.4 | 56.86±3.4 | 60.78±3.4 | 47.06±15.6 | 47.06±5.9 | 52.94±27.0 | 54.90±6.8 | 66.67±3.4 | 60.78±9.0 | 37.25±14.8 | 62.75±3.4
FNR | Grp. 0 | 83.33±28.9 | 50.00±0.0 | 66.67±28.9 | 16.67±28.9 | 16.67±28.9 | 50.00±50.0 | 16.67±28.9 | 0.00±0.0 | 33.33±28.9 | 16.67±28.9 | 50.00±50.0
Grp. 1 | 23.81±8.2 | 19.05±8.2 | 33.33±8.3 | 28.57±14.3 | 28.57±0.0 | 28.57±14.3 | 33.33±8.3 | 19.05±8.2 | 19.05±8.2 | 33.33±8.3 | 19.05±8.2
EqOdd | - | 44.93±18.2 | 56.09±5.8 | 56.07±12.4 | 64.42±1.3 | 64.42±2.0 | 67.43±10.1 | 58.11±2.7 | 64.43±10.1 | 57.70±2.1 | 72.74±10.4 | 46.75±17.5
Sex | BCE | Grp. 0 | 0.64±0.12 | 1.08±0.19 | 0.57±0.04 | 0.55±0.05 | 0.57±0.12 | 4.93±2.90 | 0.57±0.01 | 0.45±0.03 | 0.77±0.30 | 1.10±0.17 | 0.64±0.26
Grp. 1 | 0.77±0.53 | 1.13±0.59 | 0.53±0.06 | 0.47±0.00 | 0.56±0.10 | 3.61±2.61 | 0.59±0.02 | 0.42±0.05 | 0.81±0.32 | 1.08±0.14 | 0.61±0.27
ECE | Grp. 0 | 0.19±0.03 | 0.30±0.04 | 0.23±0.04 | 0.27±0.02 | 0.20±0.01 | 0.25±0.02 | 0.27±0.04 | 0.15±0.06 | 0.22±0.05 | 0.23±0.05 | 0.23±0.02
Grp. 1 | 0.24±0.06 | 0.30±0.06 | 0.19±0.01 | 0.17±0.06 | 0.18±0.03 | 0.33±0.05 | 0.26±0.03 | 0.16±0.08 | 0.18±0.02 | 0.25±0.07 | 0.22±0.03
TPR@80 TNR | Grp. 0 | 27.64±24.0 | 45.00±31.2 | 11.00±6.9 | 13.00±14.7 | 20.33±12.9 | 50.83±17.7 | 54.50±4.8 | 25.00±26.5 | 46.67±20.8 | 38.81±25.0 | 17.50±11.5
Grp. 1 | 46.67±6.1 | 35.00±9.0 | 37.16±8.0 | 30.67±4.6 | 34.11±12.3 | 56.67±18.6 | 60.27±6.7 | 40.89±7.8 | 19.71±15.7 | 40.00±10.6 | 43.67±7.5
FPR | Grp. 0 | 20.83±3.6 | 20.83±7.2 | 39.58±25.3 | 45.83±7.2 | 50.00±38.0 | 16.67±7.2 | 8.33±9.5 | 22.92±3.6 | 18.75±6.2 | 22.92±7.2 | 39.58±9.5
Grp. 1 | 31.37±3.4 | 25.49±12.2 | 41.18±27.0 | 43.14±9.0 | 47.06±32.7 | 19.61±14.8 | 21.57±9.0 | 31.37±9.0 | 23.53±10.2 | 33.33±6.8 | 41.18±5.9
FNR | Grp. 0 | 50.00±0.0 | 33.33±14.4 | 25.00±25.0 | 25.00±0.0 | 25.00±0.0 | 41.67±14.4 | 58.33±14.4 | 41.67±14.4 | 58.33±14.4 | 33.33±14.4 | 25.00±0.0
Grp. 1 | 13.33±11.5 | 33.33±11.5 | 33.33±23.1 | 0.00±0.0 | 26.67±30.6 | 33.33±11.5 | 20.00±20.0 | 13.33±23.1 | 13.33±11.5 | 20.00±0.0 | 13.33±11.5
EqOdd | - | 76.40±5.8 | 92.43±2.2 | 82.59±5.6 | 83.95±3.3 | 84.39±7.2 | 92.16±0.4 | 74.22±13.2 | 76.61±12.1 | 74.74±11.2 | 87.51±8.5 | 92.39±5.5

Table A13: Results of other metrics in out-of-distribution setting. In the
dataset column, the dataset in the first row is the training domain, and the
second row is the testing domain.

Dataset | Attr. | Metrics | Group | ERM | Resample | DomainInd | LAFTR | CFair | LNL | EnD | ODR | GroupDRO | SWAD | SAM
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
CheXpert MIMIC-CXR | Age | BCE | Grp. 0 | 0.64±0.03 | 0.91±0.09 | 1.86±0.31 | 0.68±0.05 | 0.79±0.15 | 0.76±0.09 | 1.03±0.17 | 0.84±0.14 | 0.70±0.09 | 0.82±0.03 | 0.67±0.03
Grp. 1 | 0.70±0.01 | 0.90±0.08 | 2.07±0.30 | 0.71±0.03 | 0.82±0.14 | 0.74±0.07 | 0.97±0.17 | 0.86±0.10 | 0.72±0.09 | 0.93±0.01 | 0.72±0.03
Grp. 2 | 0.72±0.01 | 0.80±0.08 | 2.16±0.23 | 0.69±0.03 | 0.73±0.09 | 0.67±0.05 | 0.76±0.12 | 0.77±0.06 | 0.69±0.08 | 0.95±0.02 | 0.71±0.03
Grp. 3 | 0.73±0.01 | 0.74±0.06 | 2.33±0.22 | 0.69±0.05 | 0.68±0.06 | 0.63±0.04 | 0.65±0.08 | 0.71±0.03 | 0.66±0.06 | 0.95±0.02 | 0.70±0.02
Grp. 4 | 0.68±0.00 | 0.64±0.05 | 2.18±0.22 | 0.62±0.05 | 0.59±0.04 | 0.56±0.04 | 0.55±0.04 | 0.61±0.02 | 0.60±0.04 | 0.86±0.02 | 0.64±0.02
ECE | Grp. 0 | 0.28±0.01 | 0.36±0.03 | 0.43±0.05 | 0.30±0.03 | 0.36±0.08 | 0.32±0.07 | 0.45±0.07 | 0.36±0.06 | 0.28±0.07 | 0.36±0.01 | 0.27±0.02
Grp. 1 | 0.29±0.00 | 0.35±0.04 | 0.43±0.03 | 0.29±0.02 | 0.34±0.07 | 0.29±0.06 | 0.40±0.07 | 0.35±0.04 | 0.27±0.08 | 0.38±0.00 | 0.29±0.02
Grp. 2 | 0.26±0.00 | 0.27±0.04 | 0.36±0.03 | 0.24±0.02 | 0.25±0.05 | 0.21±0.04 | 0.24±0.07 | 0.26±0.02 | 0.21±0.07 | 0.32±0.00 | 0.25±0.01
Grp. 3 | 0.23±0.00 | 0.22±0.03 | 0.30±0.02 | 0.21±0.02 | 0.20±0.03 | 0.17±0.03 | 0.16±0.06 | 0.21±0.01 | 0.17±0.05 | 0.27±0.00 | 0.21±0.01
Grp. 4 | 0.18±0.00 | 0.17±0.02 | 0.23±0.01 | 0.16±0.02 | 0.15±0.02 | 0.13±0.02 | 0.11±0.03 | 0.15±0.01 | 0.14±0.03 | 0.21±0.00 | 0.17±0.01
TPR@80 TNR | Grp. 0 | 73.58±6.0 | 60.45±6.0 | 63.12±12.0 | 70.07±4.0 | 72.05±4.0 | 59.42±9.0 | 65.48±8.0 | 53.00±7.0 | 56.11±7.0 | 76.22±3.0 | 53.54±3.0
Grp. 1 | 67.32±1.0 | 58.35±3.0 | 57.29±8.0 | 65.94±2.0 | 64.42±2.0 | 55.56±3.0 | 58.52±4.0 | 56.04±6.0 | 57.77±1.0 | 69.43±1.0 | 61.52±0.0
Grp. 2 | 69.51±0.0 | 62.95±3.0 | 60.63±6.0 | 70.23±1.0 | 67.61±2.0 | 62.88±1.0 | 63.95±5.0 | 61.08±9.0 | 61.46±1.0 | 71.51±1.0 | 66.99±1.0
Grp. 3 | 64.15±0.0 | 60.47±3.0 | 57.93±6.0 | 65.12±1.0 | 63.23±1.0 | 59.91±2.0 | 61.36±4.0 | 56.24±8.0 | 56.35±1.0 | 65.89±1.0 | 63.84±0.0
Grp. 4 | 55.47±1.0 | 54.00±2.0 | 51.37±5.0 | 56.47±2.0 | 55.29±2.0 | 54.18±1.0 | 54.18±2.0 | 50.95±7.0 | 48.18±2.0 | 57.50±1.0 | 55.60±0.0
FPR | Grp. 0 | 59.85±4.0 | 53.79±7.0 | 60.61±6.0 | 56.82±9.0 | 60.23±9.0 | 71.21±9.0 | 45.08±11.0 | 59.47±3.0 | 68.18±2.0 | 52.27±2.0 | 72.35±1.0
Grp. 1 | 52.75±1.0 | 48.85±2.0 | 53.50±1.0 | 49.07±4.0 | 47.43±6.0 | 58.15±3.0 | 42.23±5.0 | 54.97±5.0 | 60.03±3.0 | 46.60±1.0 | 57.02±1.0
Grp. 2 | 38.62±1.0 | 41.28±2.0 | 42.91±2.0 | 37.82±3.0 | 38.17±4.0 | 46.61±2.0 | 39.02±4.0 | 44.94±7.0 | 44.41±3.0 | 35.38±1.0 | 40.67±1.0
Grp. 3 | 24.22±1.0 | 29.32±2.0 | 30.27±3.0 | 25.07±1.0 | 28.53±2.0 | 30.35±1.0 | 31.80±4.0 | 34.99±8.0 | 28.59±2.0 | 23.94±0.0 | 25.55±1.0
Grp. 4 | 12.89±1.0 | 18.76±3.0 | 20.52±4.0 | 14.02±2.0 | 20.21±2.0 | 16.68±3.0 | 22.58±4.0 | 25.31±10.0 | 16.38±2.0 | 14.87±1.0 | 13.82±0.0
FNR | Grp. 0 | 2.26±2.0 | 11.11±7.0 | 4.86±1.0 | 3.12±3.0 | 4.86±4.0 | 4.34±1.0 | 9.98±5.0 | 4.51±2.0 | 4.51±2.0 | 3.82±1.0 | 2.95±0.0
Grp. 1 | 4.00±0.0 | 8.92±3.0 | 5.87±2.0 | 4.86±1.0 | 6.21±3.0 | 4.69±1.0 | 11.98±3.0 | 6.56±1.0 | 4.93±1.0 | 5.40±0.0 | 4.23±0.0
Grp. 2 | 10.90±1.0 | 13.28±1.0 | 13.05±3.0 | 11.55±2.0 | 12.01±2.0 | 10.85±1.0 | 13.95±2.0 | 11.79±1.0 | 13.58±1.0 | 12.43±1.0 | 12.01±0.0
Grp. 3 | 29.16±2.0 | 27.08±2.0 | 28.58±3.0 | 27.37±2.0 | 24.63±2.0 | 26.12±3.0 | 23.33±2.0 | 24.07±2.0 | 31.66±2.0 | 28.31±1.0 | 27.74±1.0
Grp. 4 | 58.00±3.0 | 47.82±4.0 | 48.38±1.0 | 53.67±2.0 | 43.78±4.0 | 51.15±5.0 | 42.17±5.0 | 40.79±8.0 | 57.51±4.0 | 51.07±0.0 | 56.45±1.0
Sex | BCE | Grp. 0 | 0.70±0.04 | 0.63±0.08 | 0.92±0.05 | 0.72±0.07 | 0.70±0.07 | 0.65±0.00 | 0.64±0.05 | 0.69±0.11 | 0.73±0.01 | 0.90±0.01 | 0.65±0.02
Grp. 1 | 0.73±0.03 | 0.66±0.09 | 0.94±0.09 | 0.76±0.08 | 0.70±0.05 | 0.68±0.03 | 0.70±0.05 | 0.71±0.14 | 0.75±0.01 | 0.92±0.00 | 0.68±0.03
ECE | Grp. 0 | 0.22±0.02 | 0.18±0.06 | 0.22±0.02 | 0.23±0.04 | 0.22±0.04 | 0.20±0.00 | 0.19±0.03 | 0.21±0.06 | 0.23±0.01 | 0.28±0.00 | 0.20±0.01
Grp. 1 | 0.25±0.02 | 0.21±0.06 | 0.25±0.04 | 0.26±0.04 | 0.24±0.03 | 0.23±0.02 | 0.23±0.03 | 0.23±0.07 | 0.26±0.01 | 0.30±0.01 | 0.24±0.02
TPR@80 TNR | Grp. 0 | 68.33±1.0 | 69.40±2.0 | 69.25±1.0 | 68.06±2.0 | 68.37±2.0 | 67.97±1.0 | 69.98±0.0 | 70.41±1.0 | 67.32±1.0 | 71.88±0.0 | 71.14±0.0
Grp. 1 | 70.98±1.0 | 71.25±1.0 | 71.25±1.0 | 70.76±1.0 | 70.03±2.0 | 68.78±2.0 | 71.31±1.0 | 71.41±1.0 | 70.04±1.0 | 72.95±0.0 | 72.33±0.0
FPR | Grp. 0 | 28.89±1.0 | 28.73±3.0 | 27.97±1.0 | 29.47±2.0 | 28.04±3.0 | 28.62±2.0 | 27.28±1.0 | 28.41±1.0 | 28.37±1.0 | 26.48±1.0 | 28.05±2.0
Grp. 1 | 31.91±1.0 | 30.96±2.0 | 31.05±1.0 | 32.19±2.0 | 36.26±2.0 | 33.96±3.0 | 27.28±1.0 | 32.02±1.0 | 31.83±2.0 | 30.20±1.0 | 31.09±2.0
FNR | Grp. 0 | 21.03±1.0 | 20.65±1.0 | 21.47±1.0 | 20.86±3.0 | 21.63±2.0 | 21.81±2.0 | 22.49±1.0 | 20.06±0.0 | 23.08±2.0 | 20.71±1.0 | 20.16±2.0
Grp. 1 | 16.45±1.0 | 16.99±1.0 | 17.40±2.0 | 16.82±1.0 | 13.45±1.0 | 16.28±2.0 | 20.53±2.0 | 15.86±1.0 | 17.45±2.0 | 16.81±1.0 | 16.04±2.0
EqOdd | - | 96.20±1.0 | 97.05±0.0 | 96.43±1.0 | 96.62±1.0 | 91.80±1.0 | 94.57±4.0 | 98.74±0.0 | 96.09±1.0 | 95.46±1.0 | 96.19±0.0 | 96.42±0.0
Race | BCE | Grp. 0 | 0.68±0.02 | 0.66±0.04 | 0.95±0.12 | 0.69±0.02 | 0.55±0.03 | 0.68±0.01 | 0.62±0.04 | 0.67±0.05 | 0.71±0.05 | 0.89±0.00 | 0.65±0.01
Grp. 1 | 0.72±0.04 | 0.72±0.08 | 0.95±0.12 | 0.75±0.04 | 0.60±0.05 | 0.75±0.01 | 0.71±0.06 | 0.74±0.06 | 0.75±0.03 | 0.97±0.01 | 0.71±0.01
ECE | Grp. 0 | 0.20±0.01 | 0.19±0.02 | 0.23±0.02 | 0.21±0.01 | 0.10±0.04 | 0.21±0.01 | 0.18±0.02 | 0.20±0.01 | 0.21±0.01 | 0.26±0.00 | 0.20±0.01
Grp. 1 | 0.24±0.02 | 0.24±0.03 | 0.27±0.03 | 0.26±0.02 | 0.15±0.05 | 0.26±0.01 | 0.25±0.03 | 0.26±0.02 | 0.26±0.01 | 0.33±0.00 | 0.25±0.01
TPR@80 TNR | Grp. 0 | 68.42±1.0 | 67.77±3.0 | 69.90±0.0 | 69.04±1.0 | 67.13±1.0 | 65.60±2.0 | 68.46±2.0 | 68.83±2.0 | 66.73±1.0 | 70.39±1.0 | 69.85±1.0
Grp. 1 | 68.99±1.0 | 68.24±4.0 | 70.34±0.0 | 68.47±1.0 | 67.01±1.0 | 65.95±2.0 | 68.04±3.0 | 68.96±2.0 | 68.33±1.0 | 70.96±0.0 | 69.97±0.0
FPR | Grp. 0 | 24.51±1.0 | 27.57±4.0 | 24.10±1.0 | 24.98±1.0 | 27.67±3.0 | 29.97±3.0 | 24.57±1.0 | 27.55±3.0 | 26.56±1.0 | 24.69±1.0 | 24.05±2.0
Grp. 1 | 36.49±1.0 | 38.83±3.0 | 36.23±2.0 | 37.09±1.0 | 39.88±3.0 | 42.18±2.0 | 35.81±1.0 | 38.82±4.0 | 38.75±1.0 | 36.10±1.0 | 36.26±2.0
FNR | Grp. 0 | 26.11±0.0 | 22.98±2.0 | 24.76±2.0 | 24.87±1.0 | 23.98±2.0 | 22.47±2.0 | 25.65±1.0 | 21.93±2.0 | 25.44±2.0 | 23.91±1.0 | 25.17±2.0
Grp. 1 | 13.78±1.0 | 12.75±0.0 | 13.60±2.0 | 13.19±1.0 | 13.83±1.0 | 12.30±0.0 | 15.17±1.0 | 12.49±1.0 | 13.46±1.0 | 13.26±0.0 | 13.49±1.0
EqOdd | - | 87.84±1.0 | 89.25±1.0 | 88.35±1.0 | 88.10±0.0 | 88.82±1.0 | 88.81±1.0 | 89.13±1.0 | 89.64±0.0 | 87.91±1.0 | 88.97±0.0 | 88.05±0.0
MIMIC-CXR CheXpert | Age | BCE | Grp. 0 | 0.48±0.03 | 0.63±0.10 | 0.80±0.11 | 0.52±0.05 | 0.48±0.04 | 0.64±0.08 | 0.47±0.04 | 0.44±0.02 | 0.50±0.02 | 0.87±0.03 | 0.91±0.18
Grp. 1 | 0.39±0.01 | 0.46±0.02 | 0.70±0.05 | 0.42±0.03 | 0.41±0.00 | 0.44±0.02 | 0.39±0.01 | 0.39±0.02 | 0.40±0.01 | 0.70±0.00 | 0.77±0.21
Grp. 2 | 0.33±0.01 | 0.35±0.01 | 0.55±0.04 | 0.35±0.02 | 0.35±0.01 | 0.36±0.02 | 0.35±0.02 | 0.33±0.02 | 0.34±0.01 | 0.61±0.00 | 0.74±0.23
Grp. 3 | 0.29±0.01 | 0.30±0.02 | 0.45±0.03 | 0.30±0.02 | 0.33±0.02 | 0.31±0.03 | 0.34±0.02 | 0.29±0.02 | 0.29±0.01 | 0.53±0.00 | 0.71±0.24
Grp. 4 | 0.23±0.01 | 0.23±0.02 | 0.25±0.01 | 0.24±0.02 | 0.28±0.03 | 0.25±0.03 | 0.30±0.02 | 0.23±0.02 | 0.23±0.01 | 0.43±0.01 | 0.63±0.20
ECE | Grp. 0 | 0.15±0.03 | 0.17±0.03 | 0.16±0.01 | 0.18±0.04 | 0.17±0.02 | 0.26±0.05 | 0.11±0.04 | 0.13±0.02 | 0.15±0.02 | 0.34±0.01 | 0.42±0.08
Grp. 1 | 0.11±0.02 | 0.11±0.02 | 0.13±0.00 | 0.13±0.03 | 0.12±0.01 | 0.14±0.03 | 0.09±0.02 | 0.09±0.03 | 0.11±0.02 | 0.28±0.00 | 0.36±0.09
Grp. 2 | 0.11±0.02 | 0.12±0.02 | 0.10±0.00 | 0.13±0.03 | 0.13±0.02 | 0.14±0.02 | 0.13±0.02 | 0.11±0.03 | 0.12±0.01 | 0.27±0.00 | 0.38±0.10
Grp. 3 | 0.12±0.01 | 0.12±0.02 | 0.08±0.00 | 0.14±0.02 | 0.15±0.03 | 0.15±0.03 | 0.17±0.01 | 0.12±0.02 | 0.13±0.01 | 0.26±0.00 | 0.40±0.11
Grp. 4 | 0.13±0.01 | 0.12±0.02 | 0.04±0.00 | 0.14±0.02 | 0.16±0.03 | 0.15±0.03 | 0.19±0.01 | 0.13±0.02 | 0.13±0.01 | 0.25±0.00 | 0.39±0.10
TPR@80 TNR | Grp. 0 | 70.66±4.0 | 66.39±8.0 | 70.65±4.0 | 71.05±1.0 | 72.94±4.0 | 56.94±5.0 | 69.71±4.0 | 71.65±5.0 | 66.65±6.0 | 75.09±3.0 | 59.25±3.0
Grp. 1 | 79.15±2.0 | 74.96±2.0 | 79.80±2.0 | 79.88±2.0 | 78.30±1.0 | 74.56±1.0 | 80.04±2.0 | 79.77±2.0 | 78.19±2.0 | 83.09±1.0 | 72.68±1.0
Grp. 2 | 79.56±1.0 | 76.94±1.0 | 79.11±1.0 | 78.12±1.0 | 77.38±3.0 | 75.84±1.0 | 76.69±1.0 | 78.49±1.0 | 77.97±1.0 | 81.50±0.0 | 71.92±1.0
Grp. 3 | 69.59±1.0 | 67.66±2.0 | 70.69±1.0 | 66.28±2.0 | 66.74±4.0 | 66.72±2.0 | 69.47±1.0 | 68.81±1.0 | 68.29±2.0 | 72.24±0.0 | 64.09±2.0
Grp. 4 | 63.10±3.0 | 63.24±3.0 | 62.55±2.0 | 60.50±5.0 | 61.11±2.0 | 63.16±2.0 | 63.92±2.0 | 61.32±3.0 | 59.01±3.0 | 66.67±4.0 | 63.60±1.0
FPR | Grp. 0 | 22.28±2.0 | 25.16±6.0 | 19.39±6.0 | 25.32±2.0 | 23.08±6.0 | 44.07±7.0 | 16.83±4.0 | 18.75±6.0 | 23.88±6.0 | 19.55±2.0 | 34.62±11.0
Grp. 1 | 16.71±1.0 | 18.78±2.0 | 16.67±2.0 | 18.10±1.0 | 17.03±1.0 | 20.09±2.0 | 14.29±2.0 | 16.98±2.0 | 16.82±2.0 | 17.77±1.0 | 21.98±2.0
Grp. 2 | 11.30±1.0 | 13.05±2.0 | 10.94±1.0 | 11.31±0.0 | 12.65±1.0 | 12.19±1.0 | 10.90±1.0 | 12.41±1.0 | 11.18±2.0 | 11.87±1.0 | 15.60±2.0
Grp. 3 | 8.20±1.0 | 9.91±2.0 | 7.80±1.0 | 7.08±0.0 | 10.10±1.0 | 8.27±1.0 | 10.08±2.0 | 9.72±1.0 | 8.12±1.0 | 7.81±1.0 | 11.92±2.0
Grp. 4 | 4.26±1.0 | 5.76±1.0 | 3.77±1.0 | 3.04±1.0 | 7.46±2.0 | 3.77±1.0 | 7.93±2.0 | 6.25±1.0 | 3.88±1.0 | 3.65±1.0 | 6.76±2.0
FNR | Grp. 0 | 26.67±3.0 | 28.89±7.0 | 28.33±5.0 | 25.56±2.0 | 24.45±7.0 | 21.11±8.0 | 36.11±8.0 | 27.22±5.0 | 30.56±7.0 | 23.33±3.0 | 30.00±3.0
Grp. 1 | 25.58±2.0 | 26.92±4.0 | 26.23±5.0 | 23.29±2.0 | 26.98±2.0 | 24.93±5.0 | 32.89±4.0 | 25.49±4.0 | 27.04±5.0 | 20.91±1.0 | 23.53±4.0
Grp. 2 | 36.34±2.0 | 36.82±2.0 | 37.16±3.0 | 39.17±2.0 | 37.77±5.0 | 37.99±3.0 | 39.80±3.0 | 35.63±3.0 | 39.30±4.0 | 33.26±1.0 | 36.14±4.0
Grp. 3 | 57.93±4.0 | 54.62±5.0 | 58.90±3.0 | 63.44±2.0 | 53.34±4.0 | 61.17±5.0 | 52.49±5.0 | 53.74±3.0 | 60.43±5.0 | 57.37±2.0 | 53.85±3.0
Grp. 4 | 76.89±2.0 | 75.13±4.0 | 77.91±5.0 | 83.59±4.0 | 66.92±4.0 | 81.56±8.0 | 66.79±7.0 | 72.47±4.0 | 81.94±4.0 | 81.31±2.0 | 70.20±6.0
Sex | BCE | Grp. 0 | 0.30±0.00 | 0.32±0.02 | 0.30±0.02 | 0.29±0.03 | 0.32±0.01 | 0.33±0.01 | 0.33±0.04 | 0.30±0.02 | 0.30±0.01 | 0.56±0.00 | 1.25±0.17
Grp. 1 | 0.30±0.00 | 0.32±0.02 | 0.28±0.02 | 0.29±0.04 | 0.31±0.01 | 0.32±0.03 | 0.30±0.03 | 0.29±0.02 | 0.29±0.01 | 0.55±0.01 | 1.18±0.18
ECE | Grp. 0 | 0.12±0.00 | 0.13±0.02 | 0.06±0.01 | 0.10±0.03 | 0.13±0.01 | 0.14±0.01 | 0.14±0.04 | 0.11±0.02 | 0.10±0.01 | 0.26±0.00 | 0.54±0.07
Grp. 1 | 0.12±0.01 | 0.14±0.02 | 0.05±0.01 | 0.11±0.04 | 0.12±0.01 | 0.14±0.02 | 0.11±0.03 | 0.10±0.03 | 0.10±0.01 | 0.26±0.00 | 0.52±0.08
TPR@80 TNR | Grp. 0 | 80.62±1.0 | 78.88±0.0 | 77.95±1.0 | 79.46±0.0 | 78.89±1.0 | 78.83±0.0 | 79.07±0.0 | 79.59±1.0 | 78.48±1.0 | 81.61±1.0 | 77.28±0.0
Grp. 1 | 76.90±1.0 | 76.31±0.0 | 75.77±1.0 | 77.08±1.0 | 76.86±2.0 | 76.19±0.0 | 76.37±1.0 | 76.50±2.0 | 75.80±1.0 | 78.75±0.0 | 74.92±2.0
FPR | Grp. 0 | 9.24±1.0 | 9.05±1.0 | 10.36±1.0 | 9.59±1.0 | 10.63±1.0 | 10.85±2.0 | 10.34±1.0 | 10.38±0.0 | 11.22±1.0 | 8.82±1.0 | 12.10±1.0
Grp. 1 | 8.69±1.0 | 9.10±0.0 | 8.82±1.0 | 9.60±3.0 | 9.64±1.0 | 10.65±4.0 | 7.86±1.0 | 9.69±1.0 | 9.48±2.0 | 8.18±1.0 | 10.33±0.0
FNR | Grp. 0 | 42.01±4.0 | 42.96±2.0 | 40.32±1.0 | 41.99±1.0 | 39.79±4.0 | 40.89±6.0 | 38.74±2.0 | 39.68±2.0 | 39.78±2.0 | 40.45±3.0 | 37.99±2.0
Grp. 1 | 43.65±3.0 | 42.55±0.0 | 44.56±3.0 | 41.57±6.0 | 41.60±2.0 | 41.53±8.0 | 46.32±4.0 | 42.22±1.0 | 42.04±4.0 | 43.58±3.0 | 41.56±1.0
EqOdd | - | 98.42±1.0 | 98.94±1.0 | 97.11±2.0 | 97.49±2.0 | 97.91±1.0 | 95.27±3.0 | 94.97±1.0 | 98.38±0.0 | 97.96±1.0 | 98.12±0.0 | 97.33±1.0
Race | BCE | Grp. 0 | 0.31±0.02 | 0.29±0.01 | 0.28±0.01 | 0.31±0.02 | 0.30±0.02 | 0.32±0.00 | 0.35±0.04 | 0.28±0.01 | 0.31±0.01 | 0.57±0.00 | 1.09±0.22
Grp. 1 | 0.32±0.03 | 0.30±0.01 | 0.29±0.01 | 0.32±0.02 | 0.31±0.01 | 0.33±0.00 | 0.36±0.04 | 0.29±0.01 | 0.32±0.01 | 0.58±0.00 | 1.09±0.22
ECE | Grp. 0 | 0.13±0.03 | 0.11±0.02 | 0.05±0.01 | 0.13±0.02 | 0.11±0.02 | 0.13±0.01 | 0.15±0.03 | 0.09±0.02 | 0.13±0.01 | 0.27±0.00 | 0.54±0.07
Grp. 1 | 0.12±0.03 | 0.11±0.02 | 0.05±0.01 | 0.13±0.02 | 0.11±0.02 | 0.13±0.00 | 0.15±0.04 | 0.08±0.02 | 0.13±0.01 | 0.27±0.00 | 0.54±0.07
TPR@80 TNR | Grp. 0 | 78.40±1.0 | 77.71±1.0 | 79.24±0.0 | 78.88±1.0 | 78.61±2.0 | 75.59±2.0 | 76.24±2.0 | 76.21±3.0 | 79.38±0.0 | 80.59±0.0 | 73.78±1.0
Grp. 1 | 78.62±1.0 | 76.37±0.0 | 77.68±1.0 | 77.54±2.0 | 76.14±2.0 | 74.87±1.0 | 73.83±2.0 | 73.89±4.0 | 78.36±0.0 | 80.33±0.0 | 73.84±1.0
FPR | Grp. 0 | 9.11±1.0 | 10.05±0.0 | 9.31±1.0 | 8.71±1.0 | 11.06±1.0 | 11.35±2.0 | 9.12±1.0 | 9.72±2.0 | 9.89±1.0 | 8.96±1.0 | 13.10±0.0
Grp. 1 | 9.45±1.0 | 10.27±0.0 | 9.58±1.0 | 9.06±2.0 | 11.29±1.0 | 11.71±1.0 | 9.49±1.0 | 9.94±2.0 | 10.47±1.0 | 9.43±0.0 | 13.68±0.0
FNR | Grp. 0 | 42.12±3.0 | 40.93±1.0 | 42.98±3.0 | 43.03±5.0 | 37.98±3.0 | 39.11±4.0 | 43.78±2.0 | 43.39±2.0 | 39.63±4.0 | 39.58±1.0 | 39.00±1.0
Grp. 1 | 43.30±3.0 | 41.58±0.0 | 44.26±3.0 | 43.99±6.0 | 39.95±2.0 | 40.25±5.0 | 46.54±2.0 | 45.88±4.0 | 40.42±5.0 | 41.08±1.0 | 36.97±1.0
EqOdd | - | 99.24±0.0 | 99.15±0.0 | 99.23±0.0 | 99.34±0.0 | 98.90±1.0 | 99.25±1.0 | 98.43±0.0 | 98.65±1.0 | 99.32±0.0 | 99.01±0.0 | 98.70±1.0

Table A14: Results of other metrics in out-of-distribution setting. In the
dataset column, the dataset in the first row is the training domain, and the
second row is the testing domain.

Dataset | Attr. | Metrics | Group | ERM | Resample | DomainInd | LAFTR | CFair | LNL | EnD | ODR | GroupDRO | SWAD | SAM
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
ADNI 1.5T ADNI 3T | Age | BCE | Grp. 0 | 0.68±0.30 | 0.72±0.30 | 1.25±0.51 | 1.05±0.57 | 0.66±0.06 | 0.64±0.26 | 0.65±0.01 | 0.60±0.14 | 0.81±0.22 | 0.51±0.12 | 0.72±0.03
Grp. 1 | 0.44±0.11 | 0.48±0.14 | 0.49±0.13 | 0.56±0.13 | 0.51±0.09 | 0.34±0.04 | 0.58±0.04 | 0.44±0.06 | 0.54±0.08 | 0.56±0.17 | 0.72±0.09
ECE | Grp. 0 | 0.23±0.08 | 0.24±0.07 | 0.29±0.05 | 0.26±0.05 | 0.19±0.05 | 0.21±0.06 | 0.22±0.10 | 0.22±0.04 | 0.26±0.04 | 0.16±0.03 | 0.19±0.02
Grp. 1 | 0.16±0.03 | 0.16±0.05 | 0.14±0.04 | 0.20±0.04 | 0.16±0.05 | 0.14±0.01 | 0.13±0.03 | 0.14±0.05 | 0.16±0.03 | 0.13±0.01 | 0.25±0.11
TPR@80 TNR | Grp. 0 | 88.88±5.0 | 79.80±12.0 | 72.16±10.0 | 66.48±11.0 | 55.28±20.0 | 78.69±17.0 | 55.28±17.0 | 77.96±8.0 | 68.08±23.0 | 78.17±9.0 | 31.94±10.0
Grp. 1 | 81.89±6.0 | 82.21±7.0 | 80.21±7.0 | 68.75±21.0 | 68.24±16.0 | 92.10±8.0 | 60.00±3.0 | 77.59±1.0 | 69.63±13.0 | 84.70±13.0 | 26.11±12.0
FPR | Grp. 0 | 16.67±9.0 | 14.94±7.0 | 13.79±8.0 | 17.82±13.0 | 32.76±30.0 | 11.49±2.0 | 39.08±24.0 | 18.39±17.0 | 18.96±15.0 | 9.19±5.0 | 81.61±17.0
Grp. 1 | 28.50±9.0 | 30.11±13.0 | 27.96±7.0 | 38.71±19.0 | 41.94±27.0 | 17.20±7.0 | 55.91±29.0 | 29.03±12.0 | 43.01±26.0 | 11.83±2.0 | 92.47±5.0
FNR | Grp. 0 | 15.97±8.0 | 22.92±9.0 | 36.81±10.0 | 33.33±16.0 | 25.00±15.0 | 34.72±17.0 | 15.28±13.0 | 23.61±17.0 | 28.48±8.0 | 31.95±5.0 | 6.95±5.0
Grp. 1 | 5.56±0.0 | 5.56±0.0 | 3.70±5.0 | 9.26±7.0 | 10.18±13.0 | 3.71±3.0 | 14.82±21.0 | 12.97±13.0 | 7.41±6.0 | 14.82±3.0 | 3.71±3.0
EqOdd | - | 88.12±5.0 | 83.74±7.0 | 76.37±8.0 | 77.52±10.0 | 83.30±8.0 | 80.26±6.0 | 88.11±7.0 | 87.00±13.0 | 77.44±10.0 | 88.75±3.0 | 92.48±8.0
Sex | BCE | Grp. 0 | 0.62±0.14 | 0.55±0.14 | 0.84±0.25 | 0.76±0.20 | 0.59±0.22 | 0.66±0.06 | 0.59±0.03 | 1.18±0.38 | 0.52±0.09 | 0.55±0.15 | 0.71±0.04
Grp. 1 | 0.63±0.14 | 0.50±0.10 | 1.00±0.48 | 0.70±0.20 | 0.66±0.24 | 0.66±0.04 | 0.54±0.01 | 1.16±0.42 | 0.53±0.10 | 0.59±0.06 | 0.73±0.05
ECE | Grp. 0 | 0.21±0.03 | 0.22±0.03 | 0.27±0.07 | 0.27±0.06 | 0.21±0.05 | 0.21±0.08 | 0.11±0.02 | 0.28±0.07 | 0.19±0.05 | 0.16±0.04 | 0.21±0.04
Grp. 1 | 0.19±0.04 | 0.17±0.05 | 0.23±0.04 | 0.20±0.05 | 0.20±0.05 | 0.15±0.05 | 0.12±0.02 | 0.27±0.07 | 0.14±0.04 | 0.15±0.02 | 0.18±0.04
TPR@80 TNR | Grp. 0 | 59.78±8.0 | 69.99±14.0 | 54.89±18.0 | 51.67±11.0 | 64.66±23.0 | 30.44±19.0 | 60.86±12.0 | 80.07±10.0 | 62.22±21.0 | 75.72±15.0 | 26.89±9.0
Grp. 1 | 71.23±11.0 | 75.58±8.0 | 67.33±14.0 | 62.35±16.0 | 64.69±10.0 | 40.41±19.0 | 74.81±10.0 | 80.66±5.0 | 78.27±12.0 | 77.17±4.0 | 17.65±5.0
FPR | Grp. 0 | 28.03±11.0 | 24.24±9.0 | 38.63±9.0 | 42.42±11.0 | 32.58±14.0 | 60.60±32.0 | 19.70±26.0 | 28.79±16.0 | 26.51±10.0 | 19.70±5.0 | 87.88±13.0
Grp. 1 | 28.51±9.0 | 16.23±11.0 | 34.21±22.0 | 34.65±14.0 | 26.76±14.0 | 59.21±26.0 | 16.67±24.0 | 14.91±5.0 | 21.05±6.0 | 17.55±8.0 | 86.85±9.0
FNR | Grp. 0 | 8.89±11.0 | 20.00±20.0 | 3.33±6.0 | 7.78±8.0 | 6.66±7.0 | 12.22±24.0 | 33.33±29.0 | 11.11±8.0 | 12.22±11.0 | 22.22±15.0 | 0.00±0.0
Grp. 1 | 19.13±12.0 | 21.60±11.0 | 14.81±4.0 | 17.90±7.0 | 23.46±11.0 | 22.22±18.0 | 20.99±12.0 | 19.75±9.0 | 19.13±7.0 | 22.22±7.0 | 8.64±6.0
EqOdd | - | 89.40±5.0 | 90.89±5.0 | 87.66±5.0 | 89.33±7.0 | 86.49±6.0 | 88.02±7.0 | 89.84±3.0 | 88.74±4.0 | 89.93±3.0 | 93.55±4.0 | 93.96±3.0

##  Appendix D Incorporate new datasets and algorithms in MEDFAIR

We implement MEDFAIR using PyTorch framework. We show example codes to
demonstrate how to incorporate new datasets and algorithms. A detailed
documentation can be found in supplementary materials.

###  D.1 Adding new datasets

We implement a base dataset class BaseDataset, and new dataset can be added by
creating a new file and inheriting it.

from datasets.BaseDataset import BaseDataset class DatasetX(BaseDataset): def
__init__(self, metadata, path_to_images, sens_name, transform):
super(DatasetX, self).__init__(metadata, sens_name, transform)

def __getitem__(self, idx): item = self.metadata.iloc[idx] img =
Image.open(path_to_images[idx])

# apply image transform/augmentation img = self.transform(img) label =
torch.FloatTensor([int(item[’binaryLabel’])])

# convert to sensitive attributes to numerical values sensitive =
self.get_sensitive(self.sens_name, self.sens_classes, item)

return idx, img, label, sensitive

###  D.2 Adding new algorithm

We implement a base algorithm class BaseNet, where contains basic
configuration and regular training/validation/testing loop. A new algorithm
can be added by inheriting it and rewriting the training loop, loss, etc. if
needed.

For example, SAM (Foret et al.,, 2020) algorithm can be added by re-
implementing the training loop. class SAM(BaseNet): def __init__(self, opt,
wandb): super(SAM, self).__init__(opt, wandb) def _train(self, loader):
self.network.train() for i, (index, images, targets, sensitive_attr) in
enumerate(loader):

enable_running_stats(self.network) outputs, _= self.network(images)

loss = self._criterion(outputs, targets) loss.mean().backward()
self.optimizer.first_step(zero_grad=True) self.scheduler.step()

disable_running_stats(self.network) outputs, _= self.network(images)
self._criterion(outputs, targets).mean().backward()
self.optimizer.second_step(zero_grad=True) self.scheduler.step()


# Wavelet-based Bi-dimensional Aggregation Network for SAR Image Change
Detection

Jiangwei Xie, Feng Gao, Xiaowei Zhou, Junyu Dong This work was supported in
part by the National Science and Technology Major Project under Grant
2022ZD0117202, in part by the Natural Science Foundation of Qingdao under
Grant 23-2-1-222-ZYYD-JCH, and in part by the Postdoctoral Fellowship Program
of CPSF under Grant GZC20241614. (Corresponding author: Xiaowei Zhou.)
Jiangwei Xie, Feng Gao, Xiaowei Zhou, and Junyu Dong are with the School of
Computer Science and Technology, Ocean University of China, Qingdao 266100,
China.

###### Abstract

Synthetic aperture radar (SAR) image change detection is critical in remote
sensing image analysis. Recently, the attention mechanism has been widely used
in change detection tasks. However, existing attention mechanisms often employ
down-sampling operations such as average pooling on the Key and Value
components to enhance computational efficiency. These irreversible operations
result in the loss of high-frequency components and other important
information. To address this limitation, we develop Wavelet-based Bi-
dimensional Aggregation Network (WBANet) for SAR image change detection. We
design a wavelet-based self-attention block that includes discrete wavelet
transform and inverse discrete wavelet transform operations on Key and Value
components. Hence, the feature undergoes downsampling without any loss of
information, while simultaneously enhancing local contextual awareness through
an expanded receptive field. Additionally, we have incorporated a bi-
dimensional aggregation module that boosts the non-linear representation
capability by merging spatial and channel information via broadcast mechanism.
Experimental results on three SAR datasets demonstrate that our WBANet
significantly outperforms contemporary state-of-the-art methods. Specifically,
our WBANet achieves 98.33%, 96.65%, and 96.62% of percentage of correct
classification (PCC) on the respective datasets, highlighting its superior
performance. Source codes are available at
https://github.com/summitgao/WBANet.

###### Index Terms:

Change detection; Synthetic aperture radar; Wavelet transform; Bi-dimensional
aggregation module.

##  I Introduction

Synthetic aperture radar (SAR) is adept at producing high-resolution images of
the Earth’s surface, even under conditions of low visibility caused by adverse
weather [1]. SAR sensors can penetrate cloud cover, making them especially
valuable for Earth observation in cloudy or rainy areas. Consequently, SAR
data has garnered significant interest from the research community, supporting
a range of applications such as object detection [2], disaster assessment [3],
change detection [4], and image classification [5]. Among these, change
detection serves as a crucial tool for identifying changes in land cover,
urban growth, and deforestation.

Recently, various convolutional neural network based models for change
detection have been developed, demonstrating significant advancements in
performance. Hou et al. [6] introduced an end-to-end dual branch architecture
that merges CNN with a generative adversarial network (GAN), enhancing the
detection of fine-grained changes. Wang et al. [7] introduced distinctive
patch convolution combined with random label propagation, achieving high
accuracy in change detection at a reduced computational cost. Zhao et al. [8]
utilized a multidomain fusion module that integrates spatial and frequency
domain features into complementary feature representations. Zhu et al. [9]
designed a feature comparison module that limits the number of feature
channels in the fusion process, enabling better utilization of fine-grained
information in the multiscale feature map for more accurate prediction.

The previously mentioned CNN-based methods have demonstrated impressive
achievements. Furthermore, Vision Transformer (ViT) [10] has showcased high
performance in various computer vision tasks, leading to the adoption of
attention mechanisms in change detection models. Zhang et al. [11] combine
convolution and attention mechanisms to improve the performance of SAR image
change detection. Although these pioneer efforts have achieved promising
performance, designing an attention-based network for SAR change detection is
still a non-trivial task, due to the following reasons: 1) High-frequency
information loss in self-attention computation. Traditional down-sampling
methods on the Key and Value components in efficient attention mechanisms,
often result in the loss of high-frequency components like texture details. 2)
Limitation in non-linear feature transformation. Existing methods require MLP-
like structure for non-linear feature transformation. However, spatial and
channel-wise attentions are rarely exploited simultaneously.

To address the above two limitations, we propose a Wavelet-based Bi-
dimensional Aggregation Network, WBANet for short, which achieves down-
sampling without information dropping and fuses both spatial and channel
information. Specifically, we design a Wavelet-based Self-attention Module
(WSM) which uses Discrete Wavelet Transform (DWT) and Inverse Discrete Wavelet
Transform (IDWT) to enable lossless and invertible down-sampling in the self-
attention computation. In addition, we develop a Bi-dimensional Aggregation
Module (BAM) to enhance the non-linear feature representation capabilities.
This module efficiently captures both spatial and channel-wise feature
dependencies.

Figure 1: An overview of the proposed Wavelet-based Bi-dimensional Aggregation
Network (WBANet). The WBANet comprises pre-classification module, wavelet-
based bi-dimensional aggregation blocks. Each wavelet-based bi-dimensional
aggregation block has two critical components: Wavelet-based Self-attention
Module and Bi-dimensional Aggregation Module.

In summary, the contributions of this letter can be summarized as follows:

  * •

We propose WSM to integrates DWT and IDWT for down-sampling without
information loss, thus preserving textures and other high-frequency details.

  * •

We develop BAM that captures both spatial and channel-wise feature
dependencies effectively. This module merges information from two branches and
enhances the non-linear feature representation capabilities.

  * •

Extensive experiments are conducted on three public SAR datasets,
demonstrating the efficacy of our proposed WBANet. We have made our code
publicly available to benefit other researchers.

##  II Methodology

The framework of the proposed WBANet is illustrated in Fig. 1. First of all,
two multitemporal SAR images ($I_{1}$ and $I_{2}$), captured at different
times over the same geographic region, are fed into the network. The objective
of the change detection task is to generate a change map, marking changed
pixels as ”1” and unchanged pixels as ”0”. Initially, the pre-classification
module uses a logarithmic ratio operator to compute a difference image for
pseudo-label generation. Subsequently, the hierarchical fuzzy c-means
algorithm [12] [13] is employed to classify pixels into changed, unchanged,
and intermediate categories. Then, some wavelet-based bi-dimensional
aggregation block process these data from the pre-classification module.
Finally, the output features from this block are passed through fully
connected layer to generate the change map.

The wavelet-based bi-dimensional aggregation block is comprised of two
components: the Wavelet-based Self-attention Block (WSM) and the Bi-
dimensional Aggregation Module (BAM). We will present the details of both
modules in the following subsections.

###  II-A Wavelet-based Self-attention Module (WSM)

The proposed WSM employs Discrete Wavelet Transform (DWT) and Inverse Discrete
Wavelet Transform (IDWT) to facilitate down-sampling in the attention
mechanism. The wavelet transform enables feature extraction at both coarse and
fine-grained scales, while also ensuring that the down-sampling is invertible.
Due to the simple structure and high computational efficiency of Haar wavelet,
it can quickly complete the downsampling operation. Furthermore, SAR change
images often exhibit considerable sharp high-frequency information while Haar
wavelet is adept at effectively capturing these high-frequency components[14].
Thus, we select Haar wavelet to conduct the downsample operation. The
structure of this module is shown in Fig. 2.

To efficiently process the input feature $X\in\mathbb{R}^{H\times W\times C}$,
we first reduce its channel dimensions to $\widetilde{X}\in\mathbb{R}^{H\times
W\times\frac{C}{4}}$ using a learnable transformation matrix
$W_{d}\in\mathbb{R}^{C\times\frac{C}{4}}$. Following this channel reduction,
we apply DWT with the Haar wavelet to down-sample $\widetilde{X}$, and
decompose it into four distinct subbands.

Haar wavelet is composed of the low-pass filter
$f_{L}=\left(\frac{1}{\sqrt{2}},\frac{1}{\sqrt{2}}\right)$ and high-pass
filter $f_{H}=\left(\frac{1}{\sqrt{2}},-\frac{1}{\sqrt{2}}\right)$. We first
encode $\widetilde{X}$ into two subbands $X_{L}$ and $X_{H}$ along the rows.
Subsequently, these subbands are processed using the same filters along the
columns, resulting in four wavelet subbands: $X_{LL},X_{LH},X_{HL},$ and
$X_{HH}$. Here,
$X_{LL}\in\mathbb{R}^{\frac{H}{2}\times\frac{W}{2}\times\frac{C}{4}}$ encodes
the low-frequency components, and contains coarse-grained structural
information.
$X_{LH},X_{HL},X_{HH}\in\mathbb{R}^{\frac{H}{2}\times\frac{W}{2}\times\frac{C}{4}}$
represent the high-frequency components, and describe fine-grained textures.

We then concatenate these four subbands along the channel dimension to get
$\hat{X}$:

$\hat{X}=\textrm{Concat}(X_{LL},X_{LH},X_{HL},X_{HH})$ (1)

Figure 2: Comparison of the traditional self-attention block in PVT(Pyramid
Vision Transformer) and the proposed wavelet-based self-attention module.
Linear denotes fully connected layers and $\otimes$ denotes matrix
multiplication.

The concatenated output $\hat{X}$ is transformed into Key ($K^{w}$) and Value
($V^{w}$) matrices through the convolutional layer, while the Query ($Q$)
remains the original input image $X$. In this case, wavelet-based multi-head
self-attention computes the interaction across these elements for each head as
follows:

$\displaystyle\text{head}_{i}$
$\displaystyle=\textrm{Attention}\left({Q}_{i},{K}_{i}^{{w}},{V}_{{i}}^{{w}}\right)$
(2)
$\displaystyle=\textrm{Softmax}\left(\frac{{Q}_{i}{K}_{i}^{{w}}}{\sqrt{D_{h}}}\right){V}_{i}^{{w}}$

where ${K}_{i}^{w}$ denotes the down-sampled key, ${V}_{i}^{w}$ denotes the
down-sampled value, and ${D_{h}}$ represents the dimension of each head.

To enhance the output of the wavelet-based self-attention block, we apply the
IDWT to $\hat{X}$ to produce $X^{\boldsymbol{r}}$. The reconstructed
$X^{\boldsymbol{r}}$ mirrors the details of the original input image,
providing excellent local contextualization and an expanded receptive field.
The final output integrates the contributions of each attention head with this
reconstructed map. This integration is essential for effectively capturing
information across multiple scales.

Figure 3: Illustration of the Bi-dimensional Aggregation Module. It includes
two branches: global channel attention and local spatial attention. \it{c}⃝
indicates to concatenate in channel axis. $\oplus$ implies element-wise
addition and $\odot$ denotes element-wise multiplication.

The overall operation can be formulated as follows:

$\textrm{WaveAttn}(X)=\textrm{Concat}\left(\textit{head}_{0},\cdots,\textit{head}_{N_{h}},{X}^{r}\right){W}^{O},$
(3)

where ${N_{h}}$ represents the number of attention heads, and ${W}^{O}$ is the
transformation matrix that combines all the heads and the reconstructed image
into a single output tensor. The use of wavelet transform in the self-
attention mechanism significantly enhances the ability to contextualize
information over longer ranges with a reduced computational load compared to
conventional self-attention modules. This approach ensures that both global
coherence and local detail are preserved and emphasized in the model’s
outputs.

###  II-B Bi-dimensional Aggregation Module (BAM)

To enhance the non-linear representation capabilities and effectively capture
both spatial and channel dependencies, we develop Bi-dimensional Aggregation
Module (BAM), as depicted in Fig. 3. This module includes two branches: the
channel aggregation branch and the spatial aggregation branch.

Channel Aggregation: In this branch, average pooling is applied in the spatial
dimension of input features $X\in\mathbb{R}^{{H\times{W}\times{C}}}$ to
aggregate global representations. Subsequently, a fully connected layer (FC)
coupled with a GELU activation function reduces the channel dimensions from
$C$ to $\frac{C}{r}$, producing an intermediate output $\hat{X}$. Here, $r$,
the reduction ratio, is set to 2. This is followed by FC layer and Sigmoid
activation function to generate the output of the channel attention branch,
$X^{C}\in\mathbb{R}^{1\times 1\times C}$.

Spatial Aggregation: First, a linear transformation, together with a GELU
activation, transforms the channel dimension to $\frac{C}{r}$, while the
spatial dimensions remain unchanged. The resulting intermediate output,
$\widetilde{X}$, is then concatenated with $\hat{X}$ to form
$\widetilde{X}^{\prime}\in\mathbb{R}^{H\times W\times\frac{2C}{r}}$. The
process culminates similarly to the channel attention branch, resulting in the
final output $X^{S}\in\mathbb{R}^{H\times W\times 1}$.

The outputs of both branches, $X^{C}$ and $X^{S}$, are merged through an
element-wise summation, ensuring the final output retains the same dimensions
as the original input $X$. This integration optimally combines the refined
channel and spatial information, enhancing the overall feature representation
while maintaining focus on both global context and local details.

##  III Experimental Results and Analysis

Figure 4: Visualized results of different change detection methods on the
three datasets: (a) Image captured at $t_{1}$. (b) Image captured at $t_{2}$.
(c) Ground truth image. (d)-(i) Results by different methods.

###  III-A Datasets and Evaluation Metrics

To validate the effectiveness of the proposed WBANet, we conducted
comprehensive experiments on three distinct SAR datasets: Chao Lake, the
Yellow River, and Sulzberger Datasets. Chao Lake Dataset: This dataset
includes images of Chao Lake in China, captured in May 2020 using the
Sentinel-1 sensor. This period coincides with the highest recorded water
levels in the lake’s history, providing a dynamic range of changes to detect.
Sulzberger Dataset: Captured by the European Space Agency’s Envisat satellite
over five days in March 2011, this dataset documents the breakup of an ice
shelf, offering a unique perspective on drastic natural events. Yellow River
Dataset: This dataset focuses on the Yellow River Estuary in China, with data
collected from June 2008 to June 2009 using the Radarset-2 SAR sensor. This
dataset is particularly challenging due to the pronounced speckle noise.

The hierarchical fuzzy c-means algorithm could classify pixels into changed,
unchanged, and intermediate categories. Pixels from the changed and unchanged
groups are randomly selected as training data, while intermediate group pixels
are the test data. For a thorough assessment of our model, we employed five
commonly used evaluation metrics: False Positives (FP), False Negatives (FN),
Overall Error (OE), Percentage of Correct Classification (PCC), and the Kappa
Coefficient (KC).

TABLE I: Change detection results of different methods on three Datasets. The best results are marked in Bold. Method | Results on the Chao Lake dataset
---|---
FP | FN | OE | PCC ($\%$) | KC ($\%$)
CWNN [15] | 7528 | 2213 | 9741 | 93.39 | 65.01
SAFNet [16] | 2231 | 1272 | 3503 | 97.62 | 85.55
DDNet [17] | 1472 | 1559 | 3031 | 97.94 | 87.04
LANTNet [18] | 1822 | 1023 | 2845 | 98.07 | 88.20
CAMixer [11] | 1867 | 906 | 2773 | 98.12 | 88.56
Proposed WBANet | 1092 | 1373 | 2465 | 98.33 | 89.38
Method | Results on the Sulzberger dataset
FP | FN | OE | PCC ($\%$) | KC ($\%$)
CWNN [15] | 2987 | 387 | 3374 | 94.85 | 86.95
SAFNet [16] | 1661 | 883 | 2544 | 96.12 | 89.80
DDNet [17] | 1835 | 585 | 2420 | 96.31 | 90.39
LANTNet [18] | 1761 | 635 | 2396 | 96.34 | 90.46
CAMixer [11] | 1105 | 1207 | 2312 | 96.47 | 90.56
Proposed WBANet | 1553 | 640 | 2193 | 96.65 | 91.23
Method | Results on the Yellow River dataset
FP | FN | OE | PCC ($\%$) | KC ($\%$)
CWNN [15] | 3052 | 1034 | 4086 | 94.50 | 82.46
SAFNet [16] | 2199 | 1467 | 3666 | 95.06 | 83.69
DDNet [17] | 1251 | 2222 | 3473 | 95.32 | 83.76
LANTNet [18] | 915 | 2343 | 3258 | 95.61 | 84.56
CAMixer [11] | 991 | 2126 | 3117 | 95.80 | 85.35
Proposed WBANet | 605 | 1905 | 2510 | 96.62 | 88.15

###  III-B Experimental Results and Discussion

We evaluated our WBANet against five state-of-the-art methods: CWNN [15],
SAFNet [16], DDNet [17], LANTNet [18], and CAMixer [11], implemented using
default parameters from their studies. All the experiments, except for the
CWNN running with Matlab, were conducted on the Google Colab platform with
Python 3.10.12, PyTorch 2.1.0, and an NVIDIA Tesla T4 GPU with 15 GB of
memory.

Quantitative results are detailed in Table I. For the Chao Lake dataset, our
WBANet excels in all metrics except false negatives (FN), with significant
improvements in the Kappa Coefficient (KC) by 24.37%, 3.83%, 2.34%, 1.18%, and
0.82% over CWNN, SAFNet, DDNet, LANTNet, and CAMixer, respectively. On the
Sulzberger dataset, WBANet outperforms other methods in overall error (OE),
PCC, and KC. Although CAMixer and CWNN show lower FP and FN rates
respectively, they both register higher OEs compared to our WBANet. Similarly,
on the Yellow River dataset, WBANet leads in all metrics apart from FN.
Notably, it enhances the KC value by 5.69%, 4.46%, 4.39%, 2.79%, and 2.35%
over CWNN, SAFNet, DDNet, LANTNet, and CAMixer, respectively. While CWNN
records a lower FN rate, it lags significantly behind in FP and OE.

Fig. 4 illustrates the visual comparison of change maps produced by different
methods on three datasets. Compared to the baseline methods, such as CWNN and
SAFNet, our WBANet generates change maps that are visually closer to the
ground truth and contain less noise. For instance, in the Yellow River
dataset, where speckle noise greatly impacts performance, it is challenging to
produce accurate change maps. Here, the performance of CWNN and SAFNet is
notably degraded, while DDNet, LANTNet, and CAMixer frequently misclassify
changed pixels as unchanged.

Experimental results on three SAR datasets confirm that our proposed WBANet
outperforms other state-of-the-art methods. The effectiveness of our WSM and
BAM demonstrates significant contributions to attention feature extraction and
non-linear representation modeling.

TABLE II: Ablation studies of the proposed WBANet. Method | PCC on different datasets ($\%$)
---|---
Chao Lake | Sulzberger | Yellow River
Basic Network | 97.46 | 96.02 | 95.67
w/o WSM | 98.13 | 96.33 | 95.92
w/o BAM | 97.82 | 96.51 | 96.17
Proposed WBANet | 98.33 | 96.65 | 96.62

###  III-C Ablation Study

To evaluate the effectiveness of the proposed Wavelet-based Self-attention
Block and Bi-dimensional Aggregation Module, ablation experiments were
performed on three datasets. We designed three variants: (1) Basic Network,
which is the WBANet without the WSM and BAM; (2) w/o WSM, which omits the
Wavelet-based Self-attention Module; and (3) w/o BAM, which lacks the Bi-
dimensional Aggregation Module. The results in Table II clearly show that both
the WSM and the BAM significantly enhance the non-linear representation
capabilities, thereby improving change detection performance.

Figure 5: Visualization of the feature representations on the Yellow River
dataset. (a) Features before the WSM. (b) Features after the WSM.

Additionally, we utilized the t-SNE [19] tool to visualize feature
characteristics before and after applying the Wavelet-based Self-attention
Block. As depicted in Fig. 5, the representations post-application display
more distinct, well-defined clusters compared to the original input.

###  III-D Analysis of the Block Number

The number of Wavelet-based Bi-dimensional Aggregation Blocks, denoted as $N$,
is a crucial parameter. We explored the relationship between $N$ and the
Percentage of Correct Classification (PCC) by varying $N$ from 0 to 8. As
illustrated in Fig. 6, PCC consistently improves with an increase in the
number of Wavelet-based Bi-dimensional Aggregation Blocks up to 5. However,
beyond this point, PCC begins to decline due to the increased model
complexity. Consequently, we optimized $N$ for different datasets: $N=5$ for
the Chao Lake dataset, $N=2$ for the Sulzberger dataset, and $N=4$ for the
Yellow River dataset.

##  IV Conclusion

In this letter, we introduce a novel WBANet for SAR image change detection
task. The WBANet utilizes DWT and IDWT to achieve down-sampling without the
loss of high-frequency details and other important information. Additionally,
we developed BAM to enhance non-linear representation capabilities by
capturing spatial and channel dependencies and refining features. Extensive
experiments on three SAR datasets have verified the effectiveness and
rationality of our solution.

Figure 6: Relationship between the number of attention blocks and PCC values.

## References

  * [1] J. Wang, F. Gao, J. Dong, S. Zhang, and Q. Du, “Change detection from synthetic aperture radar images via graph-based knowledge supplement network,” _IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing_ , vol. 15, pp. 1823–1836, 2022.
  * [2] I. Ševo and A. Avramović, “Convolutional neural network based automatic object detection on aerial images,” _IEEE Geoscience and Remote Sensing Letters_ , vol. 13, no. 5, pp. 740–744, 2016.
  * [3] A. Sarkar, T. Chowdhury, R. R. Murphy, A. Gangopadhyay, and M. Rahnemoonfar, “SAM-VQA: Supervised attention-based visual question answering model for post-disaster damage assessment on remote sensing imagery,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 61, pp. 1–16, 2023.
  * [4] T. Yan, Z. Wan, P. Zhang, G. Cheng, and H. Lu, “TransY-Net: Learning fully transformer networks for change detection of remote sensing images,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 61, pp. 1–12, 2023.
  * [5] X. Qian, F. Liu, L. Jiao, X. Zhang, Y. Guo, X. Liu, and Y. Cui, “Ridgelet-Nets with speckle reduction regularization for SAR image scene classification,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 59, no. 11, pp. 9290–9306, 2021.
  * [6] B. Hou, Q. Liu, H. Wang, and Y. Wang, “From W-Net to CDGAN: Bitemporal change detection via deep learning techniques,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 58, no. 3, pp. 1790–1802, 2020.
  * [7] J. Wang, F. Gao, J. Dong, Q. Du, and H.-C. Li, “Change detection from synthetic aperture radar images via dual path denoising network,” _IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing_ , vol. 15, pp. 2667–2680, 2022.
  * [8] C. Zhao, L. Ma, L. Wang, T. Ohtsuki, P. T. Mathiopoulos, and Y. Wang, “SAR image change detection in spatial-frequency domain based on attention mechanism and gated linear unit,” _IEEE Geoscience and Remote Sensing Letters_ , vol. 20, pp. 1–5, 2023.
  * [9] S. Zhu, Y. Song, Y. Zhang, and Y. Zhang, “ECFNet: A siamese network with fewer FPs and fewer FNs for change detection of remote sensing images,” _IEEE Geoscience and Remote Sensing Letters_ , vol. 20, pp. 1–5, 2023.
  * [10] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” in _Proceedings of International Conference on Learning Representations (ICLR)_ , 2021, pp. 1–21.
  * [11] H. Zhang, Z. Lin, F. Gao, J. Dong, Q. Du, and H.-C. Li, “Convolution and attention mixer for synthetic aperture radar image change detection,” _IEEE Geoscience and Remote Sensing Letters_ , vol. 20, pp. 1–5, 2023.
  * [12] F. Gao, J. Dong, B. Li, and Q. Xu, “Automatic change detection in synthetic aperture radar images based on PCANet,” _IEEE Geoscience and Remote Sensing Letters_ , vol. 13, no. 12, pp. 1792–1796, 2016.
  * [13] H.-C. Li, T. Celik, N. Longbotham, and W. J. Emery, “Gabor feature based unsupervised change detection of multitemporal sar images based on two-level clustering,” _IEEE Geoscience and Remote Sensing Letters_ , vol. 12, no. 12, pp. 2458–2462, 2015.
  * [14] S. Mallat, _A wavelet tour of signal processing_. Elsevier, 1999.
  * [15] F. Gao, X. Wang, Y. Gao, J. Dong, and S. Wang, “Sea ice change detection in SAR images based on convolutional-wavelet neural networks,” _IEEE Geoscience and Remote Sensing Letters_ , vol. 16, no. 8, pp. 1240–1244, 2019.
  * [16] Y. Gao, F. Gao, J. Dong, Q. Du, and H.-C. Li, “Synthetic aperture radar image change detection via siamese adaptive fusion network,” _IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing_ , vol. 14, pp. 10 748–10 760, 2021.
  * [17] X. Qu, F. Gao, J. Dong, Q. Du, and H.-C. Li, “Change detection in synthetic aperture radar images using a dual-domain network,” _IEEE Geoscience and Remote Sensing Letters_ , vol. 19, pp. 1–5, 2022.
  * [18] D. Meng, F. Gao, J. Dong, Q. Du, and H.-C. Li, “Synthetic aperture radar image change detection via layer attention-based noise-tolerant network,” _IEEE Geoscience and Remote Sensing Letters_ , vol. 19, pp. 1–5, 2022.
  * [19] L. Maaten and G. Hinton, “Visualizing data using t-SNE,” _Journal of Machine Learning Research_ , vol. 9, no. 86, pp. 2579–2605, 2008.


# Topological data analysis “hearing” the shapes of drums and bells

By Guo-Wei Wei

###### Abstract

Mark Kac asked a famous question in 1966, “Can one hear the shape of a drum?”,
a spectral geometry problem that has intrigued mathematicians for the last six
decades and is important to many other fields, such as architectural
acoustics, audio forensics, pattern recognition, radiology, and imaging
science. A related question is how to hear the shape of a drum. We show that
the answer was given in the set of 65 Zenghouyi chime bells dated back to
475-433 B.C. in China. The set of chime bells gradually varies their sizes and
weights to enable melodies, intervals, and temperaments. The same design
principle was used in many other musical instruments, such as xylophones, pan
flutes, pianos, etc. We reveal that there is a fascinating connection between
the progression pattern of many musical instruments and filtration (or
spectral sequence) in topological data analysis (TDA). We argue that
filtration-induced evolutionary de Rham-Hodge theory provides a new
mathematical foundation for musical instruments. Its discrete counterpart,
persistent Laplacians and many other persistent topological Laplacians,
including persistent sheaf Laplacians and persistent path Laplacians are
briefly discussed.

Figure 1: The progression patterns in musical instruments. a. The Zonghouyi
chime bells. b. An xylophone. c. A pan flute. d. Strings of a grand piano.
Image courtesy of Wikipedia.

“Can one hear the shape of a drum?”, a famous question posed by Mark Kac in
1966 [10], has intrigued mathematicians for generations. In other words, if
you hear the sound from a drum, i.e., the set of overtones produced by the
drum, can you infer its shape? Mathematically, the essence of the spectral
geometry question is whether the shape can be uniquely determined from the
eigenvalues of the Laplacian operator defined on the shape. There are many
examples of isospectral manifolds which are not isometric in two and higher
dimensional settings [15, 6, 17]. However, the problem is not closed yet — one
can discern the shapes of certain geometric types from their sounds [16, 9].
The question has a far-reaching impact on many fields beyond mathematics, such
as architectural acoustics, audio forensics, pattern recognition, radiology,
imaging science, and musical science [4, 3].

In musical composition, it is a common practice to use a drum set of varying
sizes, instead of a single piece of drum, to facilitate a tonic harmonic
progression, which is a foundation of harmony in modern music. The human brain
is trained from the drum set to distinguish the overtones of individual drum.
The comparative training/learning from a set of drums is the basis to hear the
shape of a drum from a drum set.

Interestingly, to create tonal harmony, the ancient Chinese built a set of 65
Zenghouyi chime bells dated back to 475-433 B.C. in the Warring States Period
in China (Figure 1a). The shape of each chime bell was designed to produce a
distinct sound. The set of 65 chime bells gradually varies in their sizes and
weights, ranging from 153.4 centimeters (60.4 in) to 20.4 centimeters (8.0 in)
in height and from 203.6 kilograms (449 lb) to 2.4 kilograms (5.3 lb) in
weight. The set of Zenghouyi chime bells covers from C2 to D7 tonal range and
can play all twelve half tones in the middle area of the tonal range [21],
enabling melodies, intervals, and temperaments. The same design principle is
used in many other musical instruments, such as xylophones (Figure 1b), pan
flutes (Figure 1c), and pianos (Figure 1d). Due to the close proximity among
chime bells, resonance among them may occur when they are struck, giving rise
to prolonged harmony. Made of bronze, one of the most precious metals
available at the time, chime bells were used in various rituals, ceremonies,
and entertainment.

There is a fascinating and apparent connection between the progression pattern
of musical instruments (Figure 1) and mathematical filtration (Figure 2a), a
spectral sequence technique [13] widely used in homological algebra and
topological data analysis (TDA). As a new branch of mathematics, TDA uses
topological and geometric concepts to understand and extract topological
patterns and structures in data. The main workhorse of TDA is persistent
homology [5, 22], a branch of algebraic topology that creates a mathematical
microscopy of a point cloud by filtration. Through the comparative analysis of
the topological invariant changes induced by the filtration, persistent
homology delineates the shape of data [12]. Paired with advanced machine
learning and deep learning algorithms, persistent homology has had tremendous
success in data science. It has been established as one of the most powerful
tools in simplifying the geometric complexity and reducing the high
dimensionality of biomolecular interactions [11], revolutionizing drug
discovery and the forecasting of emerging viral variants.

Figure 2: Illustration of filtration and persistent Laplacian spectra. a.
Filtration of a point cloud. b-c. The persistent Laplacian analysis of the
point cloud. Here, $\beta^{\alpha}_{j}$ and $\lambda^{\alpha}_{j}$ ($j=0,1$)
are persistent Betti-$j$ numbers and the first nonzero eigenvalues of the
$j$th persistent Laplacian, respectively. Note that $\lambda^{\alpha}_{j}$
capture the homotopic shape evolution of data (see the frequency changes after
$\alpha=1.5$) that is not reflected in $\beta^{\alpha}_{j}$. The similarity
between $\lambda^{\alpha}_{0}$ and $\lambda^{\alpha}_{1}$ is a coincidence.
Image courtesy of Dr. Jian Jiang.

However, persistent homology is not directly applicable to the tonal analysis
of chime bells. First, the set of chime bells may be regarded as the result of
a set of evolving chime bell manifolds, rather than that of the filtration of
a point cloud. Additionally, there is no change in topological invariants
associated the homotopic shape evolution of the set of chime bells. Finally,
persistent homology cannot present a frequency or tonal analysis of chime
bells or many other musical instruments.

Recently, an evolutionary de Rham-Hodge method has been proposed as a
multiscale generalization of the classical de Rham-Hodge theory, a landmark of
the 20th Century’s mathematics [1]. It provides a multiscale geometric and
topological analysis of filtration-induced manifolds, on which a family of
evolutionary Hodge Laplacians can be defined on the set of chime bells to
characterize their tonal evolution. In association with a family of
evolutionary de Rham complexes, evolutionary Hodge Laplacians reveal the full
set of topological invariants, such as Betti numbers, in their kernel or null
space dimensions.

The point-cloud counterpart of the evolutionary de Rham-Hodge method on
manifolds is called persistent spectral graph [18] (also known as persistent
Laplacians [14]) on simplicial complexes. Like the evolutionary de Rham-Hodge
method, persistent Laplacians not only return the full set of topological
invariants in their harmonic spectra as persistent homology does but also
capture the homotopic shape evolution of data during the filtration in their
first non-harmonic spectra (Figure 2b and 2c), for which persistent homology
cannot describe.

A generalization of persistent Laplacians was made through the sheaf theory
[8] and the resulting persistent sheaf Laplacians [20] allow the embedding of
heterogeneous characters in topological invariants, e.g., encoding non-
geometric information in a geometry-based simplicial complex. Another
generalization is persistent path Laplacians [19], built from the path complex
and path homology [7, 2]. Persistent path Laplacians are designed for directed
graphs and directed networks. These new persistent topological Laplacians not
only lay a mathematical foundation for the tonal analysis in musical science
but also significantly extend the applicable domain and power of TDA.

## References

  * [1] J. Chen, R. Zhao, Y. Tong, and G.-W. Wei.  Evolutionary de Rham-Hodge method.  Discrete and Continuous Dynamical Systems. Series B, 26(7):3785, 2021.
  * [2] S. Chowdhury and F. Mémoli.  Persistent path homology of directed networks.  In Proceedings of the Twenty-Ninth Annual ACM-SIAM Symposium on Discrete Algorithms, pages 1152–1169. SIAM, 2018.
  * [3] L. Cosmo, M. Panine, A. Rampini, M. Ovsjanikov, M. M. Bronstein, and E. Rodolà.  Isospectralization, or how to hear shape, style, and correspondence.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7529–7538, 2019.
  * [4] I. Dokmanić, R. Parhizkar, A. Walther, Y. M. Lu, and M. Vetterli.  Acoustic echoes reveal room shape.  Proceedings of the National Academy of Sciences, 110(30):12186–12191, 2013.
  * [5] H. Edelsbrunner, J. Harer, et al.  Persistent homology-A survey.  Contemporary Mathematics, 453:257–282, 2008.
  * [6] C. Gordon, D. Webb, and S. Wolpert.  Isospectral plane domains and surfaces via riemannian orbifolds.  Inventiones Mathematicae, 110(1):1–22, 1992.
  * [7] A. Grigor’yan, Y. Lin, Y. Muranov, and S.-T. Yau.  Homologies of path complexes and digraphs.  arXiv preprint arXiv:1207.2834, 2012.
  * [8] J. Hansen and R. Ghrist.  Toward a spectral theory of cellular sheaves.  Journal of Applied and Computational Topology, 3(4):315–358, 2019\.
  * [9] H. Hezari, Z. Lu, and J. Rowlett.  The dirichlet isospectral problem for trapezoids.  Journal of Mathematical Physics, 62(5):051511, 2021.
  * [10] M. Kac.  Can one hear the shape of a drum?  The American Mathematical Monthly, 73(4P2):1–23, 1966.
  * [11] J. Liu, K.-L. Xia, J. Wu, S. S.-T. Yau, and G.-W. Wei.  Biomolecular topology: Modelling and analysis.  Acta Mathematica Sinica, English Series, 38(10):1901–1938, 2022\.
  * [12] P. Y. Lum, G. Singh, A. Lehman, T. Ishkanov, M. Vejdemo-Johansson, M. Alagappan, J. Carlsson, and G. Carlsson.  Extracting insights from the shape of complex data using topology.  Scientific Reports, 3(1):1–8, 2013.
  * [13] J. McCleary.  A user’s guide to spectral sequences.  Number 58. Cambridge University Press, 2001.
  * [14] F. Mémoli, Z. Wan, and Y. Wang.  Persistent laplacians: Properties, algorithms and implications.  SIAM Journal on Mathematics of Data Science, 4(2):858–884, 2022\.
  * [15] J. Milnor.  Eigenvalues of the Laplace operator on certain manifolds.  Proceedings of the National Academy of Sciences, 51(4):542–542, 1964.
  * [16] A. W. Reid.  Isospectrality and commensurability of arithmetic hyperbolic 2-and 3-manifolds.  Duke Mathematical Journal, 65(2):215–228, 1992.
  * [17] M.-F. Vignéras.  Variétés riemanniennes isospectrales et non isométriques.  Annals of Mathematics, 112(1):21–32, 1980.
  * [18] R. Wang, D. D. Nguyen, and G.-W. Wei.  Persistent spectral graph.  International Journal for Numerical Methods in Biomedical Engineering, 36(9):e3376, 2020.
  * [19] R. Wang and G.-W. Wei.  Persistent path Laplacian.  Foundations of Data Sciences, 5(1):26–55, 2023.
  * [20] X. Wei and G.-W. Wei.  Persistent sheaf Laplacians.  arXiv preprint arXiv:2112.10906, 2021.
  * [21] Y. Yan, K. Chai, H. Liang, and L. Kong.  Physics involvement in ancient chinese chime bells.  In AIP Conference Proceedings, volume 1517, pages 43–48. American Institute of Physics, 2013.
  * [22] A. Zomorodian and G. Carlsson.  Computing persistent homology.  Discrete Comput. Geom., 33:249–274, 2005.

Guo-Wei Wei is an MSU Foundation Professor at Michigan State University. His
research concerns the mathematical foundations of biological science and data
science.