waves to be excited linearly by tidal forcing, as opposed to nonlinearly (with
respect to tidal amplitude) by the elliptical instability. Inertial waves are
by far the most efficient mechanism studied here, either those excited by the
elliptical instability for short orbital and tidal periods, or by the linear
frequency-averaged dissipation. The latter leads to $Q^{\prime}\approx
10^{3}(P_{\mathrm{rot}}/10\mathrm{hr})^{2}$ for Jupiter-like rotation periods,
which is consistent with the efficient tidal dissipation rates required to
explain the observed orbital migration of the moons of Jupiter and Saturn
(e.g. Lainey et al., 2009; Lainey et al., 2012, where tidal amplitudes are
likely to be too small for the elliptical instability to operate effectively).

All mechanisms except the frequency-averaged inertial wave mechanism are more
efficient in the Hot Jupiter model due to its larger radius, weaker rotation
and stronger convective driving. This allows the elliptical instability to be
on par or even more efficient than linearly-excited inertial waves in the
shortest-period Hot Jupiters. We conclude that inertial wave mechanisms are
probably the most efficient ones for dissipating tidal energy in giant
planets, at least those without extended stable layers.

## Acknowledgements

NBV was supported by EPSRC studentship 2528559. AJB and RH were supported by
STFC grants ST/S000275/1 and ST/W000873/1. RH would like to thank the Isaac
Newton Institute for Mathematical Sciences, Cambridge, for support and
hospitality during the programme “Frontiers in dynamo theory: from the Earth
to the stars” where work on this paper was undertaken. This work was supported
by EPSRC Grant No. EP/R014604/1. RH’s visit to the Newton Institute was
supported by a grant from the Heilbronn Institute. Simulations were undertaken
on ARC4, part of the High Performance Computing facilities at the University
of Leeds, and the DiRAC Data Intensive service at Leicester, operated by the
University of Leicester IT Services, which forms part of the STFC DiRAC HPC
Facility (www.dirac.ac.uk). The equipment was funded by BEIS capital funding
via STFC capital grants ST/K000373/1 and ST/R002363/1 and STFC DiRAC
Operations grant ST/R001014/1. DiRAC is part of the National e-Infrastructure.

## Data Availability

The simulation data used in this article will be shared on reasonable request
to the corresponding author.

## References

  * André et al. (2019) André Q., Mathis S., Barker A. J., 2019, A&A, 626, A82
  * Astoul & Barker (2022) Astoul A., Barker A. J., 2022, MNRAS, 516, 2913
  * Aubert et al. (2001) Aubert J., Brito D., Nataf H.-C., Cardin P., Masson J.-P., 2001, Physics of the Earth and Planetary Interiors, 128, 51
  * Aurnou et al. (2020) Aurnou J. M., Horn S., Julien K., 2020, Phys. Rev. Res., 2, 043115
  * Barker (2016) Barker A. J., 2016, Monthly Notices of the Royal Astronomical Society, 459, 939
  * Barker (2020) Barker A. J., 2020, MNRAS, 498, 2270
  * Barker & Astoul (2021) Barker A. J., Astoul A. A. V., 2021, MNRAS, 506, L69
  * Barker & Lithwick (2013a) Barker A. J., Lithwick Y., 2013a, Monthly Notices of the Royal Astronomical Society, 435, 3614
  * Barker & Lithwick (2013b) Barker A. J., Lithwick Y., 2013b, Monthly Notices of the Royal Astronomical Society, 437, 305
  * Barker et al. (2014) Barker A. J., Dempsey A. M., Lithwick Y., 2014, The Astrophysical Journal, 791, 13
  * Barker et al. (2016) Barker A. J., Braviner H. J., Ogilvie G. I., 2016, MNRAS, 459, 924
  * Barros et al. (2022) Barros S. C. C., et al., 2022, A&A, 657, A52
  * Benbakoura et al. (2019) Benbakoura M., Réville V., Brun A. S., Le Poncin-Lafitte C., Mathis S., 2019, A&A, 621, A124
  * Bodenheimer et al. (2001) Bodenheimer P., Lin D. N. C., Mardling R. A., 2001, The Astrophysical Journal, 548, 466
  * Böhm-Vitense (1958) Böhm-Vitense E., 1958, Z. Astrophys., 46, 108
  * Bolton et al. (2017) Bolton S. J., et al., 2017, Space Science Reviews, 213, 5
  * Bouillaut et al. (2021) Bouillaut V., Miquel B., Julien K., Aumaître S., Gallet B., 2021, Proceedings of the National Academy of Science, 118, e2105015118
  * Boyd (2001) Boyd J. P., 2001, Chebyshev and Fourier spectral methods. Courier Corporation
  * Cattaneo & Tobias (2013) Cattaneo F., Tobias S. M., 2013, Journal of Fluid Mechanics, 735, 457–472
  * Cauley et al. (2019) Cauley P. W., Shkolnik E. L., Llama J., Lanza A. F., 2019, Nature Astronomy, 3, 1128
  * Cébron et al. (2013) Cébron D., Le Bars M., Le Gal P., Moutou C., Leconte J., Sauret A., 2013, Icarus, 226, 1642
  * Chandrasekhar (1961) Chandrasekhar S., 1961, Hydrodynamic and hydromagnetic stability. Courier Corporation
  * Chandrasekhar (1967) Chandrasekhar S., 1967, Communications on Pure and Applied Mathematics, 20, 251
  * Cox & Giuli (1968) Cox J. P., Giuli R. T., 1968, Principles of stellar structure. Gordon and Breach
  * Currie et al. (2020) Currie L. K., Barker A. J., Lithwick Y., Browning M. K., 2020, Monthly Notices of the Royal Astronomical Society, 493, 5233
  * Cébron et al. (2010) Cébron D., Maubert P., Le Bars M., 2010, Geophysical Journal International, 182, 1311
  * Cébron et al. (2012) Cébron D., Le Bars M., Noir J., Aurnou J. M., 2012, Physics of Fluids, 24, 061703
  * Dewberry (2023) Dewberry J. W., 2023, MNRAS,
  * Dobbs-Dixon et al. (2004) Dobbs-Dixon I., Lin D. N. C., Mardling R. A., 2004, ApJ, 610, 464
  * Duguid et al. (2019) Duguid C. D., Barker A. J., Jones C. A., 2019, Monthly Notices of the Royal Astronomical Society
  * Duguid et al. (2020) Duguid C. D., Barker A. J., Jones C. A., 2020, Monthly Notices of the Royal Astronomical Society, 497, 3400
  * Favier et al. (2014) Favier B., Silvers L. J., Proctor M. R. E., 2014, Physics of Fluids, 26, 096605
  * French et al. (2012) French M., Becker A., Lorenzen W., Nettelmann N., Bethkenhagen M., Wicht J., Redmer R., 2012, The Astrophysical Journal Supplement Series, 202, 5
  * Fuller et al. (2016) Fuller J., Luan J., Quataert E., 2016, MNRAS, 458, 3867
  * Gastine & Wicht (2021) Gastine T., Wicht J., 2021, Icarus, 368, 114514
  * Gastine et al. (2016) Gastine T., Wicht J., Aubert J., 2016, Journal of Fluid Mechanics, 808, 690
  * Goldreich & Gold (1963) Goldreich P., Gold T., 1963, Monthly Notices of the Royal Astronomical Society, 126, 257
  * Goldreich & Nicholson (1977) Goldreich P., Nicholson P. D., 1977, Icarus, 30, 301
  * Goldreich & Soter (1966) Goldreich P., Soter S., 1966, Icarus, 5, 375
  * Goodman & Oh (1997) Goodman J., Oh S. P., 1997, The Astrophysical Journal, 486, 403
  * Guervilly et al. (2014) Guervilly C., Hughes D. W., Jones C. A., 2014, Journal of Fluid Mechanics, 758, 407
  * Guervilly et al. (2019) Guervilly C., Cardin P., Schaeffer N., 2019, Nature, 570, 368
  * Guillot et al. (2004) Guillot T., Stevenson D. J., Hubbard W. B., Saumon D., 2004, in Bagenal F., Dowling T. E., McKinnon W. B., eds, , Vol. 1, Jupiter. The Planet, Satellites and Magnetosphere. Cambridge University Press, pp 35–57
  * Han et al. (2014) Han E., Wang S. X., Wright J. T., Feng Y. K., Zhao M., Fakhouri O., Brown J. I., Hancock C., 2014, PASP, 126, 827
  * Hebb et al. (2010) Hebb L., et al., 2010, ApJ, 708, 224
  * Helled et al. (2022) Helled R., et al., 2022, Icarus, 378, 114937
  * Ingersoll & Pollard (1982) Ingersoll A. P., Pollard D., 1982, Icarus, 52, 62
  * Ireland & Browning (2018) Ireland L. G., Browning M. K., 2018, The Astrophysical Journal, 856, 132
  * Jermyn et al. (2022) Jermyn A. S., et al., 2022, arXiv e-prints, p. arXiv:2208.03651
  * Jones (2015) Jones C., 2015, in Schubert G., ed., , Treatise on Geophysics (Second Edition), second edition edn, Elsevier, Oxford, pp 115–159, doi:https://doi.org/10.1016/B978-0-444-53802-4.00141-X, https://www.sciencedirect.com/science/article/pii/B978044453802400141X
  * Kerswell (2002) Kerswell R. R., 2002, Annual Review of Fluid Mechanics, 34, 83
  * Lainey et al. (2009) Lainey V., Arlot J.-E., Karatekin O., Van Hoolst T., 2009, Nature, 459, 957
  * Lainey et al. (2012) Lainey V., et al., 2012, The Astrophysical Journal, 752, 14
  * Lainey et al. (2017) Lainey V., et al., 2017, Icarus, 281, 286
  * Lavorel & Le Bars (2010) Lavorel G., Le Bars M., 2010, Physics of Fluids, 22, 114101
  * Le Bars & Le Dizès (2006) Le Bars M., Le Dizès S., 2006, Journal of Fluid Mechanics, 563, 189–198
  * Le Reun et al. (2017) Le Reun T., Favier B., Barker A. J., Le Bars M., 2017, Phys. Rev. Lett., 119, 034502
  * Lesur & Longaretti (2007) Lesur G., Longaretti P. Y., 2007, MNRAS, 378, 1471
  * Lesur & Longaretti (2009) Lesur G., Longaretti P.-Y., 2009, A&A, 504, 309
  * Lin (2023) Lin Y., 2023, A&A, 671, A37
  * Lurie et al. (2017) Lurie J. C., et al., 2017, The Astronomical Journal, 154, 250
  * Maciejewski et al. (2016) Maciejewski G., et al., 2016, A&A, 588, L6
  * Maffei et al. (2019) Maffei S., Calkins M. A., Julien K., Marti P., 2019, Physical Review Fluids, 4, 041801
  * Mathis et al. (2016) Mathis S., Auclair-Desrotour P., Guenel M., Gallet F., Le Poncin-Lafitte C., 2016, A&A, 592, A33
  * Müller & Helled (2023) Müller S., Helled R., 2023, A&A, 669, A24
  * Müller et al. (2020) Müller S., Helled R., Cumming A., 2020, A&A, 638, A121
  * Nine et al. (2020) Nine A. C., Milliman K. E., Mathieu R. D., Geller A. M., Leiner E. M., Platais I., Tofflemire B. M., 2020, AJ, 160, 169
  * Novi et al. (2019) Novi L., von Hardenberg J., Hughes D. W., Provenzale A., Spiegel E. A., 2019, Phys. Rev. E, 99, 053116
  * Ogilvie (2012) Ogilvie G. I., 2012, Monthly Notices of the Royal Astronomical Society, 429, 613
  * Ogilvie (2014) Ogilvie G. I., 2014, ARA&A, 52, 171
  * Ogilvie & Lesur (2012) Ogilvie G. I., Lesur G., 2012, Monthly Notices of the Royal Astronomical Society, 422, 1975
  * Ogilvie & Lin (2004) Ogilvie G. I., Lin D. N. C., 2004, ApJ, 610, 477
  * Parodi et al. (2004) Parodi A., von Hardenberg J., Passoni G., Provenzale A., Spiegel E. A., 2004, Phys. Rev. Lett., 92, 194503
  * Patra et al. (2020) Patra K. C., et al., 2020, The Astronomical Journal, 159, 150
  * Paxton et al. (2011) Paxton B., Bildsten L., Dotter A., Herwig F., Lesaffre P., Timmes F., 2011, ApJS, 192, 3
  * Paxton et al. (2013) Paxton B., et al., 2013, ApJS, 208, 4
  * Paxton et al. (2015) Paxton B., et al., 2015, ApJS, 220, 15
  * Paxton et al. (2018) Paxton B., et al., 2018, ApJS, 234, 34
  * Paxton et al. (2019) Paxton B., et al., 2019, ApJS, 243, 10
  * Penev et al. (2007) Penev K., Sasselov D., Robinson F., Demarque P., 2007, ApJ, 655, 1166
  * Penev et al. (2009a) Penev K., Sasselov D., Robinson F., Demarque P., 2009a, ApJ, 704, 930
  * Penev et al. (2009b) Penev K., Barranco J., Sasselov D., 2009b, ApJ, 705, 285
  * Pontin et al. (2020) Pontin C. M., Barker A. J., Hollerbach R., André Q., Mathis S., 2020, Monthly Notices of the Royal Astronomical Society, 493, 5788
  * Pontin et al. (2023) Pontin C., Barker A., Hollerbach R., 2023, ApJ
  * Spiegel & Veronis (1960) Spiegel E. A., Veronis G., 1960, ApJ, 131, 442
  * Stevenson (1979) Stevenson D. J., 1979, Geophysical & Astrophysical Fluid Dynamics, 12, 139
  * Stevenson (2020) Stevenson D. J., 2020, Annual Review of Earth and Planetary Sciences, 48, 465
  * Terquem (2021) Terquem C., 2021, MNRAS, 503, 5789
  * Terquem et al. (1998) Terquem C., Papaloizou J. C. B., Nelson R. P., Lin D. N. C., 1998, ApJ, 502, 788
  * Tobias & Cattaneo (2013) Tobias S. M., Cattaneo F., 2013, Journal of Fluid Mechanics, 717, 347–360
  * Townsend (2022) Townsend R., 2022, MESA SDK for Mac OS, doi:10.5281/zenodo.7457723, https://doi.org/10.5281/zenodo.7457723
  * Turner et al. (2021) Turner J. D., Ridden-Harper A., Jayawardhana R., 2021, AJ, 161, 72
  * De Vries et al. (2023) De Vries N. B., Barker A. J., Hollerbach R., 2023, Physics of Fluids, 35, 024116
  * Vidal & Barker (2020a) Vidal J., Barker A. J., 2020a, Monthly Notices of the Royal Astronomical Society, 497, 4472
  * Vidal & Barker (2020b) Vidal J., Barker A. J., 2020b, The Astrophysical Journal Letters, 888, L31
  * Waleffe (1990) Waleffe F., 1990, Physics of Fluids A: Fluid Dynamics, 2, 76
  * Wei (2022) Wei X., 2022, A&A, 664, A10
  * Zahn (1966) Zahn J. P., 1966, Annales d’Astrophysique, 29, 313
  * Zahn (1977) Zahn J. P., 1977, A&A, 57, 383
  * Zahn (1989) Zahn J. P., 1989, A&A, 220, 112

##  Appendix A Resolution

Table 2: Table of resolutions used in the simulations with different Rayleigh numbers, Ekman numbers, and horizontal box size $L_{x}=L_{y}=4$ (unless otherwise specified). The same resolution was used for all ellipticities. The square brackets indicate all entries within are multiplied by the factor 5 in front. $\textrm{Ek}=5\cdot 10^{-5.5}$ | $n_{x},n_{y}$ | $n_{z}$
---|---|---
$R=-6,-4,-3,-1,-0.8,0.3,0.8,2,3,4,5,6$ | 256x256 | 96
$R=7,8$ | 256x256 | 128
$R=-10,9,10,11,12,15$ | 256x256 | 160
$R=20$ | 256x256 | 224
$\textrm{Ra}=1.3\cdot 10^{8}$ | $n_{x},n_{y}$ | $n_{z}$
$\text{Ek}=5\cdot\left[10^{-5.6},10^{-5.7},10^{-5.8},10^{-5.9}\right]$ | 256x256 | 96
$\text{Ek}=5\cdot 10^{-5.4}$ | 256x256 | 128
$\text{Ek}=5\cdot\left[10^{-5.2},10^{-5.3}\right]$ | 256x256 | 160
$\text{Ek}=\textbf{5}\cdot\left[10^{-5},10^{-5.1}\right]$ | 256x256 | 196
$\text{Ek}=5\cdot\left[10^{-4.5},10^{-4.6},10^{-4.7},10^{-4.8},10^{-4.9}\right]$ | 256x256 | 128
$R=6$ | $n_{x},n_{y}$ | $n_{z}$
$\text{Ek}=5\cdot\left[10^{-5.6},10^{-5.7},10^{-5.8},10^{-5.9},10^{-6}\right]$ | 256x256 | 96
$\text{Ek}=5\cdot\left[10^{-5.0},10^{-5.1},10^{-5.4}\right]$ | 256x256 | 128
$\text{Ek}=5\cdot\left[10^{-5.2},10^{-5.3}\right]$ | 256x256 | 160
$R=6$, $\epsilon=0$, $L_{x}=2$ (to determine $l_{c}$) | $n_{x},n_{y}$ | $n_{z}$
$\text{Ek}=5\cdot\left[10^{-5},10^{-5.1},...,10^{-5.8},10^{-5.9},10^{-6}\right]$ | 512x512 | 128
$\mathrm{Ra}=1.3\cdot 10^{8}$, $\epsilon=0$, $L_{x}=2$ (to determine $l_{c}$) | $n_{x},n_{y}$ | $n_{z}$
$\text{Ek}=5\cdot\left[10^{-4.5},10^{-4.6},....,10^{-5.8},10^{-5.9},10^{-6}\right]$ | 512x512 | 128

A table of resolutions used for our simulations is given in Table 2. High
Rayleigh number simulations were carried out with higher vertical resolutions.
Simulations with higher ellipticities were not found to require higher
resolutions. In addition, to fully resolve the heat flux spectrum for
calculations of the convective length scale we opted to use a 2x2x1 ($L=2$)
box, as indicated in the bottom two columns of Table 2.

The horizontal spectra used in the calculation of the length scale are shown
in Fig. 14. Both the heat flux $F(k_{\perp})$ and the vertical kinetic energy
$E_{z}(k_{\perp})$ spectra are plotted. The heat fluxes are shown with solid
(and dashed) lines, and the vertical kinetic energy spectra with dotted lines.
The spectra from cases with the most extreme parameters we considered are
displayed; the largest Ekman number, $\mathrm{Ek}=5\cdot 10^{-4.5}$, at fixed
$\mathrm{Ra}=1.3\cdot 10^{8}$ is plotted in solid-blue and dotted-purple, the
smallest Ekman number, $\mathrm{Ek}=5\cdot 10^{-6}$, at fixed
$\mathrm{Ra}=1.3\cdot 10^{8}$ in solid-yellow and dotted-burgundy, and the
smallest Ekman number at fixed $R=6$ in solid-green and dotted-orange. The
heat flux spectra at small Ekman number contain dashed parts, indicating
negative heat flux for these $k_{\perp}$. The heat flux tends to be
concentrated towards larger wave numbers with broad distributions. The strong
peak in solid-yellow and dotted-burgundy emerges because this simulation is
close to onset, and thus strongly follows the linear onset wave number. The
conventional rule that the power in the peak of the spectrum must be a factor
of at least $10^{3}$ larger than at the anti-aliasing scale is maintained in
the kinetic energy spectra as well as the heat flux spectra. Our simulations
are therefore likely to be spatially converged in the horizontal plane.

The main parameter in this work is the energy injection rate and the resulting
effective viscosity. Here we plot the effective viscosity at the two most
extreme parameters used in our simulations in Fig. 15. This shows the smallest
Ekman number, fixed supercriticality $R=6$ and $\epsilon=0.03$, and largest
Ekman number, fixed $\mathrm{Ra}=1.3\cdot 10^{8}$ and $\epsilon=0.02$. Low
ellipticities are specifically chosen to avoid bursts of the elliptical
instability. The effective viscosities in these simulations take different
values due to the different Rayleigh and Ekman numbers, but are further offset
for clarity. There is no change as we increase the resolution for this
quantity at either extreme of parameter space, so we conclude that the
effective viscosities we measured are numerically converged with our adopted
resolutions.

Figure 14: The heat flux $F(k_{\perp})$ (solid and dashed lines) and the
vertical kinetic energy $E_{z}(k_{\perp})$ (dotted lines) spectra as a
function of horizontal wavenumber $k_{\perp}$. The spectra are plotted at the
most extreme values of the surveyed parameter space in Ekman number. The
largest Ekman number, $\mathrm{Ek}=5\cdot 10^{-4.5}$, at fixed
$\mathrm{Ra}=1.3\cdot 10^{8}$ is plotted in solid-blue and dotted-purple, the
smallest Ekman number, $\mathrm{Ek}=5\cdot 10^{-6}$, at fixed
$\mathrm{Ra}=1.3\cdot 10^{8}$ in solid-yellow and dotted-burgundy, and the
smallest Ekman number at fixed $R=6$ in solid-green and dotted-orange. Dashed
parts of the heat flux spectra indicate negative heat flux for these scales.
Figure 15: The effective viscosity at largest $\mathrm{Ek}=5\cdot 10^{-4.5}$,
$\mathrm{Ra}=1.3\cdot 10^{8}$ in blue, orange, yellow and purple and at
smallest $\mathrm{Ek}=5\cdot 10^{-6}$, $R=6$ in green, cyan, burgundy and
black. The effective viscosities at $\mathrm{Ek}=5\cdot 10^{-6}$ are increased
by a factor of 100 to offset them for clarity. Apart from fluctuations the
effective viscosity is unchanged with increased resolution.

##  Appendix B MESA Inlists

The illustrative models in § 4.2 are based on the MESA test suite case
make_planets. We highlight here changes in the inlist files we used to obtain
these models. Any parameters not mentioned here are unchanged from the test
suite default values. The inlist_create and inlist_core are the same for both
the Jupiter and Hot Jupiter model:
inlist_create
max_model_number = 1020
initial_Y=0.27
inlist_core
dlg_core_mass_per_step=0.002d0
The differences between these models lies in inlist_evolve; for the Jupiter
model:
max_model_number = 2500
irradiation_flux = 50000.d0
inject_uniform_extra_heat = 0.0d0
max_age=4.5d9
and for the Hot Jupiter model:
max_model_number = 2500
irradiation_flux = 1.d9
inject_uniform_extra_heat = 0.05d0
max_age=4.5d9.


# Building Transportation Foundation Model via Generative Graph Transformer

Xuhong Wang1,†, Ding Wang1,†, Liang Chen1 and Yilun Lin1,∗, Member, IEEE This
work is supported by the Shanghai Artificial Intelligence Laboratory.$\dagger$
Equal contribution.*Corresponding author: Yilun Lin
(linyilun@pjlab.org.cn)1Xuhong Wang (wangxuhong@pjlab.org.cn), Ding Wang
(wangding@pjlab.org.cn), Liang Chen (chenliang@pjlab.org.cn), Yilun Lin
<EMAIL_ADDRESS>are with Urban Computing Lab, Shanghai Artificial
Intelligence Laboratory, Shanghai, China.

###### Abstract

Efficient traffic management is crucial for maintaining urban mobility,
especially in densely populated areas where congestion, accidents, and delays
can lead to frustrating and expensive commutes. However, existing prediction
methods face challenges in terms of optimizing a single objective and
understanding the complex composition of the transportation system. Moreover,
they lack the ability to understand the macroscopic system and cannot
efficiently utilize big data. In this paper, we propose a novel approach,
Transportation Foundation Model (TFM), which integrates the principles of
traffic simulation into traffic prediction. TFM uses graph structures and
dynamic graph generation algorithms to capture the participatory behavior and
interaction of transportation system actors. This data-driven and model-free
simulation method addresses the challenges faced by traditional systems in
terms of structural complexity and model accuracy and provides a foundation
for solving complex transportation problems with real data. The proposed
approach shows promising results in accurately predicting traffic outcomes in
an urban transportation setting.

##  I INTRODUCTION

The most promising approach to creating intelligent transportation systems is
the utilization of spatio-temporal data to predict traffic patterns.
Therefore, the issue of traffic prediction [1] has gained significant
attention from both academic and industry professionals. Traffic prediction
plays a crucial role in alleviating traffic congestion, enhancing traffic
safety, optimizing urban planning, and improving citizens’ travel experiences
[2].

However, there are two problems with current traffic prediction technology.
Firstly, this technology mainly targets the prediction of certain fixed
indicators, including origin-destination travel time [3], path-travel time
[4], travel demand [5], regional flow [6], network flow [7], and traffic speed
[8], which cannot establish a common knowledge system for intelligent
transportation systems. A well-known example is the large language model (LLM)
[9], such as ChatGPT [10], whose training objective never includes a specific
metric, but instead uses generative machine learning techniques to estimate
the probability distribution of all real-world language data.

The first major issue is the training objective, while the second concerns the
granularity problem of the traffic prediction model. As we all know, the
spatial-temporal data used in traffic prediction models usually come from
cameras, magnetic sensors, and smartphones in the road network. This data has
a coarse granularity and cannot model micro-interaction mechanisms between
vehicles. However, the macroscopic traffic system is composed of many
microscopic subsystems. Studying vehicle interactions at a microscopic scale
can help people gain a deeper understanding of the overall traffic system.

We believe that the most essential solution to the above two problems is to
integrate some concepts from traffic simulation into traffic prediction,
bridging the gap between these two research fields. The main feature of
traffic prediction technology is specificity. It is usually used to predict
the spatio-temporal evolution of a limited number of traffic indicators, while
the concept of traffic simulation technology has a fundamental difference from
traffic prediction.

Traffic simulation uses mathematical models to describe the behavior of
individuals or the entire system to reflect complex traffic phenomena, helping
traffic managers better understand traffic evolution. This is a bottom-up
modeling approach that not only focuses on a single prediction target, but
also leans towards the modeling of the entire transportation system. Traffic
simulation involves the development of rules and functions based on expert
knowledge and assumptions to simulate the traffic network. Traditionally,
these rules and functions are based on expert knowledge and assumptions about
traffic behavior, as well as limited real-world data. This has led to traffic
simulation being unable to leverage the large-scale data available today to
break through the bottleneck of modeling accuracy.

Figure 1: Different methods used for modeling transportation systems. The most
primitive method of modeling transportation systems was manual surveys and
field observation. Traffic simulation tends to model the behavior of each
traffic participant through physical mechanisms to ultimately form an
understanding of the entire traffic system. Traffic prediction typically uses
a fully data-driven approach to predict indicators in certain perspectives or
scenarios within the transportation system, without generating a comprehensive
understanding of the entire system.

Following the large models used in natural language processing, we attempt to
establish a basic model in the field of transportation. This model is not
intended to predict a specific metric in traffic, but rather to explore how to
model traffic interactions from the perspectives of various traffic
participants and ultimately establish a large model that can understand the
entire traffic system, and improve the performance of various traffic
prediction and simulation tasks.

We propose Transportation Foundation Model (TFM) to capture the participants
and their interactions in the transportation system using the graph structure
and dynamic graph generation algorithm. Each participant in the transportation
system (such as vehicles, pedestrians, smart traffic lights, etc.) is
represented as a node, and their relationships are represented as edges, which
together form a traffic graph representing the state of all participants in
the current transportation system. The temporal changes of the traffic graph
can represent the evolution of the entire traffic system, and this data
organization method provides a more adaptive and flexible way to model complex
transportation problems. This data-driven, and model-free simulation system
approach can address the challenges of structural complexity and model
accuracy in traditional systems, and provide a basis for complex traffic
problems with real data.

This study provides an overview of proposed Transportation Foundation Model.
Section 2 introduces related work in traffic simulation. Section 3 provides
basic introductions to the concepts and models used in TFM. Section 4 presents
a general introduction to the structure and workflow of TFM. Section 5
provides an example scenario to demonstrate the capabilities of TFM. Finally,
the study concludes with an outlook on future work, highlighting possible
directions for further research and development.

##  II RELATED WORKS

###  II-A Traffic Prediction

Traffic flow forecasting is crucial in Intelligent Transportation Systems
(ITS). Two main methods are dynamic modeling and data mining. Dynamic modeling
uses complicated formulations and theoretical assumptions to predict traffic
dynamics, while data mining finds historical data patterns for predicting
future trends. Early data mining models like Historical Average (HA) are
simple and fast but have low accuracy. Newer models, like ARIMA [11], VAR
[12], and Kalman filter [13], require low computational resources but assume
stationarity and can’t deal with non-linearity. Nonparametric and Learnable
models like support vector regression [14], k-nearest neighbor [15], and
Bayesian network [16] have emerged to tackle these problems.

With the development of deep learning in various fields, researchers have
applied deep learning models in transportation systems [17]. Deep belief
networks [18] extract high-dimensional features but can’t extract specific
spatiotemporal features. An RNN [19] can better learn temporal dependence, but
causes gradient problems. LSTM and GRU [20] are improvements but have
limitations in performance when stacked. A CNN [21] has stable gradients and
low memory requirements, but requires a stack of multiple layers and doesn’t
naturally consider spatial dependence. ST-ResNet [22] focuses on temporal
closeness, period, and trend but CNN format needs to be changed, and the
captured spatial features remain incomplete. Dilated convolution networks,
like WaveNet [23], can extract longer sequence information without increasing
parameters and are suitable for medium- and long-term traffic prediction.

###  II-B Graph Learning in Traffic Prediction

The study of traffic forecasting is an emerging topic in graph learning with
significant potential impact on daily life. DCRNN [24] combines graph
convolution with recurrent neural networks in an encoder-decoder manner, while
STGCN [25] combines graph convolution with gated temporal convolution. Graph
WaveNet [26] combines adaptive graph convolution with dilated casual
convolution to capture spatiotemporal dependencies, while ASTGCN [27] uses
both the spatial-temporal attention mechanism and the spatial-temporal
convolution. STG2Seq [28] uses multiple gated graph convolutional modules and
seq2seq architecture with attention mechanisms to make multi-step predictions.
STSGCN [29] utilizes multiple localized spatial-temporal subgraph modules to
capture localized spatial-temporal correlations directly. LSGCN [30]
integrates a novel attention mechanism and graph convolution into a spatially
gated block. AGCRN [31] utilizes node adaptive parameter learning to capture
node-specific spatial and temporal correlations in time-series data
automatically without a pre-defined graph, whereas STFGNN [32] captures hidden
spatial dependencies through a novel fusion operation of various spatial and
temporal graphs that are treated for different time periods in parallel.
Finally, Z-GCNETs [33] integrate a new time-aware zigzag topological layer
into time-conditioned GCNs, and STGODE [34] captures spatiotemporal dynamics
through a tensor-based ODE.

###  II-C Traffic Simulation

A traffic simulation tool for urban road networks is presented in 1997, which
is based on the Nagel–Schreckenberg Model [35]. Genetic algorithms have been
applied to airport ground traffic optimization [36]. A ground traffic
simulation tool is proposed and applied to Roissy Charles De Gaulle airport.
Truly agent-based traffic and mobility simulations have been studied [37]. A
real-world case study is presented, which says that multi-agent methods for
traffic are mature enough to be used alongside existing methods. ParamGrid, a
scalable and synchronized framework, was presented, and it can distribute the
simulation across a cluster of ordinary-performance, low-cost personal
computers (PCs) connected by local area network (LAN) [38]. A microscopic
model was proposed, which is able to simulate traffic situations in an urban
environment in real time for use in driving simulators [39]. The functionality
of simulation models is enhanced, and reliability is improved The proposed
approach offers significant advantages over conventional methods where
historical and outdated data is used to run poorly calibrated traffic
simulation models [40]. A multi-agent behavioral model was proposed, and it is
based on (i) the opportunistic individual behaviors that describe the norm
violation and (ii) the anticipatory individual abilities of simulated drivers
that allow critical situations to be detected [41]. A novel, real-time
algorithm for modeling large-scale has been presented, realistic traffic using
a hybrid model of both continuum and agent-based methods for traffic
simulation [42]. The latest developments concerning intermodal traffic
solutions, simulator coupling and model development and validation were
presented on the example of the open-source traffic simulator SUMO [43].

##  III METHODS

###  III-A From Traffic Networks to Transportation Graphs

The concept of the Dynamic Transportation graph defined in this article is not
the same as the traffic network mentioned in other literature. This is because
the traffic network usually only includes elements such as lanes and
intersections to form a road network, while Dynamic Transportation Graph not
only includes the traffic network, but also describes each participant in the
traffic system, such as vehicles, pedestrians, traffic lights, etc.

######  Definition III.1 (Dynamic Transportation Graph)

A traffic system can be described in a Dynamic Graph
(DG)$\mathcal{G}_{n}(\mathcal{V}_{n},\mathcal{E}_{n},\mathbf{S}_{n},\mathbf{A}_{n})$,
where $n=1,2,3,\dots$ represents the evolving steps of the system and
$\mathcal{V}_{n}$ represents all participants in it.
$\mathcal{E}_{n}\subseteq\mathcal{V}_{n}\times\mathcal{V}_{n}$ is the
relationship between those traffic
participants$e_{i}=(u_{i},v_{i})\in\mathcal{E}_{n},u_{i},v_{i}\in\mathcal{V}_{n}$,
which means each relational edge consists of a source node $u_{i}$ and a
target node$v_{i}$. The nodes and edges of a dynamic graph may have feature
attributes attached, $\mathbf{S}_{n}\in\mathcal{R}^{|\mathcal{V}_{n}|\times
D}$ represents the node state matrix of$\mathcal{V}_{n}$, where$D$ is the
dimension of the node state vector. The state attribute of each single node
$v_{i}\in\mathcal{V}_{n}$ is defined as $\mathbf{s}_{i}\in\mathcal{R}^{D}$.
Generally speaking, the features on the node represent the current state of
traffic participants, such as current vehicle speed, acceleration, traffic
light status, traffic flow of the road, etc.,
and$\mathbf{A}_{n}\in\mathcal{R}^{|\mathcal{V}_{n}|\times|\mathcal{V}_{n}|}$
represents the adjacency matrix of $\mathcal{V}_{n}$.

######  Definition III.2 (Dynamic Graph Transformation)

According to the definition of Dynamic Transportation Graph, the overall
dynamic evolution process of traffic system can be regarded as a dynamic graph
transformation process
$\mathcal{T}:\mathcal{G}_{n}\rightarrow\mathcal{G}_{n+1}$. Essentially, this
process is to learn a conditional probability function, which can be modeled
as a probability chain.

$p\left(\mathcal{G}_{n+1}\right)=\prod^{N}_{n=1}p\left(\mathcal{G}_{n+1}\mid\mathcal{G}_{n}\right)$
(1)

The parameters of the model are learned from the training data using the
maximum likelihood estimation method, which maximizes the likelihood function
of the training data.

###  III-B The Framework of Transportation Foundation Model

Due to the difficulty of directly modeling the process of Dynamic Graph
Transformation, it can be decomposed into three related sub-tasks:

  1. 1.

Dynamic graph representation
$p\left(\mathbf{Z}_{n}\mid\mathcal{G}_{n}\right)$: As all data can be
represented as a graph, learning from historical data is essentially a feature
learning of dynamic graphs. Its goal is to encode the knowledge and
information contained in historical dynamic graphs into graph feature vectors
$\mathbf{Z}_{n}$.

  2. 2.

Interaction generation $p\left(u_{i},v_{i}\mid\mathbf{Z}_{n}\right)$: The
interaction behavior between any two traffic participants can be represented
as an edge on a dynamic graph. The generation of the interaction is to obtain
the joint probability distribution of the edges $e_{i}=(u_{i},v_{i})$.

  3. 3.

State prediction $p\left(\mathbf{s}_{i}\mid\mathbf{Z}_{n}\right)$: The state
change of traffic participants can be represented as a feature change of nodes
on the graph. The model needs to use historical status and relational
information to predict the state information of each participant in the next
step.

Figure 2: The framework of Transportation Foundation Model (TFM). The
assumption of the TFM is very simple, that is, to regard the overall
transportation system as a system with Markovian properties. The model
predicts the next state $\mathcal{G}_{n}$ based on the entire system state of
the previous time step $\mathcal{G}_{n-1}$. TFM is built upon graph learning
approaches, with the ability to handle multi-scale and multi-source data
through flexible graph structure. This flexibility enables TFM to be used for
a wide range of traffic-related tasks, making it a powerful tool for accurate
and realistic simulations of urban transportation systems.

###  III-C Dynamic Graph Encoder

The function of this module is to learn the probability function
$p(\mathbf{Z}_{n}|\mathcal{G}_{n})$, which is specifically used to generate
the feature embedding matrix $\mathbf{Z}_{n}=\\{\mathbf{z}_{n,v},\forall
v\in\mathcal{V}_{n}\\}$ composed of all nodes at time n by learning the
characteristic and structural information of the graph $\mathcal{G}_{n}$. For
the sake of convenience, we will omit all subscript labels $*_{n}$ in the
description of this section.

Assuming we have a GNN n.etwork with $L=\\{1,\dots,l,\dots,L\\}$. For any node
$v$, the Encoder model aims to generate its embedding $\mathbf{z}{v}$ by
adopting an attention-based method to aggregate information from neighboring
nodes $\mathbf{z}_{u}\in\mathcal{N}^{v}$, while considering the time of each
interaction event and using a time encoding method $\Phi$ for representation.

Figure 3: The network structure of TFM’s Graph Encoder. The Encoder Layer
should be capable of encoding relational dependencies, timestamps, and
optionally edge features at the same time. The encoded node representations
are used by the forecaster in next section to generate the predicted
Interactions.

$\mathbf{z}_{v}^{(l)}=f_{\text{agg }}^{\text{enc
}}\left(\mathbf{z}_{v}^{(l-1)},\left\\{g_{\text{msg }}^{\text{enc
}}\left(\mathbf{z}_{u}^{(l-1)},\phi\left(t_{n}-t\right)\right):\left(u,t\right)\in\mathcal{N}^{v}\right\\}\right)$
(2)

The specific calculation form of formula 2 is as follows:

$\displaystyle\mathbf{z}_{v}^{(0)}$
$\displaystyle=\operatorname{MLP}(\mathbf{s}_{v})$ (3)
$\displaystyle\mathbf{z}_{v}^{(l)}$
$\displaystyle=\operatorname{MLP}\left(\mathbf{z}_{v}^{(l-1)}\|\tilde{\mathbf{z}}_{v}^{(l)}\right)$
$\displaystyle\tilde{\mathbf{z}}_{v}^{(l)}$
$\displaystyle=\operatorname{Attn}^{(l)}\left(\mathbf{q}_{v}^{(l)},\mathbf{K}^{(l)},\mathbf{V}^{(l)}\right)$
$\displaystyle\mathbf{q}_{v}^{(l)}$
$\displaystyle=\left[\mathbf{z}_{v}^{(l-1)}\|\Phi(0)\right]$
$\displaystyle\mathbf{K}^{(l)}$
$\displaystyle=\mathbf{V}^{(l)}=\left[\mathbf{z}_{u}^{(l-1)}\|\Phi\left(t_{n}-t\right),u\in\mathcal{N}^{v}\right]$
$\displaystyle\Phi(\Delta{t})$
$\displaystyle=\frac{1}{\sqrt{d_{w}}}\cos(\vec{\mathbf{w}}{\Delta{t}}+\vec{\mathbf{b}})$

where $Attn^{(l)}$ is the short of Attention layers, which is calculated by
the following equation:

$\operatorname{Attn}^{(l)}\left(\mathbf{q}_{v}^{(l)},\mathbf{K}^{(l)},\mathbf{V}^{(l)}\right)=\frac{1}{|\mathcal{N}^{v}|}\sum_{u}^{\forall\mathcal{N}^{v}}\alpha\left(\mathbf{q}_{v}^{(l)},\mathbf{K}_{u}^{(l)}\right)\cdot\mathbf{W}_{V}^{(l)}\mathbf{V}_{u}^{(l)}$
(4)

$\alpha$ represents the computational mapping function of attention weight.

$\displaystyle\alpha\left(\mathbf{q}_{v}^{(l)},\mathbf{K}_{u}^{(l)}\right)$
$\displaystyle=\mathop{SoftMax}\limits_{\forall
u\in\mathcal{N}^{v}}\left(\frac{(\textbf{W}_{Q}^{(l)}\textbf{q}_{v}^{(l)})(\textbf{W}_{K}^{(l)}\textbf{K}_{u}^{(l)})}{\sqrt{d_{q}}}\right)$
(5)
$\displaystyle=\frac{\exp\left(\frac{\left(\mathbf{W}_{Q}^{(l)}\mathbf{q}_{v}^{(l)}\right)\cdot\left(\mathbf{W}_{K}^{(l)}\mathbf{K}_{u}^{(l)}\right)}{\sqrt{d_{q}}}\right)}{\sum_{j}^{\forall\mathcal{N}^{v}}\exp\left(\frac{\left(\mathbf{W}_{Q}^{(l)}\mathbf{q}_{v}^{(l)}\right)\cdot\left(\mathbf{W}_{K}^{(l)}\mathbf{K}_{j}^{(l)}\right)}{\sqrt{d_{q}}}\right)}$

###  III-D Hierarchical-Probability Interaction Generator

Due to the complexity of solving the joint probability, the conditional
probability of a single edge is further split into the joint probabilities of
the edge by using the hierarchical probability chain method, that is, each
behavior of each traffic participant is modeled separately. It can be
expressed mathematically as follows:

$\displaystyle p\left(u_{i},v_{i}\mid\mathbf{Z}_{n}\right)=p\left(v_{i}\mid
u_{i},\mathbf{Z}_{n}\right)\cdot p\left(u_{i}\mid\mathbf{Z}_{n}\right)$ (6)
$\displaystyle
p\left(u_{i}\mid\mathbf{Z}_{n}\right)=\mathop{SoftMax}\limits_{\forall
u\in\mathcal{V}_{n}}\left(\mathrm{MLP}_{\text{u}}\left(\textbf{z}^{(i-1)}_{u}\right)\right)$
$\displaystyle p\left(v_{i}\mid
u_{i},\mathbf{Z}_{n}\right)=\mathop{SoftMax}\limits_{\forall
v\in\mathcal{V}_{n}}\left(\mathrm{MLP}_{\text{v}}\left(\textbf{z}^{(i-1)}_{u_{i}}\|\textbf{z}^{(i-1)}_{v}\right)\right)$

Based on the probabilistic output of the neural network, we generated
$(u_{i},v_{i})$ to represent an edge. Generally speaking, when an event
generates an edge, the two end nodes corresponding to the event should change
state, and then the state change will lead to the addition of new edges, which
is a continuously iterative process.

Figure 4: The hierarchical probability-chain forecaster and its workflow
relationship with the auto-regressive message passing module. The node
embeddings are learned from the GNN Encoder described in Section III-C. This
module is used to generate interactions between traffic particapants in the
next evolution time step.

###  III-E Auto-regressive Node State Updater

During the process of traffic simulation, the state of each traffic
participant can change at any time, that is, the node state on the graph will
change as time evolves. The function of this module is to learn the
probability of node state changes. The change in node state may come from the
influence of other nodes, or may come from the intention of the node itself,
or even from the state change of the neighbors of the node. In summary, there
are three main factors that influence the evolution of node state: 1. Whenever
an interaction occurs, the states of the two end nodes will change. 2.
Whenever an interaction is observed indirectly by other nodes, the states of
the other nodes themselves may change. 3. Even if the node itself has not
interacted with any other node, its own state may change over time.

Therefore, we also adopted the graph neural network method to model the impact
of the first two situations on the state of the node, using a recurrent neural
network, GRU, to handle the natural evolution of the node’s state over time.

$\displaystyle\mathbf{h}_{v}^{(i)}$ $\displaystyle=\mathbf{s}_{v}^{(i-1)}$ (7)
$\displaystyle\mathbf{h}_{v}^{(i)}$ $\displaystyle=f_{\text{agg }}^{\text{upd
}}\left(\left\\{g_{\mathrm{msg}}^{\text{upd
}}\left(\mathbf{h}_{u}^{(i)},\mathbf{h}_{v}^{(i)}\right):u\in\mathcal{N}_{\tilde{G}^{(i)}}^{v}\right\\}\right)$
$\displaystyle\mathbf{s}_{v}^{(i)}$
$\displaystyle=\operatorname{GRU}\left(\left[\mathbf{h}_{v}^{(i)}\|\phi\left(\Delta
t_{n+1}\right)\right],\mathbf{z}_{v}^{(i-1)}\right)$

###  III-F Learning Objects

Assuming that the model’s prediction for node connection is
${\bar{\mathcal{E}}_{n}}$ and the prediction for node state is
${\bar{\mathbf{S}}_{n}}$, these two components together form the overall
predicted graph ${\bar{\mathcal{G}}_{n}}$. Therefore, the optimization goal of
the model should be to minimize the difference between the predicted graph
${\bar{\mathcal{G}}_{n}}$ and the actual graph ${\mathcal{G}_{n}}$, which can
be further decomposed into two parts - the difference in graph structure and
the difference in node state.

$\mathcal{L}=\|{\mathcal{G}_{n}}-{\bar{\mathcal{G}}_{n}}\|=\|{\mathcal{E}_{n}}-{\bar{\mathcal{E}}_{n}}\|+\|{\mathbf{S}_{n}}-{\bar{\mathbf{S}}_{n}}\|$
(8)

####  III-F1 Structure loss

As minimizing the difference between graphs is highly complex and not
differentiable, we can naturally induce model learning metrics using a
probabilistic loss function. Let ${\mathcal{G}_{n}}$ be $\mathcal{T}$ and
${\mathcal{G}_{n-1}}$ be $\mathcal{S}$. The process of generating graphs can
be expressed as a sequence of events in a certain order, so the structure loss
can be set as the negloglikelihood of all graph events.

$\displaystyle p(\mathcal{T}\mid\mathcal{S})=\prod_{i}$ $\displaystyle
p\left(v_{i}\mid\mathcal{T}_{i-1},\mathcal{S}\right)$ (9)
$\displaystyle\prod_{v_{j}\in\mathcal{T}_{i-1}}$ $\displaystyle
p\left(e_{ji}\mid<v_{j},v_{i}>,\mathcal{T}_{i-1},\mathcal{S}\right)$
$\mathcal{L}_{Struc}=\operatorname{NegLikelyHood}(\mathcal{T}\mid\mathcal{S})=-\log
p(\mathcal{T}\mid\mathcal{S})$ (10)

####  III-F2 State loss

As the node states are dense feature vectors, comparing them is very easy. We
can simply use the mean square error (MSE) loss function to calculate the
differences between the node states on the graph ${\bar{\mathcal{G}}_{n}}$ and
the true graph ${\mathcal{G}_{n}}$, and obtain gradients.

$\mathcal{L}_{State}=\operatorname{MSE}({\mathbf{S}_{n}},{\bar{\mathbf{S}}_{n}})=\sum_{v\in\\{\bar{\mathcal{V}}_{n}\cap{\mathcal{V}}_{n}\\}}\|{\mathbf{s}_{v}}-{\bar{\mathbf{s}}_{v}}\|^{2}$
(11)

##  IV Case Study: A city-scale simulation

###  IV-A Data Collection

Effective transportation planning and decision-making requires accurate and
reliable simulations of large-scale traffic scenarios. Such simulations serve
to provide policymakers and engineers with a platform to evaluate traffic
management strategies and infrastructure improvements. In this study, we
propose the TFM, and its performance is demonstrated through a simulation of
Bologna city, which can be seen in Figure 5.

Bologna is a vast metropolis, similar in complexity to the SUMO example [44].
The simulation accurately models the city’s roads, intersections, traffic
lights, and a vast number of vehicles, creating a simulation that represents
real-world traffic conditions. The simulation scenario utilized consists of
11,079 vehicles, 26 traffic controllers, 268 roads, and 412 lanes, enabling
the generation of a wealth of data that can be used to evaluate the
effectiveness of various traffic management strategies.

Figure 5:  Left: The city of Bologna’s street network. Right: The graph
structure that represents the traffic system in the city of Bologna and
illustrates the links between different system agents. The graph comprises
nodes that represent roads, lanes, vehicles, and traffic lights within the
city, and edges that depict relationships between these nodes. This graph
structure facilitates a more comprehensive understanding of the transportation
network by offering a visual representation of the system.

###  IV-B Results

The MFDs generated by TFM and SUMO both reveal consistent trends with respect
to the relationship between traffic density and speed. However, while SUMO’s
MFD curves present a consistent and tight pattern, TFM’s MFD curves expose a
higher variance of data points. This disparity may be attributed to TFM’s
data-driven methodology, which allows for more precise and realistic modeling
of complex traffic behaviors across diverse regions within a city. In
contrast, SUMO assumes traffic behavior is uniform across all roads, a method
which may not always be applicable in real-world traffic. The study [45]
utilized SUMO outputs to optimize the TFM model, which may cause limitations
in modeling the many nuanced complexities of real-world traffic. As such,
further scientific investigation is essential, utilizing real-world data to
establish how the simulation can be refined to reflect local traffic
behavior’s intricacies and variability.

Figure 6: Comparison of the MFD between SUMO and TFM. The blue color
represents the results of the SUMO simulation software, while the red dots
represent the results of the proposed TFM. TFM method shows more diversity in
traffic patterns, because the TFM method can recover diverse traffic patterns
from a large amount of data of different scenarios, and even new patterns can
emerge, instead of solely relying on simple physical rules like SUMO does.

In Figure 7, we will examine the understanding and prediction of micro traffic
patterns in the TFM model from the perspective of transportation participants.
We can see that TFM accurately learns the behavioral patterns to be produced
by individual traffic participants. When the model predicts many traffic
participants simulataneously, interesting global patterns can emerge because
the graphic model can model the interaction among them. For example, the
scattered MFD shown in Figure 6.

Figure 7: Evaluating the TFM method from the micro perspective of traffic
vehicles. The blue curve represents the results of SUMO, while the red curve
represents the results of TFM. Although the relationship between speed and
acceleration is not one-to-one in the prediction results of TFM, we can see
that TFM has learned the control behavior pattern of vehicles. In practical
applications, people can focus on one of the indicators predicted by TFM and
derive other indicators through physical rules.

##  V CONCLUSIONS

In conclusion, the Transportation Foundation Model (TFM) proposed in this
paper is a novel approach that integrates traffic simulation principles into
traffic prediction. This model addresses the challenges faced by traditional
systems and provides a promising tool for solving complex transportation
problems with real data. By using graph structures and dynamic graph
generation algorithms, TFM captures the participatory behavior and interaction
of transportation system actors to produce accurate predictions of traffic
outcomes in densely populated urban areas. As a data-driven and model-free
simulation method, TFM has the ability to understand and efficiently utilize
big data, making it a promising option for optimizing the efficiency of
traffic management in modern cities.

## Data Availability

The SUMO simulation platform and data is publicly available at
https://www.eclipse.org/sumo. .

## Code availability

The code of TransWorldNG was implemented in Python using the deep learning
framework of PyTorch. Code, trained models, and scripts reproducing the
experiments of this paper are available at
https://github.com/PJSAC/TransWorldNG.

## References

  * [1] M. Liu, J. Wu, Y. Wang, and L. He, “Traffic flow prediction based on deep learning,” _Journal of System Simulation_ , vol. 30, no. 11, p. 4100, 2018\.
  * [2] W. Jiang, “Cellular traffic prediction with machine learning: A survey,” _Expert Systems with Applications_ , p. 117163, 2022.
  * [3] F. Li, Z. Gui, Z. Zhang, D. Peng, S. Tian, K. Yuan, Y. Sun, H. Wu, J. Gong, and Y. Lei, “A hierarchical temporal attention-based lstm encoder-decoder model for individual mobility prediction,” _Neurocomputing_ , vol. 403, pp. 153–166, 2020.
  * [4] T. Xing, X. Zhou, and J. Taylor, “Designing heterogeneous sensor networks for estimating and predicting path travel time dynamics: An information-theoretic modeling approach,” _Transportation Research Part B: Methodological_ , vol. 57, pp. 66–90, 2013.
  * [5] H. Yuan, G. Li, Z. Bao, and L. Feng, “An effective joint prediction model for travel demands and traffic flows,” in _2021 IEEE 37th International Conference on Data Engineering (ICDE)_. IEEE, 2021, pp. 348–359.
  * [6] A. M. Nagy and V. Simon, “Survey on traffic prediction in smart cities,” _Pervasive and Mobile Computing_ , vol. 50, pp. 148–163, 2018.
  * [7] I. Lohrasbinasab, A. Shahraki, A. Taherkordi, and A. Delia Jurcut, “From statistical-to machine learning-based network traffic prediction,” _Transactions on Emerging Telecommunications Technologies_ , vol. 33, no. 4, p. e4394, 2022.
  * [8] M. T. Asif, J. Dauwels, C. Y. Goh, A. Oran, E. Fathi, M. Xu, M. M. Dhanya, N. Mitrovic, and P. Jaillet, “Spatiotemporal patterns in large-scale traffic speed prediction,” _IEEE Transactions on Intelligent Transportation Systems_ , vol. 15, no. 2, pp. 794–804, 2013.
  * [9] I. Beltagy, K. Lo, and A. Cohan, “Scibert: A pretrained language model for scientific text,” _arXiv preprint arXiv:1903.10676_ , 2019.
  * [10] B. D. Lund and T. Wang, “Chatting about chatgpt: how may ai and gpt impact academia and libraries?” _Library Hi Tech News_ , vol. 40, no. 3, pp. 26–29, 2023.
  * [11] A. Saadallah, L. Moreira-Matias, R. Sousa, J. Khiari, E. Jenelius, and J. Gama, “Bright—drift-aware demand predictions for taxi networks,” _IEEE Transactions on Knowledge and Data Engineering_ , vol. 32, no. 2, pp. 234–245, 2018.
  * [12] Y. Chen and M. J. Zuo, “A sparse multivariate time series model-based fault detection method for gearboxes under variable speed condition,” _Mechanical Systems and Signal Processing_ , vol. 167, p. 108539, 2022.
  * [13] S. V. Kumar, “Traffic flow prediction using kalman filtering technique,” _Procedia Engineering_ , vol. 187, pp. 582–587, 2017.
  * [14] J. Ahn, E. Ko, and E. Y. Kim, “Highway traffic flow prediction using support vector regression and bayesian classifier,” in _2016 International Conference on Big Data and Smart Computing (BigComp)_. IEEE, 2016, pp. 239–244.
  * [15] P. Cai, Y. Wang, G. Lu, P. Chen, C. Ding, and J. Sun, “A spatiotemporal correlative k-nearest neighbor model for short-term traffic multistep forecasting,” _Transportation Research Part C: Emerging Technologies_ , vol. 62, pp. 21–34, 2016.
  * [16] J. Sun and J. Sun, “A dynamic bayesian network model for real-time crash prediction using traffic speed conditions data,” _Transportation Research Part C: Emerging Technologies_ , vol. 54, pp. 176–186, 2015.
  * [17] H. Yao, X. Tang, H. Wei, G. Zheng, and Z. Li, “Revisiting spatial-temporal similarity: A deep learning framework for traffic prediction,” in _Proceedings of the AAAI conference on artificial intelligence_ , vol. 33, no. 01, 2019, pp. 5668–5675.
  * [18] L. Nie, D. Jiang, S. Yu, and H. Song, “Network traffic prediction based on deep belief network in wireless mesh backbone networks,” in _2017 IEEE Wireless Communications and Networking Conference (WCNC)_. IEEE, 2017, pp. 1–5.
  * [19] C. Qiu, Y. Zhang, Z. Feng, P. Zhang, and S. Cui, “Spatio-temporal wireless traffic prediction with recurrent neural network,” _IEEE Wireless Communications Letters_ , vol. 7, no. 4, pp. 554–557, 2018.
  * [20] R. Fu, Z. Zhang, and L. Li, “Using lstm and gru neural network methods for traffic flow prediction,” in _2016 31st Youth Academic Annual Conference of Chinese Association of Automation (YAC)_. IEEE, 2016, pp. 324–328.
  * [21] D. Yang, S. Li, Z. Peng, P. Wang, J. Wang, and H. Yang, “Mf-cnn: Traffic flow prediction using convolutional neural network and multi-features fusion,” _IEICE TRANSACTIONS on Information and Systems_ , vol. 102, no. 8, pp. 1526–1536, 2019.
  * [22] J. Zhang, Y. Zheng, D. Qi, R. Li, X. Yi, and T. Li, “Predicting citywide crowd flows using deep spatio-temporal residual networks,” _Artificial Intelligence_ , vol. 259, pp. 147–166, 2018.
  * [23] C. Tian and W. K. Chan, “Spatial-temporal attention wavenet: A deep learning framework for traffic prediction considering spatial-temporal dependencies,” _IET Intelligent Transport Systems_ , vol. 15, no. 4, pp. 549–561, 2021.
  * [24] Y. Li, R. Yu, C. Shahabi, and Y. Liu, “Diffusion convolutional recurrent neural network: Data-driven traffic forecasting,” in _International Conference on Learning Representations_.
  * [25] B. Yu, H. Yin, and Z. Zhu, “Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting,” in _Proceedings of the 27th International Joint Conference on Artificial Intelligence_ , 2018, pp. 3634–3640.
  * [26] Z. Wu, S. Pan, G. Long, J. Jiang, and C. Zhang, “Graph wavenet for deep spatial-temporal graph modeling,” in _The 28th International Joint Conference on Artificial Intelligence (IJCAI)_. International Joint Conferences on Artificial Intelligence Organization, 2019.
  * [27] S. Guo, Y. Lin, N. Feng, C. Song, and H. Wan, “Attention based spatial-temporal graph convolutional networks for traffic flow forecasting,” in _Proceedings of the AAAI conference on artificial intelligence_ , vol. 33, no. 01, 2019, pp. 922–929.
  * [28] L. Bai, L. Yao, S. S. Kanhere, X. Wang, and Q. Z. Sheng, “Stg2seq: spatial-temporal graph to sequence model for multi-step passenger demand forecasting,” in _28th International Joint Conference on Artificial Intelligence, IJCAI 2019_. International Joint Conferences on Artificial Intelligence, 2019, pp. 1981–1987.
  * [29] C. Song, Y. Lin, S. Guo, and H. Wan, “Spatial-temporal synchronous graph convolutional networks: A new framework for spatial-temporal network data forecasting,” in _Proceedings of the AAAI conference on artificial intelligence_ , vol. 34, no. 01, 2020, pp. 914–921.
  * [30] R. Huang, C. Huang, Y. Liu, G. Dai, and W. Kong, “Lsgcn: Long short-term traffic prediction with graph convolutional networks.” in _IJCAI_ , vol. 7, 2020, pp. 2355–2361.
  * [31] L. Bai, L. Yao, C. Li, X. Wang, and C. Wang, “Adaptive graph convolutional recurrent network for traffic forecasting,” _Advances in neural information processing systems_ , vol. 33, pp. 17 804–17 815, 2020.
  * [32] M. Li and Z. Zhu, “Spatial-temporal fusion graph neural networks for traffic flow forecasting,” in _Proceedings of the AAAI conference on artificial intelligence_ , vol. 35, no. 5, 2021, pp. 4189–4196.
  * [33] Y. Chen, I. Segovia, and Y. R. Gel, “Z-gcnets: Time zigzags at graph convolutional networks for time series forecasting,” in _International Conference on Machine Learning_. PMLR, 2021, pp. 1684–1694.
  * [34] Z. Fang, Q. Long, G. Song, and K. Xie, “Spatial-temporal graph ode networks for traffic flow forecasting,” in _Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining_, 2021, pp. 364–373.
  * [35] J. Esser and M. Schreckenberg, “Microscopic simulation of urban traffic based on cellular automata,” _International Journal of Modern Physics C_ , vol. 8, no. 05, pp. 1025–1036, 1997.
  * [36] J.-B. Gotteland and N. Durand, “Genetic algorithms applied to airport ground traffic optimization,” in _The 2003 Congress on Evolutionary Computation, 2003. CEC’03._ , vol. 1. IEEE, 2003, pp. 544–551.
  * [37] M. Balmer, N. Cetin, K. Nagel, and B. Raney, “Towards truly agent-based traffic and mobility simulations,” in _Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004\. AAMAS 2004._ IEEE, 2004, pp. 60–67.
  * [38] R. Klefstad, Y. Zhang, M. Lai, R. Jayakrishnan, and R. Lavanya, “A distributed, scalable, and synchronized framework for large-scale microscopic traffic simulation,” in _Proceedings. 2005 IEEE Intelligent Transportation Systems, 2005._ IEEE, 2005, pp. 813–818.
  * [39] J. Maroto, E. Delso, J. Felez, and J. M. Cabanellas, “Real-time traffic simulation with a microscopic model,” _IEEE Transactions on Intelligent Transportation Systems_ , vol. 7, no. 4, pp. 513–527, 2006.
  * [40] A. Puri, K. Valavanis, and M. Kontitsis, “Statistical profile generation for traffic monitoring using real-time uav based video data,” in _2007 Mediterranean Conference on Control & Automation_. IEEE, 2007, pp. 1–6.
  * [41] A. Doniec, R. Mandiau, S. Piechowiak, and S. Espié, “A behavioral multi-agent model for road traffic simulation,” _Engineering Applications of Artificial Intelligence_ , vol. 21, no. 8, pp. 1443–1454, 2008\.
  * [42] J. Sewall, D. Wilkie, and M. C. Lin, “Interactive hybrid simulation of large-scale traffic,” in _Proceedings of the 2011 SIGGRAPH Asia Conference_ , 2011, pp. 1–12.
  * [43] P. A. Lopez, M. Behrisch, L. Bieker-Walz, J. Erdmann, Y.-P. Flötteröd, R. Hilbrich, L. Lücken, J. Rummel, P. Wagner, and E. Wießner, “Microscopic traffic simulation using sumo,” in _2018 21st international conference on intelligent transportation systems (ITSC)_. IEEE, 2018, pp. 2575–2582.
  * [44] L. Bieker, D. Krajzewicz, A. Morra, C. Michelacci, and F. Cartolano, “Traffic Simulation for All: A Real World Traffic Scenario from the City of Bologna,” in _Modeling Mobility with Open Data_ , M. Behrisch and M. Weber, Eds. Cham: Springer International Publishing, 2015, pp. 47–60, series Title: Lecture Notes in Mobility. [Online]. Available: https://link.springer.com/10.1007/978-3-319-15024-6˙4
  * [45] J. A. Laval and F. Castrillón, “Stochastic Approximations for the Macroscopic Fundamental Diagram of Urban Networks,” _Transportation Research Procedia_ , vol. 7, pp. 615–630, 2015. [Online]. Available: https://linkinghub.elsevier.com/retrieve/pii/S2352146515001003