* Gavagnin et al. (2017) Gavagnin, E., Bleuler, A., Rosdahl, J., & Teyssier, R. 2017, MNRAS , 472, 4155
  * Gerhard (2011) Gerhard, O. 2011, Memorie della Societa Astronomica Italiana Supplementi, 18, 185
  * Gladman et al. (2001) Gladman, B., Kavelaars, J. J., Petit, J.-M., et al. 2001, AJ , 122, 1051
  * Gladman et al. (2008) Gladman, B., Marsden, B. G., & Vanlaerhoven, C. 2008, Nomenclature in the Outer Solar System, ed. M. A. Barucci, H. Boehnhardt, D. P. Cruikshank, A. Morbidelli, & R. Dotson, 43
  * Glaser et al. (2020) Glaser, J. P., McMillan, S. L. W., Geller, A. M., Thornton, J. D., & Giovinazzi, M. R. 2020, AJ , 160, 126
  * Gomes et al. (2005) Gomes, R., Levison, H. F., Tsiganis, K., & Morbidelli, A. 2005, Nat , 435, 466
  * Gomes et al. (2004) Gomes, R. S., Morbidelli, A., & Levison, H. F. 2004, Icarus , 170, 492
  * Goodman et al. (1993) Goodman, J., Heggie, D. C., & Hut, P. 1993, ApJ , 415, 715
  * Grazier et al. (1999) Grazier, K. R., Newman, W. I., Kaula, W. M., & Hyman, J. M. 1999, Icarus , 140, 341
  * Gropp (2002) Gropp, W. 2002, MPICH2: A New Start for MPI Implementations, ed. D. Kranzlmüller, J. Volkert, P. Kacsuk, & J. Dongarra (Berlin, Heidelberg: Springer Berlin Heidelberg), 7–7
  * Gropp et al. (1996) Gropp, W., Lusk, E., Doss, N., & Skjellum, A. 1996, Parallel Computing, 22, 789
  * Hands & Dehnen (2020) Hands, T. O. & Dehnen, W. 2020, MNRAS , 493, L59
  * Hands et al. (2019) Hands, T. O., Dehnen, W., Gration, A., Stadel, J., & Moore, B. 2019, MNRAS , 490, 21
  * Hanse et al. (2018) Hanse, J., Jílková, L., Portegies Zwart, S. F., & Pelupessy, F. I. 2018, MNRAS , 473, 5432
  * Hartmann (1965) Hartmann, W. K. 1965, Icarus , 4, 157
  * Hartmann (1966) Hartmann, W. K. 1966, Icarus , 5, 406
  * Hayashi et al. (1985) Hayashi, C., Nakazawa, K., & Nakagawa, Y. 1985, in Protostars and Planets II, ed. D. C. Black & M. S. Matthews, 1100–1153
  * Heisler & Tremaine (1986) Heisler, J. & Tremaine, S. 1986, Icarus , 65, 13
  * Higuchi & Kokubo (2015) Higuchi, A. & Kokubo, E. 2015, AJ , 150, 26
  * Higuchi & Kokubo (2020) Higuchi, A. & Kokubo, E. 2020, MNRAS , 492, 268
  * Higuchi et al. (2007) Higuchi, A., Kokubo, E., Kinoshita, H., & Mukai, T. 2007, AJ , 134, 1693
  * Hill (1913) Hill, G. W. 1913, AJ , 27, 171
  * Hills (1981) Hills, J. G. 1981, AJ , 86, 1730
  * Hills (1984) Hills, J. G. 1984, Nat , 311, 636
  * Hunter (2007) Hunter, J. D. 2007, Computing in Science and Engineering, 9, 90
  * Hut (1984) Hut, P. 1984, Nat , 311, 638
  * Ida et al. (2000) Ida, S., Larwood, J., & Burkert, A. 2000, ApJ , 528, 351
  * Ito & Higuchi (2020) Ito, T. & Higuchi, A. 2020, in AAS/Division for Planetary Sciences Meeting Abstracts, Vol. 52, AAS/Division for Planetary Sciences Meeting Abstracts, 304.07
  * Ito & Ohtsuka (2019) Ito, T. & Ohtsuka, K. 2019, Monographs on Environment, Earth and Planets, 7, 1
  * Ito & Tanikawa (2002) Ito, T. & Tanikawa, K. 2002, MNRAS , 336, 483
  * Jetsu & Pelt (2000) Jetsu, L. & Pelt, J. 2000, A&A , 353, 409
  * Jewitt (2002) Jewitt, D. C. 2002, AJ , 123, 1039
  * Jílková et al. (2012) Jílková, L., Carraro, G., Jungwiert, B., & Minchev, I. 2012, A&A , 541, A64
  * Jílková et al. (2016) Jílková, L., Hamers, A., Hammer, M., & Portegies Zwart, S. 2016, MNRAS , 457, 4218
  * Jílková et al. (2015) Jílková, L., Portegies Zwart, S., Pijloo, T., & Hammer, M. 2015, MNRAS , 453, 3157
  * Jiménez-Torres (2020) Jiménez-Torres, J. J. 2020, Acta Astron. , 70, 53
  * Johansen & Lambrechts (2017) Johansen, A. & Lambrechts, M. 2017, Annual Review of Earth and Planetary Sciences, 45, 359
  * Johansen et al. (2007) Johansen, A., Oishi, J. S., Mac Low, M.-M., et al. 2007, Nat , 448, 1022
  * Johansen et al. (2021) Johansen, A., Ronnet, T., Bizzarro, M., et al. 2021, arXiv e-prints, arXiv:2102.08611
  * Johnstone et al. (1998) Johnstone, D., Hollenbach, D., & Bally, J. 1998, ApJ , 499, 758
  * Jurić et al. (2008) Jurić, M., Ivezić, Ž., Brooks, A., et al. 2008, ApJ , 673, 864
  * Kaib & Quinn (2008) Kaib, N. A. & Quinn, T. 2008, Icarus , 197, 221
  * Kaib et al. (2013) Kaib, N. A., Raymond, S. N., & Duncan, M. 2013, Nat , 493, 381
  * Kaib et al. (2011a) Kaib, N. A., Roskar, R., & Quinn, T. 2011a, in AAS/Division of Dynamical Astronomy Meeting #42, AAS/Division of Dynamical Astronomy Meeting, 9.04
  * Kaib et al. (2011b) Kaib, N. A., Roškar, R., & Quinn, T. 2011b, Icarus , 215, 491
  * Kavelaars et al. (2020) Kavelaars, J. J., Lawler, S. M., Bannister, M. T., & Shankman, C. 2020, Perspectives on the distribution of orbits of distant Trans-Neptunian objects, ed. D. Prialnik, M. A. Barucci, & L. Young, 61–77
  * Kenyon & Bromley (2006) Kenyon, S. J. & Bromley, B. C. 2006, AJ , 131, 1837
  * Kobayashi et al. (2005) Kobayashi, H., Ida, S., & Tanaka, H. 2005, Icarus , 177, 246
  * Kokubo & Ida (1998) Kokubo, E. & Ida, S. 1998, Icarus, 131, 171
  * Kokubo & Ida (2002) Kokubo, E. & Ida, S. 2002, The Astrophysical Journal, 581, 666
  * Korycansky & Papaloizou (1995) Korycansky, D. G. & Papaloizou, J. C. B. 1995, MNRAS , 274, 85
  * Kozai (1962) Kozai, Y. 1962, AJ , 67, 591
  * Kroupa (2001) Kroupa, P. 2001, MNRAS , 322, 231
  * Kruijer et al. (2017) Kruijer, T. S., Burkhardt, C., Budde, G., & Kleine, T. 2017, Proceedings of the National Academy of Science, 114, 6712
  * Kuiper (1951) Kuiper, G. P. 1951, in 50th Anniversary of the Yerkes Observatory and Half a Century of Progress in Astrophysics, ed. J. A. Hynek, 357
  * Lada & Lada (2003) Lada, C. J. & Lada, E. A. 2003, ARA&A , 41, 57
  * Laskar (1990) Laskar, J. 1990, Icarus , 88, 266
  * Laskar et al. (2011) Laskar, J., Fienga, A., Gastineau, M., & Manche, H. 2011, A&A , 532, A89
  * Laughlin & Adams (1998) Laughlin, G. & Adams, F. C. 1998, ApJL , 508, L171
  * Lecar et al. (2001) Lecar, M., Franklin, F. A., Holman, M. J., & Murray, N. J. 2001, ARA&A , 39, 581
  * Leto et al. (2008) Leto, G., Jakubík, M., Paulech, T., Neslušan, L., & Dybczyński, P. A. 2008, MNRAS , 391, 1350
  * Leto et al. (2009) Leto, G., Jakubík, M., Paulech, T., Neslušan, L., & Dybczyński, P. A. 2009, Earth Moon and Planets, 105, 263
  * Levison et al. (2001) Levison, H. F., Dones, L., & Duncan, M. J. 2001, AJ , 121, 2253
  * Levison et al. (2010a) Levison, H. F., Duncan, M. J., Brasser, R., & Kaufmann, D. E. 2010a, Science, 329, 187
  * Levison et al. (2008) Levison, H. F., Morbidelli, A., Van Laerhoven, C., Gomes, R., & Tsiganis, K. 2008, Icarus , 196, 258
  * Levison et al. (2010b) Levison, H. F., Thommes, E., & Duncan, M. J. 2010b, AJ , 139, 1297
  * Li et al. (2006) Li, J., Zhou, L.-Y., & Sun, Y.-S. 2006, Chinese J. Astron. Astrophys., 6, 588
  * Licht (1999) Licht, A. L. 1999, Icarus , 137, 355
  * Lidov (1962) Lidov, M. 1962, Planet. Space Sci., 9, 719
  * Lineweaver (2010) Lineweaver, C. H. 2010, in Astrobiology Science Conference 2010: Evolution and Life: Surviving Catastrophes and Extremes on Earth and Beyond, Vol. 1538, 5226
  * Lüghausen et al. (2012) Lüghausen, F., Parmentier, G., Pflamm-Altenburg, J., & Kroupa, P. 2012, MNRAS , 423, 1985
  * Luhman (2014) Luhman, K. L. 2014, ApJ , 781, 4
  * Luu & Jewitt (2002) Luu, J. X. & Jewitt, D. C. 2002, ARA&A , 40, 63
  * Lykawka & Mukai (2008) Lykawka, P. S. & Mukai, T. 2008, AJ , 135, 1161
  * Madigan & McCourt (2016) Madigan, A.-M. & McCourt, M. 2016, MNRAS , 457, L89
  * Malhotra (1993) Malhotra, R. 1993, Nat , 365, 819
  * Malhotra (1998) Malhotra, R. 1998, in American Astronomical Society Meeting Abstracts, Vol. 193, American Astronomical Society Meeting Abstracts, 96.02
  * Malhotra (2019) Malhotra, R. 2019, Geoscience Letters, 6, 12
  * Martínez-Barbosa et al. (2016) Martínez-Barbosa, C. A., Brown, A. G. A., Boekholt, T., et al. 2016, MNRAS , 457, 1062
  * Martínez-Barbosa et al. (2015) Martínez-Barbosa, C. A., Brown, A. G. A., & Portegies Zwart, S. 2015, MNRAS , 446, 823
  * Martínez-Barbosa et al. (2017) Martínez-Barbosa, C. A., Jílková, L., Portegies Zwart, S., & Brown, A. G. A. 2017, MNRAS , 464, 2290
  * McKee & Ostriker (2007) McKee, C. F. & Ostriker, E. C. 2007, ARA&A , 45, 565
  * Melott & Bambach (2010) Melott, A. L. & Bambach, R. K. 2010, MNRAS , 407, L99
  * Milani & Nobili (1992) Milani, A. & Nobili, A. M. 1992, Nat , 357, 569
  * Miller (1964) Miller, R. H. 1964, ApJ , 140, 250
  * Monari et al. (2014) Monari, G., Helmi, A., Antoja, T., & Steinmetz, M. 2014, A&A , 569, A69
  * Moore et al. (2020) Moore, N. W. H., Li, G., & Adams, F. C. 2020, ApJ , 901, 92
  * Morbidelli et al. (2009) Morbidelli, A., Brasser, R., Tsiganis, K., Gomes, R., & Levison, H. F. 2009, A&A , 507, 1041
  * Morbidelli & Levison (2004) Morbidelli, A. & Levison, H. F. 2004, AJ , 128, 2564
  * Morbidelli et al. (2005) Morbidelli, A., Levison, H. F., Tsiganis, K., & Gomes, R. 2005, Nat , 435, 462
  * Morbidelli & Nesvorný (2020) Morbidelli, A. & Nesvorný, D. 2020, Kuiper belt: formation and evolution, ed. D. Prialnik, M. A. Barucci, & L. Young, 25–59
  * Morbidelli et al. (2007) Morbidelli, A., Tsiganis, K., Crida, A., Levison, H. F., & Gomes, R. 2007, AJ , 134, 1790
  * Movshovitz et al. (2010) Movshovitz, N., Bodenheimer, P., Podolak, M., & Lissauer, J. J. 2010, Icarus , 209, 616
  * Murray & Holman (1999) Murray, N. & Holman, M. 1999, Science, 283, 1877
  * Murray & Holman (2001) Murray, N. & Holman, M. 2001, Nat , 410, 773
  * Mustill et al. (2016) Mustill, A. J., Raymond, S. N., & Davies, M. B. 2016, MNRAS , 460, L109
  * Napier et al. (2021a) Napier, K. J., Adams, F. C., & Batygin, K. 2021a, The Planetary Science Journal, 2, 53
  * Napier et al. (2021b) Napier, K. J., Gerdes, D. W., Lin, H. W., et al. 2021b, arXiv e-prints, arXiv:2102.05601
  * Neslušan (2000) Neslušan, L. 2000, A&A , 361, 369
  * Neslušan et al. (2009) Neslušan, L., Dybczyński, P. A., Leto, G., Jakubík, M., & Paulech, T. 2009, Earth Moon and Planets, 105, 257
  * Nesvorný (2020) Nesvorný, D. 2020, Research Notes of the American Astronomical Society, 4, 212
  * Neukum et al. (2001) Neukum, G., Ivanov, B. A., & Hartmann, W. K. 2001, Space Sci. Rev. , 96, 55
  * Newton (1687) Newton, I. 1687, Philosophiae Naturalis Principia Mathematica, Vol. 1
  * Nordlander et al. (2017) Nordlander, T., Rickman, H., & Gustafsson, B. 2017, A&A , 603, A112
  * Nurmi et al. (2002) Nurmi, P., Valtonen, M. J., Zheng, J. Q., & Rickman, H. 2002, MNRAS , 333, 835
  * Offner & Arce (2015) Offner, S. S. R. & Arce, H. G. 2015, ApJ , 811, 146
  * Oliphant (2006) Oliphant, T. E. 2006, A guide to NumPy, Vol. 1 (Trelgol Publishing USA)
  * Oort (1950) Oort, J. H. 1950, Bul. Astron. Inst. Neth. , 11, 91
  * Öpik (1932) Öpik, E. 1932, Proceedings of the American Academy of Arts and Sciences, 67, 169
  * Opitom et al. (2019) Opitom, C., Fitzsimmons, A., Jehin, E., et al. 2019, A&A , 631, L8
  * ’Oumuamua ISSI Team et al. (2019) ’Oumuamua ISSI Team, Bannister, M. T., Bhandare, A., et al. 2019, Nature Astronomy, 3, 594
  * Parker (2020) Parker, R. J. 2020, Royal Society Open Science, 7, 201271
  * Patterson & Smith (1987) Patterson, C. & Smith, A. B. 1987, Nat , 330, 248
  * Paulech et al. (2010) Paulech, T., Jakubík, M., Neslušan, L., Dybczyński, P. A., & Leto, G. 2010, A&A , 509, A48
  * Pelupessy et al. (2012) Pelupessy, F. I., Jänes, J., & Portegies Zwart, S. 2012, New Astron. , 17, 711
  * Perets & Kouwenhoven (2012) Perets, H. B. & Kouwenhoven, M. B. N. 2012, ApJ , 750, 83
  * Pfalzner et al. (2021) Pfalzner, S., Aizpuru Vargas, L., Bhandare, A., & Veras, D. 2021, arXiv e-prints, arXiv:2104.06845
  * Pfalzner et al. (2020) Pfalzner, S., Davies, M. B., Kokaia, G., & Bannister, M. T. 2020, ApJ , 903, 114
  * Pfalzner & Vincke (2020) Pfalzner, S. & Vincke, K. 2020, ApJ , 897, 60
  * Pfalzner et al. (2005) Pfalzner, S., Vogel, P., Scharwächter, J., & Olczak, C. 2005, A&A , 437, 967
  * Pirani et al. (2018) Pirani, S., Johansen, A., Bitsch, B., Mustill, A. J., & Turrini, D. 2018, in AAS/Division for Planetary Sciences Meeting Abstracts, Vol. 50, AAS/Division for Planetary Sciences Meeting Abstracts #50, 200.01D
  * Pirani et al. (2019) Pirani, S., Johansen, A., Bitsch, B., Mustill, A. J., & Turrini, D. 2019, A&A , 623, A169
  * Plummer (1911) Plummer, H. C. 1911, MNRAS , 71, 460
  * Popovas et al. (2018) Popovas, A., Nordlund, Å., Ramsey, J. P., & Ormel, C. W. 2018, MNRAS , 479, 5136
  * Portegies Zwart (2019) Portegies Zwart, S. 2019, A&A , 622, A69
  * Portegies Zwart (2020) Portegies Zwart, S. 2020, Nature Astronomy, 4, 819
  * Portegies Zwart (2021) Portegies Zwart, S. 2021, A&A , 647, A136
  * Portegies Zwart & McMillan (2018) Portegies Zwart, S. & McMillan, S. 2018, Astrophysical Recipes; The art of AMUSE
  * Portegies Zwart et al. (2009) Portegies Zwart, S., McMillan, S., Harfst, S., et al. 2009, New Astronomy, 14, 369
  * Portegies Zwart et al. (2020) Portegies Zwart, S., Pelupessy, I., Martínez-Barbosa, C. van Elteren, A., & McMillan, S. 2020, Communications in Nonlinear Science and Numerical Simulation, 105240
  * Portegies Zwart et al. (2018) Portegies Zwart, S., Torres, S., Pelupessy, I., Bédorf, J., & Cai, M. X. 2018, MNRAS , 479, L17
  * Portegies Zwart et al. (2018) Portegies Zwart, S., van Elteren, A., Pelupessy, I., et al. 2018, AMUSE: the Astrophysical Multipurpose Software Environment
  * Portegies Zwart (2009) Portegies Zwart, S. F. 2009, ApJL , 696, L13
  * Portegies Zwart & Boekholt (2018) Portegies Zwart, S. F. & Boekholt, T. C. N. 2018, Communications in Nonlinear Science and Numerical Simulations, 61, 160
  * Portegies Zwart & Jílková (2015) Portegies Zwart, S. F. & Jílková, L. 2015, MNRAS , 451, 4663
  * Portegies Zwart et al. (2010) Portegies Zwart, S. F., McMillan, S. L. W., & Gieles, M. 2010, ARA&A , 48, 431
  * Portegies Zwart & Verbunt (1996) Portegies Zwart, S. F. & Verbunt, F. 1996, A&A , 309, 179
  * Punzo et al. (2014) Punzo, D., Capuzzo-Dolcetta, R., & Portegies Zwart, S. 2014, MNRAS , 444, 2808
  * Rabinowitz et al. (2006) Rabinowitz, D. L., Barkume, K., Brown, M. E., et al. 2006, ApJ , 639, 1238
  * Rampino & Prokoph (2020) Rampino, M. R. & Prokoph, A. 2020, Astrobiology, 20, 1097
  * Raup & Sepkoski (1984) Raup, D. M. & Sepkoski, J. J. 1984, Proceedings of the National Academy of Science, 81, 801
  * Rawiraswattana et al. (2016) Rawiraswattana, K., Hubber, D. A., & Goodwin, S. P. 2016, MNRAS , 460, 3505
  * Rein & Liu (2012) Rein, H. & Liu, S.-F. 2012, A&A , 537, A128
  * Rein & Spiegel (2015) Rein, H. & Spiegel, D. S. 2015, MNRAS , 446, 1424
  * Rickman (2014) Rickman, H. 2014, Meteoritics and Planetary Science, 49, 8
  * Rickman et al. (2008) Rickman, H., Fouchard, M., Froeschlé, C., & Valsecchi, G. B. 2008, Celestial Mechanics and Dynamical Astronomy, 102, 111
  * Rickman et al. (1987) Rickman, H., Kamel, L., Festou, M. C., & Froeschle, C. 1987, in ESA Special Publication, Vol. 278, Diversity and Similarity of Comets, ed. E. J. Rolfe, B. Battrick, M. Ackerman, M. Scherer, & R. Reinhard, 471–481
  * Romero-Gómez et al. (2011) Romero-Gómez, M., Athanassoula, E., Antoja, T., & Figueras, F. 2011, MNRAS , 418, 1176
  * Ryder (2002) Ryder, G. 2002, Journal of Geophysical Research (Planets), 107, 5022
  * Saillenfest (2020) Saillenfest, M. 2020, Celestial Mechanics and Dynamical Astronomy, 132, 12
  * Saillenfest et al. (2019) Saillenfest, M., Fouchard, M., Ito, T., & Higuchi, A. 2019, A&A , 629, A95
  * Schlecker et al. (2020) Schlecker, M., Mordasini, C., Emsenhuber, A., et al. 2020, arXiv e-prints, arXiv:2007.05563
  * Scott (1992) Scott, D. 1992, Multivariate Density Estimation: Theory, Practice, and Visualization (Wiley Series in Probability and Statistics, John Wiley & Sons, Inc. 1992)
  * Shankman et al. (2011) Shankman, C., Gladman, B., & Kaib, N. 2011, in EPSC-DPS Joint Meeting 2011, Vol. 2011, 633
  * Shannon et al. (2019) Shannon, A., Jackson, A. P., & Wyatt, M. C. 2019, MNRAS , 485, 5511
  * Sheppard et al. (2014) Sheppard, S. S., Trujillo, C., & Williams, G. V. 2014, Minor Planet Electronic Circulars, 2014-F40
  * Sheppard et al. (2019) Sheppard, S. S., Trujillo, C. A., Tholen, D. J., & Kaib, N. 2019, AJ , 157, 139
  * Siraj & Loeb (2020) Siraj, A. & Loeb, A. 2020, ApJL , 899, L24
  * Sosa & Fernández (2009) Sosa, A. & Fernández, J. A. 2009, MNRAS , 393, 192
  * Sosa & Fernández (2011) Sosa, A. & Fernández, J. A. 2011, MNRAS , 416, 767
  * Stevenson & Lunine (1988) Stevenson, D. J. & Lunine, J. I. 1988, Icarus , 75, 146
  * Stock et al. (2020) Stock, K., Cai, M. X., Spurzem, R., Kouwenhoven, M. B. N., & Portegies Zwart, S. 2020, MNRAS , 497, 1807
  * Stöffler & Ryder (2001) Stöffler, D. & Ryder, G. 2001, Space Sci. Rev. , 96, 9
  * Sussman & Wisdom (1988) Sussman, G. J. & Wisdom, J. 1988, Science, 241, 433
  * Sussman & Wisdom (1992) Sussman, G. J. & Wisdom, J. 1992, Science, 257, 56
  * Tanikawa & Ito (2007) Tanikawa, K. & Ito, T. 2007, Publ. Astr. Soc. Japan , 59, 989
  * Tegler & Romanishin (2000) Tegler, S. C. & Romanishin, W. 2000, Nat , 407, 979
  * Thommes et al. (2008) Thommes, E., Nagasawa, M., & Lin, D. N. C. 2008, ApJ , 676, 728
  * Todorović et al. (2020) Todorović, N., Wu, D., & Rosengren, A. J. 2020, Science Advances, 6, eabd1313
  * Toomre (1964) Toomre, A. 1964, ApJ , 139, 1217
  * Toonen et al. (2016) Toonen, S., Hamers, A., & Portegies Zwart, S. 2016, Computational Astrophysics and Cosmology, 3, 6
  * Torres (2020) Torres, S. 2020, PhD thesis: Dynamics of the Oort Cloud and the Formation of Interstellar Comets (Leiden Observatory)
  * Torres et al. (2019) Torres, S., Cai, M. X., Brown, A. G. A., & Portegies Zwart, S. 2019, A&A , 629, A139
  * Torres et al. (2020a) Torres, S., Cai, M. X., Mukherjee, D., Portegies Zwart, S., & Brown, A. G. A. 2020a, Astron. Astrophys., submitted, 17
  * Torres et al. (2020b) Torres, S., Portegies Zwart, S., & Brown, A. G. A. 2020b, Astron. Astrophys., submitted, 10
  * Trujillo et al. (2000) Trujillo, C. A., Jewitt, D. C., & Luu, J. X. 2000, ApJL , 529, L103
  * Trujillo & Sheppard (2014) Trujillo, C. A. & Sheppard, S. S. 2014, Nat , 507, 471
  * Tsiganis et al. (2005) Tsiganis, K., Gomes, R., Morbidelli, A., & Levison, H. F. 2005, Nat , 435, 459
  * Tutukov, Dremova, & Dremov (2020) Tutukov A. V., Dremova G. N., Dremov V. V., 2020, ARep, 64, 936
  * Tutukov, Sizova, & Vereshchagin (2021) Tutukov A. V., Sizova M. D., Vereshchagin S. V., 2021, ARep, 65, 305
  * Valtonen et al. (2004) Valtonen, M., Mylläri, A., Orlov, V., & Rubinov, A. 2004, in Astronomical Society of the Pacific Conference Series, Vol. 316, Order and Chaos in Stellar and Planetary Systems, ed. G. G. Byrd, K. V. Kholshevnikov, A. A. Myllri, I. I. Nikiforov, & V. V. Orlov, 45
  * Valtonen & Innanen (1982) Valtonen, M. J. & Innanen, K. A. 1982, ApJ , 255, 307
  * Valtonen et al. (1992) Valtonen, M. J., Zheng, J.-Q., & Mikkola, S. 1992, Celestial Mechanics and Dynamical Astronomy, 54, 37
  * van Rossum (1995) van Rossum, G. 1995, Extending and embedding the Python interpreter, Report CS-R9527
  * Varadi et al. (2003) Varadi, F., Runnegar, B., & Ghil, M. 2003, ApJ , 592, 620
  * Vargya & Sanderson (2020) Vargya, D. & Sanderson, R. 2020, in AAS/Division of Dynamical Astronomy Meeting, Vol. 52, AAS/Division of Dynamical Astronomy Meeting, 200.05
  * Veras et al. (2014) Veras, D., Evans, N. W., Wyatt, M. C., & Tout, C. A. 2014, MNRAS , 437, 1127
  * Veras et al. (2020) Veras, D., Reichert, K., Flammini Dotti, F., et al. 2020, MNRAS , 493, 5062
  * Veras & Tout (2012) Veras, D. & Tout, C. A. 2012, MNRAS , 422, 1648
  * Veras et al. (2011) Veras, D., Wyatt, M. C., Mustill, A. J., Bonsor, A., & Eldridge, J. J. 2011, MNRAS , 417, 2104
  * Vincke et al. (2015) Vincke, K., Breslau, A., & Pfalzner, S. 2015, A&A , 577, A115
  * Vincke & Pfalzner (2016) Vincke, K. & Pfalzner, S. 2016, ApJ , 828, 48
  * Vokrouhlický et al. (2019) Vokrouhlický, D., Nesvorný, D., & Dones, L. 2019, AJ , 157, 181
  * Volk & Malhotra (2017) Volk, K. & Malhotra, R. 2017, AJ , 154, 62
  * von Zeipel (1910) von Zeipel, H. 1910, Astronomische Nachrichten, 183, 345
  * Walsh et al. (2011) Walsh, K. J., Morbidelli, A., Raymond, S. N., O’Brien, D. P., & Mandell, A. M. 2011, Nat , 475, 206
  * Wang et al. (2015) Wang, L., Spurzem, R., Aarseth, S., et al. 2015, MNRAS , 450, 4070
  * Weissman (1983) Weissman, P. R. 1983, A&A , 118, 90
  * Weissman (1996) Weissman, P. R. 1996, in Astronomical Society of the Pacific Conference Series, Vol. 107, Completing the Inventory of the Solar System, ed. T. Rettig & J. M. Hahn, 265–288
  * Williams & Cieza (2011) Williams, J. P. & Cieza, L. A. 2011, ARA&A , 49, 67
  * Winter et al. (2018) Winter, A. J., Clarke, C. J., Rosotti, G., et al. 2018, MNRAS , 478, 2700
  * Wong et al. (2019) Wong, E. W., Brasser, R., & Werner, S. C. 2019, Earth and Planetary Science Letters, 506, 407
  * Wyatt (2003) Wyatt, M. C. 2003, ApJ , 598, 1321
  * Wyatt (2008) Wyatt, M. C. 2008, ARA&A , 46, 339
  * Xiang-Gruess (2016) Xiang-Gruess, M. 2016, MNRAS , 455, 3086
  * Zderic & Madigan (2020) Zderic, A. & Madigan, A.-M. 2020, AJ , 160, 50
  * Zellner (2017) Zellner, N. E. B. 2017, Origins of Life and Evolution of the Biosphere, 47, 261
  * Zhang (2018) Zhang, Q. 2018, ApJL , 852, L13
  * Zheng et al. (1990) Zheng, J.-Q., Valtonen, M. J., & Valtaoja, L. 1990, Celestial Mechanics and Dynamical Astronomy, 49, 265
  * Zink et al. (2020) Zink, J. K., Batygin, K., & Adams, F. C. 2020, AJ , 160, 232


Comparison of Models for Training Optical Matrix Multipliers in Neuromorphic
PICs

A. Cem1*, S. Yan1,2, U.C. de Moura1, Y. Ding1, D. Zibar1, F. Da Ros1

1DTU Fotonik, Technical University of Denmark, DK-2800, Kgs. Lyngby, Denmark

2School of Optical & Electrical Information, Huazhong Univ. of Science and
Technology, 430074, Wuhan, China

<EMAIL_ADDRESS>
###### Abstract

We experimentally compare simple physics-based vs. data-driven neural-network-
based models for offline training of programmable photonic chips using Mach-
Zehnder interferometer meshes. The neural-network model outperforms physics-
based models for a chip with thermal crosstalk, yielding increased testing
accuracy.

##  1 Introduction

The increasing demand for computational power, especially for machine learning
applications, has resulted in a renewed interest in the development of new
hardware architectures. Particularly, photonics has emerged as a promising
medium for neuromorphic computing [1]. Motivated by the impressive results
obtained using artificial neural networks (ANNs), optical neural networks
(ONNs) have been suggested as a fast and low-power alternative. Each layer of
an ANN consists of two fundamental operations: a linear matrix-vector
multiplication followed by a nonlinear activation function. Photonics
implements the former particularly efficiently with a variety of designs
proposed, including using tunable microring resonators [2] and Mach-Zehnder
interferometer (MZI) meshes [3].

Focusing on the latter, the linear weights are typically adjusted by changing
the voltages applied to the thermo-optic phase shifters present in each MZI.
In order to program a MZI mesh to implement a weight matrix, the mapping
between the input voltages and MZI transmission is required. This can be
obtained using the simple physics-based model of a MZI [3, 4]. Due to
fabrication errors and neglected effects, such as thermal crosstalk (XT),
using the simple physics-based model results in poor performance [4, 5, 6].
This has lead to the development of either error correction schemes [5] and
error-resistant ONNs [6], or calibration procedures characterizing the
voltage-transmission relation of each MZI individually [4]. However, the
calibration of [4] does not take into account the thermal XT between MZIs.
This is an effect that increases as the number of MZIs per chip footprint
grows, even though complex crosstalk-minimizing chip designs [7] are being
proposed. On-line training procedures [8] address this challenge but require
re-optimization for each new target, thus they do not remove the need for
accurate models.

In this work, we compare the performances of simple physics-based models with
and without thermal XT versus an ANN approach for modelling MZI meshes, i.e.
ONN matrix multipliers. All models are trained with experimental data to
predict the weights of the MZI mesh given the voltages applied to the
individual MZIs for phase tuning. We evaluate the models using the photonic
integrated circuit (PIC) described in [9]. The ANN model is significantly more
accurate than simple physics-based models and results in improved testing
performance when used to numerically validate the training for solving a toy
classification task.

##  2 MZI Mesh Models

Fig. 1: (a) Block diagram of a photonic matrix multiplier. Experimental
measurements for $W_{21}$ when the (b) $V_{1}$ and (c) $V_{2}$ are swept,
plotted along the fitted curves for the physics-based models.

An example of an architecture for a MZI mesh implementing the linear layer of
an ONN is shown in Fig. 1(a). Each MZI has a thermo-optic phase shifter
controlled by an external heater voltage on one arm, which can be tuned to
modify the implemented weight matrix $\mathbf{W}$, relating the input and
output power vectors $\mathbf{P_{in}}$ and $\mathbf{P_{out}}$ as follows:
$\mathbf{P_{out}}=\mathbf{W}(\mathbf{V})\cdot\mathbf{P_{in}}$. In order to
program the PIC to implement a given matrix multiplication, a model describing
the mapping between the heater voltages $\mathbf{V}$ and the achieved
$\mathbf{W}$ is required.

We consider three models: two simple physics-based models without (Model 1)
and with (Model 2) thermal XT and a data-driven ANN model (Model 3). For Model
1, we have modified the model of a single MZI used in [3, 4] by adding a
finite extinction ratio $ER$. The model for the weights $W_{ij}$ of the MZI
mesh is given by Eq. 1.

$\displaystyle W_{ij}=L_{ij}\prod_{k\in
K_{ij}}\frac{1}{4}\left|\frac{\sqrt{ER}-1}{\sqrt{ER}+1}-\exp\left({\sqrt{-1}\;(\phi^{(0)}_{k}+\phi^{(2)}_{k}V_{k}^{2})}\right)\right|^{2}\,,$
(1)

where $L_{ij}$ is the total loss accumulated for the path through the PIC
corresponding to $W_{ij}$, $K_{ij}$ is the set of all MZIs in path
$i\rightarrow j$, $V_{k}$ is the voltage applied to the $k$th MZI, and
$\phi_{k}^{(0)}$ and $\phi_{k}^{(2)}$ are the fitted phase parameters
describing the initial value of the phase offset and the quadratic (thermal)
relation between $V_{k}$ and phase shift, respectively. Parameters $L_{ij}$,
$\phi_{k}^{(0)}$ and $\phi_{k}^{(2)}$ are susceptible to fabrication errors
and noise sources, so these values can be fitted during calibration, e.g. the
voltage to one MZI is swept at a time and the output power is measured. This
is equivalent to measuring $W_{ij}$ after input normalization. Fig. 1(b) shows
an example sweep along with Model 1 trained by minimizing the root mean
squared error (RMSE) between the experimentally measured and the predicted
matrix weights in dB. RMSE is the cost function used for training all models.
Even if the experimental data fits well to the analytical expression for this
case, this does not necessarily mean that the chip is modelled accurately
using Model 1. The thermal effect also applies undesired phase shifts to
neighboring MZIs due to thermal XT, as illustrated in Fig. 1(c), resulting in
errors for Model 1 when MZIs outside the path are controlled. Without
characterizing the XT explicitly, Model 1 can be extended to capture this
effect by including contributions from all voltages for the phase shifts, here
called Model 2. $W_{ij}$ for a mesh with $M$ MZIs is given in Eq. 2. Note that
$\phi^{(2)}_{ij,k,m}$ denotes the impact of $V_{m}$ on MZI $k$ for the path
corresponding to $W_{ij}$. Phase parameters were recalculated for each
$W_{ij}$.

$\displaystyle W_{ij}=L_{ij}\prod_{k\in
K_{ij}}\frac{1}{4}\left|\frac{\sqrt{ER}-1}{\sqrt{ER}+1}-\exp\left({\sqrt{-1}\;(\phi^{(0)}_{ij,k}+\sum_{m=1}^{M}\phi^{(2)}_{ij,k,m}V_{m}^{2})}\right)\right|^{2}\,.$
(2)

Alternatively, an ANN can be used to model the relation between $\mathbf{V}$
and $\mathbf{W}$ as shown in Fig. 2(a), here called Model 3. Getting
inspiration from the physics-based models, we have included the heater
voltages along with their squares in the input layer, both sets normalized
between -1 and +1 separately. The output layer consists of the matrix weights
in dB. To train an accurate model, an experimental dataset consisting of many
different combinations of input voltages is required. ANN hyperparameters are
optimized such that the RMSE between the measured and the predicted outputs is
minimized, resulting in the architecture shown in Fig. 2(a). The hyperbolic
tangent was used as the nonlinear activation function in the hidden nodes.

##  3 Experimental Setup and Results

Fig. 2: (a) Architecture of the ANN model (Model 3) for the PIC. (b)
Measurement setup for acquiring the training and testing datasets from the
PIC.

The measurement setup is shown in Fig. 2(b). A flattened power spectral
distribution from an amplified spontaneous emission (ASE) source (5-THz
bandwidth) is injected to the chip. The light is coupled from the three-core
multi-core fiber (MCF) to the chip by an on-chip grating coupler array[9].
Polarization controllers (PC) control the input signal polarization, and
$1\times 3$ and $3\times 1$ optical switches are used at the input and output
of the chip to measure the $3\times 3$ weight matrix using a power meter. The
details of the photonic chip considered for the measurements are discussed in
[9]. Here, we considered only a subset of the MZIs that resulted in a $3\times
3$ weight matrix (chip in Fig. 2(b)), with their voltages independently
controlled. To acquire a measurement dataset so that the three models can be
compared, control voltages were individually varied from 0 to 2V (half-period
of the MZIs), either systematically and one at a time (for fitting the phase
parameters of Model 1) or measuring up to 5100 sets of ($\mathbf{V}$,
$\mathbf{W}$) by randomly varying the applied voltages (for fitting the losses
in Model 1 and fitting all parameters in Models 2 and 3). 4400 measurement
samples were used for training, and the remaining 700 as testing data. For
Model 3, 700 samples from the training set were reserved for cross-validation.
Models 1 and 2 were trained on MATLAB with the interior-point algorithm, while
Model 3 was trained on PyTorch with the L-BFGS optimizer.

After training, the RMSEs between the measured and the predicted matrix
weights in the testing set are 3.26 dB, 1.44 dB, and 0.53 dB for Models 1, 2,
and 3, respectively. Fig. 3(a) shows the scatter plots of all weights for the
testing data, revealing that Model 3 achieves superior performance both for
smaller and larger weights. The probability distribution functions (PDFs) of
the errors between predicted and measured weights over the testing data are
plotted in Fig. 3(b), for all three models. While considering the thermal XT
in the physics-based model (Model 2) results in a narrower error distribution,
Models 1 and 2 are not as accurate as Model 3, especially in terms of maximum
prediction error, resulting in the long tails for their respective PDFs.

To quantify the advantage of more accurate models for off-line training in a
practical case, an example single-hidden-layer ANN was trained digitally for
the 3-input-bit XOR task described in [10] (output is 1 only when exactly one
input is 1). We assumed that the MZI mesh was used as the $3\times 3$ matrix
of weights between the input and the hidden layer (inset Fig. 3(c)). After
training, Gaussian noise was introduced to the weights of the $3\times 3$
linear layer in order to quantify the accuracy penalty that comes from using
an imperfect model for training, i.e. the applied voltages yield slightly
different weights. The noise effectively emulates prediction errors between
the training model and the chip. The hidden nodes with sigmoid activation
functions and the $3\times 1$ linear output layer were kept noise-free. The
classification accuracies for the noisy neural networks are shown in Fig.
3(c). Using Model 3 (RMSE = 0.59 dB) for training yields higher testing
accuracy ($>98$% for 20 different error realizations) than by considering the
physics-based models (76% and 59% accuracy for Models 2 and 1, respectively).

Fig. 3: (a) Scatter plot of the predicted and measured matrix weights for the
three models. (b) Probability density functions obtained by normalizing the
error histograms. (c) Classification accuracy on the 3-input XOR task with
noise ($\sigma_{noise}$) added on $\mathbf{W}$ according to the training
model’s RMSE. Error bars show the 25th and 75th percentiles for 20 noise
realizations. Inset: architecture of the ANN.

##  4 Conclusion

We describe and experimentally test an ANN-based approach for modelling
photonic neuromorphic devices. Compared to the simple physics-based models,
the ANN-based model reduces the prediction error by more than a factor of 2
for a fabricated chip, even when the thermal XT is accounted for by the
physics-based model. While our approach was tested on a specific chip, it can
also be applied to other more complex MZI-mesh architectures.

Acknowledgment Villum Foundations, Villum YI, OPTIC-AI, grant n. 29344, and
ERC CoG FRECOM, grant n. 771878.

## References

  * [1] B. Shastri et al., “Photonics for artificial intelligence and neuromorphic computing,” Nat. Phot. 15, 102-114 (2021).
  * [2] A. Tait et al., “Neuromorphic photonic networks using silicon photonic weight banks,” Sci. Rep. 7, (2017).
  * [3] Y. Shen et al., “Deep learning with coherent nanophotonic circuits,” Nat. Phot. 11, 441-446 (2017).
  * [4] D. Pérez et al., “Multipurpose silicon photonics signal processor core,” Nat. Comms. 8, (2017).
  * [5] S. Bandyopadhyay et al., “Hardware error correction for programmable photonics,” Optica 8, 1247 (2021).
  * [6] M. Fang et al., “Design of Optical Neural Networks with component imprecisions,” Opt. Express 27, 14009 (2019).
  * [7] M. Milanizadeh et al., “Control and Calibration Recipes for Photonic Integrated Circuits,” JSTQE 26, 1-10 (2020).
  * [8] H. Zhang et al., “Efficient On-Chip Training of Optical Neural Networks Using Genetic Algorithm,” ACS Photonics 8, 1662-1672 (2021).
  * [9] Y. Ding et al., “Reconfigurable SDM Switching Using Novel Silicon Photonic Integrated Circuit,” Sci. Rep. 6, (2016).
  * [10] I. Williamson et al., “Reprogrammable Electro-Optic Nonlinear Activation Functions for Optical Neural Networks,” JSTQE 26, 1-12 (2020).