* [278] K. Mitra, A.-R. Carvunis, S. K. Ramesh, and T. Ideker.  Integrative approaches for finding modular structure in biological networks.  Nature Reviews Genetics, 14(10):719–732, 2013.
  * [279] J. E. Moore, M. J. Purcaro, H. E. Pratt, C. B. Epstein, N. Shoresh, J. Adrian, T. Kawli, C. A. Davis, A. Dobin, et al.  Expanded encyclopaedias of DNA elements in the human and mouse genomes.  Nature, 583(7818):699–710, 2020.
  * [280] N. Moris, C. Pina, and A. M. Arias.  Transition states and cell fate decisions in epigenetic landscapes.  Nature Reviews Genetics, 17(11):693–703, 2016.
  * [281] R. T. Morris, T. R. O’Connor, and J. J. Wyrick.  Ceres: software for the integrated analysis of transcription factor binding sites and nucleosome positions in saccharomyces cerevisiae.  Bioinformatics, 26(2):168–174, 2010.
  * [282] D. Morselli Gysi, T. de Miranda Fragoso, F. Zebardast, W. Bertoli, V. Busskamp, E. Almaas, and K. Nowick.  Whole transcriptomic network analysis using co-expression differential network analysis (CoDiNA).  PLOS One, 15(10):e0240523, 2020.
  * [283] D. Morselli Gysi, Í. Do Valle, M. Zitnik, A. Ameli, X. Gan, O. Varol, S. D. Ghiassian, J. Patten, R. A. Davey, J. Loscalzo, et al.  Network medicine framework for identifying drug-repurposing opportunities for COVID-19.  Proceedings of the National Academy of Sciences, 118(19):e2025581118, 2021.
  * [284] R. Mosca, A. Céol, and P. Aloy.  Interactome3D: adding structural details to protein networks.  Nature Methods, 10(1):47–53, 2013.
  * [285] J. Moult, J. T. Pedersen, R. Judson, and K. Fidelis.  A large-scale experiment to assess protein structure prediction methods.  PROTEINS: Structure, Function, and Bioinformatics, 23(3):ii–iv, 1995.
  * [286] P. Mucha, T. Richardson, K. Macon, M. Porter, and J. Onnela.  Community structure in time-dependent, multiscale, and multiplex networks.  Science, 328(5980):876–878, 2010.
  * [287] K. A. Murgas, E. Saucan, and R. Sandhu.  Hypergraph geometry reflects higher-order dynamics in protein interaction networks.  Scientific Reports, 12(1):20879, 2022.
  * [288] R. Nasser and R. Sharan.  BERTwalk for integrating gene networks to predict gene-to pathway-level properties.  Bioinformatics Advances, 3(1):vbad086, 2023.
  * [289] N. Natarajan and I. S. Dhillon.  Inductive matrix completion for predicting gene-disease associations.  Bioinformatics, 30(12):i60–i68, 2014.
  * [290] E. J. Needham, B. L. Parker, T. Burykin, D. E. James, and S. J. Humphrey.  Illuminating the dark phosphoproteome.  Science Signaling, 12(565):eaau8645, 2019.
  * [291] W. Nelson, M. Žitnik, B. Wang, J. Leskovec, A. Goldenberg, and R. Sharan.  To embed or not: network embedding as a paradigm in computational biology.  Frontiers in Genetics, 10:381, 2019.
  * [292] S. Neph, A. B. Stergachis, A. Reynolds, R. Sandstrom, E. Borenstein, and J. A. Stamatoyannopoulos.  Circuitry and dynamics of human transcription factor regulatory networks.  Cell, 150(6):1274–1286, 2012.
  * [293] J. Neville, B. Gallagher, and T. Eliassi-Rad.  Evaluating statistical tests for within-network classifiers of relational data.  In Proceedings of the IEEE International Conference on Data Mining, pages 397–406, 2009.
  * [294] J. Neville, B. Gallagher, T. Eliassi-Rad, and T. Wang.  Correcting evaluation bias of relational classifiers with network cross validation.  Knowledge and Information Systems, 30:31–55, 2012.
  * [295] K. Newaz, M. Ghalehnovi, A. Rahnama, P. Antsaklis, and T. Milenković.  Network-based protein structural classification.  Royal Society Open Science, 7(6):191461, 2020.
  * [296] K. Newaz and T. Milenković.  Graphlets in network science and computational biology.  Analyzing Network Data in Biology and Medicine: An Interdisciplinary Textbook for Biological, Medical and Computational Scientists, pages 193–240, 2019.
  * [297] K. Newaz and T. Milenkovic.  Inference of a Dynamic Aging-related Biological Subnetwork via Network Propagation.  IEEE/ACM Transactions on Computational Biology and Bioinformatics, 19(2):974–988, 2022.
  * [298] K. Newaz, J. Piland, P. L. Clark, S. J. Emrich, J. Li, and T. Milenković.  Multi-layer sequential network analysis improves protein 3D structural classification.  PROTEINS: Structure, Function, and Bioinformatics, 90(9):1721–1731, 2022.
  * [299] M. Newman.  Networks.  Oxford University Press, 2018.
  * [300] M. E. Newman.  Modularity and community structure in networks.  Proceedings of the National Academy of Sciences, 103(23):8577–8582, 2006.
  * [301] J. Ni, M. Koyuturk, H. Tong, J. Haines, R. Xu, and X. Zhang.  Disease gene prioritization by integrating tissue-specific molecular networks using a robust multi-network model.  BMC Bioinformatics, 17:1–13, 2016.
  * [302] D. N. Nicholson and C. S. Greene.  Constructing knowledge graphs and their biomedical applications.  Computational and Structural Biotechnology Journal, 18:1414–1428, 2020.
  * [303] M. W. Nielsen, C. W. Bloch, and L. Schiebinger.  Making gender diversity work for scientific discovery and innovation.  Nature Human Behaviour, 2(10):726–734, 2018.
  * [304] R. Nishihara, K. Glass, K. Mima, T. Hamada, J. A. Nowak, Z. R. Qian, P. Kraft, E. L. Giovannucci, C. S. Fuchs, A. T. Chan, et al.  Biomarker correlation network in colorectal carcinoma by tumor anatomic location.  BMC Bioinformatics, 18(1):1–14, 2017.
  * [305] P. Niu, M. J. Soto, B.-J. Yoon, E. R. Dougherty, F. J. Alexander, I. Blaby, and X. Qian.  TRIMER: transcription regulation integrated with metabolic regulation.  iScience, 24(11):103218, 2021.
  * [306] M. Nykter, N. D. Price, M. Aldana, S. A. Ramsey, S. A. Kauffman, L. E. Hood, O. Yli-Harja, and I. Shmulevich.  Gene expression dynamics in the macrophage exhibit criticality.  Proceedings of the National Academy of Sciences, 105(6):1897–1900, 2008.
  * [307] A. of Us Research Program Investigators.  The “All of Us” research program.  New England Journal of Medicine, 381(7):668–676, 2019.
  * [308] O. Ourfali, T. Shlomi, T. Ideker, E. Ruppin, and R. Sharan.  SPINE: a framework for signaling-regulatory pathway inference from cause-effect experiments.  Bioinformatics, 23(13):i359–i366, 2007.
  * [309] M. Padi and J. Quackenbush.  Detecting phenotype-driven transitions in regulatory network structure.  NPJ Systems Biology and Applications, 4(1):16, 2018.
  * [310] R. D. Page.  Wikidata and the bibliography of life.  PeerJ, 10:e13712, 2022.
  * [311] S. Pai and G. D. Bader.  Patient similarity networks for precision medicine.  Journal of Molecular Biology, 430(18):2924–2938, 2018.
  * [312] S. Pai, S. Hui, R. Isserlin, M. A. Shah, H. Kaka, and G. D. Bader.  netDx: interpretable patient classification using integrated patient similarity networks.  Molecular Systems Biology, 15(3):e8497, 2019.
  * [313] S. Pan, L. Luo, Y. Wang, C. Chen, J. Wang, and X. Wu.  Unifying large language models and knowledge graphs: a roadmap.  arXiv:2306.08302, 2023.
  * [314] J. Pandey, M. Koyutürk, Y. Kim, W. Szpankowski, S. Subramaniam, and A. Grama.  Functional annotation of regulatory pathways.  Bioinformatics, 23(13):i377–i386, 2007.
  * [315] A. Pareja, G. Domeniconi, J. Chen, T. Ma, T. Suzumura, H. Kanezashi, T. Kaler, T. Schardl, and C. Leiserson.  Evolvegcn: evolving graph convolutional networks for dynamic graphs.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 5363–5370, 2020.
  * [316] J. Park, B. J. Hescott, and D. K. Slonim.  Pathway centrality in protein interaction networks identifies putative functional mediating pathways in pulmonary disease.  Scientific Reports, 9(1):5863, 2019.
  * [317] Y. Park and E. M. Marcotte.  Flaws in evaluation schemes for pair-input computational predictions.  Nature Methods, 9(12):1134–1136, 2012.
  * [318] J. Patten, P. T. Keiser, D. Morselli-Gysi, G. Menichetti, H. Mori, C. J. Donahue, X. Gan, I. do Valle, K. Geoghegan-Barek, M. Anantpadma, et al.  Identification of potent inhibitors of SARS-CoV-2 infection by combined pharmacological evaluation and cellular network prioritization.  iScience, 25(9), 2022.
  * [319] E. O. Paull, D. E. Carlin, M. Niepel, P. K. Sorger, D. Haussler, and J. M. Stuart.  Discovering causal pathways linking genomic events to transcriptional states using tied diffusion through interacting events (TieDIE).  Bioinformatics, 29(21):2757–2764, 2013.
  * [320] S. A. Peck Justice, N. A. McCracken, J. F. Victorino, G. D. Qi, A. B. Wijeratne, and A. L. Mosley.  Boosting detection of low-abundance proteins in thermal proteome profiling experiments by addition of an isobaric trigger channel to TMT multiplexes.  Analytical Chemistry, 93(18):7000–7010, 2021.
  * [321] C. Peng, F. Xia, M. Naseriparsa, and F. Osborne.  Knowledge graphs: opportunities and challenges.  Artificial Intelligence Review, pages 1–32, 2023.
  * [322] H. Peng, H. Wang, B. Du, M. Z. A. Bhuiyan, H. Ma, J. Liu, L. Wang, Z. Yang, L. Du, S. Wang, et al.  Spatial temporal incidence dynamic graph neural networks for traffic flow forecasting.  Information Sciences, 521:277–290, 2020.
  * [323] L. Perez De Souza, S. Alseekh, Y. Brotman, and A. R. Fernie.  Network-based strategies in metabolomics data analysis and interpretation: from molecular networking to biological interpretation.  Expert Review of Proteomics, 17(4):243–255, 2020.
  * [324] B. Perozzi, R. Al-Rfou, and S. Skiena.  Deepwalk: online learning of social representations.  In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 701–710, 2014.
  * [325] A. V. Persikov and M. Singh.  De novo prediction of DNA-binding specificities for Cys2His2 zinc finger proteins.  Nucleic Acids Research, 42(1):97–108, 2014.
  * [326] M. Petti and L. Farina.  Network medicine for patients’ stratification: from single-layer to multi-omics.  Wires Mechanisms of Disease, page e1623, 2023.
  * [327] L. Piersimoni, P. L. Kastritis, C. Arlt, and A. Sinz.  Cross-linking mass spectrometry for investigating protein conformations and protein-protein interactions—a method for all seasons.  Chemical Reviews, 122(8):7500–7531, 2022.
  * [328] E. Pierson, G. Consortium, D. Koller, A. Battle, and S. Mostafavi.  Sharing and specificity of co-expression networks across 35 human tissues.  PLOS Computational Biology, 11(5):e1004220, 2015.
  * [329] L. Pio-Lopez, A. Valdeolivas, L. Tichit, É. Remy, and A. Baudot.  MultiVERSE: a multiplex and multiplex-heterogeneous network embedding approach.  Scientific Reports, 11(1):8794, 2021.
  * [330] L. Pirhaji, P. Milani, M. Leidl, T. Curran, J. Avila-Pacheco, C. B. Clish, F. M. White, A. Saghatelian, and E. Fraenkel.  Revealing disease-associated pathways by network integration of untargeted metabolomics.  Nature Methods, 13(9):770–776, 2016.
  * [331] N. Pržulj.  Biological network comparison using graphlet degree distribution.  Bioinformatics, 23(2):e177–e183, 2007.
  * [332] N. Pržulj, D. G. Corneil, and I. Jurisica.  Modeling interactome: scale-free or geometric?  Bioinformatics, 20(18):3508–3515, 2004.
  * [333] N. Pržulj and N. Malod-Dognin.  Network analytics in the age of big data.  Science, 353(6295):123–124, 2016.
  * [334] P. F. Przytycki and M. Singh.  Differential allele-specific expression uncovers breast cancer genes dysregulated by cis noncoding mutations.  Cell Systems, 10(2):193–203, 2020.
  * [335] P. Purkait, T. J. Chin, A. Sadri, and D. Suter.  Clustering with hypergraphs: the case for large hyperedges.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(9):1697–1711, 2017.
  * [336] S. Pushpakom, F. Iorio, P. A. Eyers, K. J. Escott, S. Hopper, A. Wells, A. Doig, T. Guilliams, J. Latimer, C. McNamee, et al.  Drug repurposing: progress, challenges and recommendations.  Nature Reviews Drug Discovery, 18(1):41–58, 2019.
  * [337] P. Radivojac, W. T. Clark, T. R. Oron, A. M. Schnoes, T. Wittkop, A. Sokolov, K. Graim, C. Funk, K. Verspoor, A. Ben-Hur, et al.  A large-scale evaluation of computational protein function prediction.  Nature Methods, 10(3):221–227, 2013.
  * [338] M. Radovanović, A. Nanopoulos, and M. Ivanović.  Hubs in space: popular nearest neighbors in high-dimensional data.  Journal of Machine Learning Research, 11:2487–2531, 2010.
  * [339] E. Ramadan, A. Tarafdar, and A. Pothen.  A hypergraph model for the yeast protein complex network.  In Proceedings of the International Parallel and Distributed Processing Symposium, page 189, 2004.
  * [340] R. Ramola, I. Friedberg, and P. Radivojac.  The field of protein function prediction as viewed by different domain scientists.  Bioinformatics Advances, 2(1):vbac057, 2022.
  * [341] R. Rao, N. Bhattacharya, N. Thomas, Y. Duan, P. Chen, J. Canny, P. Abbeel, and Y. Song.  Evaluating protein transfer learning with tape.  In Proceedings of the Advances in Neural Information Processing Systems, volume 32, 2019.
  * [342] D. N. Reshef, Y. A. Reshef, H. K. Finucane, S. R. Grossman, G. McVean, P. J. Turnbaugh, E. S. Lander, M. Mitzenmacher, and P. C. Sabeti.  Detecting novel associations in large data sets.  Science, 334(6062):1518–1524, 2011.
  * [343] M. A. Reyna, U. Chitra, R. Elyanow, and B. J. Raphael.  NetMix: a network-structured mixture model for reduced-bias estimation of altered subnetworks.  Journal of Computational Biology, 28(5):469–484, 2021.
  * [344] G. Rhodes.  Crystallography Made Crystal Clear, Third Edition: A Guide for Users of Macromolecular Models.  Elsevier, 2010.
  * [345] A. Rider, T. Milenković, G. Siwo, R. Pinapati, S. Emrich, M. Ferdig, and N. Chawla.  Networks are important for systems biology.  Network Science, 2(02):139–161, 2014.
  * [346] M. E. Ritchie, B. Phipson, D. Wu, Y. Hu, C. W. Law, W. Shi, and G. K. Smyth.  limma powers differential expression analyses for RNA-sequencing and microarray studies.  Nucleic Acids Research, 43(7):e47–e47, 2015.
  * [347] A. Ritz, C. L. Poirel, A. N. Tegge, N. Sharp, K. Simmons, A. Powell, S. D. Kale, and T. Murali.  Pathways on demand: automated reconstruction of human signaling networks.  NPJ Systems Biology and Applications, 2(1):1–9, 2016.
  * [348] A. Ritz, A. N. Tegge, H. Kim, C. L. Poirel, and T. Murali.  Signaling hypergraphs.  Trends in Biotechnology, 32(7):356–362, 2014.
  * [349] C. H. Rodrigues and D. B. Ascher.  CSM-Potential: mapping protein interactions and biological ligands in 3D space using geometric deep learning.  Nucleic Acids Research, 50(W1):W204–W209, 2022.
  * [350] J. D. Rogers, B. A. Aguado, K. M. Watts, K. S. Anseth, and W. J. Richardson.  Network modeling predicts personalized gene expression and drug responses in valve myofibroblasts cultured with patient sera.  Proceedings of the National Academy of Sciences, 119(8):e2117323119, 2022.
  * [351] S. Roy, S. Lagree, Z. Hou, J. A. Thomson, R. Stewart, and A. P. Gasch.  Integrated module and gene-specific regulatory inference implicates upstream signaling networks.  PLOS Computational Biology, 9(10):e1003252, 2013.
  * [352] D. Ruan, A. Young, and G. Montana.  Differential analysis of biological networks.  BMC Bioinformatics, 16(1):327, 2015.
  * [353] J. Saez-Rodriguez, J. C. Costello, S. H. Friend, M. R. Kellen, L. Mangravite, P. Meyer, T. Norman, and G. Stolovitzky.  Crowdsourcing biomedical research: leveraging communities as innovation engines.  Nature Reviews Genetics, 17(8):470–486, 2016.
  * [354] N. Sahni, S. Yi, M. Taipale, J. I. F. Bass, J. Coulombe-Huntington, F. Yang, J. Peng, J. Weile, G. I. Karras, Y. Wang, et al.  Widespread macromolecular interaction perturbations in human genetic disorders.  Cell, 161(3):647–660, 2015.
  * [355] H. R. Saibil.  Cryo-EM in molecular and cellular biology.  Molecular Cell, 82(2):274–284, 2022.
  * [356] D. Salazar, C. Valencia, and N. Pržulj.  Multi-project and multi-profile joint non-negative matrix factorization for cancer omic datasets.  Bioinformatics, 37(24):4801–4809, 2021.
  * [357] C. Samieri, A. R. Sonawane, S. Lefèvre-Arbogast, C. Helmer, F. Grodstein, and K. Glass.  Using network science tools to identify novel diet patterns in prodromal dementia.  Neurology, 94(19):e2014–e2025, 2020.
  * [358] J. C. Sanghvi, S. Regot, S. Carrasco, J. R. Karr, M. V. Gutschow, B. Bolival Jr, and M. W. Covert.  Accelerated discovery via a whole-cell model.  Nature Methods, 10(12):1192–1195, 2013.
  * [359] A. Sarajlić, N. Malod-Dognin, Ö. Yaveroğlu, and N. Pržulj.  Graphlet-based characterization of directed networks.  Scientific Reports, 6, 2016.
  * [360] V. Saraph and T. Milenković.  MAGNA: maximizing accuracy in global network alignment.  Bioinformatics, 30(20):2931–2940, 2014.
  * [361] A. Sarraju, S. Ngo, and F. Rodriguez.  The leaky pipeline of diverse race and ethnicity representation in academic science and technology training in the United States, 2003–2019.  PLOS One, 18(4):e0284945, 2023.
  * [362] F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini.  The graph neural network model.  IEEE Transactions on Neural Networks, 20(1):61–80, 2008.
  * [363] E. E. Schadt.  Molecular networks as sensors and drivers of common human diseases.  Nature, 461(7261):218–223, 2009.
  * [364] G. Schiebinger, J. Shu, M. Tabaka, B. Cleary, V. Subramanian, A. Solomon, J. Gould, S. Liu, S. Lin, P. Berube, et al.  Optimal-transport analysis of single-cell gene expression identifies developmental trajectories in reprogramming.  Cell, 176(4):928–943, 2019.
  * [365] M. Schlichtkrull, T. N. Kipf, P. Bloem, R. Van Den Berg, I. Titov, and M. Welling.  Modeling relational data with graph convolutional networks.  In Proceedings of the International Semantic Web Conference, pages 593–607, 2018.
  * [366] F. Schmidt, N. Gasparoni, G. Gasparoni, K. Gianmoena, C. Cadenas, J. K. Polansky, P. Ebert, K. Nordström, M. Barann, A. Sinha, et al.  Combining transcription factor binding affinities with open-chromatin data for accurate gene expression prediction.  Nucleic Acids Research, 45(1):54–66, 2017.
  * [367] F. Schmidt, F. Kern, P. Ebert, N. Baumgarten, and M. H. Schulz.  TEPIC 2—an extended framework for transcription factor binding prediction and integrative epigenomic analysis.  Bioinformatics, 35(9):1608–1609, 2019.
  * [368] M. Schmidt, A. Niculescu-Mizil, and K. Murphy.  Learning graphical model structure using L1-regularization paths.  In Proceedings of the National Conference on Artificial Intelligence, page 1278–1283, 2007.
  * [369] A. J. Sedgewick, K. Buschur, I. Shi, J. D. Ramsey, V. K. Raghu, D. V. Manatakis, Y. Zhang, J. Bon, D. Chandra, C. Karoleski, et al.  Mixed graphical models for integrative causal analysis with application to chronic lung disease diagnosis and prognosis.  Bioinformatics, 35(7):1204–1212, 2018.
  * [370] E. Segal, D. Pe’er, A. Regev, D. Koller, N. Friedman, and T. Jaakkola.  Learning module networks.  Journal of Machine Learning Research, 6(4), 2005.
  * [371] Y. Sha, S. Wang, P. Zhou, and Q. Nie.  Inference and multiscale model of epithelial-to-mesenchymal transition via single-cell transcriptomic data.  Nucleic Acids Research, 48(17):9505–9520, 2020.
  * [372] Z. Sha, D. Schijven, S. E. Fisher, and C. Francks.  Genetic architecture of the white matter connectome of the human brain.  Science Advances, 9(7):eadd2870, 2023.
  * [373] R. Sharan and T. Ideker.  Modeling cellular machinery through biological network comparison.  Nature Biotechnology, 24(4):427–433, 2006.
  * [374] J. Shawe-Taylor and N. Cristianini.  Kernel Methods for Pattern Analysis.  Cambridge University Press, 2004.
  * [375] O. Shchur, M. Mumme, A. Bojchevski, and S. Günnemann.  Pitfalls of graph neural network evaluation.  In Proceedings of the Relational Representation Learning Workshop, 2018.
  * [376] N. Shervashidze, S. Vishwanathan, T. Petri, K. Mehlhorn, and K. Borgwardt.  Efficient graphlet kernels for large graph comparison.  In Proceedings of the Artificial Intelligence and Statistics, pages 488–495, 2009.
  * [377] S. Shit, J. C. Paetzold, A. Sekuboyina, I. Ezhov, A. Unger, A. Zhylka, J. P. Pluim, U. Bauer, and B. H. Menze.  clDice-a novel topology-preserving loss function for tubular structure segmentation.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16560–16569, 2021.
  * [378] A. F. Siahpirani and S. Roy.  A prior-based integrative framework for functional transcriptional regulatory network inference.  Nucleic Acids Research, 45(4):e21–e21, 2017.
  * [379] M. Simonovsky and N. Komodakis.  GraphVAE: towards generation of small graphs using variational autoencoders.  In Proceedings of the Artificial Neural Networks and Machine Learning, pages 412–422, 2018.
  * [380] R. Singh, J. Xu, and B. Berger.  Global alignment of multiple protein interaction networks with application to functional orthology detection.  Proceedings of the National Academy of Sciences, 105(35):12763–12768, 2008.
  * [381] S. Sledzieski, R. Singh, L. Cowen, and B. Berger.  D-script translates genome to phenome with sequence-based, structure-aware, genome-scale predictions of protein-protein interactions.  Cell Systems, 12(10):969–982, 2021.
  * [382] K. P. Smith and N. A. Christakis.  Social networks and health.  Annual Review of Sociology, 34:405–429, 2008.
  * [383] R. Solava, R. Michaels, and T. Milenković.  Graphlet-based edge clustering reveals pathogen-interacting proteins.  Bioinformatics, 18(28):i480–i486, 2012.
  * [384] A. R. Sonawane, D. L. DeMeo, J. Quackenbush, and K. Glass.  Constructing gene regulatory networks using epigenetic data.  NPJ Systems Biology and Applications, 7(1):45, 2021.
  * [385] A. R. Sonawane, J. Platig, M. Fagny, C.-Y. Chen, J. N. Paulson, C. M. Lopes-Ramos, D. L. DeMeo, J. Quackenbush, K. Glass, and M. L. Kuijjer.  Understanding tissue-specific gene regulation.  Cell Reports, 21(4):1077–1088, 2017.
  * [386] C. Stark, B.-J. Breitkreutz, T. Reguly, L. Boucher, A. Breitkreutz, and M. Tyers.  BioGRID: a general repository for interaction datasets.  Nucleic Acids Research, 34(suppl_1):D535–D539, 2006.
  * [387] H. Stärk, O. Ganea, L. Pattanaik, R. Barzilay, and T. Jaakkola.  Equibind: Geometric deep learning for drug binding structure prediction.  In Proceedings of the International Conference on Machine Learning, pages 20503–20521, 2022.
  * [388] C. Stegehuis, R. Van Der Hofstad, and J. S. Van Leeuwaarden.  Epidemic spreading on complex networks with community structures.  Scientific Reports, 6(1):1–7, 2016.
  * [389] K. R. Stevens, K. S. Masters, P. Imoukhuede, K. A. Haynes, L. A. Setton, E. Cosgriff-Hernandez, M. A. L. Bell, P. Rangamani, S. E. Sakiyama-Elbert, S. D. Finley, et al.  Fund Black scientists.  Cell, 184(3):561–565, 2021.
  * [390] J. M. Stokes, K. Yang, K. Swanson, W. Jin, A. Cubillos-Ruiz, N. M. Donghia, C. R. MacNair, S. French, L. A. Carfrae, Z. Bloom-Ackermann, et al.  A deep learning approach to antibiotic discovery.  Cell, 180(4):688–702, 2020.
  * [391] G. Stolovitzky, D. Monroe, and A. Califano.  Dialogue on reverse-engineering assessment and methods: the DREAM of high-throughput pathway inference.  Annals of the New York Academy of Sciences, 1115(1):1–22, 2007\.
  * [392] Y. Sun, J. Crawford, J. Tang, and T. Milenković.  Simultaneous optimization of both node and edge conservation in network alignment via WAVE.  In Proceedings of the Workshop on Algorithms In Bioinformatics, pages 16–39, 2015.
  * [393] Y. Sverchkov and M. Craven.  A review of active learning approaches to experimental design for uncovering biological networks.  PLOS Computational Biology, 13(6):e1005466, 2017.
  * [394] Z. E. Sychev, A. Hu, T. A. DiMaio, A. Gitter, N. D. Camp, W. S. Noble, A. Wolf-Yadlin, and M. Lagunoff.  Integrated systems biology analysis of KSHV latent infection reveals viral induction and reliance on peroxisome mediated lipid metabolism.  PLOS Pathogens, 13(3):e1006256, 2017.
  * [395] D. Szklarczyk, R. Kirsch, M. Koutrouli, K. Nastou, F. Mehryary, R. Hachilif, A. L. Gable, T. Fang, N. T. Doncheva, S. Pyysalo, et al.  The STRING database in 2023: protein–protein association networks and functional enrichment analyses for any sequenced genome of interest.  Nucleic Acids Research, 51(D1):D638–D646, 2023.
  * [396] B. Tahmasebi, D. Lim, and S. Jegelka.  Counting substructures with higher-order graph neural networks: Possibility and impossibility results.  arXiv:2012.03174, 2020.
  * [397] B. Tahmasebi, D. Lim, and S. Jegelka.  The power of recursion in graph neural networks for counting substructures.  In Proceedings of the International Conference on Artificial Intelligence and Statistics, pages 11023–11042, 2023.
  * [398] F. Tang, D. Xu, S. Wang, C. K. Wong, A. Martinez-Fundichely, C. J. Lee, S. Cohen, J. Park, C. E. Hill, K. Eng, et al.  Chromatin profiles classify castration-resistant prostate cancers suggesting therapeutic targets.  Science, 376(6596):eabe1505, 2022.
  * [399] J. Tang, M. Qu, M. Wang, M. Zhang, J. Yan, and Q. Mei.  LINE: Large-scale information network embedding.  In Proceedings of the International Conference on World Wide Web, pages 1067–1077, 2015.
  * [400] A. E. Teschendorff and A. P. Feinberg.  Statistical mechanics meets single-cell biology.  Nature Reviews Genetics, 22(7):459–476, 2021.
  * [401] C. V. Theodoris, L. Xiao, A. Chopra, M. D. Chaffin, Z. R. Al Sayed, M. C. Hill, H. Mantineo, E. M. Brydon, Z. Zeng, X. S. Liu, et al.  Transfer learning enables predictions in network biology.  Nature, pages 1–9, 2023.
  * [402] R. Torbey, N. D. Martin, J. R. Warner, and C. L. Fletcher.  Algebra I before high school as a gatekeeper to computer science participation.  In Proceedings of the ACM Technical Symposium on Computer Science Education, pages 839–844, 2020.
  * [403] R. J. Townshend, S. Eismann, A. M. Watkins, R. Rangan, M. Karelina, R. Das, and R. O. Dror.  Geometric deep learning of rna structure.  Science, 373(6558):1047–1051, 2021.
  * [404] P.-T. Tseng, Y.-S. Cheng, C.-F. Yen, Y.-W. Chen, B. Stubbs, P. Whiteley, A. F. Carvalho, D.-J. Li, T.-Y. Chen, W.-C. Yang, et al.  Peripheral iron levels in children with attention-deficit hyperactivity disorder: a systematic review and meta-analysis.  Scientific Reports, 8(1):1–11, 2018.
  * [405] J.-J. Tu, L. Ou-Yang, Y. Zhu, H. Yan, H. Qin, and X.-F. Zhang.  Differential network analysis by simultaneously considering changes in gene interactions and gene expression.  Bioinformatics, 37(23):4414–4423, 2021.
  * [406] K. Tu, P. Cui, X. Wang, F. Wang, and W. Zhu.  Structural deep embedding for hyper-networks.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
  * [407] N. Tuncbag, A. Braunstein, A. Pagnani, S.-S. C. Huang, J. Chayes, C. Borgs, R. Zecchina, and E. Fraenkel.  Simultaneous reconstruction of multiple signaling pathways via the prize-collecting steiner forest problem.  Journal of Computational Biology, 20(2):124–136, 2013.
  * [408] N. Tuncbag, S. J. Gosline, A. Kedaigle, A. R. Soltis, A. Gitter, and E. Fraenkel.  Network-based interpretation of diverse high-throughput datasets through the omics integrator software package.  PLOS Computational Biology, 12(4):e1004879, 2016.
  * [409] V. Vacic, L. M. Iakoucheva, S. Lonardi, and P. Radivojac.  Graphlet kernels for prediction of functional residues in protein structures.  Journal of Computational Biology, 17(1):55–72, 2010.
  * [410] M. G. Van Der Wijst, D. H. de Vries, H. Brugge, H.-J. Westra, and L. Franke.  An integrative approach for building personalized gene regulatory networks for precision medicine.  Genome Medicine, 10(1):1–15, 2018.
  * [411] O. Vanunu, O. Magger, E. Ruppin, T. Shlomi, and R. Sharan.  Associating genes and protein complexes with disease via network propagation.  PLOS Computational Biology, 6(1):e1000641, 2010.
  * [412] S. V. Vasaikar, P. Straub, J. Wang, and B. Zhang.  Linkedomics: analyzing multi-omics data within and across 32 cancer types.  Nucleic Acids Research, 46(D1):D956–D963, 2018.
  * [413] P. Veličković and C. Blundell.  Neural algorithmic reasoning.  Patterns, 2(7), 2021.
  * [414] P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y. Bengio.  Graph attention networks.  In Proceedings of the International Conference on Learning Representations, 2018.
  * [415] P. Veličković, W. Fedus, W. L. Hamilton, P. Liò, Y. Bengio, and R. D. Hjelm.  Deep Graph Infomax.  In Proceedings of the International Conference on Learning Representations, 2019.
  * [416] V. Vijayan, D. Critchlow, and T. Milenković.  Alignment of dynamic networks.  Bioinformatics, 33(14):i180–i189, 2017.
  * [417] V. Vijayan, E. Krebs, and T. Milenković.  Pairwise versus multiple global network alignment.  IEEE Access, 8:41961–41974, 2020.
  * [418] V. Vijayan and T. Milenković.  Multiple network alignment via multiMAGNA++.  IEEE/ACM Transactions on Computational Biology and Bioinformatics, 15(5):1669–1682, 2017.
  * [419] V. Vijayan and T. Milenković.  Aligning dynamic networks with DynaWAVE.  Bioinformatics, 34(10):1795–1798, 2018.
  * [420] V. Vijayan, V. Saraph, and T. Milenković.  MAGNA++: Maximizing Accuracy in Global Network Alignment via both node and edge conservation.  Bioinformatics, 31(14):2409–2411, 2015.
  * [421] S. V. N. Vishwanathan, N. N. Schraudolph, R. I. Kondor, and K. M. Borgwardt.  Graph kernels.  Journal of Machine Learning Research, 11:1201–1242, 2010.
  * [422] I. Voitalov, L. Zhang, C. Kilpatrick, J. B. Withers, A. Saleh, V. R. Akmaev, and S. D. Ghiassian.  The module triad: A novel network biology approach to utilize patients’ multi-omics data for target discovery in ulcerative colitis.  Scientific Reports, 12(1):21685, 2022.
  * [423] A. Waagmeester, G. Stupp, S. Burgstaller-Muehlbacher, B. M. Good, M. Griffith, O. L. Griffith, K. Hanspers, H. Hermjakob, T. S. Hudson, K. Hybiske, et al.  Wikidata as a knowledge graph for the life sciences.  eLife, 9:e52614, 2020.
  * [424] G. Wachman and R. Khardon.  Learning from interpretations: a rooted kernel for ordered hypergraphs.  In Proceedings of the International Conference on Machine Learning, pages 943–950, 2007.
  * [425] B. Wang, A. M. Mezlini, F. Demir, M. Fiume, Z. Tu, M. Brudno, B. Haibe-Kains, and A. Goldenberg.  Similarity network fusion for aggregating data types on a genomic scale.  Nature Methods, 11(3):333–337, 2014.
  * [426] B. Wang, A. Pourshafeie, M. Zitnik, J. Zhu, C. D. Bustamante, S. Batzoglou, and J. Leskovec.  Network enhancement as a general method to denoise weighted biological networks.  Nature Communications, 9(1):3108, 2018.
  * [427] H. Wang, T. Fu, Y. Du, W. Gao, K. Huang, Z. Liu, P. Chandak, S. Liu, P. Van Katwyk, A. Deac, et al.  Scientific discovery in the age of artificial intelligence.  Nature, 620(7972):47–60, 2023.
  * [428] H. Wang, D. Lian, W. Liu, D. Wen, C. Chen, and X. Wang.  Powerful graph of graphs neural network for structured entity analysis.  World Wide Web, 25(2):609–629, 2022.
  * [429] H. Wang, D. Lian, Y. Zhang, L. Qin, and X. Lin.  GoGNN: Graph of Graphs Neural Network for Predicting Structured Entity Interactions.  In Proceedings of the International Joint Conference on Artificial Intelligence, 2021.
  * [430] H. Wang, H. Zheng, and D. Z. Chen.  TANGO: A GO-term embedding based method for protein semantic similarity prediction.  IEEE/ACM Transactions on Computational Biology and Bioinformatics, 20(1):694–706, 2023.
  * [431] J. Wang, S. Lisanza, D. Juergens, D. Tischer, J. L. Watson, K. M. Castro, R. Ragotte, A. Saragovi, L. F. Milles, M. Baek, et al.  Scaffolding protein functional sites using deep learning.  Science, 377(6604):387–394, 2022.
  * [432] L. Wang, H. Liu, Y. Liu, J. Kurtin, and S. Ji.  Learning hierarchical protein representations via complete 3d graph networks.  In Proceedings of the International Conference on Learning Representations, 2022.
  * [433] Q. Wang, H. Jiang, Y. Jiang, S. Yi, Q. Nie, and G. Zhang.  Multiplex network infomax: Multiplex network embedding via information fusion.  Digital Communications and Networks, 2022.
  * [434] T. Wang, W. Shao, Z. Huang, H. Tang, J. Zhang, Z. Ding, and K. Huang.  MOGONET integrates multi-omics data using graph convolutional networks allowing patient classification and biomarker identification.  Nature Communications, 12(1):3445, 2021.
  * [435] X. Wang, D. Bo, C. Shi, S. Fan, Y. Ye, and S. Y. Philip.  A survey on heterogeneous graph embedding: methods, techniques, applications and sources.  IEEE Transactions on Big Data, 9(2):415–436, 2022.
  * [436] X. Wang, H. Ji, C. Shi, B. Wang, Y. Ye, P. Cui, and P. S. Yu.  Heterogeneous graph attention network.  In Proceedings of the World Wide Web Conference, pages 2022–2032, 2019.
  * [437] Y. Wang, H. Lee, J. M. Fear, I. Berger, B. Oliver, and T. M. Przytycka.  NetREX-CF integrates incomplete transcription factor data with gene expression to reconstruct gene regulatory networks.  Communications Biology, 5(1):1282, 2022.
  * [438] Y. Wang, Q. Peng, W. Wang, X. Guo, M. Shao, H. Liu, W. Liang, and L. Pan.  Network alignment enhanced via modeling heterogeneity of anchor nodes.  Knowledge-Based Systems, 250:109116, 2022.
  * [439] Y. Wang, Y. Zhao, N. Shah, and T. Derr.  Imbalanced graph classification via graph-of-graph neural networks.  In Proceedings of the ACM International Conference on Information and Knowledge Management, pages 2067–2076, 2022.
  * [440] D. Weighill, M. Ben Guebila, K. Glass, J. Platig, J. J. Yeh, and J. Quackenbush.  Gene targeting in disease networks.  Frontiers in Genetics, 12:649942, 2021.
  * [441] D. Weighill, M. B. Guebila, K. Glass, J. Quackenbush, and J. Platig.  Predicting genotype-specific gene regulatory networks.  Genome Research, 32(3):524–533, 2022.
  * [442] J. Wen, X. Zhang, E. Rush, V. A. Panickan, X. Li, T. Cai, D. Zhou, Y.-L. Ho, L. Costa, E. Begoli, et al.  Multimodal representation learning for predicting molecule–disease relations.  Bioinformatics, 39(2):btad085, 2023.
  * [443] J. L. Wetzel, K. Zhang, and M. Singh.  Learning probabilistic protein–DNA recognition codes from DNA-binding specificities using structural mappings.  Genome Research, 32(9):1776–1786, 2022.
  * [444] S. Windels, N. Malod-Dognin, and N. Pržulj.  Graphlet eigencentralities capture novel central roles of genes in pathways.  PLOS One, 17(1):e0261676, 2022.
  * [445] S. Windels, N. Malod-Dognin, and N. Pržulj.  Identifying cellular cancer mechanisms through pathway-driven data integration.  Bioinformatics, 38(18):4344–4351, 2022.
  * [446] S. Winkler, I. Winkler, M. Figaschewski, T. Tiede, A. Nordheim, and O. Kohlbacher.  De novo identification of maximally deregulated subnetworks based on multi-omics data with deregnet.  BMC Bioinformatics, 23(1):1–28, 2022.
  * [447] L. Wu, H. Lin, C. Tan, Z. Gao, and S. Z. Li.  Self-supervised learning on graphs: Contrastive, generative, or predictive.  IEEE Transactions on Knowledge and Data Engineering, 35:4216–4235, 2023.
  * [448] X. Wu, Q. Liu, and R. Jiang.  Align human interactome with phenome to identify causative genes and networks underlying disease families.  Bioinformatics, 25(1):98–104, 2009.
  * [449] A. Xenos, N. Malod-Dognin, S. Milinković, and N. Pržulj.  Linear functional organization of the omic embedding space.  Bioinformatics, 37(21):3839–3847, 2021.
  * [450] A. Xenos, N. Malod-Dognin, C. Zambrana, and N. Pržulj.  Integrated data analysis uncovers new COVID-19 related genes and potential drug re-purposing candidates.  International Journal of Molecular Sciences, 24(2):1431, 2023.
  * [451] Y. Xie, S. Katariya, X. Tang, E. Huang, N. Rao, K. Subbian, and S. Ji.  Task-agnostic graph explanations.  In Proceedings of the Advances in Neural Information Processing Systems, volume 35, pages 12027–12039, 2022.
  * [452] Y. Xie, Z. Xu, J. Zhang, Z. Wang, and S. Ji.  Self-supervised learning of graph neural networks: A unified review.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(2):2412–2429, 2022.
  * [453] Y. Xie, Y. Zhang, M. Gong, Z. Tang, and C. Han.  MGAT: Multi-view Graph Attention Networks.  Neural Networks, 132:180–189, 2020.
  * [454] H. Xiong, J. Yan, and L. Pan.  Contrastive multi-view multiplex network embedding with applications to robust network alignment.  In Proceedings of the ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 1913–1923, 2021.
  * [455] J. Xu, T. L. Wickramarathne, and N. V. Chawla.  Representing higher-order dependencies in networks.  Science Advances, 2(5):e1600028, 2016.
  * [456] M. Xu, Z. Zhang, J. Lu, Z. Zhu, Y. Zhang, M. Chang, R. Liu, and J. Tang.  Peer: a comprehensive and multi-task benchmark for protein sequence understanding.  In Proceedings of the Advances in Neural Information Processing Systems, volume 35, pages 35156–35173, 2022.
  * [457] Y. Yan, Q. Zhou, J. Li, T. Abdelzaher, and H. Tong.  Dissecting Cross-Layer Dependency Inference on Multi-Layered Inter-Dependent Networks.  In Proceedings of the International Conference on Information and Knowledge Management, pages 2341–2351, 2022.
  * [458] X. H. Yang, A. Goldstein, Y. Sun, Z. Wang, M. Wei, I. P. Moskowitz, and J. M. Cunningham.  Detecting critical transition signals from single-cell transcriptomes to infer lineage-determining transcription factors.  Nucleic Acids Research, 50(16):e91–e91, 2022.
  * [459] M. Yasunaga, A. Bosselut, H. Ren, X. Zhang, C. D. Manning, P. S. Liang, and J. Leskovec.  Deep bidirectional language-knowledge graph pretraining.  In Proceedings of the Advances in Neural Information Processing Systems, volume 35, pages 37309–37323, 2022.
  * [460] M. Yasunaga, J. Leskovec, and P. Liang.  Linkbert: Pretraining language models with document links.  Association for Computational Linguistics, 2022.
  * [461] Ö. Yaveroğlu, T. Milenković, and N. Pržulj.  Proper evaluation of alignment-free network comparison methods.  Bioinformatics, 31(16):2697–2704, 2015.
  * [462] E. Yeger-Lotem, L. Riva, L. J. Su, A. D. Gitler, A. G. Cashikar, O. D. King, P. K. Auluck, M. L. Geddie, J. S. Valastyan, D. R. Karger, et al.  Bridging high-throughput genetic and transcriptional data reveals cellular responses to alpha-synuclein toxicity.  Nature Genetics, 41(3):316–323, 2009.
  * [463] C. Ying, T. Cai, S. Luo, S. Zheng, G. Ke, D. He, Y. Shen, and T.-Y. Liu.  Do transformers really perform badly for graph representation?  In Proceedings of the Advances in Neural Information Processing Systems, volume 34, pages 28877–28888, 2021.
  * [464] Z. Ying, D. Bourgeois, J. You, M. Zitnik, and J. Leskovec.  Gnnexplainer: Generating explanations for graph neural networks.  In Proceedings of the Advances in Neural Information Processing Systems, volume 32, 2019.
  * [465] B.-J. Yoon, X. Qian, and E. R. Dougherty.  Quantifying the objective cost of uncertainty in complex dynamical systems.  IEEE Transactions on Signal Processing, 61(9):2256–2266, 2013.
  * [466] J. You, T. Du, and J. Leskovec.  ROLAND: graph learning framework for dynamic graphs.  In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 2358–2366, 2022.
  * [467] J. You, R. Ying, X. Ren, W. Hamilton, and J. Leskovec.  GraphRNN: Generating Realistic Graphs with Deep Auto-Regressive Models.  In Proceedings of the International Conference on Machine Learning, pages 5708–5717, 2018.
  * [468] Y. You, T. Chen, Y. Sui, T. Chen, Z. Wang, and Y. Shen.  Graph contrastive learning with augmentations.  In Proceedings of the Advances in Neural Information Processing Systems, volume 33, pages 5812–5823, 2020.
  * [469] X. Yu, Z. Liu, Y. Fang, and X. Zhang.  Learning to count isomorphisms with graph neural networks.  In Proceedings of the AAAI Conference on Artificial Intelligence, 2023.
  * [470] H. Yuan, H. Yu, J. Wang, K. Li, and S. Ji.  On explainability of graph neural networks via subgraph explorations.  In Proceedings of the International Conference on Machine Learning, pages 12241–12252, 2021.
  * [471] X. Yue, Z. Wang, J. Huang, S. Parthasarathy, S. Moosavinasab, Y. Huang, S. M. Lin, W. Zhang, P. Zhang, and H. Sun.  Graph embedding on biomedical networks: methods, applications and evaluations.  Bioinformatics, 36(4):1241–1251, 2020.
  * [472] H. Y. Yuen and J. Jansson.  Better link prediction for protein-protein interaction networks.  In Proceedings of the IEEE International Conference on Bioinformatics and Bioengineering, pages 53–60, 2020.
  * [473] C. Zambrana, A. Xenos, R. Bottcher, N. Malod-Dognin, and N. Pržulj.  Network neighbors of viral targets and differentially expressed genes in COVID-19 are drug target candidates.  Scientific Reports, 11(1):18985, 2021.
  * [474] B. Zhang and S. Horvath.  A general framework for weighted gene co-expression network analysis.  Statistical Applications in Genetics and Molecular Biology, 4(1), 2005.
  * [475] C. Zhang, D. Song, C. Huang, A. Swami, and N. V. Chawla.  Heterogeneous graph neural network.  In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, page 793–803, 2019.
  * [476] J. Zhang, L. Cammarata, C. Squires, T. P. Sapsis, and C. Uhler.  Active learning for optimal intervention design in causal models.  arXiv:2209.04744, 2022.
  * [477] L. Zhang, G. Yu, M. Guo, and J. Wang.  Predicting protein-protein interactions using high-quality non-interacting pairs.  BMC Bioinformatics, 19(19):105–124, 2018.
  * [478] R. Zhang and J. Ma.  Matcha: probing multi-way chromatin interaction with hypergraph representation learning.  Cell Systems, 10(5):397–407, 2020.
  * [479] R. Zhang, J. Ma, and J. Ma.  DANGO: Predicting higher-order genetic interactions.  bioRxiv 2020.11.26.400739, 2020.
  * [480] R. Zhang, T. Zhou, and J. Ma.  Multiscale and integrative single-cell Hi-C analysis with Higashi.  Nature Biotechnology, 40(2):254–261, 2022.
  * [481] R. Zhang, Y. Zou, and J. Ma.  Hyper-SAGNN: a self-attention based graph neural network for hypergraphs.  In Proceedings of the International Conference on Learning Representations, 2020.
  * [482] S. Zhang, H. Tong, Y. Xia, L. Xiong, and J. Xu.  NetTrans: Neural Cross-Network Transformation.  In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 986–996, 2020.
  * [483] S. Zhang, H. Tong, J. Xu, Y. Hu, and R. Maciejewski.  ORIGIN: Non-Rigid Network Alignment.  In Proceedings of the IEEE International Conference on Big Data, pages 998–1007, 2019.
  * [484] X. Zhang, M. Zeman, T. Tsiligkaridis, and M. Zitnik.  Graph-guided network for irregularly sampled multivariate time series.  In Proceedings of the International Conference on Learning Representations, 2022.
  * [485] Y. Zhang and H. Huang.  Brain connectome based complex brain disorder prediction via novel graph-blind convolutional network.  In Proceedings of the International Conference on Information Processing In Medical Imaging, 2019.
  * [486] Y. Zhang, L. Zhan, S. Wu, P. Thompson, and H. Huang.  Disentangled and proportional representation learning for multi-view brain connectomes.  In Proceedings of the Medical Image Computing and Computer Assisted Intervention, pages 508–518, 2021.
  * [487] Z. Zhang, Q. Liu, Q. Hu, and C.-K. Lee.  Hierarchical graph transformer with adaptive node sampling.  In Proceedings of the Advances in Neural Information Processing Systems, volume 35, pages 21171–21183, 2022.
  * [488] Z. Zhang, M. Xu, A. R. Jamasb, V. Chenthamarakshan, A. Lozano, P. Das, and J. Tang.  Protein representation learning by geometric structure pretraining.  In Proceedings of the International Conference on Learning Representations, 2023.
  * [489] C. Zhao, L. Zhan, P. M. Thompson, and H. Huang.  Revealing Continuous Brain Dynamical Organization with Multimodal Graph Transformer.  In Proceedings of the Medical Image Computing and Computer Assisted Intervention, pages 346–355, 2022.
  * [490] J. Zhao, Q. Wen, S. Sun, Y. Ye, and C. Zhang.  Multi-view Self-supervised Heterogeneous Graph Embedding.  In Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery In Databases, pages 319–334, 2021.
  * [491] L. Zhao, Y. Song, C. Zhang, Y. Liu, P. Wang, T. Lin, M. Deng, and H. Li.  T-GCN: A Temporal Graph Convolutional Network for Traffic Prediction.  IEEE Transactions on Intelligent Transportation Systems, 21(9):3848–3858, 2019.
  * [492] V. W. Zheng, M. Sha, Y. Li, H. Yang, Y. Fang, Z. Zhang, K.-L. Tan, and K. C.-C. Chang.  Heterogeneous embedding propagation for large-scale e-commerce user alignment.  In Proceedings of the IEEE International Conference on Data Mining, pages 1434–1439, 2018.
  * [493] D. Zhou, J. Huang, and B. Schölkopf.  Learning with hypergraphs: Clustering, classification, and embedding.  In Proceedings of the Advances in Neural Information Processing Systems, volume 19, 2006.
  * [494] G. Zhou, J. Ewald, and J. Xia.  OmicsAnalyst: a comprehensive web-based platform for visual analytics of multi-omics data.  Nucleic Acids Research, 49(W1):W476–W482, 2021.
  * [495] J. Zhou and O. G. Troyanskaya.  Predicting effects of noncoding variants with deep learning–based sequence model.  Nature Methods, 12(10):931–934, 2015.
  * [496] N. Zhou, Y. Jiang, T. R. Bergquist, A. J. Lee, B. Z. Kacsoh, A. W. Crocker, K. A. Lewis, G. Georghiou, H. N. Nguyen, M. N. Hamid, et al.  The CAFA challenge reports improved protein function prediction and new functional annotations for hundreds of genes through experimental screens.  Genome Biology, 20(1):1–23, 2019.
  * [497] P. Zhou, S. Wang, T. Li, and Q. Nie.  Dissecting transition cells from single-cell transcriptome data through multiscale stochastic dynamics.  Nature Communications, 12(1):5609, 2021.
  * [498] L. Zhu, Y. Ding, C.-Y. Chen, L. Wang, Z. Huo, S. Kim, C. Sotiriou, S. Oesterreich, and G. Tseng.  MetaDCN: meta-analysis framework for differential co-expression network detection with an application in breast cancer.  Bioinformatics, 33(8):1121–1129, 2016.
  * [499] M. Zitnik, M. Agrawal, and J. Leskovec.  Modeling polypharmacy side effects with graph convolutional networks.  Bioinformatics, 34(13):i457–i466, 2018.
  * [500] M. Zitnik and J. Leskovec.  Predicting multicellular function through multi-layer tissue networks.  Bioinformatics, 33(14):i190–i198, 2017.
  * [501] M. Zitnik, F. Nguyen, B. Wang, J. Leskovec, A. Goldenberg, and M. M. Hoffman.  Machine learning for integrating data in biology and medicine: Principles, practice, and opportunities.  Information Fusion, 50:71–91, 2019.
  * [502] E. Zotenko, K. S. Guimarães, R. Jothi, and T. M. Przytycka.  Decomposition of overlapping protein complexes: A graph theoretical method for analyzing static and dynamic protein associations.  Algorithms for Molecular Biology, 1(1):1–11, 2006.


#  Finding essential parts of the brain in rs-fMRI can improve diagnosing ADHD
by Deep Learning

Byunggun Kim1, Jaeseon Park1, Taehun Kim 1, and Younghun Kwon1,2
1Department of Applied Artificial Intelligence
1Department of Applied Physics
{byunggunkim, wotjs307, taehunkim<EMAIL_ADDRESS>
Hanyang University, Ansan, Kyunggi-Do, 425-791, Republic of Korea

###### Abstract

Attention-Deficit/Hyperactivity Disorder(ADHD) is considered a very common
psychiatric disorder, but it is difficult to establish an accurate diagnostic
method for ADHD. Recently, with the development of computing resources and
machine learning methods, studies have been conducted to classify ADHD using
resting-state functional magnetic resonance(rs-fMRI) imaging data. However,
most of them utilized all areas of the brain for training the models. In this
study, as a different way from this approach, we conducted a study to classify
ADHD by selecting areas that are essential for using a deep learning model.
For the experiment, rs-fMRI data provided by ADHD-200 global competition was
used. To obtain an integrated result from the multiple sites, each region of
the brain was evaluated with ‘Leave-one-site-out’ cross-validation. As a
result, when we only used 15 important region of interest(ROIs) for training,
an accuracy of 70.6% was obtained, significantly exceeding the existing
results of 68.6% from all ROIs. In addition, to explore the new structure
based on SCCNN-RNN, we performed the same experiment with three modified
models: (1) Separate Channel CNN - RNN with Attention (ASCRNN), (2) Separate
Channel dilate CNN - RNN with Attention (ASDRNN), (3) Separate Channel CNN -
slicing RNN with Attention (ASSRNN). As a result, the ASSRNN model provided a
high accuracy of 70.46% when training with only 13 important region of
interest (ROI). These results show that finding and using the crucial parts of
the brain in diagnosing ADHD by Deep Learning can get better results than
using all areas.

_K_ eywords ADHD $\cdot$ Deep learning $\cdot$ rs-fMRI $\cdot$ AAL116 $\cdot$
ROI

##  1 Introduction

Attention-Deficit/Hyperactivity Disorder(ADHD) is known as a psychiatric
disorder that frequently appears in children. (Barkley (1997); Faraone et al.
(2003); Polanczyk et al. (2007)) However, an accurate diagnostic method for
ADHD is not known yet. (Barkley (1997)) To overcome this difficulty, there
have been efforts to find a biomarker between healty control(HC) and ADHD. For
example, functional connectivity (FC) is extracted from fMRI data based on
machine learning methods. (Greicius (2008); Zhu et al. (2008); Dey et al.
(2014); Guo et al. (2014); Lin et al. (2014); Sen et al. (2018); Tang et al.
(2021)) They tried to classify ADHD using FC.

In recent years, the models with deep neural networks (Riaz et al. (2017); Zou
et al. (2017); Riaz et al. (2018); Mao et al. (2019); Zhang et al. (2020)) are
also used to get the feature. These methods obtained high accuracies compared
to the traditional methods. Most of the studies mentioned so far used the
whole regions of the brain.

However, we need to focus on the results that the difference feature between
ADHD and HC might be in a specific or some of the region of interest (ROI).
(Eloyan et al. (2012); Lin et al. (2014)) Therefore, it is natural that we
have a question ‘Is it good to cover all brain areas in the diagnosis of ADHD
through a neural network model?’

In this study, we want to find a specific answer to the question. The chosen
dataset, evaluation method, and models are as follows. At first, because the
measure parameter was different with each site, we combined all sites of data
as the training dataset to avoid biased results. Next, we evaluated the
trained model with Leave-One-Site-Out Cross-validation(LOSO) (Zhang et al.
(2020)) and compared it, using the ROIs that we used for the training. The
model architectures for this experiment are ‘Separated channel CNN -
RNN(SCCNN-RNN)’ proposed by (Zhang et al. (2020)) and the new architecture
based on it. Each of the models has the same number of trainable parameters
regardless of the number of ROIs. It means that we can control the variance of
the result from the model capacity. So we can get a more accurate result
related to the importance of the ROIs.

The experimental procedures were also designed to examine the existence of
some critical areas in the identification of ADHD. First of all, with the
SCCNN-RNN model, we examine the importance of the individual ROI by using only
one ROI feature for the model training. Then, the ROIs were ranked according
to the results. In the second experiment, By selecting some of the ranked ROI
that have a considerable influence on the diagnosis of ADHD, we investigated
how it affects the classification accuracy. We found in the second experiment
that using only high-ranked ROIs is much better than using whole ROIs for the
classification. We conducted experiments on three other new architecture
models based on ‘SCCNN-RNN’ to supplement this result.

In conclusion, even with a small amount of ROIs, the evaluation of the result
show 70.6% accuracy. In the next 2, 3 sections, we will explain the data
selection and descriptions of the models for the experiment.

##  2 Method

###  2.1 To diagnosis the ADHD, is it necessary that we take the whole regions
of the brain?

As mentioned in the Introduction, several studies have been conducted to find
the biomarker between HC and ADHD patients. In recent years, the models with
deep neural networks have been used to understand the biomarker. In order to
do it, they used the whole regions of the brain.

However, there were the results that the difference feature between ADHD and
HC might be in a specific or some of the ROIs. (Eloyan et al. (2012); Lin et
al. (2014)) Along this line, we conduct a study to answer the question:Is it
good to cover all brain areas in the diagnosis of ADHD through a neural
network model?

Fortunately, in our study, we may answer the question. As the result of our
investigation, we can show that the SCCNN-RNN model using a small amount of
ROIs provides 70.6% accuracy, which exceeds the existing results of 68.6%
using all ROIs.

###  2.2 Data selection for experiment (Why we use the AAL 116 template)

The rs-fMRI, as the 4-dimensional structure, has both spatial and temporal
information of the brain. Therefore, the raw data obtained from a single
subject contains a large amount of low-dimensional (x, y, z, t) features. It
means that we need lots of data samples to learn a meaningful hidden feature
of ADHD with a neural network. Fortunately, ‘Neuro Bureau’ (Bellec et al.
(2017)) provides many data samples($\sim$1k) preprocessed with various methods
(Athena, NIAK, Burner) used in the ADHD-200 Global competition. Nevertheless,
there are some difficulties in using them directly for the experiment. For the
first reason, as we said before, The number of data samples is still small
enough to use low-dimensional data for training directly.

Furthermore, the second reason is ADHD-200 Global competition’s dataset
consisted of several sites. In other words, Each site of fMRI data was
collected with different parameters of MRI devices. If one trains the model
with the fMRI data samples from the specific site, it occurs biased result to
the initial setting of the measuring device.

In this study, we try to use as many data samples as possible and obtain
results that are not dependent on the measuring device. To overcome these
situations, we constructed a training dataset from multiple sites together.
And we extracted feature vectors that are less sensitive to the unique
biological information (phenotype) and measure parameters. In previous
studies, handcraft feature extraction (Brown et al. (2012); Dai et al. (2012))
was frequently used. However, these can depend on the context in which the
fMRI data are measured. So it is not a proper method for our situations. For
that reason, we take the Automated Anatomical Labelling(AAL 116) (Tzourio-
Mazoyer et al. (2002)) for the feature extraction method.

If we use the AAL 116 template, we expect it is possible to effectively
extract features, according to contribution on the 116 interest regions (ROIs)
in low-dimensional fMRI data. In summary, we choose the dataset from the five
sites (NYU, Peking, OHSU, KKI, NI) preprocessed by the NIAK pipeline(Lavoie-
Courchesne et al. (2012)) from The Neuro Bureau ADHD-200 preprocessed
Repository. And then, we extract high-level features using the AAL template.
Through these processes, we showed that a specific ROI is useful for
diagnosing ADHD in general fMRI data.

###  2.3 Separate Channel CNN - RNN Architecture

Figure 1: Architecture of SCCNN - RNN. After passing the SCCNN part that
extracts the bold signal of each ROI through convolution layers and the RNN
part that learns the relationship between multiple ROIs, ADHD is determined
from the output of the last step of RNN through two fully connected layers.

As we mentioned before, we want to find out the meaningful ROIs for diagnosing
ADHD and HCs. To this end, we selected ‘the Separate Channel CNN - RNN(SCCNN-
RNN)’ proposed by (Zhang et al. (2020)) as the base model architecture for the
experiment, which satisfies the following two reasons.

The first reason is that ‘SCCNN-RNN’ can extract spatial and temporal
information from fMRI data. Specifically, ‘SCCNN-RNN’ can be divided into two
parts with different purposes. The SCCNN part can extract the feature of the
BOLD signal in each ROI with 1-D CNN. The RNN part can learn the spatial
relation of the ROIs.

And the second reason is that ‘SCCNN-RNN’ architecture can always keep the
same number of learnable weights regardless of the change in the input data
dimension. For example, we train several models with different ROIs and
compare them with evaluation results. If the input’s shape changes the number
of trainable parameters, the model’s learning capacity can also be changed. It
means that we compared the influence of the ROI with inconsistent results.
Therefore, to avoid this situation, we controlled the trainable parameters.

Now, we explain a detailed setting of the SCCNN-RNN for our experiments. It is
shown in Fig 1. In the SC-CNN part, we stack four layers with 1D CNN. The
convolution layer’s channel number is ‘32’, ‘64’, ‘96’, ‘96’, respectively.
And the stride size is ‘1’, and the filter size is ‘3’ as common parameters on
the convolution layers. In the RNN part, we used the Bidirectional LSTM
cell(Graves and Schmidhuber (2005)) because it’s mechanisms were proved the
performance in many sequence data domains such as speech recognition (Graves
et al. (2013)) and language model (Sundermeyer et al. (2012)) and so on. The
hidden state numbers of each step are set to ‘128’. The output of the last
T-step of the RNN are connected with one fully connected layer with ‘128’
neurons. At last, the classification layer is put as the last layer with a
softmax activation to output the two-dimension vectors as the probability of
ADHD and HCs.

As we said, the ‘SCCNN-RNN’ structure is a proper and straightforward model to
learn the spatial and temporal features from the rs-fMRI data for our
experiment. Therefore, in the next section, we expand this structure to
explore the more efficient result.

###  2.4 Other modified models based on the ‘SCCNN - RNN’ architecture

Figure 2: Differences in the proposed model architectures. (a) Separate
Channel CNN - RNN with Attention (ASCRNN) (b) Separate Channel dilate CNN -
RNN with Attention (ASDRNN) (c) Separate Channel CNN - slicing RNN with
Attention(ASSRNN). All models contain the attention mechanism
($c_{i}=\sum_{j=1}^{N_{R}}\alpha_{ij}h_{j}$$\colon$ attention weight,
$h_{j}\colon$ output in $j$ step on the RNN, $R_{N}\colon$ Number of ROIs used
in training) in common. In (a), only attention mechanism from SCCNN - RNN
structure is applied. In (b), the Dilate CNN and skip connection on the SCCNN
part based on the ASCRNN is applied. In (c), the slicing BiLSTM based on the
ASCRNN is applied.

To make the modified models from the ‘SCCNN - RNN’ structure, we applied some
ideas from the speech emotion recognition(SER) model (Meng et al. (2019); Xie
et al. (2019); Peng et al. (2020)). In other words, we chose ideas of the
three different speech emotion recognition models. The reason is that speech
emotion recognition using a neural network also requires a structure for
obtaining emotion information in a long sequence with a small amount of data.
It is similar to our problems. For this reason, this study was conducted to
investigate the neural network structure to obtain features from fMRI data and
various SCCNN-RNN-based model structures. And it also shows that selecting and
using a region with high impact is more helpful in disease identification than
using all regions.

As you can see in Fig 2, these three modified models from ‘SCCNN-RNN’ have
slight differences. However, the attention mechanism is applied in common. The
attention mechanism usually improves the performance of the models if the
training data is sequential. In ‘SCCNN-RNN’ structure, it only uses the last
hidden state outputs from BiLSTM as the next layer’s inputs. Therefore, it is
structurally difficult to learn the importance of ROIs, To overcome it, we put
into the attention mechanism to focus on the important ROI with all step’s
hidden states.

In the recent years, the attention mechanism has been used as several
ways(Bahdanau et al. (2014); Luong et al. (2015); Vaswani et al. (2017)). In
our study, we chose attentive attention method proposed in (Zhang et al.
(2020)). Specifically, to learn the correlation between the two reference
ROIs, after linear transformation of comparison vector with learnable weight
matrix , map into non-linear function is considered. It should be noted that
in this study, despite using the same method as the previous, there is
difference. In (Zhang et al. (2020)), the attention was stacked after ‘SCCNN’
part. but we stack in order of SCCNN part, RNN part, and attention mechanism.
The reason for applying the attention method in this way is to consider not
only the relationship between two areas, but also the relationship between
several areas. For example, $h_{j}$ of jth BiLSTM’s hidden state consists of
the forward hidden state $\vec{h}_{j}$ ,contained with information from $1$st
ROI to $j$th ROI, and backward hidden state
${\mathchoice{\vbox{\offinterlineskip\halign{#\cr\reflectbox{$\displaystyle\vec{}\mkern
4.0mu$}\cr\kern-4.30554pt\cr$\displaystyle
h$\cr}}}{\vbox{\offinterlineskip\halign{#\cr\reflectbox{$\textstyle\vec{}\mkern
4.0mu$}\cr\kern-4.30554pt\cr$\textstyle
h$\cr}}}{\vbox{\offinterlineskip\halign{#\cr\reflectbox{$\scriptstyle\vec{}\mkern
4.0mu$}\cr\kern-3.01389pt\cr$\scriptstyle
h$\cr}}}{\vbox{\offinterlineskip\halign{#\cr\reflectbox{$\scriptscriptstyle\vec{}\mkern
4.0mu$}\cr\kern-2.15277pt\cr$\scriptscriptstyle h$\cr}}}}_{j}$, contained with
information from $R_{N}$ th ROI to $j$th ROI in reverse order. And we learn
the correlation $\alpha_{ij}$ between $h_{i}$ and $h_{j}$ with the attention
layer. Therefore all ROIs that we choose can be considered in one step. The
equations 9, 10 and 11 describe the process. We call this model as ‘Separate
Channel CNN - RNN with Attention’ (ASCRNN). This attention is also applied to
the two models proposed later in the same way.

$\displaystyle
h_{j}=\left[\vec{h}_{j},{\mathchoice{\vbox{\offinterlineskip\halign{#\cr\reflectbox{$\displaystyle\vec{}\mkern
4.0mu$}\cr\kern-4.30554pt\cr$\displaystyle
h$\cr}}}{\vbox{\offinterlineskip\halign{#\cr\reflectbox{$\textstyle\vec{}\mkern
4.0mu$}\cr\kern-4.30554pt\cr$\textstyle
h$\cr}}}{\vbox{\offinterlineskip\halign{#\cr\reflectbox{$\scriptstyle\vec{}\mkern
4.0mu$}\cr\kern-3.01389pt\cr$\scriptstyle
h$\cr}}}{\vbox{\offinterlineskip\halign{#\cr\reflectbox{$\scriptscriptstyle\vec{}\mkern
4.0mu$}\cr\kern-2.15277pt\cr$\scriptscriptstyle h$\cr}}}}_{j}\right],\quad
1\leq j\leq N_{R},$ (9)
$\displaystyle\alpha_{ij}=attentive\left(h_{i},h_{j}\right)$ (10)
$\displaystyle c_{i}=\sum_{j=1}^{N_{R}}\alpha_{ij}h_{j}$ (11)

The next modified model called ‘Separate Channel dilate CNN - RNN with
Attention(ASDRNN)’ is focused on the BOLD signal extraction. It differs from
the SCCNN part. From previous studies (Mao et al. (2019)), it can be seen that
learning with only the relationship within a specific frame of the BOLD signal
is helpful in the diagnosis of ADHD. With this fact, we replace the 1-D CNN
with dilation 1-D CNN (Yu and Koltun (2015)). Also, to treat the gradient
vanishing problem that emerged from the deep neural networks, we applied the
skip connection (He et al. (2016)) after the last batch normalization layer.
The detailed structure of Separate Channel dilated CNN can be seen in Fig 2
and 3. In dilation 1-D CNN, dilate rate is set as ‘2’.

Figure 3: The comparison of separate channel CNN and Separate channel dilate
CNN with skip connection architectures.

Fig 2 is the last modified model named Separate Channel CNN - slicing RNN with
Attention(ASSRNN). This model takes a segmented area as input instead of
giving the entire ROI area as input to the RNN to get the relationship between
partitioned regions. To do that, instead of BiLSTM in the ASCRNN model, we
applied the slicing BiLSTM(Peng et al. (2020)). The slicing BiLSTM’s inputs
are the series of subsequences splatted from the original sequence with
constant window size$(w)$ and stride size$(l)$. This can be expected to focus
on the relationship within the small number of ROIs and avoid the gradient
vanishing problem that may appear as the time step of the input becomes longer
in the RNN structure. This process can be found in below 12, 15 and 16
equations

$\displaystyle s=\left\\{0,1,...,\lceil{N_{R}-l\over w}\rceil\right\\}$ (12)
$\displaystyle
x^{(s)}=\left\\{\begin{array}[]{rcl}\left[x_{1+sw},...,x_{l+sw}\right]&\mbox{if}&0\leq
s\leq\lceil{N_{R}-l\over w}\rceil-1\\\
\left[x_{N_{R}-l+1},...,x_{N_{R}}\right]&\mbox{else}&\end{array}\right.$ (15)
$\displaystyle h_{s}=BiLSTM\left(x^{(s)}\right)$ (16)

##  3 Experiment Result and Discussion

###  3.1 Settings in Training and Evaluation

To analysis the result to choice of the ROIs, all parameters required for
learning were chosen as identical ones in our experiments. The parameter
setting is based on the results obtained through the experiment. Specifically,
we used Adam optimizer (Kingma and Ba (2014)). The learning rate was chosen to
be 1e-4. Xavier initialization(Glorot and Bengio (2010)) was used as the
Initialization method for all trainable weights. Also, to prevent the
overfitting problem, we used l2 regularization with factor 0.0005. The leaky
ReLU (Maas et al. (2013)) with 0.1 slope coefficient was chosen as the
activation function. Next, to avoid bias due to an imbalanced dataset
corresponding to ADHD and HC, the same number of each class (ADHD, HC) was
sampled for all mini-batch. The mini-batch size was set to be 32. And then, we
set binary cross entropy for the loss function. The evaluation method of the
accuracy of the models was chosen by ‘Leave-one-site-out cross
validation(LOSO)’. By proceeding with the evaluation with the data set of
another site not used for training the model, it is possible to avoid the
dependent characteristics (parameters of the measurement device). And the
experiment results for the ROIs that play an essential role in diagnosis can
be general.

###  3.2 Detail of the dataset composition

We used rs-fMRI data samples provided by the ADHD-200 competition (Bellec et
al. (2017)) for the experiment. They consist of the data from eight
institutions (NYU : New York University child study center, Peking : Peking
University, OHSU : Oregon Health Sciences University, KKI : Kennedy Krieger
Institute, NI : NeuroIMAGE, BHBU : Bradley/ Brown University, Pitt :
University of Pittsburgh, WUSTL : Washington University at Saint Louis) Among
them, BHBU, Pitt, and WUSTL are difficult to use for supervised learning, so
five other sites (NYU, Peking, OHSU, KKI, NI) are used. Moreover, to
compensate for the lack of data, all data (training data, test data) of each
site were combined and considered one site dataset. So, the overall structure
of the data we used is shown in Table 1. And rs-fMRI sample is preprocessed,
as we mentioned before. See in section 2.2.

| NYU | Peking | OHSU | KKI | NI | Total
---|---|---|---|---|---|---
ADHD | 147 | 101 | 43 | 25 | 36 | 352
HC | 110 | 143 | 70 | 69 | 37 | 429
Total | 257 | 244 | 133 | 94 | 73 | 781
Table 1: The data composition of each site that we used in our experiments

###  3.3 Analysis selecting an important region of the brain according to AAL
116 ROI using a neural network model of SCCNN-RNN

To determine the importance of individual ROIs for ADHD discrimination, we
evaluated the accuracy through independent neural network model trained with
specific ROI. For the experiment, we used ‘SCCNN-RNN’ models with LSTM. Fig 4
shows the diagnosis accuracy trained with specific ROI. The distribution of
the accuracy obtained in the experiment is in a large range(minimum accuracy :
61.93% with index=77, minimum accuracy : 68.47% with index=28). This indicates
a meaningful result when learning with only one ROI because it has fewer fMRI
features than all ROIs. It can be expected that there may be a region that
plays an important role in discriminating ADHD. Based on these results, in
next experiment, we will show how learning by selecting a few important ROIs
differs from using all ROIs.

Figure 4: The accuracy according to individual ROI when we trained the SCCNN-
RNN with only one individual ROI.

###  3.4 Accuracy analysis for ADHD discrimination according to ROI ranking in
SCCNN-RNN

The experiment is performed to determine how the association between important
ROIs can be understood in the ADHD discrimination. The specific progress of
the experiment is as follows. According to the ROI ranking from the result of
section 3.3 , the regions at the top are selected sequentially, and the number
of ROI features is gradually increased. And the experiment is carried out with
the independent neural network models. Also, the same model(SCCNN-RNN) is used
in order to reduce the difference that will appear due to the change in the
number of parameters of the neural network model according to the increase of
the input’s data size.

Fig 5 shows the result of the model’s accuracy trained with ranking ROIs. From
the (blue line) on Fig 5, two facts can be observed. First, it can be
confirmed that high accuracy is obtained when learning by selecting ROIs in a
certain order is performed, where the learning is done by selecting ROIs in
the order obtained. It can be seen that some ROIs play an important role for
the diagnosis. As shown in Fig 5, the ranking that achieves high accuracy is
the case of using the ROI in the top 20. In particular, an accuracy of 70.6%
was obtained when ROI up to 15th Rank was used for training. Another fact is
that as the ROI used for learning increases, it is hard to tell areas that
provide higher results than before. This means that using lots of ROIs can be
a hindrance in diagnosing ADHD. When selecting the ROI in reverse order, we
can see the results with the (orange line) on Fig 5. Relatively low accuracy
is obtained when using up to 20th rank of reverse ranks, but it can be seen
that the accuracy tends to be increased when gradually more ROIs is used. In
other words, It means that learning with ROIs of low importance determined by
accuracy does not help much in the diagnosis, but if the number of ROIs used
for subsequent learning increases, it could be helpful. That is, ROIs with a
high rank complement ROIs with a low rank.

Figure 5: The accuracy as we increase the number of learning ROIs when we
trained the SCCNN-RNN. (blue line) The accuracy in the case of learning by
selecting in order of ROIs with the highest Rank. (orange line) The accuracy
in the case of learning by selecting in order of ROI with the lowest Rank

###  3.5 Accuracy analysis for ADHD discrimination according to ROI ranking in
modified models

To investigate the impact on importance of ROI, we perform experiments with
other modified models (ASCRNN, ASDRNN, ASSRNN) that we mention in section 2.4.
The experiment is performed in the same way as described in section 3.4.
However, the number of ROI used for learning was up to the 20th rank, as a
result of Fig5. With Fig 6 and Table 2, we can describe the result as
follows:First, when learned with selected ROIs according to its rank, higher
results can be expected than using all ROIs. Further, all models provide
accuracy around 68% to 70% despite using fewer areas than all 116 areas. In
particular(Table 2), in the case of SCCNN-RNN and ASSRNN, the accuracies of
70.6(15)% and 70.46(13)% can be obtained, which are exceeding 70% accuracy.
This is a result that exceeds the previous result (68.6%) by 2%. Second (Fig
6), with all the modified models based on SCCNN-RNN, the number of ROIs
providing good accuracy lies between 10 and 20. This is slightly different
from the case where only a small number of areas (ex. 1-5 rank ROI) are used.
To summarize, it seems that a specific part of the brain must be used to
improve the diagnosis. The best accuracy was obtained for the simplest
model(SCCNN - RNN). There is an improvement of about 7%, in the case of using
ROIs with higher rankings.

Figure 6: The accuracy when we trained the other modified models(ASCRNN, ASDRNN, ASSRNN) as we increase the number of ROIs based the rank order Proposed paper | Model name | LOSO accuracy with all ROls | LOSO accuracy with specific ROIs
---|---|---|---
(Number of ROIs used in the experiment)
Riaz et al. (2017) | FCNet | 60.4 | -
Riaz et al. (2018) | DeepfMRI | 67.9 | -
Zhang et al. (2020) | SCCNN - Attention | 68.6 | -
Zhang et al. (2020) | SCCNN - RNN | 63.6 | 70.6(15)
- | ASCRNN | 65.2 | 69.97(20)
- | ASDRNN | 68.4 | 68.05(17)
- | ASSRNN | 66.86 | 70.46(13)
Table 2: Comparison of accuracy in ADHD-200 classification models using AAL
template

##  4 Conclusion

Recently, the diagnosis accuracy of ADHD and HC has been gradually improved
with rs-fMRI data through deep learning methods. However, there is difficulty
improving the performance of deep learning. Even though there are many
reasons, one of them is the limitation of medical data related to ADHD. Also,
it is hard to find the medical biomarker that distinguishes between ADHD and
normal. Nevertheless, rs-fMRI data are frequently used because they contain
various information about the brain. In this study, we showed that finding ROI
providing important areas in the brain can help diagnose ADHD. Moreover, it
was found that only using 15 ROIs in the importance order can provide a more
outstanding performance improvement (70.6%) in the diagnosis than using all
areas. It implies that establishing the good criteria for the importance of
the ROI can give diagnostic accuracy. Thus, we will consider a critical area
detection method using deep neural networks in the future study. Further, in
deep learning models, their decision process seems to be a black box. However,
this can be a major drawback in diagnosing diseases. Therefore, we will look
at how deep learning models can understand ADHD by studying the decision
process of various deep learning models.

## Acknowledgments

This work is supported by the Basic Science Research Program through the
National Research Foundation of Korea funded by the Ministry of Education,
Science and Technology (NRF2018R1D1A1B07049420) and Institute of Information &
communications Technology Planning & Evaluation(IITP) grant funded by the
Korea government(MSIT) (No.2020-0-01343, Artificial Intelligence Convergence
Research Center (Hanyang University ERICA).

## References

  * Bahdanau et al. [2014] D. Bahdanau, K. Cho, and Y. Bengio.  Neural machine translation by jointly learning to align and translate.  _arXiv preprint arXiv:1409.0473_ , 2014.
  * Barkley [1997] R. A. Barkley.  Behavioral inhibition, sustained attention, and executive functions: constructing a unifying theory of adhd.  _Psychological bulletin_ , 121(1):65, 1997.
  * Bellec et al. [2017] P. Bellec, C. Chu, F. Chouinard-Decorte, Y. Benhajali, D. S. Margulies, and R. C. Craddock.  The neuro bureau adhd-200 preprocessed repository.  _Neuroimage_ , 144:275–286, 2017.  ISSN 1053-8119.
  * Brown et al. [2012] M. R. Brown, G. S. Sidhu, R. Greiner, N. Asgarian, M. Bastani, P. H. Silverstone, A. J. Greenshaw, and S. M. Dursun.  Adhd-200 global competition: diagnosing adhd using personal characteristic data can outperform resting state fmri measurements.  _Frontiers in systems neuroscience_ , 6:69, 2012.  ISSN 1662-5137.
  * Dai et al. [2012] D. Dai, J. Wang, J. Hua, and H. He.  Classification of adhd children through multimodal magnetic resonance imaging.  _Frontiers in systems neuroscience_ , 6:63, 2012.  ISSN 1662-5137.
  * Dey et al. [2014] S. Dey, A. R. Rao, and M. Shah.  Attributed graph distance measure for automatic detection of attention deficit hyperactive disordered subjects.  _Frontiers in neural circuits_ , 8:64, 2014.  ISSN 1662-5110.
  * Eloyan et al. [2012] A. Eloyan, J. Muschelli, M. B. Nebel, H. Liu, F. Han, T. Zhao, A. D. Barber, S. Joel, J. J. Pekar, and S. H. Mostofsky.  Automated diagnoses of attention deficit hyperactive disorder using magnetic resonance imaging.  _Frontiers in systems neuroscience_ , 6:61, 2012.  ISSN 1662-5137.
  * Faraone et al. [2003] S. V. Faraone, J. Sergeant, C. Gillberg, and J. Biederman.  The worldwide prevalence of adhd: is it an american condition?  _World psychiatry_ , 2(2):104, 2003.
  * Glorot and Bengio [2010] X. Glorot and Y. Bengio.  Understanding the difficulty of training deep feedforward neural networks.  In _Proceedings of the thirteenth international conference on artificial intelligence and statistics_ , pages 249–256. JMLR Workshop and Conference Proceedings, 2010.
  * Graves and Schmidhuber [2005] A. Graves and J. Schmidhuber.  Framewise phoneme classification with bidirectional lstm and other neural network architectures.  _Neural networks_ , 18(5-6):602–610, 2005.
  * Graves et al. [2013] A. Graves, A.-r. Mohamed, and G. Hinton.  Speech recognition with deep recurrent neural networks.  In _2013 IEEE international conference on acoustics, speech and signal processing_ , pages 6645–6649. Ieee, 2013.
  * Greicius [2008] M. Greicius.  Resting-state functional connectivity in neuropsychiatric disorders.  _Current opinion in neurology_ , 21(4):424–430, 2008.  ISSN 1350-7540.
  * Guo et al. [2014] X. Guo, X. An, D. Kuang, Y. Zhao, and L. He.  Adhd-200 classification based on social network method.  In _International Conference on Intelligent Computing_ , pages 233–240. Springer, 2014.
  * He et al. [2016] K. He, X. Zhang, S. Ren, and J. Sun.  Deep residual learning for image recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 770–778, 2016.
  * Kingma and Ba [2014] D. P. Kingma and J. Ba.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_ , 2014.
  * Lavoie-Courchesne et al. [2012] S. Lavoie-Courchesne, P. Rioux, F. Chouinard-Decorte, T. Sherif, M.-E. Rousseau, S. Das, R. Adalat, J. Doyon, C. Craddock, D. Margulies, et al.  Integration of a neuroimaging processing pipeline into a pan-canadian computing grid.  In _Journal of Physics: Conference Series_ , volume 341, page 012032\. IOP Publishing, 2012.
  * Lin et al. [2014] P. Lin, J. Sun, G. Yu, Y. Wu, Y. Yang, M. Liang, and X. Liu.  Global and local brain network reorganization in attention-deficit/hyperactivity disorder.  _Brain imaging and behavior_ , 8(4):558–569, 2014\.  ISSN 1931-7565.
  * Luong et al. [2015] M.-T. Luong, H. Pham, and C. D. Manning.  Effective approaches to attention-based neural machine translation.  _arXiv preprint arXiv:1508.04025_ , 2015.
  * Maas et al. [2013] A. L. Maas, A. Y. Hannun, and A. Y. Ng.  Rectifier nonlinearities improve neural network acoustic models.  In _ICML Workshop on Deep Learning for Audio, Speech, and Language Processing_ , 2013.
  * Mao et al. [2019] Z. Mao, Y. Su, G. Xu, X. Wang, Y. Huang, W. Yue, L. Sun, and N. Xiong.  Spatio-temporal deep learning method for adhd fmri classification.  _Information Sciences_ , 499:1–11, 2019.  ISSN 0020-0255.
  * Meng et al. [2019] H. Meng, T. Yan, F. Yuan, and H. Wei.  Speech emotion recognition from 3d log-mel spectrograms with deep learning network.  _IEEE access_ , 7:125868–125881, 2019.
  * Peng et al. [2020] Z. Peng, X. Li, Z. Zhu, M. Unoki, J. Dang, and M. Akagi.  Speech emotion recognition using 3d convolutions and attention-based sliding recurrent networks with auditory front-ends.  _IEEE Access_ , 8:16560–16572, 2020.
  * Polanczyk et al. [2007] G. Polanczyk, M. S. De Lima, B. L. Horta, J. Biederman, and L. A. Rohde.  The worldwide prevalence of adhd: a systematic review and metaregression analysis.  _American journal of psychiatry_ , 164(6):942–948, 2007.
  * Riaz et al. [2017] A. Riaz, M. Asad, S. M. R. Al-Arif, E. Alonso, D. Dima, P. Corr, and G. Slabaugh.  Fcnet: a convolutional neural network for calculating functional connectivity from functional mri.  In _International Workshop on Connectomics in Neuroimaging_ , pages 70–78. Springer, 2017.
  * Riaz et al. [2018] A. Riaz, M. Asad, S. M. R. Al Arif, E. Alonso, D. Dima, P. Corr, and G. Slabaugh.  Deep fmri: An end-to-end deep network for classification of fmri data.  In _2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018)_ , pages 1419–1422. IEEE, 2018.
  * Sen et al. [2018] B. Sen, N. C. Borle, R. Greiner, and M. R. Brown.  A general prediction model for the detection of adhd and autism using structural and functional mri.  _PloS one_ , 13(4):e0194856, 2018.  ISSN 1932-6203.
  * Sundermeyer et al. [2012] M. Sundermeyer, R. Schlüter, and H. Ney.  Lstm neural networks for language modeling.  In _Thirteenth annual conference of the international speech communication association_ , 2012.
  * Tang et al. [2021] Y. Tang, C. Wang, Y. Chen, N. Sun, A. Jiang, and Z. Wang.  Identifying adhd individuals from resting-state functional connectivity using subspace clustering and binary hypothesis testing.  _Journal of attention disorders_ , 25(5):736–748, 2021.  ISSN 1087-0547.
  * Tzourio-Mazoyer et al. [2002] N. Tzourio-Mazoyer, B. Landeau, D. Papathanassiou, F. Crivello, O. Etard, N. Delcroix, B. Mazoyer, and M. Joliot.  Automated anatomical labeling of activations in spm using a macroscopic anatomical parcellation of the mni mri single-subject brain.  _Neuroimage_ , 15(1):273–289, 2002.  ISSN 1053-8119.
  * Vaswani et al. [2017] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin.  Attention is all you need.  In _Advances in neural information processing systems_ , pages 5998–6008, 2017.
  * Xie et al. [2019] Y. Xie, R. Liang, Z. Liang, C. Huang, C. Zou, and B. Schuller.  Speech emotion classification using attention-based lstm.  _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ , 27(11):1675–1685, 2019.
  * Yu and Koltun [2015] F. Yu and V. Koltun.  Multi-scale context aggregation by dilated convolutions.  _arXiv preprint arXiv:1511.07122_ , 2015.
  * Zhang et al. [2020] T. Zhang, C. Li, P. Li, Y. Peng, X. Kang, C. Jiang, F. Li, X. Zhu, D. Yao, and B. Biswal.  Separated channel attention convolutional neural network (sc-cnn-attention) to identify adhd in multi-site rs-fmri dataset.  _Entropy_ , 22(8):893, 2020.
  * Zhu et al. [2008] C.-Z. Zhu, Y.-F. Zang, Q.-J. Cao, C.-G. Yan, Y. He, T.-Z. Jiang, M.-Q. Sui, and Y.-F. Wang.  Fisher discriminative analysis of resting-state brain function for attention-deficit/hyperactivity disorder.  _Neuroimage_ , 40(1):110–120, 2008.  ISSN 1053-8119.
  * Zou et al. [2017] L. Zou, J. Zheng, C. Miao, M. J. Mckeown, and Z. J. Wang.  3d cnn based automatic diagnosis of attention deficit hyperactivity disorder using functional and structural mri.  _IEEE Access_ , 5:23626–23636, 2017.  ISSN 2169-3536.