The advent of advanced LLMs such as OntoGPT [92] and BrainGPT [93] heralds a transformative shift in how scientific information can be processed, understood, and utilized. These models have demonstrated a remarkable ability to distill and predict complex patterns from vast datasets, suggesting a potential role in enhancing user interaction with neuroscientific databases. AmadeusGPT showcases an innovative application in this direction, using LLMs to convert natural language descriptions of animal behaviors into executable analysis code, thereby facilitating interactive behavioral research, and increasing its accessibility [94]. Tools like these exemplify the potential of LLMs to help bridge gaps between complex biological knowledge and expertise in computational analysis, enhancing scientists' ability to access and analyze neuroscience data in their research. Discussions at the symposium also touched upon how LLMs could help researchers engage more effectively with existing scientific knowledge, for example through enhanced literature searches and dynamic knowledge base augmentation via scientific journal content distillation.

For example, BrainGPT has been specifically trained to anticipate the outcomes of neuroscience experiments by ingesting extensive portions of the neuroscientific literature. Its proficiency, as demonstrated by the BrainBench benchmark, surpasses that of human experts in distinguishing between true experiment results and modified abstracts [93]. Capabilities like these suggest a future where LLMs could be used to reliably navigate and summarize existing scientific knowledge. It is important to note, however, that LLMs are subject to hallucinations. For this reason, BrainGPT is not currently enabled to perform this type of task [95], and this potential use remains a matter of conjecture for the moment. Likewise, another potential use for LLMs is in inductive reasoning [96]. In the case of open data, an LLM trained in this way could be harnessed for hypothesis generation and experiment planning.

OntoGPT's approach to enhancing knowledge bases through natural language processing highlights another very promissing application for LLMs [92]. By helping construct and refine knowledge bases, LLMs can facilitate more accurate and dynamic querying of complex data structures. When it comes to managing extensive open neuroscientific data repositories, integrating LLMs could dramatically improve the precision and scope of data retrieval processes, enabling researchers to generate interconnected insights from disparate datasets.

Practical Applications of LLMs in open neuroscience.

* Enhanced Literature Search: LLMs could be utilized to refine literature search mechanisms, enabling researchers to rapidly locate relevant studies and datasets. By processing queries using LLMs trained on the latest research and reviews, we could offer more contextually aware search results, reducing the time spent on literature reviews and increasing the relevance of the information retrieved.

* Knowledge Base Augmentation: Using LLMs like OntoGPT to assist in the ongoing development and expansion of neuroscientific knowledge bases could help ensure that data curation and query management follow standard nomenclature and are consistent with open neuroscience practices. For example, LLMs can assist in linking new data entries with existing ontologies and suggest updates to improve the comprehensiveness and utility of a database.

Future Directions. As LLMs evolve, we should be able to leverage them not only to manage and query existing data, but also to anticipate and prepare for future research developments. Continuous updates to LLM training sets and algorithms and fine-tuning, using methods like LoRA [97] and Retrieval Augmented Generation (RAG) [98] techniques, will be essential to maintaining their effectiveness and relevance to the neuroscientific context. Overall, incorporating LLMs into open data in neuroscience promises to enhance utility as a dynamic, forward-looking tool that not only serves current user needs, but also adapts to and anticipates future scientific challenges.

§.§ Addressing Community Needs

For open science to advance in neuroscience, it is critical to understand and address the needs of the community. This includes continuing to develop resources and tools that make open science practices easier to adopt for users of all backgrounds, while also ensuring that funding and incentives are in place to enable users to invest the time and effort that is still required. Table <ref> summarizes key community needs identified during the symposium.

Community needs and actions for advancing open science.

Category     Actions     Key Concepts

* Provide community guidance on sharing methodologies, datatypes (raw, processed).

* Standardize required and recommended metadata types.

* Select and unify ontologies for metadata standardization.

* Define essential provenance information for shared data.

Provenance, shared methodologies, standardized metadata, unified ontologies
Tool Development

* Enhance tools for data compression, conversion, sharing, and analysis.

* Develop cloud-based data access and analysis solutions.

* Establish benchmarking platforms for model and theory evaluation.

* Develop platforms for tool comparison.

* Support large-scale data pooling and annotation.

* Simplify metadata entry through user-friendly interfaces.

* Improve automated metadata capture tools.

* Enable on-the-fly data annotation of anomalies during experiments.

* Improve ability to detect and filter anomalous data.

Cloud solutions, data compression, data pooling, metadata entry, tool benchmarking

* Improve models for understanding complex data.

* Create benchmarks and metrics for model evaluation.

* Develop data quality assurance metrics.

* Innovate automated data labeling for enhanced data reuse.

Advanced model zoo, automated data labeling, data quality metrics, model benchmarks

* Maintain centralized databases for datasets, methodologies, and tools.

* Facilitate community feedback mechanisms for shared resources.

Centralized databases
Knowledge Graphs

* Create knowledge graphs for describing entities and their relationships, and for linking disparate databases.

Knowledge graphs

* Continue to develop online resources and training for data processing and analysis tools.

Online resources, training workshops
Funding & Incentives

* Support community engagement and multi-laboratory collaborations.

* Fund technical personnel for open-source software maintenance.

* Encourage and facilitate adoption of new technologies and open science practices.

* Invest in scaling data storage solutions.

Community engagement, multi-laboratory collaboration, open-source support

§.§ Recommendations for the Practicing Neuroscientist

As we continue to innovate and advocate for community-wide advancements in open science, practicing neuroscientists have several opportunities to engage with the existing open science practices. This section makes recommendations that span the entire lifespan of a project and can greatly improve the reproducibility and efficiency of one's research. Depending on their projects and access to resources, individual research groups may find certain recommendations more relevant, helpful or feasible to implement than others. We recommend identifying these priorities and approaching the adoption of open science practices incrementally.

Data Management and Sharing Plan. It is important to prepare a data management and sharing plan early in the research process. Funders like the NIH and many scientific journals now require open sharing of data collected under their grants and for publication, respectively. Deciding early on which repository to use, understanding its data format requirements, and planning the workflow from data acquisition to publication can greatly facilitate the process. Adopting standards such as NWB early in data acquisition can also streamline the process and save time by ensuring consistency is maintained through data processing, analysis, and publication. Steps to follow are (see Fig. <ref>):

* Identify the repository where your data will be deposited.

* Understand the data format requirements of the chosen repository. Determine whether your data needs to conform to specific standards such as NWB.

* Plan how you will manage, process, analyze, and visualize your data:

* What software tools will you use?

* What data formats do these tools accept, and what formats do they output?

* Plan for and implement the use of common data standards as early as possible in the data lifecycle: from acquisition through processing, analysis, and up to publication and sharing. Early adoption streamlines workflows, avoids the need to refactor custom code down the road, and enhances the reusability of data, saving time and resources.

* Standardize the process of converting your data into the required standard format. If done at the acquisition step, make sure that the proper metadata is included. If certain post-processing steps are required routines in your laboratory, make sure to track the details of how they were run and include the relevant information during data conversion to standardized formats.

Data management plan flowchart.

Documentation and Metadata.

* Detailed Documentation: Providing thorough and structured metadata is essential for enabling effective use of your data and reuse by researchers who are not familiar with your project. When storing and sharing data, aim to (see Fig. <ref>):

* Document the source script and any other processes used to generate the dataset, even if they are not mandatory fields in your chosen data standard.

* Include comprehensive details about the devices, software versions, and analysis algorithms used during the experiments.

* Record any stimuli presented during the experiments, and include a detailed table specifying which stimuli were presented when.

* Clearly describe how neural, behavioral and stimulus data streams were aligned temporally.

* Record key subject descriptors, like genotype, referencing external databases for standard definitions where applicable.

* Annotate any anomalies or unusual occurrences during data collection that might affect subsequent analyses.

* Utilize tools like NWB GUIDE for user-friendly and automated capture of important metadata to minimize effort and enhance standardization.

* As AI/ML methods become increasingly integrated into neurophysiology data analysis, ensuring your datasets are AI-ready with rich metadata will greatly enhance their reuse and the reliability of subsequent findings.

Documentation/Metadata flowchart.

Utilizing Existing Tools. Several steps can help optimize your use of existing tools (see Fig. <ref>):

* Choosing Open-Source Software: Due to the complexity of neurophysiological data analysis, it is advisable to use established open-source software packages, when applicable. These are less prone to errors and are continually vetted by the community. Examples include:

* Spike Sorting and Processing: Consider tools like SpikeInterface and KiloSort.

* Calcium Imaging Data Processing: Consider tools like suite2p and CaImAn.

* Pose Estimation: Consider tools such as DeepLabCut and SLEAP.

* Contributing to Tool Development: If existing tools lack certain features or could be improved, contribute your enhancements back to the project. This type of collaboration:

* Allows the community to verify the robustness of the new feature.

* Enhances tool functionality and utility for the entire community.

* Accelerates scientific discovery and increases the robustness of research outcomes.

* Builds a culture of reuse and improvement, aligning with open science principles.

Developing New Tools. If you develop new tools from scratch, it can be very valuable to share these with the broader community. To maximize the robustness, usability, and findability of these tools, it is particularly helpful to:

* Share the code on a platform like GitHub that enables robust version-control, as well as user feedback and contributions, ideally under a license that is highly permissive for code reuse and adaptation.

* Document the code by including at minimum a README explaining the tool's intended use, the programming language it is designed in, its dependencies, usage examples, and ideally an interactive tutorial users can run in the cloud.
More detailed recommendations can be found in previously published articles[99].

* Make a plan for long-term maintenance and promotion of the tool. This may require investing financial resources and hiring of dedicated personnel, but is generally critical for the longevity and usability of an open-source tool.

Tooling flowchart.

§ CONCLUDING REMARKS

[colback=blue!5!white, colframe=blue!75!black, title=Key Takeaways for Advancing Open Science in Neurophysiology]

* Education and training of neuroscientists at every level is crucial for ensuring open data practices are effectively adopted and utilized.

* Funding investment in the development, dissemination, and maintenance of open-source tools and infrastructure is necessary to support long-term sustainability and reliability of research outputs. Sponsors that value open data must be prepared to fund the health of the ecosystem, which includes supporting practicing neuroscientists, tool disseminators, and continuous maintenance/development to keep tools up to date.

* Improving research methodologies by establishing benchmarks and standardized methodologies for data analysis and model evaluation will improve the reproducibility and comparability of research findings. Appropriately harnessing large language models (LLMs) and AI tools to enhance data analysis, literature search, and hypothesis generation could also significantly improve research quality and pertinence.

* Development of robust and user-friendly tools for data management, analysis, and sharing is essential to support the adoption of open science practices across laboratories with varying resources. Enhancing metadata quality and standardization is critical for the reusability and reproducibility of shared datasets, and comprehensive searchable metadata will greatly improving data utility.
Such practices have the potential to help address important ethical considerations, like animal use in neurophysiology research, as optimal reuse of existing datasets can help keep animal use to a minimum.

* Alternative career paths should be established within academia to support individuals skilled in data management and analysis, whose work is less focused on specific research hypotheses. Such positions would provide much needed job opportunities and security, while helping bridge the gap between traditional academic hierarchies and the increasingly complex technical landscape of neuroscience research.

* Progressive changes in culture and social infrastructure are necessary and must occur alongside changes in incentives and credit assignment. Tool development and dataset contributions should receive greater appreciation and formal recognition (e.g. by hiring and promotion committees). Participating in this transformation is essential for fostering an environment that values and rewards open science.

Key takeaways.

This first ODIN symposium highlighted a growing momentum in neurophysiology research to incorporate the values and practices of open science.
The key takeaways from the symposium are summarized in Fig. <ref>.
Innovations revolutionizing the quality and quantity of data we collect have been complemented by the development of robust data standardization and sharing platforms, along with a variety of computational resources for data processing, analysis and visualization. Large-scale data collection efforts are tackling the challenges of reproducibility and reliability in the field, with centralized approaches providing access to high quality data collection pipelines and decentralized ones encouraging collaborative protocol and analysis designs. However, significant challenges remain, particularly for laboratories with limited resources, where incorporating open science practices can be daunting and time-consuming. Furthermore, when laboratories do invest in adopting these practices, the time and effort required are often not sufficiently recognized by the traditional incentive structures of academic research.

Funding sponsors, publishers, and institutions wield the power to drive collaborative progress and sustain momentum. Their active support and recognition of researchers’ time and effort invested in open science initiatives are crucial for enabling this pivotal change in modern neuroscience. By acknowledging the value of open science practices, they elevate the entire field. Through incentivizing open data practices, funding robust infrastructure, and promoting tool dissemination, they create an environment where open science becomes a central pillar of neurophysiology research.

Meanwhile, we encourage researchers to actively engage in open science practices and leverage existing resources. By participating in the communities that build and use advanced tools, individuals can discover solutions to challenges they face, and tap into valuable community support. When solutions are lacking, researchers can provide feedback reflecting their specific research needs, increasing the likelihood that future iterations will address those needs. Thus, while transformative impact arises from collective action with the much needed support of funders and institutions, it is essential to recognize the power of individual voices in shaping this action.

Overall, we anticipate gradual, collaborative progress in the field, rather than an overnight transformation, engaging researchers, sponsors and institutions. We advocate for acknowledging and celebrating symbiotic developments, which together will propel us toward more open, transparent, and impactful science. In this context, the ODIN symposium (intended as a bi-annual event) can serve as a vital platform for sustaining momentum, sharing novel developments, and addressing the evolving needs of the community.

§ AUTHOR INFORMATION

C.J.G., C.B., R.L., and N.D. wrote the paper, with contributions from the other authors.

The organizers of ODIN would like to thank all the speakers for their engaging presentations. We also thank all the participants for their thoughtful engagement in the ODIN symposium and ensuing discussions. We wish to acknowledge support for the ODIN symposium from DANDI and MIT McGovern Institute for Brain Research. This work is supported by NIH Grant R24MH117295. C.J.G. is supported by the Wellcome Trust (Wellcome Trust Investigator Award 200790/Z/16/Z awarded to Claudia Clopath). R.L. is supported by NIH grant U24NS120057 and The Kavli Foundation. M.S. is supported by NIH grant U19NS132720, a C.V. Starr Fellowship, and a Burroughs Wellcome Fund's Career Award at the Scientific Interface.

[1]
S. Gražulis, A. Daškevič, A. Merkys, D. Chateigner, L. Lutterotti,
M. Quirós, N. R. Serebryanaya, P. Moeck, R. T. Downs, and A. Le Bail,
“Crystallography Open Database (COD): an open-access collection of
crystal structures and platform for world-wide collaboration,” Nucleic
Acids Research, vol. 40, pp. D420–427, Jan. 2012.

[2]
M. R. Munafò, B. A. Nosek, D. V. M. Bishop, K. S. Button, C. D. Chambers,
N. P. du Sert, U. Simonsohn, E.-J. Wagenmakers, J. J. Ware, and J. P. A.
Ioannidis, “A manifesto for reproducible science,” Nature Human
Behaviour, vol. 1, p. 0021, Jan. 2017.

[3]
I. J. Bruno and C. R. Groom, “A crystallographic perspective on sharing data
and knowledge,” Journal of Computer-Aided Molecular Design, vol. 28,
no. 10, pp. 1015–1022, 2014.

[4]
T. M. Porter and M. Hajibabaei, “Over 2.5 million COI sequences in
GenBank and growing,” PLoS One, vol. 13, no. 9, p. e0200177, 2018.

[5]
D. A. Benson, M. Cavanaugh, K. Clark, I. Karsch-Mizrachi, D. J. Lipman,
J. Ostell, and E. W. Sayers, “GenBank,” Nucleic Acids Research,
vol. 41, pp. D36–42, Jan. 2013.

[6]
J. Jumper, R. Evans, A. Pritzel, T. Green, M. Figurnov, O. Ronneberger,
K. Tunyasuvunakool, R. Bates, A. Žídek, A. Potapenko, A. Bridgland,
C. Meyer, S. A. A. Kohl, A. J. Ballard, A. Cowie, B. Romera-Paredes,
S. Nikolov, R. Jain, J. Adler, T. Back, S. Petersen, D. Reiman, E. Clancy,
M. Zielinski, M. Steinegger, M. Pacholska, T. Berghammer, S. Bodenstein,
D. Silver, O. Vinyals, A. W. Senior, K. Kavukcuoglu, P. Kohli, and
D. Hassabis, “Highly accurate protein structure prediction with
AlphaFold,” Nature, vol. 596, pp. 583–589, Aug. 2021.

[7]
J. L. Saunders, “Decentralized infrastructure for (neuro)science,” arXiv, 2022.

[8]
M. Schottdorf, G. Yu, and E. Y. Walker, “Data science and its future in large
neuroscience collaborations,” bioRxiv, p. 2024.03.20.585936, Mar.

[9]

[10]
Y. Tchoe, A. M. Bourhis, D. R. Cleary, B. Stedelin, J. Lee, K. J. Tonsfeldt,
E. C. Brown, D. A. Siler, A. C. Paulk, J. C. Yang, H. Oh, Y. G. Ro, K. Lee,
S. M. Russman, M. Ganji, I. Galton, S. Ben-Haim, A. M. Raslan, and S. A.
Dayeh, “Human brain mapping with multithousand-channel PtNRGrids
resolves spatiotemporal dynamics,” Science Translational Medicine,
vol. 14, no. 628, p. eabj1441, 2022.

[11]
J. J. Jun, N. A. Steinmetz, J. H. Siegle, D. J. Denman, M. Bauza, B. Barbarits,
A. K. Lee, C. A. Anastassiou, A. Andrei, Ç. Aydın, M. Barbic, T. J.
Blanche, V. Bonin, J. Couto, B. Dutta, S. L. Gratiy, D. A. Gutnisky,
M. Häusser, B. Karsh, P. Ledochowitsch, C. M. Lopez, C. Mitelut, S. Musa,
M. Okun, M. Pachitariu, J. Putzeys, P. D. Rich, C. Rossant, W.-l. Sun,
K. Svoboda, M. Carandini, K. D. Harris, C. Koch, J. O'Keefe, and T. D.
Harris, “Fully integrated silicon probes for high-density recording of
neural activity,” Nature, vol. 551, pp. 232–236, Nov 2017.

[12]
J. Demas, J. Manley, F. Tejera, K. Barber, H. Kim, F. M. Traub, B. Chen, and
A. Vaziri, “High-speed, cortex-wide volumetric recording of neuroactivity at
cellular resolution using light beads microscopy,” Nature Methods,
vol. 18, no. 9, pp. 1103–1111, 2021.

[13]
Y. Adam, J. J. Kim, S. Lou, Y. Zhao, M. E. Xie, D. Brinks, H. Wu, M. A.
Mostajo-Radji, S. Kheifets, V. Parot, S. Chettih, K. J. Williams, B. Gmeiner,
S. L. Farhi, L. Madisen, E. K. Buchanan, I. Kinsella, D. Zhou, L. Paninski,
C. D. Harvey, H. Zeng, P. Arlotta, R. E. Campbell, and A. E. Cohen, “Voltage
imaging and optogenetics reveal behaviour-dependent changes in hippocampal
dynamics,” Nature, vol. 569, pp. 413–417, May 2019.

[14]
For more information on CERN data management, see <https://castor.web.cern.ch/castor/>.

[15]
O. Rübel, A. Tritt, R. Ly, B. K. Dichter, S. Ghosh, L. Niu, P. Baker,
I. Soltesz, L. Ng, K. Svoboda, L. Frank, and K. E. Bouchard, “The
Neurodata Without Borders ecosystem for neurophysiological data
science,” eLife, vol. 11, p. e78362, oct 2022.

[16]
For more information on NWB, see <https://www.nwb.org/>.

[17]
For more information on DANDI, see <https://dandiarchive.org/>.

[18]
D. Yatsenko, T. Nguyen, S. Shen, K. Gunalan, C. A. Turner, R. Guzman,
M. Sasaki, D. Sitonic, J. Reimer, E. Y. Walker, and A. S. Tolias,
“DataJoint elements: Data workflows for neurophysiology,” bioRxiv, 2021.

[19]
E. C. Johnson, T. T. Nguyen, B. K. Dichter, F. Zappulla, M. Kosma, K. Gunalan,
Y. O. Halchenko, S. Q. Neufeld, M. Schirner, P. Ritter, M. E. Martone,
B. Wester, F. Pestilli, and D. Yatsenko, “A maturity model for operations in
neuroscience research,” arXiv, 2023.

[20]
For more information on DataJoint, see <https://github.com/datajoint>.

[21]
The MICrONS Consortium, J. A. Bae, M. Baptiste, C. A. Bishop, A. L. Bodor,
D. Brittain, J. Buchanan, D. J. Bumbarger, M. A. Castro, B. Celii, E. Cobos,
F. Collman, N. M. da Costa, S. Dorkenwald, L. Elabbady, P. G. Fahey,
T. Fliss, E. Froudarakis, J. Gager, C. Gamlin, W. Gray-Roncal, A. Halageri,
J. Hebditch, Z. Jia, E. Joyce, J. Joyce, C. Jordan, D. Kapner, N. Kemnitz,
S. Kinn, L. M. Kitchell, S. Koolman, K. Kuehner, K. Lee, K. Li, R. Lu,
T. Macrina, G. Mahalingam, J. Matelsky, S. McReynolds, E. Miranda,
E. Mitchell, S. S. Mondal, M. Moore, S. Mu, T. Muhammad, B. Nehoran,
O. Ogedengbe, C. Papadopoulos, S. Papadopoulos, S. Patel, X. Pitkow,
S. Popovych, A. Ramos, R. Clay Reid, J. Reimer, P. K. Rivlin, V. Rose, C. M.
Schneider-Mizell, H. S. Seung, B. Silverman, W. Silversmith, A. Sterling,
F. H. Sinz, C. L. Smith, S. Suckow, M. Takeno, Z. H. Tan, A. S. Tolias,
R. Torres, N. L. Turner, E. Y. Walker, T. Wang, A. Wanner, B. A. Wester,
G. Williams, S. Williams, K. Willie, R. Willie, W. Wong, J. Wu, C. Xu,
R. Yang, D. Yatsenko, F. Ye, W. Yin, R. Young, S.-c. Yu, D. Xenes, and
C. Zhang, “Functional connectomics spanning multiple areas of mouse visual
cortex,” bioRxiv, 2023.

[22]
For more information on Figurl, see <https://github.com/flatironinstitute/figurl>.

[23]
For more information on Neurosift, see <https://github.com/flatironinstitute/neurosift>.

[24]
For more information on Dendro, see <https://dendro.vercel.app/>.

[25]
J. Hata, K. Nakae, H. Tsukada, A. Woodward, Y. Haga, M. Iida, A. Uematsu,
F. Seki, N. Ichinohe, R. Gong, T. Kaneko, D. Yoshimaru, A. Watakabe, H. Abe,
T. Tani, H. T. Hamda, C. E. Gutierrez, H. Skibbe, M. Maeda, F. Papazian,
K. Hagiya, N. Kishi, S. Ishii, K. Doya, T. Shimogori, T. Yamamori, K. Tanaka,
H. J. Okano, and H. Okano, “Multi-modal brain magnetic resonance imaging
database covering marmosets with a wide age range,” Scientific Data,
vol. 10, p. 221, Apr 2023.

[26]
H. Okano and P. Mitra, “Brain-mapping projects using the common marmoset,”
Neuroscience Research, vol. 93, pp. 3–7, 2015.

[27]
H. Skibbe, M. F. Rachmadi, K. Nakae, C. E. Gutierrez, J. Hata, H. Tsukada,
C. Poon, M. Schlachter, K. Doya, P. Majka, M. G. P. Rosa, H. Okano,
T. Yamamori, S. Ishii, M. Reisert, and A. Watakabe, “The Brain/MINDS
marmoset connectivity resource: An open-access platform for cellular-level
tracing and tractography in the primate brain,” PLoS Biology, vol. 21,
pp. 1–37, 06 2023.

[28]
For more information on OpenScope, see <https://alleninstitute.org/division/neural-dynamics/openscope/>.

[29]
S. E. de Vries, J. H. Siegle, and C. Koch, “Sharing neurophysiology data from
the Allen Brain Observatory,” eLife, vol. 12, p. e85550, jul

[30]
C. Koch, K. Svoboda, A. Bernard, M. A. Basso, A. K. Churchland, A. L. Fairhall,
P. A. Groblewski, J. A. Lecoq, Z. F. Mainen, M. W. Mathis, S. R. Olsen, J. w.
Phillips, A. Pouget, S. Saxena, J. H. Siegle, and A. M. Zador,
“Next-generation brain observatories,” Neuron, vol. 110,
pp. 3661–3666, Nov 2022.

[31]
For more information on the OpenScope Databook, see <https://alleninstitute.github.io/openscope_databook/intro.html>.

[32]
For more information on the IBL, see <https://www.internationalbrainlab.com/>.

[33]
International Brain Laboratory, V. Aguillon-Rodriguez, D. Angelaki, H. Bayer,
N. Bonacchi, M. Carandini, F. Cazettes, G. Chapuis, A. K. Churchland, Y. Dan,
E. Dewitt, M. Faulkner, H. Forrest, L. Haetzel, M. Häusser, S. B. Hofer,
F. Hu, A. Khanal, C. Krasniak, I. Laranjeira, Z. F. Mainen, G. Meijer, N. J.
Miska, T. D. Mrsic-Flogel, M. Murakami, J.-P. Noel, A. Pan-Vazquez,
C. Rossant, J. Sanders, K. Socha, R. Terry, A. E. Urai, H. Vergara, M. Wells,
C. J. Wilson, I. B. Witten, L. E. Wool, and A. M. Zador, “Standardized and
reproducible measurement of decision-making in mice,” eLife, vol. 10,
p. e63711, may 2021.

[34]
International Brain Laboratory, B. Benson, J. Benson, D. Birman, N. Bonacchi,
M. Carandini, J. A. Catarino, G. A. Chapuis, A. K. Churchland, Y. Dan,
P. Dayan, E. E. DeWitt, T. A. Engel, M. Fabbri, M. Faulkner, I. R. Fiete,
C. Findling, L. Freitas-Silva, B. Gerçek, K. D. Harris, M. Häusser,
S. B. Hofer, F. Hu, F. Hubert, J. M. Huntenburg, A. Khanal, C. Krasniak,
C. Langdon, P. Y. P. Lau, Z. F. Mainen, G. T. Meijer, N. J. Miska, T. D.
Mrsic-Flogel, J.-P. Noel, K. Nylund, A. Pan-Vazquez, A. Pouget, C. Rossant,
N. Roth, R. Schaeffer, M. Schartner, Y. Shi, K. Z. Socha, N. A. Steinmetz,
K. Svoboda, A. E. Urai, M. J. Wells, S. J. West, M. R. Whiteway, O. Winter,
and I. B. Witten, “A brain-wide map of neural activity during complex
behaviour,” bioRxiv, 2023.

[35]
C. Findling, F. Hubert, International Brain Laboratory, L. Acerbi, B. Benson,
J. Benson, D. Birman, N. Bonacchi, M. Carandini, J. A. Catarino, G. A.
Chapuis, A. K. Churchland, Y. Dan, E. E. DeWitt, T. A. Engel, M. Fabbri,
M. Faulkner, I. R. Fiete, L. Freitas-Silva, B. Gerçek, K. D. Harris,
M. Häusser, S. B. Hofer, F. Hu, J. M. Huntenburg, A. Khanal, C. Krasniak,
C. Langdon, P. E. Latham, P. Y. P. Lau, Z. Mainen, G. T. Meijer, N. J. Miska,
T. D. Mrsic-Flogel, J.-P. Noel, K. Nylund, A. Pan-Vazquez, L. Paninski,
J. Pillow, C. Rossant, N. Roth, R. Schaeffer, M. Schartner, Y. Shi, K. Z.
Socha, N. A. Steinmetz, K. Svoboda, C. Tessereau, A. E. Urai, M. J. Wells,
S. J. West, M. R. Whiteway, O. Winter, I. B. Witten, A. Zador, P. Dayan, and
A. Pouget, “Brain-wide representations of prior information in mouse
decision-making,” bioRxiv, 2023.

[36]
C. Tannenbaum, R. P. Ellis, F. Eyssel, J. Zou, and L. Schiebinger, “Sex and
gender analysis improves science and engineering,” Nature, vol. 575,
no. 7781, pp. 137–146, 2019.

[37]
T. Sharf, T. van der Molen, S. M. K. Glasauer, E. Guzman, A. P. Buccino,
G. Luna, Z. Cheng, M. Audouard, K. G. Ranasinghe, K. Kudo, S. S. Nagarajan,
K. R. Tovar, L. R. Petzold, A. Hierlemann, P. K. Hansma, and K. S. Kosik,
“Functional neuronal circuitry and oscillatory dynamics in human brain
organoids,” Nature Communications, vol. 13, p. 4403, Jul 2022.

[38]
S. M. Peterson, S. H. Singh, N. X. R. Wang, R. P. N. Rao, and B. W. Brunton,
“Behavioral and neural variability of naturalistic arm movements,” eNeuro, vol. 8, no. 3, 2021.

[39]
S. M. Peterson, S. H. Singh, B. Dichter, M. Scheid, R. P. N. Rao, and B. W.
Brunton, “AJILE12: Long-term naturalistic human intracranial neural
recordings and pose,” Scientific Data, vol. 9, p. 184, Apr 2022.

[40]
For more information on AJILE12, see <https://github.com/neurovium/Neuromatch-AJILE12>.

[41]
For more information on MICrONS, see <https://www.microns-explorer.org/>.

[42]
E. Y. Walker, F. H. Sinz, E. Cobos, T. Muhammad, E. Froudarakis, P. G. Fahey,
A. S. Ecker, J. Reimer, X. Pitkow, and A. S. Tolias, “Inception loops
discover what excites neurons most using deep predictive models,” Nature Neuroscience, vol. 22, pp. 2060–2065, Dec 2019.

[43]
R. G. Natan, J. J. Briguglio, L. Mwilambwe-Tshilobo, S. I. Jones, M. Aizenberg,
E. M. Goldberg, and M. N. Geffen, “Complementary control of sensory
adaptation by two types of cortical interneurons,” eLife, vol. 4,
p. e09868, oct 2015.

[44]
M. Tobin, J. Sheth, K. C. Wood, E. K. Michel, and M. N. Geffen, “Differential
modulation of cortical codes for sounds of varying intensity by distinct
inhibitory neurons,” bioRxiv, 2024.

[45]
A. Mathis, P. Mamidanna, K. M. Cury, T. Abe, V. N. Murthy, M. W. Mathis, and
M. Bethge, “DeepLabCut: markerless pose estimation of user-defined
body parts with deep learning,” Nature Neuroscience, vol. 21,
pp. 1281–1289, Sep 2018.

[46]
For more information on DeepLabCut, see <https://github.com/DeepLabCut/DeepLabCut>.

[47]
S. Schneider, J. H. Lee, and M. W. Mathis, “Learnable latent embeddings for
joint behavioural and neural analysis,” Nature, vol. 617,
pp. 360–368, May 2023.

[48]
For more information on Cebra, see <https://cebra.ai/>.

[49]
A. B. Wiltschko, T. Tsukahara, A. Zeine, R. Anyoha, W. F. Gillis, J. E.
Markowitz, R. E. Peterson, J. Katon, M. J. Johnson, and S. R. Datta,
“Revealing the structure of pharmacobehavioral space through motion
sequencing,” Nature Neuroscience, vol. 23, pp. 1433–1443, Nov 2020.

[50]
For more information on MoSeq, see <https://dattalab.github.io/moseq2-website>.

[51]
A. P. Buccino, C. L. Hurwitz, S. Garcia, J. Magland, J. H. Siegle, R. Hurwitz,
and M. H. Hennig, “SpikeInterface, a unified framework for spike
sorting,” eLife, vol. 9, p. e61834, nov 2020.

[52]
For more information on SpikeInterface, see <https://github.com/SpikeInterface>.

[53]
A. P. Buccino, O. Winter, D. Bryant, D. Feng, K. Svoboda, and J. H. Siegle,
“Compression strategies for large-scale electrophysiology data,” Journal of Neural Engineering, vol. 20, p. 056009, sep 2023.

[54]
A. Navas-Olive, R. Amaducci, M.-T. Jurado-Parras, E. R. Sebastian, and L. M.
de la Prida, “Deep learning-based feature extraction for prediction and
interpretation of sharp-wave ripples in the rodent hippocampus,” eLife, vol. 11, p. e77772, sep 2022.

[55]
A. Navas-Olive, A. Rubio, S. Abbaspoor, K. L. Hoffman, and L. M. de la Prida,
“A machine learning toolbox for the analysis of sharp-wave ripples reveals
common waveform features across species,” Communications Biology,
vol. 7, p. 211, Mar 2024.

[56]
For more information on SWR detection tools, see <https://github.com/PridaLab/cnn-ripple> and <https://github.com/PridaLab/rippl-AI>.

[57]
For more information on EBRAINS, see <https://www.ebrains.eu/>/.

[58]
S. Appukuttan, L. L. Bologna, F. Schürmann, M. Migliore, and A. Davison,
“EBRAINS live papers - interactive resource sheets for computational
studies in neuroscience,” Neuroinformatics, August 2022.

[59]
L. Bologna, R. Smiriglia, C. Lupascu, S. Appukuttan, A. Davison, I. G., J.-D.
Courcol, and M. Migliore, “The EBRAINS Hodgkin-Huxley neuron builder:
An online resource for building data-driven neuron models,” Frontiers
in Neuroinformatics, vol. 16, p. 991609, September 2022.

[60]
G. Tuckute, A. Sathe, S. Srikant, M. Taliaferro, M. Wang, M. Schrimpf, K. Kay,
and E. Fedorenko, “Driving and suppressing the human language network using
large language models,” Nature Human Behaviour, vol. 8, pp. 544–561,
Mar 2024.

[61]
M. Schrimpf, P. McGrath, E. Margalit, and J. J. DiCarlo, “Do topographic deep
ANN models of the primate ventral stream predict the perceptual effects of
direct IT cortical interventions?,” bioRxiv, 2024.

[62]
P. Bashivan, K. Kar, and J. J. DiCarlo, “Neural population control via deep
image synthesis,” Science, vol. 364, no. 6439, p. eaav9436, 2019.

[63]
M. Schrimpf, J. Kubilius, H. Hong, N. J. Majaj, R. Rajalingham, E. B. Issa,
K. Kar, P. Bashivan, J. Prescott-Roy, F. Geiger, K. Schmidt, D. L. K. Yamins,
and J. J. DiCarlo, “Brain-Score: Which artificial neural network for
object recognition is most brain-like?,” bioRxiv, 2018.

[64]
M. Schrimpf, J. Kubilius, M. J. Lee, N. A. R. Murty, R. Ajemian, and J. J.
DiCarlo, “Integrative benchmarking to advance neurally mechanistic models of
human intelligence,” Neuron, 2020.

[65]
For more information on Brain-Score, see <https://www.brain-score.org/>.

[66]
S. Saxena, A. A. Russo, J. Cunningham, and M. M. Churchland, “Motor cortex
activity across movement speeds is predicted by network-level strategies for
generating muscle activity,” eLife, vol. 11, p. e67620, may 2022.

[67]
M. N. Almani and S. Saxena, “Recurrent neural networks controlling
musculoskeletal models predict motor cortex activity during novel limb
movements,” in 2022 44th Annual International Conference of the IEEE
Engineering in Medicine & Biology Society (EMBC), pp. 3350–3356, 2022.

[68]
M. N. Almani, J. Lazzari, A. Chacon, and S. Saxena, “µSim: A
goal-driven framework for elucidating the neural control of movement through
musculoskeletal modeling,” bioRxiv, 2024.

[69]
A. H. Williams, E. Kunz, S. Kornblith, and S. Linderman, “Generalized shape
metrics on neural representations,” in Advances in Neural Information
Processing Systems (M. Ranzato, A. Beygelzimer, Y. Dauphin, P. Liang, and
J. W. Vaughan, eds.), vol. 34, pp. 4738–4750, Curran Associates, Inc., 2021.

[70]
L. R. Duong, J. Zhou, J. Nassar, J. Berman, J. Olieslagers, and A. H. Williams,
“Representational dissimilarity metric spaces for stochastic neural
networks,” arXiv, 2023.

[71]
D. A. Pospisil, B. W. Larsen, S. E. Harvey, and A. H. Williams, “Estimating
shape distances on neural representations with limited samples,” in The
Twelfth International Conference on Learning Representations, 2024.

[72]
S. E. Harvey, B. W. Larsen, and A. H. Williams, “Duality of bures and shape
distances with implications for comparing neural representations,” in UniReps: the First Workshop on Unifying Representations in Neural
Models, 2023.

[73]
For more information on BRAIN 2025, see <https://braininitiative.nih.gov/vision/nih-brain-initiative-reports/brain-2025-scientific-vision>.

[74]
For more information on data standards, see <https://resources.data.gov/standards/concepts/##data-standard>.

[75]
D. Nüst, V. Sochat, B. Marwick, S. J. Eglen, T. Head, T. Hirst, and B. D.
Evans, “Ten simple rules for writing Dockerfiles for reproducible data
science,” PLOS Computational Biology, vol. 16, pp. 1–24, 11 2020.

[76]
C. A. Köhler, D. Ulianych, S. Grün, S. Decker, and M. Denker,
“Facilitating the sharing of electrophysiology data analysis results through
in-depth provenance capture,” eNeuro, 2024.

[77]
D. Yatsenko, E. Y. Walker, and A. S. Tolias, “DataJoint: A simpler
relational data model,” arXiv, 2018.

[78]
Provenance information refers to the origin, history, and lineage of data. When
comparing analyses, having detailed provenance data is crucial because it
allows researchers to trace how the data was generated, processed, and
transformed. This transparency ensures reproducibility and helps validate
results. In comparison, when dealing with raw data, the primary requirement
is good metadata and annotations. Metadata provides essential context about
the data, such as its source, collection methods, and relevant variables.
Annotations enhance the raw data by adding descriptive labels or

[79]
L. Deng, “The MNIST database of handwritten digit images for machine
learning research [best of the web],” IEEE Signal Processing Magazine,
vol. 29, no. 6, pp. 141–142, 2012.

[80]
G. Chen, S. Chai, G. Wang, J. Du, W.-Q. Zhang, C. Weng, D. Su, D. Povey,
J. Trmal, J. Zhang, M. Jin, S. Khudanpur, S. Watanabe, S. Zhao, W. Zou,
X. Li, X. Yao, Y. Wang, Y. Wang, Z. You, and Z. Yan, “GigaSpeech: An
evolving, multi-domain ASR corpus with 10,000 hours of transcribed audio,”
arXiv, 2021.

[81]
R. Ly, M. Avaylon, M. Wulf, A. Kepecs, and O. Rübel, “Structured
behavioral data format: An NWB extension standard for task-based behavioral
neuroscience experiments,” bioRxiv, 2024.

[82]
N. Bigdely-Shamlo, J. Cockfield, S. Makeig, T. Rognon, C. La Valle,
M. Miyakoshi, and K. A. Robbins, “Hierarchical Event Descriptors
(HED): Semi-structured tagging for real-world events in large-scale eeg,”
Frontiers in Neuroinformatics, vol. 10, 2016.

[83]
O. Esteban, D. Birman, M. Schaer, O. O. Koyejo, R. A. Poldrack, and K. J.
Gorgolewski, “MRIQC: Advancing the automatic prediction of image quality
in MRI from unseen sites,” PLoS One, vol. 12, pp. 1–21, 09 2017.

[84]
P. Barthó, H. Hirase, L. Monconduit, M. Zugaro, K. D. Harris, and
G. Buzsáki, “Characterization of neocortical principal cells and
interneurons by network interactions and extracellular features,” Journal of Neurophysiology, vol. 92, no. 1, pp. 600–608, 2004.
PMID: 15056678.

[85]
A. Peyrache, N. Dehghani, E. N. Eskandar, J. R. Madsen, W. S. Anderson, J. A.
Donoghue, L. R. Hochberg, E. Halgren, S. S. Cash, and A. Destexhe,
“Spatiotemporal dynamics of neocortical excitation and inhibition during
human sleep,” Proceedings of the National Academy of Sciences,
vol. 109, no. 5, pp. 1731–1736, 2012.

[86]
B. Teleńczuk, N. Dehghani, M. Le Van Quyen, S. S. Cash, E. Halgren, N. G.
Hatsopoulos, and A. Destexhe, “Local field potentials primarily reflect
inhibitory neuron activity in human and monkey cortex,” Scientific
Reports, vol. 7, p. 40211, Jan 2017.

[87]
A. A. Liu, S. Henin, S. Abbaspoor, A. Bragin, E. A. Buffalo, J. S. Farrell,
D. J. Foster, L. M. Frank, T. Gedankien, J. Gotman, J. A. Guidera, K. L.
Hoffman, J. Jacobs, M. J. Kahana, L. Li, Z. Liao, J. J. Lin, A. Losonczy,
R. Malach, M. A. van der Meer, K. McClain, B. L. McNaughton, Y. Norman,
A. Navas-Olive, L. M. de la Prida, J. W. Rueckemann, J. J. Sakon, I. Skelin,
I. Soltesz, B. P. Staresina, S. A. Weiss, M. A. Wilson, K. A. Zaghloul,
M. Zugaro, and G. Buzsáki, “A consensus statement on detection of
hippocampal sharp wave ripples and differentiation from other fast
oscillations,” Nature Communications, vol. 13, p. 6000, Oct 2022.

[88]
J. F. Ramirez-Villegas, N. K. Logothetis, and M. Besserve, “Diversity of
sharp-wave–ripple LFP signatures reveals differentiated brain-wide
dynamical events,” Proceedings of the National Academy of Sciences,
vol. 112, no. 46, pp. E6379–E6387, 2015.

[89]
N. Dehghani, S. S. Cash, and E. Halgren, “Emergence of synchronous EEG
spindles from asynchronous MEG spindles,” Human Brain Mapping,
vol. 32, no. 12, pp. 2217–2227, 2011.

[90]
L. Kaulen, J. T. C. Schwabedal, J. Schneider, P. Ritter, and S. Bialonski,
“Advanced sleep spindle identification with neural networks,” Scientific Reports, vol. 12, p. 7686, May 2022.

[91]
T. Kelder, M. P. van Iersel, K. Hanspers, M. Kutmon, B. R. Conklin, C. T.
Evelo, and A. R. Pico, “WikiPathways: Building research communities on
biological pathways,” Nucleic Acids Research, vol. 40,
pp. D1301–1307, Jan. 2012.

[92]
J. H. Caufield, H. Hegde, V. Emonet, N. L. Harris, M. P. Joachimiak,
N. Matentzoglu, H. Kim, S. A. T. Moxon, J. T. Reese, M. A. Haendel, P. N.
Robinson, and C. J. Mungall, “Structured prompt interrogation and recursive
extraction of semantics (SPIRES): A method for populating knowledge bases
using zero-shot learning,” arXiv, 2023.

[93]
X. Luo, A. Rechardt, G. Sun, K. K. Nejad, F. Yáñez, B. Yilmaz, K. Lee, A. O.
Cohen, V. Borghesani, A. Pashkov, D. Marinazzo, J. Nicholas, A. Salatiello,
I. Sucholutsky, P. Minervini, S. Razavi, R. Rocca, E. Yusifov, T. Okalova,
N. Gu, M. Ferianc, M. Khona, K. R. Patil, P.-S. Lee, R. Mata, N. E. Myers,
J. K. Bizley, S. Musslick, I. P. Bilgin, G. Niso, J. M. Ales, M. Gaebler,
N. A. R. Murty, L. Loued-Khenissi, A. Behler, C. M. Hall, J. Dafflon, S. D.
Bao, and B. C. Love, “Large language models surpass human experts in
predicting neuroscience results,” arXiv, 2024.

[94]
S. Ye, J. Lauer, M. Zhou, A. Mathis, and M. W. Mathis, “AmadeusGPT: a
natural language interface for interactive animal behavioral analysis,” Thirty-seventh Conference on Neural Information Processing Systems, 2023.

[95]
L. Huang, W. Yu, W. Ma, W. Zhong, Z. Feng, H. Wang, Q. Chen, W. Peng, X. Feng,
B. Qin, and T. Liu, “A survey on hallucination in large language models:
Principles, taxonomy, challenges, and open questions,” arXiv, 2023.

[96]
R. Wang, E. Zelikman, G. Poesia, Y. Pu, N. Haber, and N. D. Goodman,
“Hypothesis search: Inductive reasoning with language models,” arXiv, 2023.

[97]
E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and
W. Chen, “LoRA: Low-rank adaptation of large language models,” arXiv, 2021.

[98]
P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler,
M. Lewis, W. tau Yih, T. Rocktäschel, S. Riedel, and D. Kiela,
“Retrieval-augmented generation for knowledge-intensive NLP tasks,” arXiv, 2021.

[99]
S. J. Eglen, B. Marwick, Y. O. Halchenko, M. Hanke, S. Sufi, P. Gleeson, R. A.
Silver, A. P. Davison, L. Lanyon, M. Abrams, T. Wachtler, D. J. Willshaw,
C. Pouzat, and J.-B. Poline, “Toward standard practices for sharing computer
code and programs in neuroscience,” Nature Neuroscience, vol. 20,
no. 6, pp. 770–773, 2017.