* Jenner et al. (2023) Erik Jenner, Adrià Garriga-alonso, and Egor Zverev.  A comparison of causal scrubbing, causal abstractions, and related methods.  _AI Alignment Forum_ , June 2023.
  * Jermyn et al. (2023) Adam Jermyn, Chris Olah, and T Henighan.  Circuits updates - May 2023: Attention Head Superposition.  _Transformer Circuits Thread_ , 2023.
  * Jermyn et al. (2022) Adam S. Jermyn, Nicholas Schiefer, and Evan Hubinger.  Engineering Monosemanticity in Toy Models.  _CoRR_ , November 2022.
  * Jozdien (2022) Jozdien.  Conditioning Generative Models for Alignment.  _AI Alignment Forum_ , July 2022.
  * Jumelet (2023) Jaap Jumelet.  Evaluating and Interpreting Language Models.  _NLP Lecture_ , November 2023.
  * Kasioumis et al. (2021) Theodoros Kasioumis, Joe Townsend, and Hiroya Inakoshi.  Elite BackProp: Training Sparse Interpretable Neurons.  _International Workshop on Neuro-Symbolic Learning and Reasoning_ , 2021.
  * Kirchner (2023) Jan Kirchner.  Neuroscience and Natural Abstractions.  _LessWrong_ , March 2023.
  * Kornblith et al. (2019) Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton.  Similarity of Neural Network Representations Revisited.  _ICML_ , July 2019.
  * Kramár et al. (2024) János Kramár, Tom Lieberum, Rohin Shah, and Neel Nanda.  AtP*: An efficient and scalable method for localizing LLM behaviour to components.  _CoRR_ , March 2024.
  * Kulveit et al. (2023) Jan Kulveit, Clem von Stengel, and Roman Leventov.  Predictive Minds: LLMs As Atypical Active Inference Agents.  _CoRR_ , November 2023.
  * Lamparth & Reuel (2023) Max Lamparth and Anka Reuel.  Analyzing And Editing Inner Mechanisms Of Backdoored Language Models.  _CoRR_ , 2023.
  * Lan & Barez (2023) Michael Lan and Fazl Barez.  Locating Cross-Task Sequence Continuation Circuits in Transformers.  _CoRR_ , November 2023.
  * Lang et al. (2024) Leon Lang, Davis Foote, Stuart Russell, Anca Dragan, Erik Jenner, and Scott Emmons.  When Your AIs Deceive You: Challenges with Partial Observability of Human Evaluators in Reward Learning.  _CoRR_ , March 2024.
  * Lange et al. (2023) Georg Lange, Alex Makelov, and Neel Nanda.  An Interpretability Illusion for Activation Patching of Arbitrary Subspaces.  _AI Alignment Forum_ , August 2023.
  * Langedijk et al. (2023) Anna Langedijk, Hosein Mohebbi, Gabriele Sarti, Willem Zuidema, and Jaap Jumelet.  DecoderLens: Layerwise Interpretation of Encoder-Decoder Transformers.  _CoRR_ , 2023.
  * Lau et al. (2023) Edmund Lau, Daniel Murfet, and Susan Wei.  Quantifying degeneracy in singular models via the learning coefficient.  _CoRR_ , August 2023.
  * Leahy (2023) Connor Leahy.  Barriers to Mechanistic Interpretability for AGI Safety.  _AI Alignment Forum_ , 2023.
  * Leavitt & Morcos (2020) Matthew L. Leavitt and Ari Morcos.  Towards falsifiable interpretability research.  _CoRR_ , October 2020.
  * Lecomte et al. (2023) Victor Lecomte, Kushal Thaman, Trevor Chow, Rylan Schaeffer, and Sanmi Koyejo.  Incidental Polysemanticity.  _CoRR_ , 2023.
  * Lee et al. (2024) Andrew Lee, Xiaoyan Bai, Itamar Pres, Martin Wattenberg, Jonathan K. Kummerfeld, and Rada Mihalcea.  A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicity.  _CoRR_ , 2024.
  * Lee et al. (2006) Honglak Lee, Alexis Battle, Rajat Raina, and Andrew Ng.  Efficient sparse coding algorithms.  _NeurIPS_ , 2006.
  * Li et al. (2023a) Kenneth Li, Aspen K. Hopkins, David Bau, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg.  Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task.  _ICLR_ , 2023a.
  * Li et al. (2023b) Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg.  Inference-Time Intervention: Eliciting Truthful Answers from a Language Model.  _NeurIPS Spotlight_ , July 2023b.
  * Li et al. (2015) Yixuan Li, Jason Yosinski, Jeff Clune, Hod Lipson, and John Hopcroft.  Convergent Learning: Do different neural networks learn the same representations? _NIPS Workshop on Feature Extraction_ , December 2015.
  * Lieberum et al. (2023) Tom Lieberum, Matthew Rahtz, János Kramár, Neel Nanda, Geoffrey Irving, Rohin Shah, and Vladimir Mikulik.  Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchilla.  _CoRR_ , July 2023.
  * Lindner et al. (2023) David Lindner, János Kramár, Sebastian Farquhar, Matthew Rahtz, Thomas McGrath, and Vladimir Mikulik.  Tracr: Compiled Transformers as a Laboratory for Interpretability.  _CoRR_ , 2023.
  * Liu et al. (2021) Leo Z. Liu, Yizhong Wang, Jungo Kasai, Hannaneh Hajishirzi, and Noah A. Smith.  Probing Across Time: What Does RoBERTa Know and When? _EMNLP_ , September 2021.
  * Liu & Tegmark (2023) Ziming Liu and Max Tegmark.  A Neural Scaling Law from Lottery Ticket Ensembling.  _CoRR_ , 2023.
  * Liu et al. (2022a) Ziming Liu, Ouail Kitouni, Niklas Nolte, Eric J. Michaud, Max Tegmark, and Mike Williams.  Towards Understanding Grokking: An Effective Theory of Representation Learning.  _NeurIPS_ , 2022a.
  * Liu et al. (2022b) Ziming Liu, Eric J. Michaud, and Max Tegmark.  Omnigrok: Grokking Beyond Algorithmic Data.  _ICML_ , 2022b.
  * Liu et al. (2023a) Ziming Liu, Eric Gan, and Max Tegmark.  Seeing is Believing: Brain-Inspired Modular Training for Mechanistic Interpretability.  _Entropy_ , June 2023a.
  * Liu et al. (2023b) Ziming Liu, Mikail Khona, Ila R. Fiete, and Max Tegmark.  Growing Brains: Co-emergence of Anatomical and Functional Modularity in Recurrent Neural Networks.  _CoRR_ , 2023b.
  * Liu et al. (2023c) Ziming Liu, Ziqian Zhong, and Max Tegmark.  Grokking as compression: A nonlinear complexity perspective.  _CoRR_ , 2023c.
  * Locatello et al. (2019) Francesco Locatello, Stefan Bauer, Mario Lucic, Gunnar Rätsch, Sylvain Gelly, Bernhard Schölkopf, and Olivier Bachem.  Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations.  _CoRR_ , June 2019.
  * Marks et al. (2023) Luke Marks, Amir Abdullah, Luna Mendez, Rauno Arike, Philip Torr, and Fazl Barez.  Interpreting Reward Models in RLHF-Tuned Language Models Using Sparse Autoencoders.  _CoRR_ , October 2023.
  * Marshall & Kirchner (2024) Simon C. Marshall and Jan H. Kirchner.  Understanding polysemanticity in neural networks through coding theory.  _CoRR_ , January 2024.
  * Mazeika et al. (2022) Mantas Mazeika, Andy Zou, Akul Arora, Pavel Pleskov, Dawn Song, Dan Hendrycks, Bo Li, and David Forsyth.  How Hard is Trojan Detection in DNNs? Fooling Detectors With Evasive Trojans.  _CoRR_ , September 2022.
  * McDougall et al. (2023) Callum McDougall, Arthur Conmy, Cody Rushing, Thomas McGrath, and Neel Nanda.  Copy Suppression: Comprehensively Understanding an Attention Head.  _CoRR_ , October 2023.
  * McGrath et al. (2022) Thomas McGrath, Andrei Kapishnikov, Nenad Tomašev, Adam Pearce, Martin Wattenberg, Demis Hassabis, Been Kim, Ulrich Paquet, and Vladimir Kramnik.  Acquisition of chess knowledge in AlphaZero.  _PNAS_ , November 2022.
  * McGrath et al. (2023) Thomas McGrath, Matthew Rahtz, Janos Kramar, Vladimir Mikulik, and Shane Legg.  The Hydra Effect: Emergent Self-repair in Language Model Computations.  _CoRR_ , July 2023.
  * Meng et al. (2022a) Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov.  Locating and Editing Factual Associations in GPT.  _NeurIPS_ , 2022a.
  * Meng et al. (2022b) Kevin Meng, Arnab Sen Sharma, Alex Andonian, Yonatan Belinkov, and David Bau.  Mass-Editing Memory in a Transformer.  _ICLR_ , 2022b.
  * Merrill et al. (2023) William Merrill, Nikolaos Tsilivis, and Aman Shukla.  A Tale of Two Circuits: Grokking as Competition of Sparse and Dense Subnetworks.  _CoRR_ , 2023.
  * Merullo et al. (2023) Jack Merullo, Carsten Eickhoff, and Ellie Pavlick.  A Mechanism for Solving Relational Tasks in Transformer Language Models.  _CoRR_ , May 2023.
  * Michaud et al. (2023) Eric J. Michaud, Ziming Liu, Uzay Girit, and Max Tegmark.  The Quantization Model of Neural Scaling.  _CoRR_ , March 2023.
  * Michaud et al. (2024) Eric J. Michaud, Isaac Liao, Vedang Lad, Ziming Liu, Anish Mudide, Chloe Loughridge, Zifan Carl Guo, Tara Rezaei Kheirkhah, Mateja Vukelić, and Max Tegmark.  Opening the AI black box: program synthesis via mechanistic interpretability.  _CoRR_ , February 2024.
  * Mikolov et al. (2013) Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean.  Distributed Representations of Words and Phrases and their Compositionality.  _NeurIPS_ , October 2013.
  * Miller & Neo (2023) Joseph Miller and Clement Neo.  We Found An Neuron in GPT-2.  _AI Alignment Forum_ , February 2023.
  * Miller (2019) Tim Miller.  Explanation in artificial intelligence: Insights from the social sciences.  _Artificial Intelligence_ , February 2019.
  * Monea et al. (2023) Giovanni Monea, Maxime Peyrard, Martin Josifoski, Vishrav Chaudhary, Jason Eisner, Emre Kıcıman, Hamid Palangi, Barun Patra, and Robert West.  A Glitch in the Matrix? Locating and Detecting Language Model Grounding with Fakepedia.  _CoRR_ , 2023.
  * Mousi et al. (2023) Basel Mousi, Nadir Durrani, and Fahim Dalvi.  Can LLMs facilitate interpretation of pre-trained language models? _EMNLP_ , 2023.
  * Mu & Andreas (2020) Jesse Mu and Jacob Andreas.  Compositional Explanations of Neurons.  _NeurIPS_ , June 2020.
  * Nainani (2024) Jatin Nainani.  Evaluating Brain-Inspired Modular Training in Automated Circuit Discovery for Mechanistic Interpretability.  _CoRR_ , January 2024.
  * Nakkiran et al. (2019) Preetum Nakkiran, Gal Kaplun, Dimitris Kalimeris, Tristan Yang, Benjamin L. Edelman, Fred Zhang, and Boaz Barak.  SGD on Neural Networks Learns Functions of Increasing Complexity.  _NeurIPS_ , May 2019.
  * Nanda (2022a) Neel Nanda.  200 COP in MI: Looking for Circuits in the Wild.  _Neel Nanda’s Blog_ , 2022a.
  * Nanda (2022b) Neel Nanda.  200 COP in MI: Analysing Training Dynamics.  _Neel Nanda’s Blog_ , 2022b.
  * Nanda (2022c) Neel Nanda.  200 COP in MI: Studying Learned Features in Language Models.  _Neel Nanda’s Blog_ , 2022c.
  * Nanda (2022d) Neel Nanda.  A Comprehensive Mechanistic Interpretability Explainer & Glossary.  _Neel Nanda’s Blog_ , December 2022d.
  * Nanda (2023a) Neel Nanda.  200 COP in MI: Exploring Polysemanticity and Superposition.  _Neel Nanda’s Blog_ , January 2023a.
  * Nanda (2023b) Neel Nanda.  200 COP in MI: Techniques, Tooling and Automation.  _Neel Nanda’s Blog_ , 2023b.
  * Nanda (2023c) Neel Nanda.  Actually, Othello-GPT Has A Linear Emergent World Representation.  _Neel Nanda’s Blog_ , March 2023c.
  * Nanda (2023d) Neel Nanda.  Attribution Patching: Activation Patching At Industrial Scale.  _Neel Nanda’s Blog_ , February 2023d.
  * Nanda (2023e) Neel Nanda.  How to Think About Activation Patching.  _AI Alignment Forum_ , April 2023e.
  * Nanda (2023f) Neel Nanda.  Mechanistic Interpretability Quickstart Guide.  _Neel Nanda’s Blog_ , January 2023f.
  * Nanda et al. (2023a) Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, and Jacob Steinhardt.  Progress measures for grokking via mechanistic interpretability.  _ICLR_ , January 2023a.
  * Nanda et al. (2023b) Neel Nanda, Andrew Lee, and Martin Wattenberg.  Emergent Linear Representations in World Models of Self-Supervised Sequence Models.  _BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP_ , September 2023b.
  * Ngo et al. (2022) Richard Ngo, Lawrence Chan, and Sören Mindermann.  The alignment problem from a deep learning perspective.  _CoRR_ , December 2022.
  * Nichani et al. (2024) Eshaan Nichani, Alex Damian, and Jason D. Lee.  How Transformers Learn Causal Structure with Gradient Descent.  _CoRR_ , February 2024.
  * NicholasKees & janus (2022) NicholasKees and janus.  Searching for Search.  _AI Alignment Forum_ , November 2022.
  * nostalgebraist (2020) nostalgebraist.  interpreting GPT: the logit lens.  _AI Alignment Forum_ , August 2020.
  * Olah (2023) Chris Olah.  Distributed Representations: Composition & Superposition.  _Transformer Circuits Thread_ , 2023.
  * Olah & Carter (2017) Chris Olah and Shan Carter.  Research Debt.  _Distill_ , March 2017.
  * Olah & Jermyn (2024) Chris Olah and Adam Jermyn.  Reflections on Qualitative Research.  _Transformer Circuits Thread_ , March 2024.
  * Olah et al. (2017) Chris Olah, Alexander Mordvintsev, and Ludwig Schubert.  Feature Visualization.  _Distill_ , November 2017.
  * Olah et al. (2018) Chris Olah, Arvind Satyanarayan, Ian Johnson, Shan Carter, Ludwig Schubert, Katherine Ye, and Alexander Mordvintsev.  The Building Blocks of Interpretability.  _Distill_ , March 2018.
  * Olah et al. (2020) Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter.  Zoom In: An Introduction to Circuits.  _Distill_ , March 2020.
  * Olah (2022) Christopher Olah.  Mechanistic Interpretability, Variables, and the Importance of Interpretable Bases.  _Transformer Circuits Thread_ , 2022.
  * Olshausen & Field (1997) B. A. Olshausen and D. J. Field.  Sparse coding with an overcomplete basis set: a strategy employed by V1? _Vision Research_ , December 1997.
  * Olsson et al. (2022) Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Scott Johnston, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah.  In-context Learning and Induction Heads.  _Transformer Circuits Thread_ , 2022.
  * O’Mahony et al. (2023) Laura O’Mahony, Vincent Andrearczyk, Henning Muller, and Mara Graziani.  Disentangling Neuron Representations with Concept Vectors.  _CVPR Workshops_ , April 2023.
  * Ouyang et al. (2022) Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe.  Training language models to follow instructions with human feedback.  _CoRR_ , 2022.
  * Pal et al. (2023) Koyena Pal, Jiuding Sun, Andrew Yuan, Byron C. Wallace, and David Bau.  Future Lens: Anticipating Subsequent Tokens from a Single Hidden State.  _CoNLL_ , 2023.
  * Palit et al. (2023) Vedant Palit, Rohan Pandey, Aryaman Arora, and P. Liang.  Towards vision-language mechanistic interpretability: A causal tracing tool for BLIP.  _ICCVW_ , 2023.
  * Park et al. (2023) Kiho Park, Yo Joong Choe, and Victor Veitch.  The Linear Representation Hypothesis and the Geometry of Large Language Models.  _NeurIPS Workshop on Causal Representation Learning_ , November 2023\.
  * Pearl (2009) Judea Pearl.  _Causality_.  Cambridge University Press, 2009.
  * Power et al. (2022) Alethea Power, Yuri Burda, Harri Edwards, Igor Babuschkin, and Vedant Misra.  Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets.  _CoRR_ , January 2022.
  * Quirke & Barez (2023) Philip Quirke and Fazl Barez.  Understanding Addition in Transformers.  _CoRR_ , October 2023.
  * Quirke et al. (2024) Philip Quirke, Clement Neo, and Fazl Barez.  Increasing Trust in Language Models through the Reuse of Verified Circuits.  _CoRR_ , February 2024.
  * Ravfogel et al. (2020) Shauli Ravfogel, Yanai Elazar, Hila Gonen, Michael Twiton, and Yoav Goldberg.  Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection.  _ACL_ , July 2020.
  * Ren et al. (2023) Jie Ren, Mingjie Li, Qirui Chen, Huiqi Deng, and Quanshi Zhang.  Defining and Quantifying the Emergence of Sparse Concepts in DNNs.  _CoRR_ , April 2023.
  * Ribeiro et al. (2016) Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin.  "Why Should I Trust You?": Explaining the Predictions of Any Classifier.  _NAACL_ , August 2016.
  * __RicG__ (2023) __RicG__.  AGI-Automated Interpretability is Suicide.  _LessWrong_ , May 2023.
  * Richens & Everitt (2024) Jonathan Richens and Tom Everitt.  Robust agents learn causal world models.  _ICLR Oral_ , February 2024.
  * Riegel et al. (2020) Ryan Riegel, Alexander Gray, Francois Luus, Naweed Khan, Ndivhuwo Makondo, Ismail Yunus Akhalwaya, Haifeng Qian, Ronald Fagin, Francisco Barahona, Udit Sharma, Shajith Ikbal, Hima Karanam, Sumit Neelam, Ankita Likhyani, and Santosh Srivastava.  Logical Neural Networks.  _NeurIPS_ , June 2020.
  * Rudin (2019) Cynthia Rudin.  Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead.  _Nature Machine Intelligence_ , May 2019.
  * Ruthenis (2022) Thane Ruthenis.  Internal Interfaces Are a High-Priority Interpretability Target.  _AI Alignment Forum_ , December 2022.
  * Ruthenis (2023) Thane Ruthenis.  World-Model Interpretability Is All We Need.  _AI Alignment Forum_ , January 2023.
  * Räuker et al. (2023) Tilman Räuker, Anson Ho, Stephen Casper, and Dylan Hadfield-Menell.  Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks.  _TMLR_ , August 2023.
  * Sajjad et al. (2022) Hassan Sajjad, Nadir Durrani, and Fahim Dalvi.  Neuron-level Interpretation of Deep NLP Models: A Survey.  _TACL_ , November 2022.
  * Sakarvadia et al. (2023a) Mansi Sakarvadia, Aswathy Ajith, Arham Khan, Daniel Grzenda, Nathaniel Hudson, André Bauer, Kyle Chard, and Ian Foster.  Memory Injections: Correcting Multi-Hop Reasoning Failures during Inference in Transformer-Based Language Models.  _CoRR_ , September 2023a.
  * Sakarvadia et al. (2023b) Mansi Sakarvadia, Arham Khan, Aswathy Ajith, Daniel Grzenda, Nathaniel Hudson, André Bauer, Kyle Chard, and Ian Foster.  Attention Lens: A Tool for Mechanistically Interpreting the Attention Head Information Retrieval Mechanism.  _CoRR_ , October 2023b.
  * Salin et al. (2022) Emmanuelle Salin, Badreddine Farah, Stéphane Ayache, and Benoit Favre.  Are Vision-Language Transformers Learning Multimodal Representations? A Probing Perspective.  _AAAI_ , June 2022.
  * Salvatori et al. (2023) Tommaso Salvatori, Ankur Mali, Christopher L. Buckley, Thomas Lukasiewicz, Rajesh P. N. Rao, Karl Friston, and Alexander Ororbia.  Brain-Inspired Computational Intelligence via Predictive Coding.  _CoRR_ , 2023.
  * Saphra (2023) Naomi Saphra.  Interpretability Creationism.  _The Gradient_ , 2023.
  * Schaeffer et al. (2023) Rylan Schaeffer, Brando Miranda, and Sanmi Koyejo.  Are Emergent Abilities of Large Language Models a Mirage? _CoRR_ , May 2023.
  * Scherlis et al. (2023) Adam Scherlis, Kshitij Sachan, Adam S. Jermyn, Joe Benton, and Buck Shlegeris.  Polysemanticity and Capacity in Neural Networks.  _CoRR_ , July 2023.
  * Selvaraju et al. (2016) Ramprasaath R. Selvaraju, Abhishek Das, Ramakrishna Vedantam, Michael Cogswell, Devi Parikh, and Dhruv Batra.  Grad-CAM: Why did you say that? Visual Explanations from Deep Networks via Gradient-based Localization.  _ICCV_ , 2016.
  * Shapley (1988) Lloyd S. Shapley.  A value for n -person games.  _Cambridge University Press_ , October 1988.
  * Sharkey (2022) Lee Sharkey.  Circumventing interpretability: How to defeat mind-readers.  _CoRR_ , December 2022.
  * Sharkey (2023) Lee Sharkey.  A technical note on bilinear layers for interpretability.  _CoRR_ , May 2023.
  * Sharkey et al. (2022a) Lee Sharkey, Sid Black, and beren.  Current themes in mechanistic interpretability research.  _AI Alignment Forum_ , November 2022a.
  * Sharkey et al. (2022b) Lee Sharkey, Dan Braun, and Beren Millidge.  Taking features out of superposition with sparse autoencoders.  _AI Alignment Forum_ , 2022b.
  * Sharma et al. (2023) Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R. Bowman, Newton Cheng, Esin Durmus, Zac Hatfield-Dodds, Scott R. Johnston, Shauna Kravec, Timothy Maxwell, Sam McCandlish, Kamal Ndousse, Oliver Rausch, Nicholas Schiefer, Da Yan, Miranda Zhang, and Ethan Perez.  Towards Understanding Sycophancy in Language Models.  _CoRR_ , October 2023.
  * Shrikumar et al. (2017) Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje.  Learning Important Features Through Propagating Activation Differences.  _ICML_ , 2017.
  * Simon et al. (2023) James B. Simon, Maksis Knutins, Liu Ziyin, Daniel Geisz, Abraham J. Fetterman, and Joshua Albrecht.  On the Stepwise Nature of Self-Supervised Learning.  _ICML_ , May 2023.
  * Smilkov et al. (2017) Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg.  SmoothGrad: removing noise by adding noise.  _CoRR_ , June 2017.
  * Srivastava et al. (2014) Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov.  Dropout: A Simple Way to Prevent Neural Networks from Overfitting.  _JMLR_ , 2014.
  * Stander et al. (2023) Dashiell Stander, Qinan Yu, Honglu Fan, and Stella Biderman.  Grokking Group Multiplication with Cosets.  _CoRR_ , 2023.
  * Steinhardt (2023) Jacob Steinhardt.  Emergent Deception and Emergent Optimization.  _Bounded Regret_ , February 2023.
  * Stolfo et al. (2023) Alessandro Stolfo, Yonatan Belinkov, and Mrinmaya Sachan.  A Mechanistic Interpretation of Arithmetic Reasoning in Language Models using Causal Mediation Analysis.  _EMNLP_ , October 2023.
  * Sucholutsky et al. (2023) Ilia Sucholutsky, Lukas Muttenthaler, Adrian Weller, Andi Peng, Andreea Bobu, Been Kim, Bradley C. Love, Erin Grant, Iris Groen, Jascha Achterberg, Joshua B. Tenenbaum, Katherine M. Collins, Katherine L. Hermann, Kerem Oktar, Klaus Greff, Martin N. Hebart, Nori Jacoby, Qiuyi Zhang, Raja Marjieh, Robert Geirhos, Sherol Chen, Simon Kornblith, Sunayana Rane, Talia Konkle, Thomas P. O’Connell, Thomas Unterthiner, Andrew K. Lampinen, Klaus-Robert Müller, Mariya Toneva, and Thomas L. Griffiths.  Getting aligned on representational alignment.  _CoRR_ , November 2023.
  * Sundararajan et al. (2017) Mukund Sundararajan, Ankur Taly, and Qiqi Yan.  Axiomatic Attribution for Deep Networks.  _ICML_ , June 2017.
  * Syed et al. (2023) Aaquib Syed, Can Rager, and Arthur Conmy.  Attribution Patching Outperforms Automated Circuit Discovery.  _CoRR_ , October 2023.
  * Tegmark & Omohundro (2023) Max Tegmark and Steve Omohundro.  Provably safe systems: the only path to controllable AGI.  _CoRR_ , September 2023.
  * Tenney et al. (2019) Ian Tenney, Dipanjan Das, and Ellie Pavlick.  BERT Rediscovers the Classical NLP Pipeline.  _ACL_ , August 2019.
  * Thilak et al. (2022) Vimal Thilak, Etai Littwin, Shuangfei Zhai, Omid Saremi, Roni Paiss, and Joshua Susskind.  The Slingshot Mechanism: An Empirical Study of Adaptive Optimizers and the Grokking Phenomenon.  _CoRR_ , 2022.
  * Tigges et al. (2023) Curt Tigges, Oskar John Hollinsworth, Atticus Geiger, and Neel Nanda.  Linear Representations of Sentiment in Large Language Models.  _CoRR_ , October 2023.
  * Todd et al. (2023) Eric Todd, Millicent L. Li, Arnab Sen Sharma, Aaron Mueller, Byron C. Wallace, and David Bau.  Function Vectors in Large Language Models.  _CoRR_ , 2023.
  * Trivedi et al. (2021) Dweep Trivedi, Jesse Zhang, Shao-Hua Sun, and Joseph J. Lim.  Learning to Synthesize Programs as Interpretable and Generalizable Policies.  _NeurIPS_ , 2021.
  * Turner et al. (2023) Alexander Matt Turner, Lisa Thiergart, David Udell, Gavin Leech, Ulisse Mini, and Monte MacDiarmid.  Activation Addition: Steering Language Models Without Optimization.  _CoRR_ , September 2023.
  * TurnTrout et al. (2023) TurnTrout, peligrietzer, Ulisse Mini, montemac, and David Udell.  Understanding and controlling a maze-solving policy network.  _AI Alignment Forum_ , November 2023.
  * Vaintrob et al. (2024) Dmitry Vaintrob, jake_mendel, and Kaarel.  Toward A Mathematical Framework for Computation in Superposition.  _AI Alignment Forum_ , 2024.
  * Variengien & Winsor (2023) Alexandre Variengien and Eric Winsor.  Look Before You Leap: A Universal Emergent Decomposition of Retrieval Tasks in Language Models.  _CoRR_ , December 2023.
  * Varma et al. (2023) Vikrant Varma, Rohin Shah, Zachary Kenton, János Kramár, and Ramana Kumar.  Explaining grokking through circuit efficiency.  _CoRR_ , September 2023.
  * Verma et al. (2019a) Abhinav Verma, Hoang M. Le, Yisong Yue, and Swarat Chaudhuri.  Imitation-Projected Programmatic Reinforcement Learning.  _NeurIPS_ , 2019a.
  * Verma et al. (2019b) Abhinav Verma, Vijayaraghavan Murali, Rishabh Singh, Pushmeet Kohli, and Swarat Chaudhuri.  Programmatically Interpretable Reinforcement Learning.  _CoRR_ , April 2019b.
  * Vig et al. (2020) Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber.  Investigating Gender Bias in Language Models Using Causal Mediation Analysis.  _NeurIPS_ , 2020.
  * Vilas et al. (2023) M. Vilas, Timothy Schaumlöffel, and Gemma Roig.  Analyzing vision transformers for image classification in class embedding space.  _CoRR_ , 2023.
  * Voita & Titov (2020) Elena Voita and Ivan Titov.  Information-Theoretic Probing with Minimum Description Length.  _EMNLP_ , March 2020.
  * Voita et al. (2023) Elena Voita, Javier Ferrando, and Christoforos Nalmpantis.  Neurons in Large Language Models: Dead, N-gram, Positional.  _CoRR_ , September 2023.
  * von Oswald et al. (2023) Johannes von Oswald, Eyvind Niklasson, Maximilian Schlegel, Seijin Kobayashi, Nicolas Zucchet, Nino Scherrer, Nolan Miller, Mark Sandler, Blaise Agüera y Arcas, Max Vladymyrov, Razvan Pascanu, and João Sacramento.  Uncovering mesa-optimization algorithms in Transformers.  _CoRR_ , September 2023.
  * Voss et al. (2021) Chelsea Voss, Gabriel Goh, Nick Cammarata, Michael Petrov, Ludwig Schubert, and Chris Olah.  Branch Specialization.  _Distill_ , April 2021.
  * Wang et al. (2023) Kevin Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt.  Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small.  _ICLR_ , 2023.
  * Warstadt et al. (2020) Alex Warstadt, Alicia Parrish, Haokun Liu, Anhad Mohananey, Wei Peng, Sheng-Fu Wang, and Samuel R. Bowman.  BLiMP: The Benchmark of Linguistic Minimal Pairs for English.  _Transactions of the Association for Computational Linguistics_ , 2020\.
  * Watanabe (2009) Sumio Watanabe.  _Algebraic Geometry and Statistical Learning Theory_.  Cambridge Monographs on Applied and Computational Mathematics. Cambridge University Press, 2009.
  * Wei et al. (2022) Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus.  Emergent Abilities of Large Language Models.  _TMLR_ , October 2022.
  * Wentworth (2022) John Wentworth.  How To Go From Interpretability To Alignment: Just Retarget The Search.  _AI Alignment Forum_ , August 2022.
  * Whittington et al. (2022) James C. R. Whittington, Will Dorrell, Surya Ganguli, and Timothy E. J. Behrens.  Disentangling with Biological Constraints: A Theory of Functional Cell Types.  _CoRR_ , September 2022.
  * Wong et al. (2023) Lionel Wong, Gabriel Grand, Alexander K. Lew, Noah D. Goodman, Vikash K. Mansinghka, Jacob Andreas, and Joshua B. Tenenbaum.  From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thought.  _CoRR_ , June 2023.
  * Wu et al. (2022) Baoyuan Wu, Hongrui Chen, Mingda Zhang, Zihao Zhu, Shaokui Wei, Danni Yuan, and Chao Shen.  BackdoorBench: A Comprehensive Benchmark of Backdoor Learning.  _NeurIPS Datasets and Benchmarks_ , October 2022.
  * Wu et al. (2023) Zhengxuan Wu, Atticus Geiger, Christopher Potts, and Noah D. Goodman.  Interpretability at Scale: Identifying Causal Mechanisms in Alpaca.  _CoRR_ , May 2023.
  * Yu et al. (2023) Qinan Yu, Jack Merullo, and Ellie Pavlick.  Characterizing Mechanisms for Factual Recall in Language Models.  _CoRR_ , October 2023.
  * Yun et al. (2021) Zeyu Yun, Yubei Chen, Bruno A. Olshausen, and Yann LeCun.  Transformer visualization via dictionary learning: contextualized embedding as a linear superposition of transformer factors.  _NAACL Workshop DeeLIO_ , 2021.
  * Zeiler & Fergus (2014) Matthew D. Zeiler and Rob Fergus.  Visualizing and Understanding Convolutional Networks.  _ECCV_ , 2014.
  * Zhang et al. (2021) Biao Zhang, Ivan Titov, and Rico Sennrich.  Sparse Attention with Linear Units.  _EMNLP_ , October 2021.
  * Zhang et al. (2017) Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals.  Understanding deep learning requires rethinking generalization.  _ICLR_ , February 2017.
  * Zhang & Nanda (2023) Fred Zhang and Neel Nanda.  Towards Best Practices of Activation Patching in Language Models: Metrics and Methods.  _ICLR_ , 2023.
  * Zhang et al. (2019) Quanshi Zhang, Yu Yang, Haotian Ma, and Ying Nian Wu.  Interpreting CNNs via Decision Trees.  _CVPR_ , 2019.
  * Zhong et al. (2023) Ziqian Zhong, Ziming Liu, Max Tegmark, and Jacob Andreas.  The Clock and the Pizza: Two Stories in Mechanistic Explanation of Neural Networks.  _CoRR_ , 2023.
  * Zou et al. (2023) Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J. Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J. Zico Kolter, and Dan Hendrycks.  Representation Engineering: A Top-Down Approach to AI Transparency.  _CoRR_ , October 2023.