Anthropic’s Interpretability Research

A Toy Model of Mechanistic (Un)Faithfulness

When transcoders go awry.

Tracing Attention Computation Through Feature Interactions

Kamath et al., 2025

We describe and apply a method to explain attention patterns in terms of feature interactions, and integrate this information into attribution graphs.

A Toy Model of Interference Weights

Unpacking "interference weights" in some more depth.

Sparse mixtures of linear transforms

We investigate sparse mixture of linear transforms (MOLT), a new approach to transcoders.

Circuits Updates — July 2025

A collection of small updates: revisiting A Mathematical Framework and applications of interpretability to biology.

Automated Auditing

A note on using agents to perform automated alignment audits, including using interpretability tools.

Circuits Updates — April 2025

A collection of small updates: jailbreaks, dense features, and spinning up on interpretability.

Progress on Attention

An update on our progress studying attention.

On the Biology of a Large Language Model

Lindsey et al., 2025

We investigate the internal mechanisms used by Claude 3.5 Haiku — Anthropic's lightweight production model — in a variety of contexts.

Circuit Tracing: Revealing Computational Graphs in Language Models

Ameisen et al., 2025

We describe an approach to tracing the "step-by-step" computation involved when a model responds to a single prompt.

Insights on Crosscoder Model Diffing

A preliminary note on using crosscoders to diff models.

Circuits Updates — January 2025

A collection of small updates: dictionary learning optimization techniques.

Stage-Wise Model Diffing

A preliminary note on model diffing through dictionary fine-tuning.

Sparse Crosscoders for Cross-Layer Features and Model Diffing

A preliminary note on a way to get consistent features across layers, and even models.

Using Dictionary Learning Features as Classifiers

A preliminary note comparing feature-based and raw-activation based harmfulness classifiers.

Circuits Updates — September 2024

A collection of small updates: investigating successor heads, oversampling data in SAEs.

Circuits Updates — August 2024

A collection of small updates: interpretability evals, reproducing self-explanation.

Circuits Updates — July 2024

A collection of small updates: five hurdles, linear representations, dark matter, pivot tables, feature sensitivity.

Circuits Updates — June 2024

A collection of small updates: topk and gated SAE investigation.

Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet

Templeton et al., 2024

Using a sparse autoencoder, we extract a large number of interpretable features from Claude 3 Sonnet. Some appear to be safety-relevant.

Circuits Updates — April 2024

A collection of small updates from the Anthropic Interpretability Team.

Circuits Updates — March 2024

A collection of small updates from the Anthropic Interpretability Team.

Reflections on Qualitative Research

Some opinionated thoughts on why interpretability research may have qualitative aspects be more central than we're used to in other fields.

Circuits Updates — February 2024

A collection of small updates from the Anthropic Interpretability Team.

Circuits Updates — January 2024

A collection of small updates from the Anthropic Interpretability Team.

Towards Monosemanticity: Decomposing Language Models With Dictionary Learning

Bricken et al., 2023

Using a sparse autoencoder, we extract a large number of interpretable features from a one-layer transformer.

Circuits Updates — July 2023

A collection of small updates from the Anthropic Interpretability Team.

Circuits Updates — May 2023

A collection of small updates from the Anthropic Interpretability Team.

Interpretability Dreams

Our present research aims to create a foundation for mechanistic interpretability research. In doing so, it's important to keep sight of what we're trying to lay the foundations for.

Distributed Representations: Composition & Superposition

An informal note on how "distributed representations" might be understood as two different, competing strategies — "composition" and "superposition" — with quite different properties.

Privileged Bases in the Transformer Residual Stream

Our mathematical theories of the Transformer architecture suggest that individual coordinates in the residual stream should have no special significance, but recent work has shown that this observation is false in practice. We investigate this phenomenon and provisionally conclude that the per-dimension normalizers in the Adam optimizer are to blame for the effect.

Superposition, Memorization, and Double Descent

Henighan et al., 2023

We have little mechanistic understanding of how deep learning models overfit to their training data, despite it being a central problem. Here we extend our previous work on toy models to shed light on how models generalize beyond their training data.

Toy Models of Superposition

Elhage et al., 2022

Neural networks often seem to pack many unrelated concepts into a single neuron - a puzzling phenomenon known as 'polysemanticity'. In our latest interpretability work, we build toy models where the origins and dynamics of polysemanticity can be fully understood.

Softmax Linear Units

An alternative activation function increases the fraction of neurons which appear to correspond to human-understandable concepts.

Mechanistic Interpretability, Variables, and the Importance of Interpretable Bases

An informal note on intuitions related to mechanistic interpretability.

In-Context Learning and Induction Heads

Olsson et al., 2022

An exploration of the hypothesis that induction heads are the primary mechanism behind in-context learning. We also report the existence of a previously unknown phase change in transformers language models.

paper

A Mathematical Framework for Transformer Circuits

Elhage et al., 2021

Our early mathematical framework for reverse engineering models, demonstrated by reverse engineering small toy models.

paper

Exercises

Some exercises we've developed to improve our understanding of how neural networks implement algorithms at the parameter level.

note, exercises

Videos

Very rough informal talks as we search for a way to reverse engineering transformers.

links, videos

PySvelte

One approach to bridging Python and web-based interactive diagrams for interpretability research.

github link, infrastructure

Garcon

A description of our tooling for doing interpretability on large models.

note, infrastructure

Original Distill Circuits Thread

Our exploration of Transformers builds heavily on the original Circuits thread on Distill.