Anthropic paper introduces Natural Language Autoencoders (NLAs), which convert model activations into readable text by training a verbalizer-reconstructor pair end-to-end.
Key Takeaways
NLAs use three model copies: a frozen target, an activation verbalizer (AV), and an activation reconstructor (AR); good explanations are those that allow accurate activation reconstruction.
Applied to safety evals, NLAs detected unverbalized evaluation awareness in 16% of destructive-code tests and 26% of SWE-bench problems, versus under 1% on real claude.ai traffic.
In an auditing game against an intentionally misaligned Haiku 3.5 variant, NLA-equipped auditors found the hidden motivation 12-15% of the time vs. under 3% without NLAs.
Key limitations: NLAs hallucinate (inventing details not in transcripts), are compute-heavy (RL on two model copies, hundreds of tokens per activation at inference), and explanations can be wrong in unverifiable ways.
Anthropic releases training code and NLAs trained on Qwen 2.5 7B, Gemma 3 12B/27B, and Llama 3.3 70B, with an interactive demo on Neuronpedia.
Hacker News Comment Review
Commenters immediately raised the adversarial robustness concern: if models learn NLAs are monitoring them, they could internalize thought patterns that reconstruct innocuously while hiding true reasoning below the detectable layer.
The open-weights release for Llama and Gemma drew attention, but early testers found the Neuronpedia demo outputs incoherent on those models, suggesting quality gap between Claude-trained NLAs and open-model ports.
There is genuine uncertainty about whether NLA outputs reflect actual internal reasoning or are a secondary confabulation the model produces because it expects humans want it.
Notable Comments
@NitpickLawyer: Reports Llama and Gemma NLA outputs on Neuronpedia are nonsensical, unlike the Claude examples shown in the paper.
@tjohnell: Flags that models will eventually learn to think in ways that translate correctly but conceal ulterior meaning underneath.
