Learning to Read Out
Unembedding Dynamics in Language Model Pretraining
2026
A model’s outputs can lag behind what it already represents.
Evaluations usually judge a model by what it says. But during training, everything it says passes through a readout that is still being learned, and that is the confound we identify: it’s like reading a ruler while the markings are still being drawn. Probing hidden states directly, we found task information there before the model’s answers showed it, and on harder tasks the gap never fully closed. If you’re watching outputs for early warning signs, you may find out late, or not at all.