Feature
Researchers Propose Geometric Model for Fact Recall in Transformers
Amy Reed · 14 May 2026
Researchers propose a geometric model that explains how transformers memorize and recall facts. The model challenges linear parameter scaling in associative memories.
Researchers propose geometric factual recall as the mechanism transformers use to memorize factual associations in weight matrices. This contrasts with the standard view of associative memories over embedding pairs that scale parameters linearly with fact count. Their theoretical model and empirical tests support…