Feature
Study Reveals Scaling Laws for Merging Large Language Models
Amy Reed · 12 May 2026
Researchers identify empirical scaling laws for language model merging, measured by cross-entropy loss. A compact power law predicts returns from adding experts or scaling model size.
Researchers uncovered empirical scaling laws for language model merging, using cross-entropy to measure performance. They derived a compact power law relating model size to the number of experts added. This establishes the first quantitative predictor for merging returns at scale.