Skip to content
Feature

Study Reveals Scaling Laws for Merging Large Language Models

Amy Reed · 12 May 2026

Researchers identify empirical scaling laws for language model merging, measured by cross-entropy loss. A compact power law predicts returns from adding experts or scaling model size.

Researchers uncovered empirical scaling laws for language model merging, using cross-entropy to measure performance. They derived a compact power law relating model size to the number of experts added. This establishes the first quantitative predictor for merging returns at scale.

Keep reading, free

Join the My AI Guide email to read the full story, what it means for you and the sources. One email every Monday.

One email every Monday. We will send a quick confirm first, so we know it is really you. Privacy Policy.