Loading…
Glossary term
Empirical relationships showing how model performance improves as size, data and compute grow.
Scaling laws are measured relationships showing that a model's error falls in a predictable, power-law way as you increase model size, data and compute. The gains are steady, but each doubling buys a smaller improvement.
Labs use them to forecast how a big, expensive run will perform before starting it.
They guided the move to ever larger models and let companies plan multi-million-dollar training runs.