Glossary term

Regularization

Techniques that discourage a model from becoming too complex, reducing overfitting.

How it works

Regularization adds a pressure that keeps a model from becoming too complex. A common form, L2 regularization, adds a penalty to the loss for large weights, nudging the model toward simpler solutions.

Other forms include dropout, early stopping and data augmentation.

Without and with regularization

Without

Wiggly fit to every point
Large weights
Poor on new data

With

Smoother fit
Weights kept small
Better on new data

In code

L2 regularization added to the loss

l2_strength = 0.01
loss = data_loss + l2_strength * sum((w ** 2).sum() for w in weights)

Why it matters

It is a main defense against overfitting, especially with limited data.

Related terms

Overfitting Dropout Loss function