Glossary term

Epoch

One full pass through the training dataset.

How it works

One epoch is a complete pass through the training dataset. Training usually runs for many epochs, with the model seeing every example repeatedly.

Very large language models are often trained for only about one pass over huge datasets, while smaller models may use dozens of epochs.

Each epoch is a full pass over the data

In code

Epochs and batches

for epoch in range(10):               # 10 passes over the data
    for x, y in train_loader:         # one batch at a time
        step(x, y)
    print("epoch", epoch, "done")

Why it matters

Too few epochs underfit and too many can overfit, so epoch count is a key training setting.

Related terms

Batch size Training Overfitting