Loading…
Glossary term
A regularization method that randomly switches off some units during training so the network does not over-rely on any one of them.
During training, dropout randomly switches off a fraction of units on each pass. The network cannot rely on any single unit, so it learns more robust, spread-out features. At prediction time all units are used.
A different random subset is switched off every step.
Dropout in PyTorch
import torch.nn as nn
layer = nn.Sequential(
nn.Linear(128, 64),
nn.ReLU(),
nn.Dropout(p=0.2), # drop 20% of units while training
)Dropout is a simple, widely used trick that reduces overfitting in neural networks.