Loading…
Glossary term
A setting chosen by the developer before training, such as learning rate, rather than learned from data.
Hyperparameters are settings chosen before training, such as learning rate, batch size, number of layers and dropout rate. Parameters, by contrast, are learned from the data.
They are often tuned by trying many combinations and comparing results on a validation set.
A hyperparameter configuration
config = {
"learning_rate": 3e-4,
"batch_size": 64,
"num_layers": 12,
"dropout": 0.1,
"epochs": 5,
}Good hyperparameters can be the difference between a model that works and one that does not.