Loading…
Glossary term
A neural network made of several fully connected layers of units.
A multilayer perceptron stacks several fully connected layers with non-linear activations between them. Every unit in one layer connects to every unit in the next.
Adding hidden layers lets it learn patterns a single perceptron cannot.
An MLP in PyTorch
import torch.nn as nn
mlp = nn.Sequential(
nn.Linear(3, 5), nn.ReLU(),
nn.Linear(5, 5), nn.ReLU(),
nn.Linear(5, 2),
)MLPs are a basic building block, and the feed-forward part of every transformer is one.