Loading…
Glossary term
Reusing a model trained on one task as the starting point for a different, related task.
Instead of training from scratch, you begin with a model already trained on a large, general task and adapt it to yours. The early layers have already learned broadly useful patterns, so you often need far less data and compute.
Fine-tuning a pretrained language model is a form of transfer learning.
Reuse a pretrained image model
import torch.nn as nn
from torchvision import models
m = models.resnet18(weights="DEFAULT") # pretrained
for p in m.parameters():
p.requires_grad = False # freeze what it already knows
m.fc = nn.Linear(512, 10) # new head for your 10 classesIt is why small teams can build capable systems without frontier-scale budgets.