ResearchOct 19
ScaleNav: Scalable Internet Video Pretraining for Language-Conditioned Local Navigation
Upvotes84Authors10
Loading…
Glossary term
The first, large-scale training stage in which a model learns general patterns from broad data.
Pretraining is the first and largest training stage. The model reads a huge corpus and learns to predict the next token, absorbing grammar, facts and patterns of reasoning. It consumes most of the compute used to build a model.
Pretraining gives the model its broad knowledge. Later stages shape how that knowledge is used.
ResearchOct 19
Upvotes84Authors10