Loading…
Glossary term
The set of tokens a model can read and produce.
A model's vocabulary is the fixed list of tokens it can read and write, commonly between tens of thousands and a few hundred thousand entries. Each has an ID and a learned embedding.
A larger vocabulary shortens text into fewer tokens but increases the size of the embedding and output layers.
| Token | ID | |
|---|---|---|
| Entry 1 | the | 262 |
| Entry 2 | cat | 9246 |
| Entry 3 | ing | 278 |
The vocabulary shapes how efficiently a model handles different languages and kinds of text.