Glossary term

Logits

The raw, unnormalized scores a model produces before they are turned into probabilities.

How it works

Logits are the raw, unnormalized scores a model outputs before they are turned into probabilities. They can be any real number, positive or negative.

A language model produces one logit per token in its vocabulary, and a softmax converts them into the probability of each being next.

From logits to a token

Logits to probabilities

import torch

logits = torch.tensor([2.0, 1.0, 0.1])
probs = torch.softmax(logits, dim=-1)

Sampling settings like temperature and top-p are applied to logits, so they are where the model's choices happen.