Loading…
Glossary term
A decoding method that picks the next token from only the k most likely candidates.
Top-k keeps only the k most likely tokens, renormalizes their probabilities and samples among them. With k = 1 it is greedy decoding.
It is simple, but a fixed k can be too loose when the model is confident or too tight when it is not.
Top-k filtering
import numpy as np
def top_k(probs, k=3):
keep = np.argsort(probs)[-k:]
out = np.zeros_like(probs)
out[keep] = probs[keep]
return out / out.sum()It is one of the standard controls on how adventurous generation is.