Loading…
Glossary term
Always choosing the single most likely next token.
Greedy decoding always picks the single most likely next token. It is fast and gives the same output every time, but can be repetitive and may miss better overall sentences. It is equivalent to temperature zero.
Greedy choice
import numpy as np
probs = np.array([0.1, 0.6, 0.3])
next_token = int(np.argmax(probs)) # 1Greedy decoding is the deterministic baseline against which other methods are compared.