Loading…
Glossary term
A setting that controls how random a model's output is; lower values are more predictable.
Temperature rescales a model's logits before the softmax. A low temperature makes the most likely tokens even more dominant, giving focused, repeatable output. A high temperature flattens the odds, producing more varied and sometimes odd text. At zero it is effectively greedy decoding.
Applying temperature
import numpy as np
def probs(logits, temperature=1.0):
z = np.array(logits) / temperature
e = np.exp(z - z.max())
return e / e.sum()
print(probs([2.0, 1.0, 0.1], 0.5).round(2)) # sharper
print(probs([2.0, 1.0, 0.1], 2.0).round(2)) # flatterIt is the main dial between reliable and creative output.