Glossary term

Softmax

A function that turns a list of scores into probabilities that sum to one.

How it works

Softmax turns a list of raw scores into probabilities. It exponentiates each score and divides by the total, so every output is between 0 and 1 and they add up to 1.

Larger scores get disproportionately more probability, and a temperature setting can sharpen or flatten the result.

Softmax of scores [2.0, 1.0, 0.1], as probabilities

Class A66%
Class B24%
Class C10%

In code

Softmax

import numpy as np

def softmax(x):
    e = np.exp(x - np.max(x))   # subtract max for numerical stability
    return e / e.sum()

print(softmax(np.array([2.0, 1.0, 0.1])).round(2))  # [0.66 0.24 0.1 ]

Why it matters

Softmax is how a model converts its final scores into a probability for each possible next token or class.

Related terms

Logits Temperature Attention mechanism