Loading…
Glossary term
Reinforcement learning from human feedback: tuning a model using human preference judgments.
Reinforcement learning from human feedback has three steps. First, a model is fine-tuned on good examples. Second, people compare pairs of its answers and a reward model learns to predict which they prefer. Third, the model is trained with reinforcement learning to produce answers that score highly with the reward model.
It is a major reason chat assistants feel helpful and polite.
RLHF turned raw language models into usable assistants, though it can also encourage flattery.