Loading…
Glossary term
Work to make AI systems behave in line with human intentions and values.
Alignment is the work of making an AI system's goals and behavior match what its users and society actually intend. That is harder than it sounds, because it is difficult to state precisely what we want, and systems can find unexpected ways to satisfy an instruction.
Practical alignment today includes instruction tuning, preference learning such as RLHF, red teaming and ongoing monitoring.
As systems become more capable and autonomous, the cost of a mismatch between what we ask and what we want grows.