Loading…
Glossary term
Research into understanding how a model's internal workings produce its behavior.
Interpretability research tries to understand what is happening inside a model: what its internal components represent and how they combine to produce an output.
It ranges from simple visualizations to detailed reverse engineering of circuits.
If we can see why a model does something, we can better trust, debug and align it.