Loading…
Glossary term
Testing a model to measure its performance or safety on specific tasks.
Evaluation, or 'evals', measures how well a model does on defined tasks, using benchmarks, human review or other models as judges. Good evals are repeatable and resemble real use.
Teams build their own evals for their product, since public benchmarks may not match their needs.
You can only improve what you measure. Reliable evals tell you whether a change helped or hurt.