What are evals in AI?
Evals are repeatable tests that score an AI system's outputs against known-good answers or criteria, so you can measure quality and catch regressions before changes ship.
In more detail
An eval set is a collection of real inputs with the answer you expect, plus a scoring method: exact match for fields, a rubric for text, or a second model acting as judge with spot checks by a person. Run it whenever the prompt, model, retrieval or tools change. Without evals you find regressions through unhappy users. Build the set from real cases, include the awkward ones (skewed scans, odd formats, missing fields) and hold some back so you do not tune to the test. Report accuracy per field and per document type, not one headline number. A 97 percent average can hide 60 percent on the one supplier that matters.
Example
Before an invoice extraction system goes live, a team scores it on 300 past invoices with hand-verified fields. They track total, tax, PO number and line-item accuracy separately and set a pass threshold for each.