Sourced explainer
The test champion still lost to your printer
A medal in one exercise does not describe an entire working day.
At an imaginary championship the AI solves immaculate puzzles. Then somebody hands it a crooked invoice, an ambiguous instruction and a file called “final-really-3”. The printer watches. It knows this competition already.
An evaluation depends on the tasks and conditions selected. NIST recommends documenting generalisation limits and avoiding broad capability claims based on narrow or anecdotal observations. A result can inform without predicting a different use.
When reading a score, look for the task, the definition of success and errors concealed by an average. For your own comparison, we suggest keeping representative examples, awkward cases included, and recording the corrections required. A successful demonstration remains an example, not a guarantee.
Comedy starts when the trophy replaces the work: the robot lays its medal on the scanner and waits for it to do the rest. To return to reality, remove the medal and describe a specific task. Less spectacular, but easier to check.
Sources without clown shoes : NIST AI 600-1 — Generative AI Profile
