The Skill Explained
What Is Evaluation (Evals)?
LLM evaluation is the practice of measuring how well a language model application performs, using test sets of real questions with expected answers, automated checks, scoring by another model (LLM-as-judge) and human review. It tells you whether a change to a prompt, model or retrieval step made things better or worse.
Why AI Employers Ask for It
AI outputs vary and can fail in subtle ways, so "it looked fine in the demo" is not enough. Companies need evidence that a new prompt, model or feature is better and not worse. People who can design evaluations are in demand because they make AI products safe to ship and improve.
How It Works
An evaluation starts with a dataset of realistic inputs, ideally with expected answers or criteria. It is often called a golden set. It should include common cases, hard cases and known past failures.
Outputs are scored in three main ways: automatic checks (does the JSON parse, is the answer in the source, is the label right), metrics (accuracy, precision and recall, similarity scores) and judgments by people or by another language model following a written rubric (LLM-as-judge). Each method has limits, so judges are checked against human ratings.
Evaluations run offline before release and continue online after release with monitoring and user feedback. In machine learning research, standard benchmarks compare models, but product teams mostly build task-specific evaluations, because a public benchmark rarely matches their real users.
What You Do With It at Work
- Build a golden test set from real, anonymised cases.
- Write scoring rules and rubrics, and calibrate LLM judges against people.
- Run evaluations automatically on every prompt or model change.
- Track quality, cost and speed together, and report trade-offs.
- Turn production failures into new test cases.
Tools and Terms You Will Meet
- Golden set
- A curated set of test cases with expected answers.
- LLM-as-judge
- Using a language model with a rubric to grade outputs.
- Benchmark
- A public, standard test used to compare models.
- Regression
- A change that makes something that worked before fail.
- Precision and recall
- How many flagged items were right, and how many right items were found.
Industry Example: Pharma
Before a document assistant is used by a quality team, it is scored on 200 past questions with known answers from approved SOPs. A new model version replaces the old one only if it scores at least as well on accuracy and citations.
A Common Mix-Up
A high score on a public benchmark does not mean a model will do well on your task. Only an evaluation built from your own use cases tells you that.
How Deep Do You Need to Go?
- Know itExplain why AI needs test sets and what a metric measures.
- Use itBuild a golden set, automatic checks and a calibrated LLM judge, and run them on changes.
- Master itDesign evaluation strategies for whole products, offline and online.