How to evaluate LLM output, step by step

You have a prompt running in your product and a nagging doubt about whether the model behind it is the right one. Testing it by hand never settles the doubt, because you only try a few inputs and your read on the answers shifts each time you look.

Scoring the output properly takes five steps, and the last one hands you the model to ship. This post walks through all five on one real task, showing what you put in and what you get back at each step, so you can run the same process on your own.

Step 1: pick a real task

An evaluation is only worth as much as the task you run it on. A toy prompt tells you nothing about how a model behaves on the work your product actually does, so start with a real request your users send.

This is a good task to score because there is a clearly right reply and several ways to get it wrong, which is exactly where a model’s quality shows.

Step 2: write down what a good answer is

Before you run anything, decide what you would accept as a good reply, because without that there is nothing to score against. For a task with a clear right answer, the simplest standard is to write the ideal reply yourself.

Each model’s reply will be scored on how close it comes to this.

Step 3: run the candidates together

Send the same task to several models at once, so their replies land on one scale and you compare them directly instead of judging each in isolation. Here is what three models wrote.

Step 4: read the result

The evaluation scores each reply against your ideal answer and returns the numbers side by side.

DeepSeek V4 Flash scores highest, because it committed to the refund and matched the ideal reply most closely. The other two land close together and lower, because both asked the customer to confirm instead of acting. The score now measures the gap you would have spotted reading them yourself, and it sits next to the cost of each answer, so the choice can account for both.

Step 5: ask for the cheapest answer that is still good enough

The highest score is not automatically the one to ship. DeepSeek is best here, but it costs twice what GPT-4o Mini does, and for a lot of tasks a slightly lower score at half the price is the better call. The way to make that trade-off without eyeballing it is to set a quality bar, the lowest score you would accept for this task, and let the evaluation return the cheapest model that clears it.

The right bar depends on the job. An internal draft only you will read might be fine at 30. A customer-facing reply needs more. A legal or medical answer might not be acceptable below 99. Set the bar for this refund reply to 60 and the result names the pick for you.

All three cleared 60, so the cheapest of them wins, and GPT-4o Mini at a hundredth of a cent is the pick, not the top scorer you would have reached for by instinct. Raise the bar to 70 and the answer changes on its own, because now only DeepSeek is good enough. You move the bar to match how much quality the task really needs, and the model follows.

That is the whole loop. Pick a real task, write down the good answer, run the models together, read the result, and let the quality bar name the cheapest model you can ship. The one judgement you make is what good looks like, and from there the choice is consistent every time you run it, on far more inputs than you could ever check by hand. For the ideas behind each step, and when a rubric beats a written answer, see how to evaluate an LLM.