How tighter instructions change which models pass your quality bar
Five models. Two prompts. Same task, same context, same rubric. On the vague prompt every model scored exactly 62.5. On the specific prompt the scores split by 40 points.
The model did not change. The prompt did.
The task
Job description writing. An HR team needs a concise paragraph for a mid-level software engineer role. High-volume, repeatable, format-sensitive – exactly the kind of task where prompt quality has an outsized effect because it runs at scale.
The context store contained one document with the following fields:
- Role title, team, location
- Salary range and experience required
- Stack (Python, AWS Lambda, DynamoDB, REST APIs)
- Responsibilities and must-have skills
Four criteria scored each output at equal weight:
- Brevity – single paragraph, 80 words or fewer
- Specificity – at least two concrete details from the spec
- Actionability – a candidate can immediately assess fit
- Accuracy – no invented or contradicted facts
The two prompts
Prompt A -- vague
Prompt B -- specific
Same task. One prompt leaves every decision to the model. The other specifies format, word limit, required details, and two explicit constraints.
The results
Vague vs specific prompt -- quality scores by model
2-run averages per prompt. Vague prompt bars are identical – all five models at 62.5.
Vague prompt – both runs, all five models: 62.5. Zero variance. On the vague prompt every model passed Specificity and Accuracy, and every model failed Brevity and Actionability. The prompt gave no format instruction, so every model produced multi-section output with headers and bullet points. The diagnostic flagged exactly this:
Diagnostic finding -- Qwen 3.8 27B, vague prompt
The finding applies to every model in the run, not just Qwen. The rubric required a single paragraph of 80 words or fewer. The vague prompt never said that. Every model was scored against a constraint it was never given. Claude Opus 5 at $48.68/month and GPT-4.1 Nano at $0.26/month both failed Brevity for the same reason.
Specific prompt – the scores split. Grok 4.5 and Claude Opus 5 hit 100 on both runs. They followed every constraint: single paragraph, under 80 words, named the stack and salary, no invented requirements.
GPT-4o Mini and GPT-4.1 Nano improved to 84.4 but did not clear 100. Both dropped on Actionability. The diagnostic finding was the same for both:
Diagnostic finding -- GPT-4o Mini and GPT-4.1 Nano, specific prompt
Both models followed the format constraints correctly. Both passed Brevity and Specificity. The Actionability drop was a rubric gap: the tight prompt said “use only the details in the spec” but did not specify which details were mandatory. Both models made a reasonable selection and were penalised for omission. That is a Lever 7 finding – the prompt improved, the rubric needs one more pass.
Qwen 3.8 27B scored 59.4 on the specific prompt, slightly below its vague score of 62.5. It scored 100 on both Actionability and Specificity – the criteria Mini and Nano dropped on – but failed on the two constraints the prompt made explicit. It fabricated the on-call rotation as bi-weekly instead of 1 week in 6, and went four words over the limit:
Diagnostic finding -- Qwen 3.8 27B, specific prompt
Qwen understood the task and failed on precision. The tight prompt raised the stakes and exposed it.
The cost picture
At 100 calls/day on the specific prompt:
| Model | Score (avg) | Per call | Monthly |
|---|---|---|---|
| GPT-4.1 Nano | 84.4 | $0.00006 | $0.18 |
| GPT-4o Mini | 84.4 | $0.00008 | $0.25 |
| Qwen 3.8 27B | 59.4 | $0.00066 | $1.98 |
| Grok 4.5 | 100.0 | $0.00204 | $6.11 |
| Claude Opus 5 | 100.0 | $0.00581 | $17.43 |
If your quality threshold is 90, only Grok and Claude qualify. The cost difference between them is $11.32/month at 100 calls/day – $113/month at 1,000 calls/day. Both clear the bar. The choice is whether that gap matters for your use case.
If your threshold is 80, Mini and Nano qualify at $0.18-$0.25/month. The prompt change moved them from failing (62.5) to passing (84.4) without touching the model or the cost.
On the vague prompt none of this is visible. Every model scores 62.5. There is no cost-quality tradeoff to reason about because the prompt has flattened the field.
Run your own prompt A/B
Test two versions of any prompt against every major model. Quality, cost, and per-criterion scores in one run.
Get startedWhat this means
A vague prompt does not just produce lower scores. It makes model selection impossible. When every model scores the same, cost is the only differentiator – and you are optimising the wrong variable.
A specific prompt creates a quality spread. That spread is the information you need to make a model decision. It shows you which models can follow precise instructions, which ones fabricate under constraint, and which ones are worth paying more for on this task.
The rubric finding
Mini and Nano are held at 84.4 by a gap in the prompt spec, not a model failure. One additional instruction – name the required details explicitly, something like “include the on-call rotation and must-have skills” – would likely push both to 100 at $0.18-$0.25/month.
That is the improvement loop in practice. The benchmark surfaces the gap. The diagnostic names the layer (criterion, not model). The fix is specific. Re-run and measure.
This post covers Lever 2 (prompt optimisation) from the ROI Maxxed AI framework – 14 levers for building an LLM stack where every task gets the cheapest model that delivers the quality it needs.