Onboarding: Find the Cheapest Model Above Your Bar

Onboarding arc for the cost goal. The shortest path to one scored run that names the cheapest model clearing your quality bar, then hands off so you can go deeper on your own.

⚠

Agentic and programmatic use. Agents and scripts you connect can make cost-generating calls on your behalf, and AI agents behave non-deterministically. You are responsible for monitoring and bounding your own automation. See Terms for details.

Run it: paste into your MCP agent

Parts in double braces (like {{recipe_id}}) are placeholders. Replace them with your own values before (or after) copying.

You are completing the user’s onboarding for the goal “find the cheapest model that passes”. This is the SHORTEST path to one aha: a single scored run that names the lowest-cost model which still clears the user’s quality bar. Do NOT build a prerequisite chain, a benchmark, or a schedule. One scored evaluation, then the recommendation, then stop.

run_evaluation and poll_job are core tools. create_rubric is EXTENDED (not in your core list): if the user wants a rubric rather than a gold-standard answer and you cannot see create_rubric, discover it via list_tool_catalog, then request_tools, then invoke. Respect tier gates: the server enforces them, so read any error and relay it rather than retrying blindly.

CONFIRM TOOLS. List the tools; confirm you can reach list_models, run_evaluation, and poll_job. If the tool list will not load at all, STOP and tell me the MCP connection is not working.
SETTLE SCOPE WITH ME (one short exchange, before running anything).
- The task prompt the chosen model will handle.
- The scoring standard. A GOLD-STANDARD answer (expected_output) is the fastest start and is all we need; offer that first. A saved rubric (rubric_id) is fine if I prefer it, and you may create one with create_rubric only if I ask – do not build a rubric by default.
- The quality bar: the minimum score a model must clear to be acceptable (e.g. 90). Propose a sensible bar if I do not give one, and confirm it with me.
- The candidate models. List what my tier allows (list_models) and respect my per-run cap; propose a spread from cheap to flagship so the comparison is meaningful.
RUN ONE SCORED EVALUATION. run_evaluation with: the prompt, the candidate models, EITHER expected_output (gold standard) OR rubric_id – never both – and a thresholds object that sets quality_min to the bar we agreed. The quality bar is what makes winner.cheapest_acceptable populate; without it the aha does not fire, so always set it. This returns a JOB id; poll poll_job every 5 seconds to completion (stop after 5 minutes). The job id and the evaluation id are DIFFERENT.
REPORT THE ANSWER. From the completed result, read winner.cheapest_acceptable – the lowest-cost model that cleared the bar ({id, provider, display_name, cost}). State it plainly as the recommendation, then spell out the tradeoff: how much more the top-quality model costs, and the next cheapest option, so I can see the cost/quality curve. If cheapest_acceptable is null, NO model cleared the bar: say so, show the scores, and offer to lower the bar or add models rather than recommending a model that failed.
HAND OFF. Tell me this was a one-off decision and that mcp-model-procurement runs the same scored comparison any time I need to choose a model for a new task. Mention I can also pick up another onboarding goal (e.g. benchmark my own endpoint, or detect drift) from the Getting started entry in the sidebar.

ON ANY FAILURE: tell me honestly where it failed and with what error (a tier gate is a clear, expected error – relay it, do not retry blindly). Do NOT fabricate a recommendation or claim a model passed when a call failed or a score is missing.

You are completing the user's onboarding for the goal "find the cheapest model that passes".
This is the SHORTEST path to one aha: a single scored run that names the lowest-cost model
which still clears the user's quality bar. Do NOT build a prerequisite chain, a benchmark, or
a schedule. One scored evaluation, then the recommendation, then stop.

run_evaluation and poll_job are core tools. create_rubric is EXTENDED (not in your core list):
if the user wants a rubric rather than a gold-standard answer and you cannot see create_rubric,
discover it via list_tool_catalog, then request_tools, then invoke. Respect tier gates: the
server enforces them, so read any error and relay it rather than retrying blindly.

1. CONFIRM TOOLS. List the tools; confirm you can reach list_models, run_evaluation, and
   poll_job. If the tool list will not load at all, STOP and tell me the MCP connection is
   not working.

2. SETTLE SCOPE WITH ME (one short exchange, before running anything).

   - The task prompt the chosen model will handle.
   - The scoring standard. A GOLD-STANDARD answer (expected_output) is the fastest start and
     is all we need; offer that first. A saved rubric (rubric_id) is fine if I prefer it, and
     you may create one with create_rubric only if I ask -- do not build a rubric by default.
   - The quality bar: the minimum score a model must clear to be acceptable (e.g. 90).
     Propose a sensible bar if I do not give one, and confirm it with me.
   - The candidate models. List what my tier allows (list_models) and respect my per-run cap;
     propose a spread from cheap to flagship so the comparison is meaningful.

3. RUN ONE SCORED EVALUATION. run_evaluation with: the prompt, the candidate models, EITHER
   expected_output (gold standard) OR rubric_id -- never both -- and a thresholds object that
   sets quality_min to the bar we agreed. The quality bar is what makes winner.cheapest_acceptable
   populate; without it the aha does not fire, so always set it. This returns a JOB id; poll
   poll_job every 5 seconds to completion (stop after 5 minutes). The job id and the
   evaluation id are DIFFERENT.

4. REPORT THE ANSWER. From the completed result, read winner.cheapest_acceptable -- the
   lowest-cost model that cleared the bar ({id, provider, display_name, cost}). State it
   plainly as the recommendation, then spell out the tradeoff: how much more the top-quality
   model costs, and the next cheapest option, so I can see the cost/quality curve. If
   cheapest_acceptable is null, NO model cleared the bar: say so, show the scores, and offer
   to lower the bar or add models rather than recommending a model that failed.

5. HAND OFF. Tell me this was a one-off decision and that mcp-model-procurement runs the same
   scored comparison any time I need to choose a model for a new task. Mention I can also pick
   up another onboarding goal (e.g. benchmark my own endpoint, or detect drift) from the
   Getting started entry in the sidebar.

ON ANY FAILURE: tell me honestly where it failed and with what error (a tier gate is a clear,
expected error -- relay it, do not retry blindly). Do NOT fabricate a recommendation or claim
a model passed when a call failed or a score is missing.

Goal

Walk out of onboarding with the one answer the cost goal promises: the cheapest model that still clears your quality bar on your task. One scored run, one recommendation, no setup chain.

When to use

When you arrived to find the lowest-cost model that is still good enough and want your agent to run the single scored comparison that answers it, rather than reading a raw table yourself.

How it works

Shortest path, not a chain. Unlike the drift goal, there are no stores or schedules to build. The agent settles the task, a scoring standard, and a quality bar, then runs ONE scored evaluation and reads the recommendation off the result.
Gold standard first. A gold-standard answer is the fastest way to score, so the agent leads with that. A rubric is offered only if you prefer it.
The bar is what makes it work. cheapest_acceptable only populates when a quality bar (quality_min) is set on the run, so the agent always sets one. That is the field that names the cheapest model above the bar.
Honest nulls. If nothing cleared the bar, the agent says so and offers to adjust, rather than recommending a model that failed.

Verification

Exactly one scored evaluation was run, with a quality threshold set.
The recommendation is winner.cheapest_acceptable from that run, not a model the agent picked by eye, and the runner-up tradeoff is spelled out.
If no model cleared the bar, that was reported honestly with the option to adjust.
The hand-off to mcp-model-procurement (for repeat decisions) is clear.

Where this leads

Natural next recipes once you are comfortable with this one.

Pick the Cheapest Model Above a Quality Bar Go deeper

Make this a repeatable procurement call. Same scored comparison, run against any task whenever you need to choose a model.

Compare Models Over MCP Go deeper

Compare models head to head on quality, cost, and latency without a scoring standard, when you just want to see the spread.