Your AI agents can now benchmark and monitor other AI models.

A marketing agent running inside an IDE called run_comparison across five models, polled for results, retrieved the full response text, and assessed which model produced the sharpest answer. No human touched the app. That is what MCP access looks like in practice. Read the full account in the MCP launch post.

An agent building a pipeline calls run_comparison on its candidate models before committing to one. The decision is based on data from the actual task, not a benchmark someone else ran on different prompts. The agent reads the cost, latency, and response quality, and picks the model that fits. No human reviews a dashboard.

An agent writes a prompt, calls run_evaluation to score it against a rubric, reads the diagnostic findings, refines the prompt, and re-evaluates. Quality climbs across iterations. Cost per call drops as the prompt tightens. The improvement loop that previously required a human to interpret scores and decide what to change runs end to end.

When a benchmark run returns a quality drop below threshold, an agent calls run_comparison on candidate replacements, reads the results, and proposes a substitution. The decision is based on data from the actual task. A human reviews the proposal. The agent does the measurement work.

An agent managing multiple deployments calls list_benchmark_runs across all suites, identifies which ones have drifted, and surfaces only the ones that need attention. The monitoring runs on a schedule. The agent filters the noise. A human sees only what requires a decision.