Drift

In how to evaluate LLM output we showed that picking a model is a trade-off you control. You set a quality bar for the task and take the cheapest model that still clears it, and sometimes the fastest one too when your pipeline is serving users live. You evaluated your models that way, picked one, and shipped it.

That score was true the day you measured it, but it does not stay true. The company behind the model updates it without telling you, someone edits the prompt, a document the model reads from changes, and the quality you signed off on slips without a single alarm going off.