LLM Model Routing: Cost per Successful Workflow
New model pricing makes routing attractive, but the useful metric is cost per successful workflow rather than tokens alone.
Enterprise AI4 min read
The invoice for an AI agent does not end at the model call. An unsuccessful run may consume tools, retries, human review and another model invocation. A fast, cheap response that requires a person to repair the result can cost more than a slower, more capable response that completes the task once. That is why model pricing has to be read in the context of the workflow.
OpenAI released GPT-6.1 Sol on 29 September 2026, positioning it close to its higher-priced Astra model on several coding, computer-use and professional-task evaluations. The published API prices are $2 per million standard input tokens, $0.10 per million cached input tokens and $10 per million output tokens. OpenAI reports favourable cost-per-task results on its selected evaluations; these are vendor-reported measurements, not a guarantee for a particular enterprise workload. Its 2 October model guide recommends matching models and reasoning levels to tasks, using caching and compaction where appropriate, and measuring task success, latency and cost per successful task. Sources: GPT-6.1 Sol announcement and GPT-6 model guide.
That framing is useful for an orchestration platform. A portfolio assistant that classifies incoming work may need speed and predictable structured output. A model drafting a complex migration plan may need deeper reasoning and more context. A production incident agent may need to examine a repository, logs and dependencies, then stop for approval before changing anything. Sending all three jobs to the same model at the same reasoning setting is convenient, but it hides the economics of each step.
A good router begins with the task, not a vendor leaderboard. It should know the required output, the allowed tools, sensitivity of the data, latency target and cost envelope. It can choose a model for the first attempt, validate the result against business rules and escalate only when evidence says the attempt is insufficient. Escalation can mean a stronger model, more reasoning, a different tool path or a human decision. It should never quietly turn a failed validation into a confident answer.
Caching has a strong appeal for repeated enterprise tasks. Stable instructions, schemas and reference context can be reused, lowering input cost. But a cached prompt is valuable only if it still represents current policy and data. A bank’s workflow needs a versioned relationship between the cached context, the source documents and the decision made. Otherwise, an optimization can become a stale rule that is difficult to detect.
The relevant unit of measurement is an end-to-end case. For each workflow, record the proportion completed correctly, time to a usable outcome, model and tool spend, number of retries, human review minutes and severity of errors. Compare routes on representative tasks, including ambiguous and adverse cases. A low token bill with a high review burden is not a win. Neither is a premium model that produces elegant prose but takes the wrong action.
For Archy, this argues for model choice as a policy inside the workflow designer rather than a single tenant-wide configuration. A customer could require one deployment boundary for sensitive banking data, set a cheaper path for routine summaries, and reserve stronger reasoning for architecture decisions or unresolved exceptions. The control plane would record why a route was selected and whether the outcome justified the expense. That positioning is a product inference from the pricing and guidance, not a promise that any particular model will outperform another in Archy’s tasks.
The next step is a small evaluation set drawn from real work: Jira issue analysis, document synthesis, tool-driven updates and a few failure cases. Compare several model routes with the same acceptance criteria. The result may be counterintuitive. That is precisely why the metric must be a completed, defensible job rather than a cheap million tokens.
For the related architecture decisions, see Enterprise AI Total Cost: Cost per Business Outcome; AI Agent Observability with OpenTelemetry.
