Agent Routing Is Becoming a Control-Plane Problem

The smallest question can control the whole workflow

A request arrives: investigate a production alert, answer a customer question, review a pull request, process a claim, or summarize a document. The visible work may take many model calls and tools. But the first decision is often smaller and more consequential: is the request in scope, which route is appropriate, and what level of authority should the system receive? If that first decision is wrong, a capable agent can be sent to the wrong environment, granted unnecessary tools, or kept away from the human who should have reviewed it.

OpenAI's Decisions API is designed for precisely this class of bounded question. It evaluates text, images, or both and returns typed answers that can be used to classify content, select among predefined options, or prioritize work. The service is a public beta and currently supports the gpt-6-luna model through a dedicated decisions endpoint.^[1][2]

This is not a traditional decision-tree builder

A conventional decision tree encodes rules in advance: if a request is in category A, send it to queue B. The value of that approach is determinism. The limitation is that real requests rarely arrive with clean, pre-labeled categories. A model-based decision layer interprets the request and returns a typed answer. That makes it useful for ambiguous language and images, but it also means the result is probabilistic and should be treated as evidence for a route, not as an unquestionable fact.

The Decisions API exposes three answer types. A predicate estimates whether a condition is true. A choice selects from options supplied by the developer and includes confidence. A score evaluates input against named levels or a numeric rubric. The API can ask multiple questions about the same input in one request. That combination makes it natural for tasks such as support triage, request classification, review prioritization, eligibility checks, and selecting the next bounded step in an agent workflow.^[1][2]

1. Predicate: Is a condition met?Use a probability for a bounded question, such as whether an issue describes a security concern or whether a claim includes evidence of damage.
2. Choice: Which approved route applies?Select among a small set of explicit options, such as knowledge-base answer, specialist agent, human review, or reject as out of scope.
3. Score: How should work be prioritized?Assess urgency, quality, or risk against a defined rubric, then use a policy rule to determine the next permitted action.

A route is often an authority decision

It is tempting to frame routing as a cost optimization: send simple work to a smaller model and difficult work to a larger one. That matters, but it is not the whole story. A route can also determine whether an agent is allowed to search private data, call a tool that changes a record, open a browser, run code, or contact a person. The route therefore changes the system's effective authority.

That is why an agent should not be able to translate a low-confidence classifier result directly into a consequential action. A classifier can recommend a path. A separate control layer should enforce which tools, permissions, rate limits, approval requirements, and environment are available on that path. For a meaningful action, the system needs to preserve the difference between a model's interpretation and the policy that authorizes the next step.

• A low-risk request can receive a retrieval-only answer with no write access.
• A request that may affect a customer record can be routed to a drafted recommendation and require human approval before any change.
• A request that looks like a security incident can be routed to a restricted environment, a dedicated response workflow, and a named escalation owner.
• An ambiguous or low-confidence request can be paused, clarified, or sent to review instead of being forced into a confident-looking route.

Record enough evidence to challenge the route later

Routing is easy to overlook because it happens before the work most people notice. That makes it especially important to record. When an agent causes unnecessary cost, misses an escalation, or takes the wrong action, the investigation should not begin with guesswork about why it was given that path. A useful record connects the input, the decision question, the available routes, the confidence or score, the policy threshold, and the outcome of the selected workflow.

Request
  What was the input, task class, owner, and sensitivity level?

Decision
  Which predicate, choice, or score was requested? What answer and confidence came back?

Policy
  Which threshold and routing rule applied? Which routes were permitted or blocked?

Authority
  Which model, tools, data scope, environment, and approval mode did the selected route receive?

Outcome
  Did the task meet an independently checked success criterion? Was the route later corrected, escalated, or reversed?

Measure downstream outcomes, not just model confidence

Confidence is an input to a policy, not a proof that the policy worked. A well-calibrated routing system should be evaluated against the task outcome and against the failure modes that matter to the business. The measurement changes by workflow, but the principle does not: the route needs to earn trust through the quality of the work it enables and the harm it prevents.

• False approvals: how often did a route allow action that should have been reviewed or blocked?
• Missed escalations: how often did a high-risk or complex request receive an insufficient workflow?
• Unnecessary escalation: how often did a safe, routine request consume specialist or human time?
• Verified completion: how often did the selected route produce a result that passed an independent check?
• Recovery quality: when the route was wrong, did the system stop safely, preserve evidence, and direct the request to the right owner?

OpenAI distinguishes the Decisions API from Structured Outputs and function calling. That distinction is useful in architecture reviews. Use a decision when you need a bounded classification, choice, or score. Use Structured Outputs when an application needs a custom data object or explanation. Use function calling when the model needs to request a tool with arguments. Keeping those jobs separate makes a system easier to test, observe, and govern.^[1]

Put the control plane before the autonomy

The next generation of agent systems will not be defined only by how well models reason after work has begun. It will also be defined by how responsibly systems decide what work should begin, where it should run, and what it may touch. A small routing decision can quietly set the cost, scope, and risk of everything that follows.

OpenAI's Decisions API makes this layer easier to build. The engineering responsibility remains the same. Define the routes before deployment. Bound each route's authority. Send uncertainty somewhere safe. Verify the downstream result. Then use the evidence to improve the thresholds and policies, rather than accepting the model's confidence as the final word.^[1][2]