There is a version of “AI-powered workflow” that demos beautifully and cannot be put into production. It looks like this: a request arrives, a language model reads it, decides what should happen next, and the process does that.
It is genuinely impressive in a sales meeting. It is also, in most regulated enterprise contexts, unshippable — and the reason has nothing to do with how good the model is.
The question you cannot answer
Six months after go-live, someone asks why claim 88213 was routed to enhanced review while claim 88214, which looks the same, was not.
If a deterministic rule made that decision, the answer is a lookup. You retrieve the flow version that was live that day, read the condition, and show that the two claims differed on a field. The conversation lasts four minutes.
If a model made that decision, the honest answer is “the model weighed the inputs differently.” You cannot reproduce it exactly — sampling, model version, and context all moved. You cannot show a rule, because there wasn’t one. In an audit, a compliance review, or a dispute, that is not an answer. It is the absence of one.
This is the core issue, and it survives every improvement in model quality. A more capable model produces better decisions; it does not produce reproducible ones, and reproducibility is what the audit is asking for.
Design time and run time are different problems
The useful reframing is to stop asking “where does AI go in this process?” and ask instead which of two very different jobs you want it to do.
Design time is authorship. Someone is building or changing a workflow. The work is interpretive, tolerant of iteration, and — critically — every output is reviewed by a human before it has any effect. Getting it slightly wrong costs an undo.
Run time is execution. A real case is moving through a real process with real consequences. The work must be deterministic, reproducible, and explainable months later. Getting it wrong costs a customer, a fine, or a court date.
These have opposite requirements. Design time rewards flexibility and judgement; run time punishes both. The mistake is treating “AI in the workflow” as one decision instead of two.
The rule: no model in the run path
Our default is blunt and has held up well: the model authors the process; it never executes it.
At design time, an assistant can do genuinely valuable work. Describe a process in a sentence and have it drafted on the canvas. Ask what happens when a step times out and have the gap identified. Ask it to add a parallel audit branch and watch it wire the split and join correctly. Every one of those outputs lands in a draft that a human inspects, edits and publishes.
At run time, the router executes a frozen snapshot of that published flow using a deterministic condition language. Given the same run data, it produces the same routing decision today, next quarter, and in the post-incident review. There is no sampling, no model version, no temperature.
The value is not diminished by this boundary. Authoring a workflow is the expensive, skill-gated bottleneck — that is where help is worth paying for. Evaluating amount > 5000 was never the hard part.
Making the boundary structural
Stated as a principle, “no model in the run path” erodes. Someone will need one exception, and the exception will be reasonable, and then there will be a second one. The boundary has to be enforced by structure rather than intention.
Give the assistant a tool surface, not an API
The assistant should not manipulate application state directly. It should call named commands with schema’d payloads — add_node, connect, update_node, set_variables, validate_flow — the same surface the user interface uses. Two things follow. First, the assistant cannot do anything the tool surface cannot express, so its blast radius is defined by design rather than by prompt. Second, if there is no runtime command in the registry, there is no way for the assistant to reach the run path even if it tries.
Make tool calls visible and reversible
Render every command the assistant invokes, with its payload, in the interface. Not a summary — the actual call. Users trust an assistant they can watch, and they catch its mistakes far earlier. Snapshot before each turn so any change can be reverted in one action.
Route model output through the same validation as human input
An assistant-authored flow gets no privileges. It passes the identical publish gate: well-formed graph, schemas satisfied, loops guarded, no unbound tasks. If the model produces something invalid, validation catches it exactly as it would catch a person’s mistake. This is also the honest answer to “what if the AI makes something wrong” — the gate does not care who authored it.
Keep one swap seat
Everything the application knows about interpreting a request should live behind one function. Which implementation sits behind it — a hosted model, a local one, a deterministic phrase parser — is decided in exactly one place. This keeps the model from spreading through the codebase, and it means the product still works when there is no API key, no network, or no budget.
What this buys you
An architecture where the model is confined to design time has a property that matters more than any benchmark: you can explain it to an auditor in one sentence. “The AI helps our administrators draw the process. It has no role in executing it, and here is the frozen version that ran.”
That sentence is worth more than a percentage point of routing accuracy. It is the difference between a pilot that impresses people and a system that is allowed to touch production.
It also ages well. Models will keep improving, and the authoring assistant will keep getting better essentially for free. Meanwhile the run path — the part with legal and financial consequences — stays exactly as deterministic as it was on day one.
Where the boundary gets argued
Stating the rule is easy. It gets tested in four specific places, and it is worth deciding each in advance rather than in the moment.
“The model is only making a suggestion”
A suggestion that is accepted by default is a decision. If the interface pre-selects the model’s recommendation and the human clicks through, the model routed the case and the audit trail says a person did — which is worse than the model having routed it openly, because now the record is misleading.
The test is not what the interface calls it. It is: if the human does nothing, what happens? If the answer is “the model’s choice takes effect,” the model is in the run path.
“It is only classification, not routing”
This one is subtler and the answer is genuinely yes — with a condition. A model that labels a document as an invoice has produced data. A deterministic rule then routes on that label. That is the correct shape.
The condition is that the label must be reviewable. If a misclassification silently sends a case down a path nobody inspects, the fact that a rule did the routing is a technicality. Classification that drives consequential routing needs either a confidence threshold that escalates to a human, or a sampling regime that catches drift.
“We need it to handle the exceptions”
The most seductive version, because exceptions are exactly where deterministic rules are painful and models look strong. The reasoning is that the happy path is well understood, and the model should handle the messy remainder.
This inverts the risk. Exceptions are the cases with the highest stakes, the least oversight and the worst outcomes when handled wrongly. They are the last place to put an unexplainable decision. The right shape is the opposite: automate the well-understood path deterministically, and route the exceptions to humans — with the model helping those humans understand the case faster.
“Regulation allows it”
Sometimes true. But the question that actually gets asked is not whether the decision was permitted — it is whether you can explain it. A model in the run path can produce a decision that is entirely lawful and entirely unexplainable, and that combination is expensive precisely when it matters.
What you can safely do at run time
The rule is not “never call a model during a live case.” It is that the model must not decide. Several genuinely useful things sit comfortably inside that boundary:
- Produce data that a rule then acts on — a classification, an extraction, a normalisation. Deterministic, reviewable, auditable downstream.
- Draft artefacts a human sends — a response, a summary, a letter. The human is the actor; the record shows what they approved.
- Explain the case to the person handling it — what is here, what is missing, what policy applies. Pure assistance, no state change.
- Rank a queue — with the caveat that ranking is soft routing. Acceptable when every item is eventually handled and the ordering only affects sequence; not acceptable when low-ranked items are effectively dropped.
The pattern across all four: the model’s output is an input to something else, and that something else is deterministic, or human, or both.
The version question nobody asks early enough
One practical consequence deserves its own note. Model versions change, and a version that is deprecated will be withdrawn on the provider’s schedule rather than yours.
If the model is confined to design time, this is an inconvenience: your authoring assistant behaves slightly differently, and the flows it previously authored are unaffected because they are frozen data. If the model is in the run path, a version change silently alters production behaviour for cases in flight, and your audit trail now spans two decision-making regimes with nothing recording where the boundary fell.
Should you keep model calls at run time — for classification or extraction, as above — record the model identifier alongside the output, in the run data, permanently. It costs nothing and it is the difference between being able to explain a historical batch and not.
EFTEDRA builds workflow automation on IBM Business Automation Workflow and Claude — assistant tasks and coach views that install into the processes you already run. See what we build, or try the live demo.