Keep a step out of AI when its rule can be written as a check, when its answer has to hold every time, when its job is copying information between tools that should be one system, or when a wrong answer commits you to something before a person sees it. Give a model the steps that read or write open-ended language, where a good-enough answer is useful and something checks it.
The AI features we build are mostly ordinary software with a model doing one or two of the steps, and deciding which steps those are comes before choosing a model. Here are the four questions we ask, with the evidence behind each.
One request, several steps
A request to “use AI for our quotes” names a whole process, and most of that process already has rules. Reading the customer’s description of the job is language work, and so is drafting the email that goes back. Pulling prices from the price list, adding up the total, and checking what discount the business allows are rules someone could write down, and a quote over a set amount may need a person to approve it.
So a model earns two of those steps, a person may need one, and the rest are ordinary code. When the steps are known ahead of time like this, the result is a workflow with a model inside it, cheaper to build and test than a system that works out its own steps.
When not to use AI: four questions for each step
Take the steps one at a time. Each of these questions can move a step away from the model, to code, to a person, or to a better arrangement of your systems, and the model keeps the rest.
- Can you write the rule as a check? Then code runs it on every output.
- Does it have to hold every time? Then code holds the limit, even where a model writes the words.
- Is the step copying information between tools? Then fix where the information lives before automating the copying.
- What does a wrong answer cost, and who sees it first? If it commits you before a person looks, put a limit in code or a person in front of it.
Two more reasons can keep a step from a model, outside these four: data you may not send to a model provider (what to check first), and a decision you have to explain step by step, which is easier for a rule than for a model’s answer.
Can you write the rule as a check?
If a rule can be written as a check that needs no judgement, such as a length, a required field, or a phrase that must not appear, code can run it on every output, and a model asked to make the same check will sometimes pass the output that breaks the rule.
We saw this on a client’s story tool, which turns a writer’s answers into a three-page summary for studios and buyers. Its prompt already forbade dialogue. In a September 2026 test of ten stories, two drafts each, a grader model (a cheaper, faster model from the same family as the one that drafted them) gave all 20 summaries a perfect score for prose, a score that covered dialogue. Two of the drafts quoted a line anyway, and on both, the grader’s notes described the prose as written “without any dialogue.”
Those 2 were among 13 drafts with flaws a plain text check catches; the other 11 announced their own structure, with lines like “the midpoint arrives as…” or “the story is no longer about…”, which reads like the writer’s outline, not the story a buyer wants, and which the prompt had not forbidden. The second phrase echoed the prompt’s own description of the turn (“changes what the story is about”), so we reworded that line, spelled out both rules, and the tool’s test suite now checks every draft for both in code. The same stories, run again, for $3.32 in model calls across both runs, a small price for measuring the fix:
| Ten stories, two drafts each | Before the fix | After |
|---|---|---|
| Drafts the structure check caught | 11 of 20 | 0 of 20 |
| Drafts the dialogue check caught | 2 of 20 | 0 of 20 (a two-word quote got past it) |
The structure check came after the first run; both runs’ saved drafts were graded with it. A check holds a rule only as far as its pattern spells it out: the dialogue check counts quoted passages of five words or more, so a character’s two-word “thank you” in one later draft got past it. A grader model belongs on the questions no rule can score, and even there its scores need checking against people’s judgement. A rule that a check can test needs no model’s opinion at all.
Our free AI citeability checker applies the same idea to a whole product: it is a tool about AI search with no model in it. Whether robots.txt lets a given AI crawler in, whether the page’s words are in the HTML before any JavaScript runs, whether the page carries a date: each of those questions has one right answer, so each check is a written rule with its threshold in the code, and the rules have tests. A model would add a bill, and the chance of a different answer, to questions that already have one.
Does it have to hold every time?
Anything that must hold every time is enforced in code, with the model working inside the limit, because the same input can come back worded differently on two calls, and sometimes with a different answer. The providers say so themselves: Anthropic’s documentation warns that identical inputs may produce different outputs across calls, and Google’s that a small amount of variation is still possible at its most predictable setting.
We saw it in our own October 2026 test of two models from different developers on one hosted service: 12 support emails for a made-up tour company, each sent 20 times under each of three settings: the defaults, temperature (the randomness setting) at zero, and zero with a fixed seed. The six clear emails went to the expected queue in all 720 of their answers. For “My mother died, so I can’t come on the 16th. What happens to my booking?”, each model split its 20 answers between two queues under every setting. Routing can absorb that when someone fixes a misrouted email; a refund amount cannot.
An August 2026 test on the same story tool’s shorter summaries put this line in the prompt: “At most 1400 characters total. Count them. Going over is a failure.” A model reads and writes in tokens, chunks of text that average a few characters each in English, so a character count is not a unit it works in, and asking for three paragraphs of 60 to 75 words instead, step one in the order we try fixes in, helped but still left summaries over the target, on the same model and stories:
| Summaries over the 1,400-character target (minimums) | Rule in characters | Rule in words and paragraphs |
|---|---|---|
| All runs that completed | 48 of 98 | 13 of 74 |
| The six stories with nine or ten usable runs under both | 30 of 58 | 13 of 60 |
Provider errors cost the second version 26 calls, unevenly across the stories, so the second row compares like with like, and one of those stories still ran over in 8 of its 10 runs: the prompt alone could not hold the target.
The prompt sets a target, and the code holds the limit. For those summaries that is the 1,500 characters their field can store, and the code that saves one cuts anything longer back to its last full sentence. That cut is also why the counts above are minimums: the test saved each summary after it, so a reply cut back under 1,400 characters counted as within the target. Cutting is acceptable for a summary, and for text whose last line carries a price, a condition, or a next step, the safer rule is to reject the draft and ask again.
Is the step copying information between tools?
When a step exists to move information between systems that should be one, handing it to a model keeps the arrangement and adds one more part to maintain, while consolidating the systems removes the step.
On an operations platform we built, five tools became one, and data entry errors from copying between systems went to zero, because the step that produced them no longer existed. Any automation that reads one tool and writes to another, with a model in it or without, leaves someone maintaining every field mapping, every change to one tool’s export format, and every case where two systems disagree. That is why we consolidate first and automate what is left.
Consolidating is not always the right call: when the tools are only an annoyance and the workflow through them is not what the business depends on, fixing the connection between them costs less.
What does a wrong answer cost, and who sees it first?
If a wrong answer commits the business to money or a promise before a person sees it, the model can suggest and code or a person decides.
In 2024 a British Columbia tribunal held Air Canada responsible for its website chatbot’s wrong answer about claiming a bereavement fare after travel, which the airline’s own policy page did not allow, and ordered it to pay C$650.88 in damages. It rejected the airline’s argument that the chatbot was responsible for its own actions: “it makes no difference whether the information comes from a static page or a chatbot.”
We ran a version of that risk ourselves in October 2026, with a made-up tour company’s refund policy in the prompt and the same two models answering 24 customer messages ten times each. Neither repeated the airline’s mistake, whatever that chatbot was built on: all 60 answers to requests for a bereavement refund after the trip refused, and no answer gave money the policy did not allow. The three wrong decisions all gave less, and twice a customer cancelling exactly 30 days out was told it was 29 days. But 13 of the 240 refusals offered help rebooking, which the policy never mentions, and an offer outside the policy is what a limit in code or a person should catch.
The design lesson travels further than the ruling: where an answer states a policy, as that one did, it should quote the policy’s own text or hand the question to a person. In our test, asking for the quote changed no decision we could measure; its value is that the customer reads the policy’s own words. In a quoting or support feature we build, the model can draft a refund, propose a discount, or suggest a delivery date, and code decides what it is allowed to carry out, within limits set when the feature is designed. On live phone calls, where a promise is made before anyone can catch it, complaints and billing disputes go to a person.
Where a model earns its place
A model belongs on the steps that read or write open-ended language, where an approximate answer is useful and something checks it before it counts:
- Reading what follows no format. Emails, free-text form fields, and documents that differ every time, sorted or searched for the details a rule then acts on, where rule-based automation breaks down.
- Drafting for a person to finish. A first reply, a summary, or a product description that someone edits and sends.
- Answering from your own documents, with retrieval supplying the passages and the answer pointing back to them.
- Judging what no rule can score, such as whether a summary reads well, with a person checking the judge.
That split, a model on one or two steps, code on the rest, and a person wherever a mistake would commit the business, is the first thing we settle when we scope AI integration work, because it decides what the feature costs to run and how often it can be wrong.
Put the model where the judgement is
Any AI proposal, including one you write yourself, can be tested with two questions: which steps does the model take, and what checks each of them? A proposal that answers “the AI handles it” has not yet been split into steps, and the four questions above are how to split it.