A vendor walks you through a demo. An email arrives from a customer asking where a late order is, and the system reads it, finds the account, checks the shipment, decides the delay warrants a refund, issues one, and writes back. Nobody touches it end to end, and it works perfectly.
The question that rarely gets asked in that room is what happens on the run where it finds the wrong account. A demo shows you one attempt at one task, and the thing being sold will run that task a few thousand times against your real records.
What an AI agent actually is
A chatbot generates text. An agent is a model that has been given tools and permission to use them, so it can look up an order, create a ticket, send an email, or update a record, and it chooses which of those to do and in what order until the job is finished.
The useful line to draw is between fixed steps and chosen ones. In most software the sequence is written by a developer ahead of time and the model fills in a piece of it. In an agent, the model directs its own process and tool use, deciding what to do next based on what it found last. That is a real capability, and it is genuinely new in the last couple of years.
It is also worth separating from the question of where the information comes from. An assistant that answers accurately from your own documents is doing retrieval, which is a different mechanism and a much smaller commitment. Retrieval changes what a system knows. Tools change what it can do.
Why every vendor is suddenly selling one
Because the word moves deals, and in a demo a chatbot with a new label is hard to tell apart from the real thing.
Gartner named the practice agent washing, and its estimate is that roughly 130 of the thousands of vendors claiming agentic capability actually have it, with the rest rebranding assistants, chatbots, and existing process automation. The same research expects more than 40% of agentic AI projects to be canceled by the end of 2027, on escalating costs, unclear business value, and inadequate risk controls.
None of that means the category is empty. We build these, and the ones that work are worth building. It means the label carries almost no information, so the questions below are what you have instead.
What it is allowed to do without you
This is the part that decides how a project goes, and it is a decision the buyer owns rather than the vendor. Autonomy works as a set of permissions somebody grants rather than a rating a system earns by being clever, and it comes in rungs.
It reads and drafts. The system looks things up, summarizes, and prepares a reply or a record for a person to send. The worst outcome is a wrong answer that someone catches before it goes anywhere.
It acts, and asks first. The refund is calculated, the reply is written, the calendar change is prepared, and a person approves before any of it lands. The worst outcome is review time spent on work that gets thrown away.
It acts alone, inside bounds. It executes without approval, within limits somebody set: refunds under a certain amount, these record types and not those, this list of customers, working hours only. The worst outcome is bounded and, if the bounds were drawn properly, reversible.
It acts alone, unbounded. Rare in practice, and almost never what a business should buy first.
The asymmetry that makes this worth thinking about carefully is that reading is recoverable and writing is not. An agent that reads the wrong record produces a bad answer. An agent that writes to the wrong record has changed something a customer can see, and undoing it means knowing it happened. Most first builds belong on the second rung, and the good ones move up one rung at a time as the evidence comes in.
Why the tenth step is where it goes wrong
Agents work in steps, and reliability compounds down a chain in a way that single-shot testing hides. A step that succeeds 95 times out of 100 sounds solid. Ten of those steps in sequence finish cleanly about 6 times out of 10.
Two findings put numbers on that. On a benchmark that gives agents real tools, a simulated customer, and a written policy to follow, a leading model of its day solved fewer than half the retail tasks on a single attempt, and solved a task on all eight attempts less than a quarter of the time. The gap between those two figures is the gap between a demo and a deployment, because a demo is one attempt and a deployment is every attempt.
The second is about length. Measured across a set of research and engineering tasks, models came close to perfect on work that takes a person a few minutes, and succeeded less than 10% of the time on work that takes a person more than about four hours. The curve is steep, and it is the reason a system that handles a two-minute lookup beautifully can fall apart on an afternoon’s work that no single step of it finds difficult.
Neither number is a verdict on the technology, and both keep moving. What they tell you is where to spend your caution: on the length of the chain you hand over and the number of times it will run, rather than on how impressive any one run looks.
What it takes to run one
An agent is the same production scaffolding as any other AI feature, with the stakes raised, because a mistake now changes something instead of merely saying something. That means guardrails, so a conversation that goes off-script does not become an action that goes off-script. It means evals, because “it worked in the demo” is a sample size of one, and running the same task repeatedly is the only way to see the consistency the benchmarks measure. It means observability, since a system that acts on its own needs a record of what it did and what it was looking at when it decided to. Cost belongs on the list too, as an agent spends several model calls per task where a chatbot spends one, which changes the arithmetic on running costs more than people expect.
The item that is genuinely new here is the reversal path. Every action the system can take on its own needs an answer to “how do we undo this and how do we know it happened,” decided before the permission is granted rather than during the incident. That question is dull, and it is the one that separates the deployments that survive their first bad week from the ones that get switched off. The rest of the scaffolding is the standard set that any production AI feature needs.
When you want an agent, and when you want a workflow
Most of what gets pitched as an agent is a workflow, and that is not a criticism. If the steps are known ahead of time, writing them down is cheaper to build, faster to run, easier to test, and far easier to explain to an auditor. Handing those steps to a model that rediscovers them on every run buys nothing and costs latency, money, and predictability. The advice from the people building the models is the same: find the simplest thing that works and add complexity only where it earns its place.
Two examples, one on each side. Routing an inbound enquiry to the right queue and drafting a first reply is a workflow with a model inside it, because the shape never changes. Working out why a shipment is late when the answer might be in the order system, the carrier’s tracking, an email thread, or a warehouse note is closer to genuine agent territory, because the next step depends on what the last one turned up.
This is the build-versus-buy question in a new costume, and complexity nobody needed is the most common way an AI project stops being affordable.
How it connects to the systems you already run
An agent is only as useful as the tools it can reach, and connecting a model to your systems used to be custom work per vendor, which meant redoing it every time you changed models. That has largely standardized, and the protocol most of the industry now uses was donated to a Linux Foundation body at the end of 2025. For a buyer, the practical effect is that integration work built against the standard is less of a bet on one vendor staying the best or staying reasonably priced, which is the case against lock-in applied one layer up from the model itself.
Ask what happens on the run where it is wrong
When the demo ends, the question that tells you the most is what the system is permitted to do unsupervised, what bounds that permission has, and what happens on the run where it is wrong. What it can do is the easier question, and the demo already answered it.
A vendor who has built one of these will have answers ready, because those decisions are most of the work. A vendor who renamed a chatbot will answer with capabilities. We build agents where the task genuinely needs one, as part of the AI integration work we ship, and the conversation starts in the same place every time: which actions are worth handing over, and which ones stay in front of a person.