AI Agent vs Chatbot:What’s the Difference for Your Store?

AI agent vs chatbot sounds like a naming argument until a vendor sends you a quote. At that point it’s a purchasing decision. The two cost different amounts, fail in different ways and can do very different things to your ecommerce store. Gartner has a name for relabeling an old product as a new one. It calls it agent washing. In June 2025 the firm estimated that only about 130 of the thousands of agentic AI vendors were real. So part of what you’re offered is a chatbot with a new label. Below is a way to tell the difference, and a way to decide which one your store needs.

What actually separates an AI agent from a chatbot

Anthropic draws the cleanest line we’ve seen. Workflows are “systems where LLMs and tools are orchestrated through predefined code paths.” Agents are “systems where LLMs dynamically direct their own processes and tool usage.”

Apply that to a store. A bot that answers “what’s your return policy” from your policy page is a chatbot. A bot with a “track my order” button that always calls your order system and prints the result is a workflow. It uses a tool, but a person wrote every step.

It becomes an agent when the model decides the steps. Say a shopper writes that their parcel hasn’t arrived, the tracking page says delivered, and they want to know what happens next. An agent looks up the order, reads the carrier status, checks your policy on lost parcels and picks a next move. A workflow can only follow the path somebody coded.

So two questions matter more than the label. Who chooses the next step, the model or your developer? And what can it change in your systems, apart from the chat window?

Most “AI chatbots” with order lookup sit in the middle. They’re workflows with a language model on the front. That’s fine, but we wouldn’t pay agent prices for one.

Four questions for the sales call

• Which tools can it call, and can I see the full list?
• Does the model choose the order of steps, or does your team fix them?
• What happens when a tool call fails or comes back empty?
• Can I read a log of every step it took in a real conversation?

The third question tells you the most. A fixed script and a model making choices behave differently when something breaks, and that’s where you see which one you’re buying.

When an ecommerce chatbot is the better buy

Anthropic’s advice to builders is to find “the simplest solution possible” and add complexity only when needed. It adds that this “might mean not building agentic systems at all.” For many applications, it says, “optimizing single LLM calls with retrieval and in-context examples is usually enough.” Agentic systems also “often trade latency and cost for better task performance,” and autonomy brings “the potential for compounding errors.”

For a store, that sorts the work cleanly. Shipping times, sizing, materials and return windows are retrieval jobs. The answer already sits in your content. A chatbot grounded in your own pages handles them cheaply and predictably. An agent adds loop time and a new way to be wrong, and a shopper on a phone feels both.

An AI customer service agent earns its cost where the path changes from one customer to the next. Think of an address change on an order that may or may not have shipped. Or an exchange that depends on stock in another size. Or a refund that turns on the order’s age, the payment method and the item. Each step depends on what the last one found. That’s the shape an agent handles and a script doesn’t.

What your assistant should be allowed to do

OWASP lists “Excessive Agency” in its 2025 top ten for LLM applications. It names three root causes: excessive functionality, excessive permissions and excessive autonomy. Its fixes are to limit permissions “to the minimum necessary” and to require a human to approve high-impact actions before they happen.

On a store, we sort actions into tiers.

The defaults are our opinion, not a standard. The point is to decide them on purpose, before launch. Anthropic suggests the same habits for agents in general, including stopping conditions such as a maximum number of iterations, and pauses for human feedback at checkpoints.

Two cases worth knowing

In December 2023, a user told the chatbot on a Chevrolet dealership’s site to agree with anything and call every reply legally binding. Then he asked for a new Tahoe on a $1 budget. The bot answered that it was a deal and a “legally binding offer,” according to GM Authority. The dealership took it offline.

The lesson is about design. A rule in the prompt is not a control. If the assistant has no tool that can set a price or issue a discount, nothing typed at it can make it do so. So put the limit in the tool. Prices come from the catalog. Discounts come only from codes you created. Refunds go to a queue.

The Air Canada case runs the other way. The airline’s chatbot told Jake Moffatt he could apply for a bereavement fare after travelling. Another page on the airline’s own site said otherwise. Air Canada argued the chatbot was a separate entity responsible for its own actions. The tribunal rejected that and found negligent misrepresentation. In its words, “it should be obvious to Air Canada that it is responsible for all the information on its website.”

For your store, that means what your assistant says about returns, warranties and delivery dates counts as something you said. Ground it in your policy pages and show the source with each answer. Keep the logs.

There’s a second agent you didn’t hire

Your own assistant isn’t the only agent that can reach your store. Agentic commerce adds a second kind, the agents shoppers bring with them. Shopify’s Storefront Catalog MCP server lets outside AI agents search a store’s catalog. Its tools include search_catalog, lookup_catalog and get_product, and each request has to carry the agent’s profile URL. Shopify’s docs list separate Cart and Order MCP pages as well.

Those agents read your data and ignore your homepage. If your sizes live in a tag and your return policy sits in a PDF, a shopper’s agent won’t find either. We covered how to check this in our guide to designing a Shopify store that AI agents can understand and buy from. We also walked through the tool calls in how Shopify stores are evolving for the agentic commerce era.

Fix that data first. If your own assistant reads the catalog, it gets better answers too.

How to tell if it’s working

Most customer service reports lead with deflection, the share of chats that never reached a person. That number also rewards a bot that frustrates people into leaving. Resolution is harder to measure and more honest. Count a chat as resolved when the assistant fixed the issue and the customer didn’t contact you again about it within a set window, such as seven days. Anthropic makes a similar point about support, where success “can be clearly measured through user-defined resolutions.”

Test it like any other change

Offer the assistant to some visitors and not to others. Don’t compare people who used it with people who didn’t. Someone who opens a chat already has a question, so the two groups differ before the bot says a word. Compare everyone who was offered it with everyone who wasn’t.

Pick one main number, such as conversion rate or orders per visitor. Add guardrails like support tickets per order, refund rate, cancellations and average order value. If conversion rises and refunds rise with it, that’s worth a hard look before you roll out. Our post on CRO program metrics beyond win rate covers how to choose them.

Plan for a slow test. Most visitors never open a chat widget, so the effect gets spread thin across everyone you offered it to. That takes more traffic or more time, and our guide on how long an A/B test should run shows how to work out how much.

Gartner expects more than 40 percent of agentic AI projects to end up canceled by 2027. Its reasons are “escalating costs, unclear business value or inadequate risk controls.” A guarded test answers the second reason. The approval tiers and logs answer the third.

If you’re weighing an assistant for your store and want a second opinion on what it should be allowed to do, we’ll look at it with you. Talk to our team about your AI agent plans.

FAQs

What is the difference between an AI agent and a chatbot?

A chatbot answers questions, usually from a script or a set of pages. An AI agent chooses its own steps and uses tools to act on your systems, such as looking up an order or updating a cart. The test is who decides the next step, the model or your developer.

Can a chatbot become an AI agent?

Yes. Under Anthropic’s definition, a chatbot becomes an agent when you give it tools and let the model choose its own steps. Many store bots stay on the workflow side, because developers fix every step in code.

Does a small ecommerce store need an AI agent?

Usually not at first. Anthropic recommends the simplest solution that works, and for answering product and policy questions a chatbot grounded in your own pages is often enough. Add agent features later, one task at a time, starting with read-only ones like order status.

What is agent washing?

Gartner uses the term for vendors that rebrand assistants, robotic process automation or chatbots as agentic products without substantial agentic capabilities.