Your agent checkout works. Your shipping answer doesn't.

Every check you can see is green

Picture the store the week before a big promotion. Someone runs an agent-readiness checker and it comes back looking great. There's an /agents.md file. The store answers an MCP tool listing. An AI assistant can put an item in a cart and price a checkout. Nobody on the team built any of it, which feels like a gift.

Then a shopper asks an assistant the most ordinary question in retail: what does shipping to Boston cost, and when will it get here? The assistant asks your store. Your store has nothing useful to say.

That is roughly what Bill Hunt of Bisan Digital found when he audited 122 of the top Shopify storefronts between August and September 2026. He published the results on Search Engine Journal on October 14. The plumbing works. The answers a buyer needs before they commit mostly don't come through. If you run a Shopify store, the useful takeaway isn't that agents are overhyped or that Shopify did something wrong. It's that "ready by default" describes what the platform switched on, not what your store can actually tell a customer.

What the 122-store study actually found

Start with the credit, because it's real. Shopify turned agent tools on for Liquid storefronts, and merchants didn't have to write a line of code. In Hunt's sample, 89 of 122 stores answered a standard MCP tool listing. On every one of the 74 stores that exposed a Universal Commerce Protocol (UCP) checkout, his non-completing test worked: create a checkout for an in-stock item, read it back, cancel it, and have a repeat cancel refused with a coded error. That's 74 of 74.

Now the other half. Hunt put buyer-style questions to each store's own policy tool. Of the 88 stores where that tool responded:

• Return window for this item: a usable answer from 70 of 88.
• Shipping cost to Boston and arrival date: 1 store returned anything at all, and 0 of 88 gave a usable answer.
• Can I cancel after ordering, and by when: 2 of 88 usable.
• Can used or open-box items be returned: all 88 returned something, and only 5 were usable.

A few more findings make the pattern clear:

• 89 stores serve /agents.md, but 83 of those files are near-identical copies of Shopify's template. Only five stores in the whole sample wrote their own.
• All 93 stores with a UCP fulfillment declaration list shipping as the only method. That includes one store that documents Pickup in Store yet tells agents "No pickup locations available," and another that offers Click & Collect and declares shipping only.
• None of the 74 live checkouts returned an arrival estimate or a carrier.
• 51 of 122 stores block /policies/ in robots.txt, a leftover from an old robots.txt.liquid theme template.
• No store passed all nine of Hunt's purchase-decision checks. The best three got to seven of nine, and all three missed the same two: no stated cancellation window and no way to cancel.

Keep one thing in proportion: every checkout came back needing buyer input before it could be completed. On Shopify today, the agent builds a priced cart and a person commits the money. So the real risk isn't a rogue agent spending money. It's an agent that walks a ready buyer to the last step with the wrong idea about delivery, pickup or returns, or one that can't answer at all.

Why a working checkout still gives empty answers

In Hunt's sample, most stores do publish the facts. The trouble is that they're written as prose on a policy page and the tool can't get at them when the question is phrased the way a shopper phrases it. Ask the policy tool "What is your shipping policy?" and it finds the page. Ask "What does shipping cost to Boston?" and, in his sample, it almost never does. One store's FAQ says free standard ground shipping on U.S. orders over $75, and that sentence never came back for a question about shipping cost.

Empty isn't even the worst case. When asked about cancellation, two stores returned their shipping FAQ instead. An agent has no reliable way to tell that a confident answer is about the wrong thing, so it passes the mistake along. Pickup is the same problem in machine-readable form: a store that offers collection tells the agent, cleanly and wrongly, that it ships only. Shoppers who could have picked up today get sent to delivery.

The part worth taking to your next ops meeting is who owns each gap. Hunt sorted them, and almost none belong to the web team:

• Pickup availability and cutoff: fulfillment and stores.
• Delivery estimate: fulfillment.
• Cancellation window and the ability to cancel: order management.
• Used and open-box exceptions: legal and customer service.
• Return window as structured data: legal.

The checks almost every store passes are the ones the platform generates for you: catalog, variants, price, checkout. The ones almost every store fails need a person outside the web team to write down a commercial rule in a form a machine can use. A new markdown file won't fix that. Someone deciding that the cancellation window is one hour, and saying so in plain numbers, will.

Ask your store the questions a buyer would

You don't need a full audit to find out where you stand. Hunt's own advice is that most of this takes an afternoon, and the first step is one command. Swap in your domain and send your store's policy tool a real buyer question:

```bash
curl -s https://YOURSTORE.com/api/mcp
-H 'Content-Type: application/json'
-d '{"jsonrpc":"2.0","id":1,"method":"tools/call",
"params":{"name":"search_shop_policies_and_faqs",
"arguments":{"query":"Can I cancel an order after placing it, and by when?"}}}'
```

Then run it again for each of these, phrased the way your customers actually write:

1. What is the return window for this item?
2. What does shipping cost to [a real city you ship to], and when will it arrive?
3. Can I pick this up in store today?
4. Can I cancel after ordering, and by when?
5. Can I return something I've opened or used?

Read each answer as if you were the agent. An empty list, a link to a page, or an answer about a different topic is exactly what an assistant gets. Copy the results into one sheet with a column for "usable," "empty," and "wrong topic," and put the owner from the list above next to every line that isn't usable.

Then check two files. Each takes seconds. Open /.well-known/ucp.json and look at the fulfillment methods. If you offer pickup and it says [["shipping"]] only, that's a gap to raise with Shopify. Then open robots.txt and look for Disallow: /policies/. If it's there, it's coming from an old robots.txt.liquid template, and Shopify's documentation says deleting that template restores the default.

Write the rules down where agents can read them

Once you can see the gaps, the fixes are mostly writing and settings rather than development:

• Set return and cancellation rules in Shopify admin, then repeat the same values as plain numbers on your refund policy, shipping policy and FAQ. "30 days from delivery" works. "Hassle-free returns" doesn't.
• Turn on delivery dates if you can actually meet them. None of the 74 checkouts in Hunt's study returned an arrival estimate, and from outside he couldn't tell whether stores had dates switched off or the dates weren't being passed through. Turn them on, then ask again and see whether they show up.
• Make pickup visible. If you offer it and your UCP declaration says shipping only, get it fixed before a holiday shopper is told collection doesn't exist.
• Rewrite /agents.md with your real return window, cancellation rule, free-shipping threshold and pickup options. Only five of the 122 stores Hunt looked at had done this, so it's an easy way to stand out.
• Add MerchantReturnPolicy and OfferShippingDetails to product structured data, with itemCondition if you sell used or refurbished stock.
• Label every size and option selector. Hunt found unlabelled size or option dropdowns on 65% of the stores that use them, which leaves a browser agent guessing which menu sets size and which sets color.

One thing you can't fix on your own: in Hunt's sample, no store exposed a way for an agent to cancel, change or return an order after it's placed. The protocol doesn't define those operations yet. Until it does, make sure the policy text is clear about how a person can cancel and by when, so an agent can at least say so and hand off to a human.

Put this into practice before peak

This week, run the five buyer questions against your own store and keep the results. Anything that isn't usable gets an owner and a date, and the owner is usually in fulfillment, order management or customer service rather than the web team. Fix robots.txt and check the pickup declaration first, because those take minutes. Write your rules as numbers next. Then run the same questions again and confirm the answers changed, because a readiness checker staying green doesn't tell you much.

If you already opened Shopify's Agentic settings page and decided which channels get your catalog, this is the next step. Those settings decide who can reach your store. This test shows what those visitors hear when they ask a real question.

When the gaps run into stock that has to stay accurate across channels, delivery promises that depend on carrier cutoffs, or pickup that only some locations offer, that's the ecommerce and fulfillment operations work we do, alongside the agent side in AI applications and agents. If your store's answers came back empty and you're not sure who should fix them, send us what your policy tool said.

Related work: Paddle8.