When your AI support agent promises a refund that never happened

The problem

An AI support agent that can talk about refunds will, sooner or later, tell a customer their money is on the way. If the refund action did not run, failed, or ran for a different amount, the customer is the first to find out, usually a week later on their bank statement.

The conversation looks perfect, so support QA that reads conversations rarely catches it. The mistake only shows when you put the promise next to the order.

Four ways it goes wrong

• Promised, never done: the agent says "I've refunded you" but no refund action ran, often because the action is not switched on or needs approval.
• Done for a different amount: the agent promises $60 and the refund in Shopify is $30.
• Done on the wrong order: the customer has two orders and the refund lands on the other one.
• Done but never said: a refund went out that nobody told the customer about, which is a cost nobody approved.

How to check by hand

• Pick the tickets from last week that your AI agent answered and that mention a refund.
• For each one, find the amount and order number the agent wrote.
• Open the order in Shopify and look at its timeline for a refund with that amount.
• Note every ticket where the two don't match, and fix the customer first.

Twenty tickets take about twenty minutes. It is worth doing once, even if you never do it again, to know whether you have the problem.

The guidance line that stops it

Add this to your AI agent's guidance: "Only confirm a refund to the customer after the refund action has run and succeeded. If you cannot run it, do not promise it: tell the customer a teammate will handle it, and hand over the ticket."

In our tests with Gorgias AI Agent, when no refund action was set up, the agent already refused to promise one and handed the ticket over. The risk grows when actions are switched on, so that is when to start checking.

Check one reply now

Paste a reply your AI agent sent into our free promise checker to see every promise in it. To check every ticket against Shopify and Stripe automatically, ask for a free 30-day audit.