The happy-path demo is a liar. You call your agent, ask the exact question it was built for, it answers perfectly, and you think you're done. Then a real caller mumbles, interrupts, asks something you never anticipated, and your client watches it fall apart. Testing is the difference between an agent that survives contact with real people and one that generates refund requests. Here's the checklist.
The five test categories
1. Happy path. The calls it's designed for. Book the appointment, answer the FAQ, take the message. These should be flawless — but they're the least important test, because they're the ones you already know work.
2. Edge cases. The plausible-but-unusual. Someone changes their mind mid-call ("Tuesday — actually Thursday"). Someone gives a wrong number then corrects it. Someone asks for two things at once. Check that the final summary captures the corrected state, not the first thing said.
3. Adversarial inputs. Try to break it. Ask something plausible but false ("do you also do [service you don't offer]?"). Demand a firm price on something that needs a quote. Ask an out-of-scope question. A good agent defers or hands off; a bad one invents an answer. This is where the accuracy objection is won or lost.
4. Real-world conditions. Accents. Background noise. A bad cell connection. Talking fast. Talking over the agent (barge-in). The agent that sounds perfect on your laptop softphone often chokes on a real PSTN call from a moving car. Test from an actual phone, on cellular, not your office WiFi.
5. Escalation. Get angry. Say "let me speak to a person." Say "this is an emergency." Every one of these must route to a human cleanly — no looping, no arguing, no dead end.
How to actually run it
- Call it yourself, ten times, being difficult. You are the cheapest, meanest test user available. Use it.
- Recruit two other people with different voices/accents to call. You are blind to your own agent's failure modes.
- Score each call against the five categories. Any call that ends in a confident wrong answer, a loop, or a dead end is a fail — fix the prompt and re-run.
- Test at volume if it matters. For outbound or high-traffic inbound, run many short calls (tools like n8n or a simple script can drive this) to catch failures that only appear under load or across variations.
What breaks most often
- Endpointing — the agent cuts people off or leaves dead air. Tune this first; it's the #1 "feels robotic" cause.
- Out-of-scope hallucination — traces straight back to a thin knowledge section and a missing "don't guess" rule.
- Barge-in — the agent talks over the caller instead of stopping to listen. Deaf feels robotic.
- Summary drift — the post-call summary captures the wrong detail when the caller corrected themselves.
The rule before you hand it to a client
Do not ship an agent you haven't personally tried to break. If you can't make it fail after ten hostile calls, it's probably ready. If you haven't tried, it's not — you've just built a demo, and the client's customers will do your testing for you, badly, in public.
Testing isn't a phase you do once. Every prompt change, every new knowledge doc, every business update reopens the risk. Keep a saved list of your test scenarios and re-run the important ones after any change. The agents that build a client's trust are the ones that were broken a hundred times in private so they never break once in public.