When a prospect asks "what if it makes something up," they're describing a knowledge-base problem, not an AI problem. An agent hallucinates when it's asked something it wasn't given and isn't told to defer. Fix the KB and the accuracy objection mostly disappears. Here's how to scope, structure, and test it so the agent stays on rails.
The principle: scope beats scale
A bigger knowledge base is not a better one. The agent should answer from a tight, authoritative set of facts about this business — hours, services, pricing rules, policies, FAQs — and do something deliberate when a question falls outside it. The failure mode isn't "not enough knowledge"; it's "no defined edge," so the model improvises. Define the edge.
Structure the KB the way the caller thinks
Don't dump the client's website into the agent. Organize around the questions callers actually ask:
- Identity & hours — who you are, when you're open, where, after-hours behavior.
- Services & eligibility — what you do, what you don't, who qualifies. Be explicit about the "don'ts" — "we do not handle X" stops the agent from inventing that you do.
- Pricing rules — not a price list to recite, but rules: "quotes require a site visit," "starting at $X," "never give a firm price over the phone."
- Policies — cancellation, deposits, insurance, guarantees. The exact wording you'd want a human to use.
- FAQs — the 15 questions that make up 80% of calls, answered the way the owner would.
- Escalation map — what's explicitly out of scope and what to do (transfer, take a message, book a callback).
Short, declarative, current. Stale hours and wrong prices are worse than no answer.
The three behaviors at the edge
For anything outside the KB, the agent should do exactly one of three things — and you decide which per topic:
- Confirm — read back anything that matters (name, number, time, address) before acting. A confirmed booking can't be a hallucinated one.
- Defer — "That's a great question — let me have someone from the team follow up on that." Honest, safe, and it captures the lead.
- Hand off — route to a human for anything emotional, high-stakes, or explicitly flagged.
Write these into the system prompt as rules, not hopes. "If you do not have the information, do not guess — say you'll have someone follow up and capture their details."
Test it like an adversary
A KB you didn't try to break is a KB that will break in front of a customer. Before launch, call in and:
- Ask something plausible but false ("Do you also do [adjacent service you don't offer]?"). It should say no, not invent a yes.
- Ask for a firm price on something that needs a quote. It should follow the pricing rule, not make up a number.
- Ask an out-of-scope question. It should defer or hand off cleanly.
- Give a wrong detail then correct it ("Tuesday — actually, make it Thursday"). The summary should capture the final state.
Score each. If any ends in a confident wrong answer, the KB has a gap — fill it or tighten the rule.
Maintenance is the real work
The KB is not a launch artifact; it's a living document. Two habits keep it honest:
- Mine the transcripts weekly. Every "let me have someone follow up" is a question the KB should probably answer next week. The agent tells you exactly where its edges are — listen.
- Update on every business change. New service, new hours, price change, new policy → KB first, same day. A drifted KB is how a great agent slowly becomes a liability.
The payoff
Get this right and two things happen. The agent stops guessing, so the accuracy objection loses its teeth on sales calls — you can hand the prospect the phone and dare them to break it. And the client trusts the system, because it never says something wrong about their business. Accuracy isn't a model you buy; it's a knowledge base you maintain.