Go-live is the start of the job, not the end of it. An agent that worked perfectly in testing drifts: the business changes, callers do new things, a provider tweaks a model. The agencies that keep clients are the ones who catch problems in the transcripts before the client catches them on a bad call. Here's what to watch, and how often.
The mindset shift
A voice agent is not "set and forget." It's a living system running in a changing environment. Monitoring isn't overhead — it's the retention work. Every week you spend fifteen minutes in the transcripts is a week you find the small problems before they become a churn conversation. Skip it and you'll find out something broke when the client calls you angry.
The metrics to watch weekly
- Containment rate — the share of calls the agent handled without escalating. Trending down means the KB has new gaps or the business changed.
- Escalation rate + reasons — how often it hands off, and why. A spike in "out of scope" points to a specific missing capability.
- Latency — median response time. It drifts. If it creeps past ~800ms, callers start feeling it before anyone reports it.
- Failed/unknown intents — every "let me have someone follow up" is a question the KB should probably answer next week. This is your roadmap.
- Booking/action success — did the agent actually complete the job (booked appointment, captured lead) or just talk?
- Call volume vs. expectation — sudden changes can mean a broken number, a marketing push, or a config issue.
Read the transcripts, not just the dashboard
Numbers tell you that something changed; transcripts tell you what. Every week, read a sample — especially the escalations and the short/abandoned calls. That's where you find the agent cutting people off, misunderstanding a common phrase, or confidently saying something slightly wrong. The dashboard will never surface the sentence that's quietly costing your client trust.
The maintenance loop
- Weekly: review the metrics + a transcript sample. Patch the top KB gap. Note anything drifting.
- On every business change: new service, new hours, price change, new policy → update the KB the same day. A drifted KB is how a great agent slowly becomes a liability.
- Monthly: a short report to the client — calls handled, bookings captured, what you improved. This is also your retention rhythm (they see the value) and your upsell opening.
Set alerts for the failures that matter
Don't rely on eyeballs alone. Alert on: the agent erroring/going down, a latency spike, an escalation path that failed (no human available, transfer failed), and volume dropping to zero (a broken number). The worst outages are the silent ones — a dead number nobody notices for three days is a client's worst week and your worst call.
The payoff
Monitoring is how "we set up an agent" becomes "we run your front line." It's the difference between a one-time build fee and a retainer that renews for years. Watch the transcripts, patch the gaps before the client feels them, and send the monthly report that proves the value. That discipline is worth more to your MRR than any new feature.