A professional services firm moved an AI agent into its live billing support queue after an eight-week pilot. No client got anything they should not have. But within weeks the agent had drifted into limbo, because the pilot never tested what surrounds an agent: an audit trail, a review process and a way to show what it delivered.
The rebuild put one person in charge, logged every decision and narrowed the scope to one measurable job.
Details as told to aibl by the firm
Three things in this case. There was no audit trail, so tracing one billing query took two people most of an afternoon. Review varied once eight people shared it instead of three. And nobody could point to a metric that had moved.
The agent drafts and flags, and a person approves before anything touches a client’s account. One named person owns what the agent may do, how outputs are reviewed and when the rules change.
Narrow the scope until one number can move. The firm picked clients querying invoice discrepancies, measured the current first-response time and set a target, so it could prove one improvement before funding the next.
If an agent is heading from pilot into live work, the full case study covers what went wrong here and the controls that fixed it:
This case study first appeared in aiblBRIEF. Read the original article: What broke when an agent moved from pilot to production.
