What is AI agent governance?

AI agent governance is the set of controls that keep an agent doing what the business intends. In Umang Shah's workshop that meant scripted steps where wording must be exact, written business rules, personal data redaction, approved models, synthetic testing before launch and a traceable record of every decision afterwards.

Should every step of an AI agent use a large language model?

No. Umang Shah of Kore AI chooses for each agent and each step. Let the model reason where customers describe things in their own words. Script simple data capture, regulated wording and final decisions such as credit approval, so the outcome can be audited and repeated.

Can an AI agent use procedures written for human staff?

Yes. Umang Shah said you can upload the procedures your staff already follow, many at once, and the builder learns from them to design the agents. A short procedure works for a first test, but it will raise warnings where information is missing.

Is real-time voice safe for regulated customer service?

It needs care. With speech-to-speech, audio reaches the model before personal data can be redacted, and the model provider is a third party. Umang Shah said some organisations refused it for that reason. Speech-to-text with redaction, or models run inside your own boundary, avoid the problem.

AI agent governance guide: build and govern a customer service agent

Downloads | AI workshop guides

Good AI agent governance starts in the build: decide which steps the AI reasons through, script the ones that must be exact, and test before customers arrive. Umang Shah, SVP Sales Engineering International at Kore AI, showed how in a live workshop at aiblLIVE London 2026.

He turned a written procedure for a fictional electronics shop into a team of customer service agents, then tested and governed it on screen. This free guide sets out the method so your team can run it.

How do you turn a procedure into a customer service AI agent?

Give the builder your standard operating procedure and let it propose a design. In the demo, it created a supervisor agent that routes each request to specialists for orders, product advice and human handover. You then review the draft, accept it or ask for changes, and fix the warnings it raises.

How do you decide which steps the AI should reason through?

Let the model reason where customers describe things in their own words, and script anything that must be exact. Umang’s examples: capture a stressed customer’s accident report with reasoning, but script a licence plate request, regulated wording and the credit decision itself. Make the choice for each agent and each step.

Not every decision needs to be made by the LLM.

Umang Shah, SVP Sales Engineering International, Kore AI

How do you keep customer data away from the AI model?

Redact personal data, or swap it for stand-in values, before it reaches the model, on inputs and outputs. Run redaction globally and at the steps where you expect personal data. Call your own systems outside the model where you can. Be careful with real-time voice, where audio reaches the model before any redaction.

How do you test an AI agent before it goes live?

Generate synthetic customers and run every combination. In the demo, the platform created three personas and three scenarios, such as returns and product search, giving nine test conversations. It also showed token use and estimated cost. After launch, set a goal such as cost per call and trace every decision.

Did we say the exact words? No, we didn’t, because LLM decided otherwise.

Umang Shah, SVP Sales Engineering International, Kore AI

Key takeaways

Start from your real procedure. It is the best first brief for an agent.
Mark every step reasoning or scripted. Script anything that must be exact.
Write the rules into the build. Refund limits and approvals become explicit conditions.
Test with synthetic customers first. Cover every persona and scenario pair.

About the workshop

Umang Shah is SVP Sales Engineering International at Kore AI, which builds and governs AI agents for customer service across digital and voice channels. He ran this workshop on the Customer track at aiblLIVE London on 20 October 2026.

What’s in the full guide

If you need an agent you can defend to an auditor, this is a method your team can start next week. The full PDF goes step by step.

An eight-step method from written procedure to governed, tested agent.
A decision rule for which steps the AI reasons through and which follow a script.
Templates and a checklist for the procedure, step design, business rules and data protection.
Five actions to take back to your own organisation.

For customer service, operations and risk leads, and the teams who build their agents. Seven pages, about a nine-minute read.