Ask a leadership team how many AI agents are running across its business, and the answer is often difficult to come by. Then ask who owns the agents, what they cost or whether they are creating more value than they consume, and it’s harder still to find the data to support that answer.
It is at this point you know for certain that experimentation has become an operating model problem.
Most organisations have agents working somewhere in the business. Each of them was built for a particular purpose and may work perfectly well. The difficulty comes as more are launched over time and the people involved in creating them move on, eventually leaving nobody with a complete view of what is running.
Such a situation is often treated as an infrastructure challenge. Yes, data, integration, and capacity are important. But the underlying issue here is that we are relying on management systems that were designed either for software, which is deployed and maintained, or for people, who are managed and developed. Agents are neither. Instead, the systems we deploy need to account for the fact that agents act with some independence, incur costs whenever they run and change the work around them.
Why Multi-agent AI Creates a Governance Challenge
We have identified four disciplines that can prevent organisations from losing control, and they are far easier to establish while agent numbers remain small.
Give every agent a named human owner: This should be one person, rather than a team or function, who is accountable for what the agent does, whether it is still needed and what it costs. When output degrades or a process changes, there is someone responsible for noticing and acting. As we set out in The Trust Imperative, organisations that establish ownership when an agent is created are far better placed to remain in control as their use of agents scales.
Enforce boundaries. Every agent needs explicit limits: what it can access, what it may decide alone and what it must escalate. Most organisations think through these boundaries and document them, but only a few go further. Boundaries need to be built into the platform that runs the agent, so that any attempt to cross one is stopped and recorded. Such enforcement shifts AI risk from a debate about intentions to a review of evidence.
Keep a central register of what is running: Record every agent, its owner, purpose, boundaries, status and running cost. This may sound overly administrative, but it ensures the organisation can answer the questions that leaders will eventually ask: how many agents do we have, which parts of the business depend on them, what are they costing and which are earning their keep? It also enables agents to be retired when they are no longer needed.
Measure the outcome, rather than the agents themselves: An agent can perform its own task well and still leave the business worse off. A document extraction agent that completes 98% of its tasks correctly may look impressive in KPI terms, but if the remaining 2% creates errors that require costly checking and rework downstream, then the agent has been unsuccessful in outcome terms.
Business Outcomes Matter More Than Agent Performance
That example points to the unit we need to measure, and the one that matters is the value stream: the complete path to an outcome, across every team and system it touches. It is at this level that we establish the baseline and targets, and identify a named owner who is accountable for the result. Agents are measured underneath it as contributors. Their measures help diagnose where a problem sits; they do not, alone, tell us whether the business is better off. This is the argument we make in The Operating Layer.
And cost belongs at the same level. This includes the costs of the models and platform, but also the costs of human supervision, checking, rework and governance. Comparing these costs with the previous cost of delivering the outcome informs whether to invest, redesign or stop. In contrast, the token bill alone tells you very little.
The organisations that scale agentic AI successfully will not be those with the most agents. They will be those that can say, at any moment, what is running, who is accountable, what it costs and what business outcome it improves.
