Every CISO has a version of the same story by now: someone in finance pasted a draft term sheet into a public chatbot to tidy up the language, or a product manager fed customer data into an AI tool to write a brief, and nobody in IT knew until the logs turned up months later. That story has become boardroom shorthand for “shadow AI,” and it has done real damage. IBM’s 2025 Cost of a Data Breach Report found that shadow AI added an average of $670,000 to the cost of a breach, and that 63% of organisations surveyed had no policy at all governing the use of unapproved AI tools. Among companies that suffered an AI-related security incident, 97% lacked proper access controls on their AI systems. Those are damning numbers, and most large enterprises have spent the past eighteen months responding to them: acceptable-use policies, approved-tool lists, DLP filters tuned to catch prompts containing anything that looks like a contract clause or a customer record.
The trouble is that this response, however overdue, is aimed at yesterday’s problem. The employee pasting text into a chatbot is a human being who can be trained, monitored and, if necessary, disciplined. The next wave of exposure isn’t human at all, and it is arriving at a scale that most governance functions were never built to count, let alone control.
The identity math nobody has done
Every AI agent a company deploys, whether it is a customer-service bot, a code-review assistant, a procurement workflow, or a fraud-monitoring system, needs an identity: credentials, permissions, an entry in some access-control system, a way of proving it is who it says it is when it touches a database or calls another system’s API. These are “non-human identities,” and they have existed in enterprise IT for years in the form of service accounts and API keys. What has changed is the multiplier. According to the Cloud Security Alliance’s 2026 State of Non-Human Identity and AI Security survey, non-human identities now outnumber human employees by roughly 45 to 1 in the average enterprise. A company with ten thousand employees, on that ratio, is nominally responsible for something close to half a million machine identities with standing access to its systems.
The problem isn’t the number itself; it’s what the same survey found sitting behind it. Ninety-two percent of respondents said their existing identity and access management tools cannot properly manage AI and non-human identity risk. Roughly half reported no clear ownership or accountability for agent identities at all, meaning that if something goes wrong, there may be no obvious person whose job it was to have prevented it. Most tellingly, only 28% of organisations said they could trace an agent’s actions back to an accountable human across all of their environments. In other words, most companies cannot currently answer a very basic question: when an autonomous system does something, who authorised it, and who is responsible for what happens next.
That question used to be trivial. It is now the central governance problem of the AI era, and it will not be solved by better chatbot policies.
Why locking it down makes it worse
The instinctive corporate response to this kind of sprawl is to tighten control: centralise approval, restrict who can deploy an agent, subject every new use case to security review. Gartner’s research suggests this instinct, applied uniformly, tends to backfire. In a May 2026 analysis, the firm predicted that 40% of enterprises will demote or decommission autonomous AI agents by 2027 specifically because governance gaps were only discovered after something had already gone wrong in production, and it identified the root cause as organisations treating agent oversight as a binary switch, locked down or fully trusted, rather than a graduated system matched to what an agent is actually allowed to do. As Gartner analyst Shiva Varma put it, over-restriction of low-risk agents “slows delivery and drives shadow development,” pushing frustrated teams to build unauthorised agents outside official channels, while under-restriction of genuinely autonomous systems raises operational and compliance risk directly. Either failure mode recreates the exact sprawl that governance was meant to prevent; it just moves the evasion from the tool layer, where shadow AI lived, to the agent layer, where it is much harder to see.
Gartner’s proposed fix is a four-level autonomy framework, running from agents that only observe and log, through agents that advise a human decision-maker, to agents that act with a human sign-off, up to agents that act independently within defined guardrails. The point for a CEO or CIO isn’t the specific taxonomy; it’s the underlying principle that governance has to be calibrated to what an agent can actually do and where it operates, not applied as a single blanket policy across an organisation that now has more autonomous actors than staff.
India’s regulators are already there
Two Indian regulatory tracks, developed independently, have converged on almost exactly this problem, and both put the accountability question squarely at board level rather than treating it as an IT ticket. The Reserve Bank of India’s FREE-AI framework, published in August 2025, sets out seven guiding principles for AI in financial services, including Accountability and Explainability, translated into 26 recommendations spanning six areas from infrastructure and policy to governance and assurance. It explicitly asks regulated entities to establish formal AI governance and accountability mechanisms before deployment, not after an incident forces the issue.
Separately, India’s Digital Personal Data Protection Rules, notified in November 2025 and phasing in through May 2027, require Significant Data Fiduciaries to verify, under Rule 13, that the algorithmic software they use does not pose a risk to the rights of the individuals whose data it processes, and require specific consent before personal data can be used to train AI models. For India’s IT services firms and global capability centres, which increasingly build and run agentic workflows on behalf of clients across multiple jurisdictions, this creates a distinctive compounding exposure: they are simultaneously the deployer of agentic systems for global principals and a data fiduciary in their own right under Indian law, answerable to two accountability regimes for the same underlying automation. Neither framework asks companies to stop building agents. Both ask them to be able to say, credibly, who is responsible when one of those agents acts.
The competitive question underneath the compliance one
It is tempting to read all of this as a compliance burden to be minimised. That undersells what is actually at stake. The organisations that build a genuine identity and accountability layer for their non-human workforce, one that can say which agent did what, under whose authorisation, with what data, will be able to deploy autonomous systems faster and with less regulatory friction than competitors who are still discovering their exposure one incident at a time. Insurers are already pricing the difference: IBM’s $670,000 breach premium for ungoverned AI is an early signal of what underwriters will charge organisations that cannot answer basic accountability questions about their own automation.
The uncomfortable truth for boards is that most of them approved AI adoption as a productivity initiative and delegated its governance to security teams as an implementation detail. The numbers now suggest the reverse ordering was correct. A company’s non-human workforce is on track to outnumber its human one by an order of magnitude, and the executives who treat that as an org-design and accountability question, not a ticketing queue, are the ones who will not end up in next year’s breach report.
