GovernanceDatapane.ai

“We Have a Governance Policy” Is Not the Same as Being Governed

Prabhat Kumar Goyal, VP of Customer Success at Datapane.Ai

6 min read - Published

“We Have a Governance Policy” Is Not the Same as Being Governed

Most enterprises believe they have data governance under control. In my experience, this belief almost never survives an audit.

The gap between having a governance policy and actually being governed is one of the most consequential and least discussed risks in enterprise AI today. Enterprises are moving fast to deploy AI against their most sensitive data assets, and the security controls they think are protecting them are often not doing the job they think they're doing. The numbers are stark: 97% of organizations that experienced AI-related breaches lacked proper access controls at the time of the incident. Not 97% of small companies or unprepared startups. 97% of the ones that got breached, across industries, including many that almost certainly had a governance document on file.

This is the governance illusion: the policy exists, the controls don't.

The Scale of the Problem

Enterprise AI adoption has surged to outpace the ability to secure it. Workforce AI adoption jumped from 22% in 2023 to 75% in 2024, a behavioral change that occurred irrespective of whether leadership had approved the tools or enforced policies. 47% of organizations using GenAI faced at least one adverse consequence in 2025, ranging from hallucinated outputs to exposing privacy and intellectual property leaks.

The regulatory environment is converging rapidly in the opposite direction. Federal agencies have imposed 59 new AI-related regulations in 2024 more than double the number in the previous year. Fines for AI governance failures reached $2.3 billion globally in 2024. The coming into force of the EU AI Act is looming. GDPR, CCPA, and HIPAA are being applied to AI-generated outputs that were never contemplated when those frameworks were written.

This creates a simple but serious hazard: organizations are adopting AI faster than they can govern it, within a regulatory environment that is actively seeking the gap.

Why Access Controls Are the Real Vulnerability

When one thinks of AI security, the conversation often turns to model security: jailbreaks, prompt injection, adversarial inputs, etc. However, in 97% of cases where breaches occurred due to AI, the victims had improper access controls in place, meaning the issue was not with the model's security but rather the fact that unauthorized users had access to the system

This is an access control issue, not an AI problem. This issue is especially acute in the enterprise due to the nature of how AI systems tend to aggregate access: a single agent or query interface can access multiple data repositories, schemas, and systems that previously required separate credentials and approvals. If the access controls at the agent layer are not granular, then every individual who can query the AI can query everything the AI can see.

Only 5% of organizations that deploy AI systems report confidence in their ability to secure AI models and data pipelines. 77% lack foundational data and AI security practices. These are not niche organizations; they represent the mainstream of enterprise AI deployment at the moment.

The Shadow AI Problem

In addition to the access control issue, companies are seeing shadow AI, employees using unsanctioned AI tools for their work and faster processing, unknowingly pasting data into a consumer AI product for processing, with no controls on where the data is going. 20% of organizations have had a security breach due to shadow AI already and this causes an extra $670,000 in breach costs, on average, compared to standard data breaches.

Shadow AI is happening because the sanctioned AI solutions are not able to be useful enough. If an enterprise business analyst can get a better or faster answer using a consumer AI product, they will do so, taking the data with them to get that answer. The answer to shadow AI is not more policy it is a better, sanctioned solution that is able to satisfy their need.

Governance You Can Prove

Here's the test I'd apply to any enterprise AI governance claim: could you prove it, right now, to an auditor?

Not "we have a policy that says access is restricted," but rather, for any given query, could you show who asked it, what data it accessed, what definition the model used, and whether the person who asked had the right to know the answer the model gave? Only 32% of the organizations in the study do regular audits of their models, and only 22% do the same for their frameworks. In short, most organizations are flying blind with their governance controls.

I think that's the real problem with governance controls - not that they're absent, but that organizations believe they have them when they don't. The auditor's report is what exposes that fiction, and the question is who finds it first: the auditor or the regulator?

What Enterprise-Secure Actually Looks Like

Real data security in an enterprise AI context comprises three non-negotiable properties, which have to be present in any solution, and they should be present everywhere

Encryption by default. Data in transit and at rest with no exceptions for convenience

Fine-grained access controls. Not role-based access at the system level but query-level, field-level, context-level controls to ensure the right person sees the right data for the right reason and no more

Immutable audit trails on every answer. Every query, every result, every definition applied, and every data source touched is logged, time-stamped and traceable. This is a property of the system, not an afterthought

These properties are essential for enterprise security and governance and are typically found in systems where security and governance are designed into the solution. Organizations with integrated security and governance approaches report 45% fewer compliance violations and 60% faster incident resolution times than those with security and governance functions operating as separate entities.

The Bottom Line

Nearly all organizations, 98% expect AI governance budgets to increase. This is a strong indicator that companies want to move from compliance mentality to proactive governance. But increased governance budget is not the same as filling the governance illusion gap. The way to truly address the governance illusion gap is to give teams confidence in the answers coming from the system by making sure that every answer includes an audit trail. In other words, it should be possible to demonstrate that the correct governance controls were applied to any particular output.

Datapane is designed from the ground up to address this requirement. Every query can be traced back to its source, and every answer includes lineage, definitions, and permissions related to that specific output. With Datapane, security and governance are built into the system by default, rather than bolted on as an afterthought.

After all, governance that cannot be proven does not exist.