The AI Risk Your Governance Framework Isn’t Built For
Most organisations approving AI adoption strategies right now are solving the wrong problem. Their governance frameworks are oriented outward — defending against deepfakes, AI-powered phishing, synthetic fraud from external actors. The threat modelling is, in almost every case, built around what an adversary outside your organisation might do with these tools.
A recent case in the UK should reorient that thinking immediately. A Derbyshire police officer is under criminal investigation — the first known case of its kind in the UK — over allegations of using artificial intelligence to fabricate evidential material across multiple cases, and perverting the course of justice. This is not a nation-state actor. This is not a sophisticated external adversary. This is a trusted insider, operating inside a controlled institutional environment, using generative AI to manipulate material that the system around them was not designed to detect.
That is the scenario your audit committee is not discussing.
The Inside Threat Is Structurally Invisible
The reason insider AI misuse doesn’t appear in most governance frameworks is straightforward: those frameworks were written to manage what people imagined AI risk looked like when the technology felt distant. Now that every knowledge worker in your organisation has access to capable generative tools — through sanctioned platforms, personal devices, or simply a browser tab — the attack surface for insider misuse is enormous and almost entirely unmonitored.
This isn’t a technology observation. It’s a governance observation.
Consider what fabrication or manipulation looks like in a business context. A procurement officer inflating competitive quotes. A compliance analyst generating documentation that supports a finding the evidence doesn’t actually support. A manager fabricating performance records. A professional services firm generating client deliverables with AI and billing them at human-effort rates without disclosure. These are not hypothetical — they are the predictable, low-drama extensions of what a police officer allegedly did with evidential material, applied to the environments where your employees work every day.
None of these scenarios require technical sophistication. None of them require access to restricted systems. All of them are, right now, essentially undetectable in most organisations.
Most AI Governance Frameworks Don’t Address This
When boards ask their security leadership or their consultants about AI governance, they receive frameworks built around three categories: data privacy (what data goes into AI systems), model risk (what outputs the organisation relies on), and external threat (what adversaries do with AI). These are legitimate concerns. They are also not the concern this article is about.
The insider misuse vector sits in an uncomfortable gap. It is too operational for most AI governance frameworks, which tend to be policy documents rather than detection programmes. It is too behavioural for most cybersecurity programmes, which are built around technical controls. And it is too unfamiliar for most audit functions, which have not yet developed methodology for assessing AI-assisted misconduct.
The result is that the vast majority of Australian organisations — and this is not an observation specific to any sector — have no meaningful detection capability for this class of problem. No log review. No output provenance. No workflow controls that would surface anomalous use of generative tools in sensitive processes. The policy might say “use AI responsibly.” The policy is not a control.
What This Looks Like in Australian Regulated Environments
Australian boards operating under APRA CPS 234, or executives accountable under the SOCI Act, should be particularly uncomfortable here. Both frameworks place accountability for information integrity directly on the institution. CPS 234 requires entities to maintain information security capability commensurate with the threat. If the threat now includes AI-assisted fabrication by internal actors — and it does — that framework expectation applies.
The Privacy Act reforms moving through Australian regulatory debate have focused heavily on the rights of individuals relative to data held about them. That conversation is necessary. But it has almost nothing to say about what happens when the records themselves — the documents, assessments, reports, and analyses inside an organisation — are quietly contaminated by generative AI misuse. The OAIC has no current posture on this. Neither does the ASD’s Essential Eight, which was not designed for this problem.
This is not a criticism of those frameworks for failing to anticipate something that has just emerged. It is an observation that the gap exists and is not being filled by anyone with authority to fill it.
The Vendor Accountability Problem
One reason this risk remains invisible is that the vendors selling AI tools into enterprise environments have no commercial incentive to surface it. The pitch is productivity. The pitch is competitive advantage. The pitch is responsible AI, by which vendors almost universally mean fairness and bias considerations, not insider misuse detection. There is no major enterprise AI vendor offering customers meaningful telemetry on how their employees are using generative tools in sensitive workflows. There is no market pressure making them build it.
Consultants are not much better. AI governance engagements tend to produce frameworks because frameworks are billable. Detecting whether your legal team is using AI to fabricate contract precedents, or whether your finance function is generating numbers it then presents as analytically derived, requires a different kind of engagement — one that is slower, more confrontational, and more likely to produce findings that make clients uncomfortable.
What Boards Actually Need to Do
There are three questions every board and audit committee should put to management before approving any further expansion of AI tools inside the organisation.
First: which workflows in this organisation produce material that is consequential — legally, financially, regulatorily, reputationally — and what controls exist to verify that material has not been AI-generated or AI-manipulated without disclosure?
Second: if an employee used a generative AI tool to fabricate or misrepresent something in one of those workflows six months ago, would we know? Not in theory. In practice. Would a log exist? Would an audit trail surface it? Would anyone have reviewed it?
Third: does our AI governance framework contain a single control — not a policy, a control — designed to detect insider misuse of generative tools?
Most management teams cannot answer these questions satisfactorily. That is the finding. The appropriate response is not to pause AI adoption — the productivity case is real and the competitive pressure is real. The appropriate response is to stop pretending that a responsible use policy is governance.
The Derbyshire case will be treated by most organisations as a curiosity — a policing story, a UK story, someone else’s problem. That is the wrong read. It is the first confirmed public instance of something that is almost certainly already occurring in organisations with far less scrutiny than a police force. The difference is that nobody is investigating yours yet.