The AI Risk Your Governance Framework Isn’t Built For
Most organisations approving AI adoption strategies right now are solving the wrong problem. Their governance frameworks are oriented outward — defending against deepfakes, AI-powered phishing, synthetic fraud from external actors. The threat modelling is, in almost every case, built around what an adversary outside your organisation might do with these tools.
A recent case in the UK ought to reorient that thinking. A Derbyshire police officer is under criminal investigation, the first known case of its kind in the UK, over allegations of using artificial intelligence to fabricate evidential material across multiple cases and perverting the course of justice. No adversary was involved. This was a trusted insider inside a controlled institutional environment, using generative tools to manipulate material that the system around them was never built to check — which is the scenario your audit committee has not discussed.
The Inside Threat Is Structurally Invisible
Insider AI misuse is missing from most governance frameworks for a simple reason. Those frameworks were drafted to manage what AI risk looked like from a distance, before the tools were on every desk. Now that every knowledge worker can reach capable generative models through a sanctioned platform, a personal phone or a browser tab, the surface for internal misuse is very large and effectively unwatched. That is a governance problem rather than a technical one.
Think about what fabrication looks like away from a police station. A procurement officer inflating competing quotes. A compliance analyst producing documentation that supports a finding the underlying evidence does not. A manager backfilling performance records after the fact. A professional services firm generating deliverables with AI and billing them at human-effort rates without telling anyone. None of it takes technical sophistication or access to a restricted system, and in most organisations none of it would be detected.
Most AI Governance Frameworks Don’t Address This
When boards ask their security leadership or their consultants about AI governance, they receive frameworks built around three categories: data privacy (what data goes into AI systems), model risk (what outputs the organisation relies on), and external threat (what adversaries do with AI). These are legitimate concerns. They are also not the concern this article is about.
Insider misuse falls between all three. It is too operational for AI governance frameworks, which are policy documents rather than detection programmes; too behavioural for cyber security programmes built around technical controls; and too unfamiliar for audit functions, which have no settled methodology for assessing AI-assisted misconduct.
So most Australian organisations, across every sector, have no detection capability worth the name for this class of problem. No log review, no provenance on generated output, nothing in the workflow that would flag unusual use of these tools in a sensitive process. There is generally a policy asking people to use AI responsibly. A policy is not a control.
What This Looks Like in Australian Regulated Environments
Australian boards operating under APRA CPS 234, or executives accountable under the SOCI Act, should be particularly uncomfortable here. Both frameworks place accountability for information integrity directly on the institution. CPS 234 requires entities to maintain information security capability commensurate with the threat. If the threat now includes AI-assisted fabrication by internal actors — and it does — that framework expectation applies.
The Privacy Act reforms moving through Australian regulatory debate have focused heavily on the rights of individuals relative to data held about them. That conversation is necessary. But it has almost nothing to say about what happens when the records themselves — the documents, assessments, reports, and analyses inside an organisation — are quietly contaminated by generative AI misuse. The OAIC has no current posture on this. Neither does the ASD’s Essential Eight, which was not designed for this problem.
This is not a criticism of those frameworks for failing to anticipate something that has just emerged. It is an observation that the gap exists and is not being filled by anyone with authority to fill it.
The Vendor Accountability Problem
Part of the reason this stays invisible is that nobody selling AI into enterprises gains anything by raising it. The pitch is productivity, competitive advantage, and responsible AI — by which the industry almost universally means fairness and bias, not misuse detection. No major vendor offers customers useful telemetry on how staff are using generative tools inside sensitive workflows, and no market pressure exists to make one build it.
Consultants are barely better. AI governance engagements produce frameworks because frameworks are billable and pleasant to present. Working out whether the legal team is generating contract precedents it has not verified, or whether finance is producing numbers it then presents as analysis, is slower work, more confrontational, and far more likely to end in findings a client did not want.
What Boards Need to Do
Three questions are worth putting to management before any further expansion of AI tooling gets approved.
Which of our workflows produce consequential material — legally, financially, reputationally — and what verifies that the material was not generated or altered by AI without disclosure? Then the harder one: if someone had fabricated something in one of those workflows six months ago, would we know today? Not in principle. In practice: would a log exist, would an audit trail carry it, would anyone have looked? And finally, does our AI governance framework contain one actual control, as distinct from a policy statement, aimed at detecting insider misuse?
Most management teams cannot get through those three. That is the finding, and the answer to it is not to pause adoption — the productivity case is real and so is the competitive pressure. The answer is to stop treating a responsible-use policy as governance.
Most organisations will file the Derbyshire case as a curiosity: a policing story, a British one, somebody else’s. It is better understood as the first confirmed public instance of something happening in plenty of places with far less scrutiny than a police force operates under. Nobody is investigating yours.