01. The definition

The definition

AI governance engineering is the application of engineering practice — systems thinking, product thinking and code — to the governance of AI systems.

Read the sentence in three parts. Engineering practice means we build, run and measure governance the way engineers build, run and measure anything else: as versioned systems with tests, telemetry and owners, not as documents. Systems thinking and product thinking mean we treat governance as a whole that spans data, model, pipeline, runtime and organisation, delivered as a product to the engineers who are its users. The governance of AI systems is the subject: the whole span of governance, risk and assurance for AI — including autonomous agents — not one narrow slice of it.

The framing is borrowed, deliberately. GRC engineering defines itself as “the application of software engineering practice, systems thinking and product thinking to governance, risk and compliance”¹. AI governance engineering is that same move, aimed at the governance of AI. It is the parent discipline pointed at a faster, stranger target.

Three clarifiers

It covers governance, risk and assurance of AI systems — including agents. The scope is not “compliance.” It runs from setting the rules (governance), through identifying and reducing what can go wrong (risk), to producing the evidence that the controls work (assurance). It explicitly includes autonomous and agentic AI, because that is where the hardest governance problems now live: an agent that browses, executes code, calls APIs and acts under delegated authority is the object that legacy governance can least see.

It is a capability, not a job title. You do not need “AI governance engineer” on your business card to do this work, and having the title does not mean you are doing it. It is a set of practices — policy-as-code, eval gates, agent registries, continuous assurance — that a security engineer, a privacy engineer, an MLOps engineer or a governance lead can each develop. The market is forming the role (technical AI-governance roles in the tech sector report a median near USD 221,000, the highest band in IAPP’s survey², and the AI governance platform market is forecast to grow from tens of millions of dollars to well over a billion by 2030³) — but the discipline is defined by the capability, not by the vacancy.

It is measured by realised risk reduction and audit-ready evidence. There are exactly two tests. Did the risk actually fall — measurably, in production, not on a maturity slide? And can a regulator or auditor read the proof — as machine-readable evidence, not a reassembled screenshot? A control that passes neither test is theatre. Framework coverage, number of policies written and committees held are inputs at best; they are never the measure.

The disambiguation cluster

The discipline is defined as much by what it is not as by what it is. Eight neighbours are routinely confused with it. Each shares a border; none is the same thing.

In prose: the frontier lies where a discipline stops. AI safety research asks whether a model is safe; we ask whether the deployed system is governed, and prove it. MLOps answers “is the model serving?”; we answer “is it allowed to serve, and what evidence says so?” — governing the very pipeline MLOps runs. Model risk management in the SR 11-7 tradition validates a model at points in time; we govern the system continuously, including agents that have no analogue in a credit model. AI compliance and legal tell you what the law requires; we build the control that meets it and the evidence that shows it, and we depend on lawyers to tell us we got the obligation right. Responsible AI and AI ethics set the values; without engineering, those values stay on a poster. GRC engineering is the parent method, and we are its AI specialisation. We inherit three of the five stack layers almost unchanged — govern-as-code, machine-readable evidence and continuous assurance — along with the “green dashboard over a broken control is theatre” test; what AI forces us to add is the other two, evals and red-teaming as controls and agent identity and runtime control, because a model whose behaviour must be tested and an autonomous actor that acts under delegated authority have no analogue in classic GRC. AI security engineering is the sibling we overlap with most, and the overlap is a feature, not a boundary dispute: the same person often wears both hats. The line is not “framing versus fences” but deliverable. Security engineering’s deliverable is a defended system — it stops the attack. AI governance engineering’s deliverable is a governed and evidenced system: the rights-and-obligations record (which control answers which article, with the proof attached) and continuous assurance as a product an auditor or regulator can query. A red-team eval is security work and governance work at once; it becomes governance when its result is filed as evidence against an obligation. Security asks “is it safe from attack?”; we ask “is it governed, and can we prove it?” — and we usually need the security answer as an input to ours. And Visure’s use of the identical phrase points the other way entirely: governing the AI that assists engineering work, not engineering the governance of AI. We claim the second meaning and disambiguate the first on sight.

The object of governance

What, concretely, does this discipline govern? Five nested objects, each needing different controls:

• Models. The trained artefacts — foundation models, fine-tunes, classifiers — with their provenance, capabilities, evaluations and known failure modes. Governed with model cards, evals and AIBOM.
• Systems. The application around the model: prompts, retrieval, tools, orchestration, the human and machine users. Most risk is here, not in the raw model.
• Agents. Systems that act — browse, execute code, call APIs, move money, delegate to other agents. Governed with identity, bounded scope, tool mediation, runtime guardrails and kill switches. This is the hardest and newest object, and the one legacy governance cannot see.
• Data. Training data, retrieval corpora, prompts and outputs — with their lawful basis, rights, provenance and retention. Governed with data cards, DPIAs and lineage.
• The organisation. The roles, decision rights, escalation paths and accountability that surround all of the above. Governed with an operating model, RACI and an incident pipeline. A control with no owner is not a control.

The discipline is coherent only when it addresses all five. A model card with no agent registry, or an agent registry with no runtime data path, governs one object and leaves the others open.

The three questions

At any moment, an AI governance engineering function must be able to answer three questions about production — instantly, from live systems, not from a document last updated a quarter ago:

1. What AI is running? Which models, systems and agents are live, in which version, owned by whom. This is the job of the inventory and the agent registry — and it must be fed by a runtime data path, not typed into a spreadsheet.
2. What is it allowed to do? The scope, permissions, guardrails and policy that bound each system and agent. This is the job of governance-as-code and runtime controls — identity before autonomy, scope before action.
3. What evidence proves it? The machine-readable, audit-ready record that the controls fired and the risk fell. This is the job of evals-as-evidence and continuous assurance — evidence as a by-product of the build.

These three questions are the spine of the whole Body of Knowledge. The five-layer stack (chapter 04) is built to answer them: Govern-as-Code and Inventory & Transparency answer what is running and what it may do; Evals & Red Teaming and Assurance & Continuous Compliance answer what evidence proves it; Runtime Controls & Observability keeps all three answers true as the system changes. The threats those controls are built against — prompt injection, tool misuse, agent identity and privilege abuse, rogue agents — are catalogued in OWASP’s Top 10 for Agentic Applications⁵, and the patterns that answer them are in chapter 05.

The limits of the eval gate

This book leans hard on evals as controls, so it owes the reader the same “theatre” test it applies to everything else. An eval gate is necessary; it is not sufficient. Take it seriously and its limits follow directly:

• It is point-in-time and sampling-bound. An eval proves the system passed these cases at this version. It says nothing about the inputs it did not sample, and nothing about tomorrow’s model.
• It is Goodhartable. The moment a threshold gates a release, there is pressure to tune the model to the suite or the threshold to the model. A gate optimised against becomes a number that rises while the risk it stood for does not move.
• It catches regressions, not novelty. A suite tests known failure modes. A novel jailbreak or an attack the suite never imagined passes green, because nothing in the gate was built to see it.

None of this argues against the gate; it argues for how the gate must be run. The eval suite is itself an artefact to be governed: its coverage measured, its cases maintained adversarially, its thresholds traced to named failure modes rather than round numbers. And a passing gate obliges runtime monitoring — layer 04 — rather than replacing it. An eval is the control at build time; the guardrail and the trace are the control at run time, against the inputs no eval anticipated. A discipline that treats a green gate as proof of safety has rebuilt framework theatre with a faster pipeline.

Maps to: EU AI Act Art. 9 (risk management), Art. 11/49/71 (documentation and registration), Art. 55 (GPAI systemic-risk duties) · ISO/IEC 42001 (AI management system) · NIST AI RMF (Govern, Map, Measure, Manage) · OWASP Top 10 for Agentic Applications 2026. Mappings are illustrative, not a claim of conformity.

Sources

1. [1] “What is GRC Engineering” (Ayoub Fandi). GRC Engineer. 2025. https://grcengineer.com/what-is-grc-engineering/ (verified: primary)
2. [2] Salary & Jobs Report 2025-26. IAPP. 2025-08-03. https://iapp.org/resources/article/salary-survey-summary/ (verified: primary)
3. [3] “Global AI Regulations Fuel Billion-Dollar Market for AI Governance Platforms”. Gartner. 2026-02-17. https://www.gartner.com/en/newsroom/press-releases/2026-02-17-gartner-global-ai-regulations-fuel-billion-dollar-market-for-ai-governance-platforms (verified: primary)
4. [4] “AI Governance Engineering”. Visure Solutions. 2026. https://visuresolutions.com/ai-engineering/ai-governance-engineering/ (verified: primary)
5. [5] Top 10 for Agentic Applications 2026. OWASP GenAI Security Project. 2025-12-09. https://genai.owasp.org/resource/owasp-top-10-for-agentic-applications-for-2026/ (verified: primary)