Ceremonial Compliance

A governance process that runs on schedule, produces records, and closes without escalation describes itself. It does not, by that fact alone, describe the system it is supposed to govern. Ceremonial Compliance is the belief that because the oversight exists and operates, it must be constraining what the system does.

Australia's Robodebt scheme is the case. For four years an automated program issued benefit-debt notices by averaging annual income across every fortnight, a method lawyers inside the government had warned was unlawful before launch. The scheme had everything a governed program is supposed to have: internal committees, audit cycles, a complaints process, an Ombudsman with jurisdiction, two Senate inquiries, and legal advice on file. About 443,000 of those debt notices were wrong.

Every one of those structures operated. The committees met. The Ombudsman investigated and recommended improvements. A Senate committee told the government to pause the scheme. The legal advice sat in the department. And the scheme kept running through all of it, because each channel reached its own conclusion and closed, and none of them had a mechanism that actually stopped the machine between conclusions.

That is the trap. Oversight that produces records is easy to mistake for oversight that produces constraint. The records are real. The meetings happened. But a meeting that cannot halt the system, an audit that cannot change it, and a report the system outlives are governance in form only. The activity becomes its own evidence, and the question of effect never gets asked.

AI raises the stakes, because the records are even easier to generate and even easier to trust. Model cards, review boards, audit logs, and sign-off checklists all accumulate. If none of them can stop a deployment that is causing harm, they are ceremony. The test of governance is not whether it ran. It is whether, on the day the system needed to be stopped, anything in the structure could stop it.