Failure Museum.Made with ❤️ by Aniruddh Jangid
Submit a case

EXHIBIT 072 · PUBLIC POLICY

Robodebt: automation gave a weak inference authority

The Royal Commission examined an unlawful debt-recovery scheme that used income averaging.

OrganizationAustralian government agencies
Period2015
LocationAustralia
StatusInvestigated
ImpactHigh
EvidenceOfficial investigation
Compare

Illustrative draft · Source-backed synopsis · Museum analysis marked separately · Updated 5 October 2026

Entrance to the Centrelink building in Burnie, Tasmania, in August 2016; contextual image of the welfare agency.
Entrance to the Centrelink building in Burnie, Tasmania, in August 2016; contextual image of the welfare agency.Gary Houston / Wikimedia Commons. Resized. Photograph · Context photograph; not the incident itself. · CC0 · Resized for display.
01 / EXECUTIVE SUMMARY

The outcome is only the beginning.

The Royal Commission examined an unlawful debt-recovery scheme that used income averaging. [1]

THE PRINCIPLE TO CARRY FORWARD

Validate legal authority and individual-level evidence before automation, and preserve accessible human review.

02 / THE PROMISE

What the system was trying to do.

Recover welfare overpayments efficiently.

Museum interpretation

The assumption beneath the promise

Museum hypothesis to examine: A population-level or averaged data relationship is adequate evidence for a consequential decision about an individual. This is an analytical proposition to test against the record, not an attributed statement by the organization.

Success should be assessed against the intended service and its safety, reliability, or integrity constraints. A headline outcome cannot tell us whether those constraints were visible, tested, or owned before the event.

03 / WHAT HAPPENED

A sequence, not a single moment.

The intended system

Recover welfare overpayments efficiently.

Source-backed synopsis

The event or review

The Royal Commission examined an unlawful debt-recovery scheme that used income averaging. [1]

Source-backed synopsis

The documented aftermath

The scheme ended and a Royal Commission issued findings and recommendations. [1]

Source-backed synopsis
Timeline of documented milestones. Gaps are not evidence that nothing happened. Sources: [1]
04 / WARNING SIGNS

What could be seen at the time?

Documented in cited record

Legal and evidentiary concerns did not reliably stop the programme. [1]

Who could see it?

See the cited investigation for named teams and the information they held. The synopsis does not infer awareness by every stakeholder.

Why was it not enough?

Signals need interpretation, authority, and a route to action. This is an analytical question, not proof that a warning was deliberately ignored.

05 / KEY DECISIONS

Choices made within constraints.

Museum analysis: A data-processing technique needed a lawful basis and evidence adequate for an individual debt.

Available alternatives

Pause, test, narrow the operating envelope, seek independent review, or build a recovery option. These are proposed analytical alternatives; feasibility at the time is not established.

Information available

Legal and evidentiary concerns did not reliably stop the programme. [1]

Incentives & constraints

Delivery pressure, cost, authority, and incomplete knowledge may shape a decision. Their specific weight is not established by this synopsis.

Risk accepted

Ask whether the relevant risk was recognized, who had authority to accept it, and whether affected people understood its implications.

06 / SYSTEM MAP

How conditions connected.

Select a node to inspect its evidence. Links show a proposed analytical relationship, not a measured causal effect.

05 / External context04 / Warning signal03 / Decisions02 / Technical conditions01 / Trigger06 / Consequences
Triggering event

The Royal Commission examined an unlawful debt-recovery scheme that used income averaging. [1]

Legend: numbered nodes = analytical categories; dashed arrows = proposed influence or propagation. Museum interpretation informed by the sources below. Feedback and omitted influences require further investigation.
07 / CONSEQUENCES

Beyond the immediate event.

People faced wrongful demands, distress, and burdens of disproving claims. [1]

Financial impactCustomer impactRegulatory impact

Categories identify documented or relevant consequences. Financial, human, and environmental totals are not estimated here when the source base does not support them.

08 / WHAT THE MODELS EXPLAIN

More than one lens.

Analytical lens

Systems thinking

Examine relationships, boundaries, feedback, and incentives across a whole system.

In this case, use the lens to examine invalid inference. This application is museum interpretation, not a finding of the original investigation.

Limit: A broad lens needs explicit boundaries and evidence to avoid explaining everything after the fact.

Analytical lens

Robust decision-making

Choose strategies that perform acceptably across many plausible futures.

In this case, use the lens to examine invalid inference. This application is museum interpretation, not a finding of the original investigation.

Limit: Introductory guide. This lens generates questions; it does not establish causation or replace case-specific evidence.

Analytical lens

Principal-agent problem

Study conflicts between delegated decision-makers and those affected by their choices.

In this case, use the lens to examine invalid inference. This application is museum interpretation, not a finding of the original investigation.

Limit: Introductory guide. This lens generates questions; it does not establish causation or replace case-specific evidence.

09 / COUNTERFACTUALS

What could have changed the outcome?

Counterfactuals are hypotheses. They identify possible intervention points without claiming that a different choice would certainly have prevented the event.

INTERVENTION 2 / During early testing

Test the operating envelope

Exercise realistic boundary conditions and shared dependencies rather than validating components only in isolation.

Cost & feasibility

Requires time and independent review before commitment.

Likely effectiveness

May reduce exposure or consequences. Not quantified; feasibility and effect must be assessed against evidence available at the time.

10 / LESSONS

From hindsight to a usable practice.

FOR FOUNDERS

Validate legal authority and individual-level evidence before automation, and preserve accessible human review.

Before accelerating hiring or spending, name the assumptions that remain untested and set evidence thresholds for the next commitment.

11 / AFTERWARD

Recovery is another investigation.

The scheme ended and a Royal Commission issued findings and recommendations. [1]

A corrective action is evidence of a response; it is not, by itself, evidence that the wider pattern has disappeared.

12 / SOURCES & EVIDENCE

Follow the record.

Documented claims are linked to sources. Analytical applications, lessons, and intervention proposals are museum interpretation. This draft does not establish motives or a single complete causal explanation.

  1. 1
Editorial history & limits

Version 1 · 4 October 2026 — source-backed illustrative synopsis created. Independent editorial review is pending. No claim of exhaustive investigation. New sources may alter the analysis.