A retrieval system that answers a legal question correctly, cites a real article of a real regulation that is genuinely in force, and is completely wrong. Five checks pass. Nothing raises an error.
Indonesian minimum wage is recalculated every year by a formula fixed in government regulation. Payroll vendors, PEOs and compliance tools all need that formula. It is a narrow, factual, well-sourced question, the easy case for retrieval.
I believe most AI demos are built backwards: they show a system being confident, when the useful thing to show is a system that knows where it has to stop. Below is a real run against the real corpus. Nothing is simulated: every document, status, article number and quoted line was pulled from the government's own legal database and is reproducible with the commands in the appendix.
Idle. Press Answer it to run the query the way a competent RAG stack would: status filter, vector retrieval, verbatim citation.
None of these five layers malfunctioned. Each did exactly what it was built to do, reported success, and handed a corrupted result to the next one. That is what makes the failure expensive: there is no error to alert on, no exception to catch, and no confidence score that drops.
The gate does not need a better model. It needs the source chain to be checkable, and it needs permission to refuse when it is not:
Each claim on this page traces to a command. The chain, statuses and relations come from the government's public legal database; the page counts and text coverage come from the published PDFs themselves.
# the amendment chain, walked from the live source curl -s "https://peraturan.bpk.go.id/Details/161909/pp-no-36-tahun-2021" \ | grep -oE 'href="/Details/[0-9]+/[a-z0-9-]+"' | sort -u # the parent regulation: 75 pages, text on 2 of them curl -sL -o pp36.pdf "https://peraturan.bpk.go.id/Download/154587" pdfinfo pp36.pdf | grep Pages pdftotext -layout pp36.pdf - | tr -d '[:space:]' | wc -c # one article, two tokens, because OCR read 1 as l curl -sL -o pp51.pdf "https://peraturan.bpk.go.id/Download/328142" pdftotext -layout pp51.pdf - | grep -c 'Pasal 7l' pdftotext -layout pp51.pdf - | grep -c 'Pasal 71'