On 18 June an AI agent researching government spending on medicines got into the Medicare statistics portal. By OpenAI’s own account it reached non-public parts of the system, retrieved credentials and wrote files. No personal records were involved. At a parliamentary hearing on 6 October, OpenAI’s chief strategy officer called the way in “not super-sophisticated”.
I wanted to know how unsophisticated, so I read the portal the way a model would, through what it had published about itself. The Internet Archive holds its pages and the vendor publishes its manuals. The portal’s own scripts wrote the server’s directory layout into every visitor’s browser console. One sentence in the vendor’s documentation says how every anonymous visitor was signed in. And the address was still humanservices.gov.au, seven years after the Department of Human Services became Services Australia. Almost every estate I have worked in has a system like that, and it keeps its old name for the same reason it keeps its old defaults: changing either one means somebody deciding something about it.
The way in was configured fifteen months before the agent arrived, by a project doing its job. By the public reconstruction of its archived code, a migration in March 2025 moved the portal onto a newer release and switched on guest access for every anonymous visitor, so the public statistics would stay public. No published vulnerability is needed to explain what followed, and nothing that would have stopped it was invented this year.
The industry has a name for refusing anything built elsewhere: not invented here. The version that does the damage now runs the other way. It is software nobody here invented, nobody here owns and nobody here can describe, and since nobody can describe it nobody can say which parts are not needed, so all of it stays online. Taking it off is a decision, and it belongs to whoever signs the budget.
Projects end and systems don’t
Guest access was a sensible call, and the vendor documents what it means: every anonymous visitor runs as one shared service account. From that day the audit trail could not tell an AI agent from a health economist.
Organisations fund software as projects and run it as systems, and only one of those has an end date. Up to go-live there is a team that knows exactly what it configured and why. Then the project closes and the budget line becomes a support renewal, which buys patches but not the people who knew why. The same thing happens when a partner’s contract ends: the code and the runbook go across, and what the system is for leaves with the people. From then on the vendor’s defaults are the organisation’s policy, and defaults reach further than the purpose.
The public side of that portal ran on three things: the report pages, the one anonymous endpoint those pages called and the static files they loaded. The rest of the platform was reachable from the same address, and the Internet Archive holds a copy of its login page to prove it. A load balancer already sat in front, and it could have refused everything outside those three paths the day somebody wrote them down. That needed no purchase, which is why it is on no vendor’s roadmap.
Patience was the control
The portal’s real defence was an assumption about its callers. A person reading statistics asks a few questions and leaves, and a script asks the same few every month, so a system can run for years on the bet that nobody will be patient enough to ask for everything else. Patience was the control, and nobody wrote it down, so nobody saw it expire.
An agent is patient, pays almost nothing per attempt and arrives having read the manual. Buying software has always meant somebody else knows more about it than you do. That was safe while the somebody was the vendor and a stranger had to work to learn the same things. A model has done that work in advance, and has probably read more about the product you bought than anyone still on your staff. AI is a refiner, as I argued in The Option to Abandon: it finds what is already there, and every path left reachable will be asked about, politely and at length. That is all “not super-sophisticated” means. Every unused path was always a cost. Patience kept it off the bill.
After an incident like this a steering committee usually funds three things: a patch, a filter in front and an upgrade behind. This portal needed no patch, already had a load balancer in front and had been upgraded fifteen months earlier. I’ve spent most of my career in content delivery, and nothing I could have built or bought to sit in front of it would have changed the outcome. An edge stops strangers who look like strangers, and a patient caller asking politely for things a system should never answer does not look like one. A filter has to guess what an attacker looks like. A list of three paths only has to know what the system is for. Banning the tools does no better. It only stops one side reading.
What an owner knows
The hard part is deciding what comes off the internet, because “does not need” is a claim about purpose and only an owner can make it. The vendor’s manual says what a system can do. Owning it means being able to say what it should do in your organisation: which requests it serves, which jobs run, where it may reach and whose name is against it. That is the one thing no model has read, and the only information advantage you have left.
Try it on one system nobody here wrote. The first answers come quickly, because anyone in the room could name them from memory. Somewhere past the first few dozen, someone stops and says they are not sure what that one does any more. It is usually a scheduled job or an account with standing access, set up by someone who has left, and it stays switched on because nobody can prove what would break. That pause is the finding. Everything after it is surface you are carrying for no one.
Written down in a document, that answer drifts the same way the system did, because the next person to change the system will not open it. It only outlasts its owner once it is the control. For the portal the answer is three lines, and the same three lines in the load balancer would have been both its description and its defence. The lasting version needs no engine at all: generate the monthly reports on a schedule and serve them as files, which have almost nothing to attack. Publish the answers, not the engine. Once the portal was switched off, the minister’s answer for what it was for was data.gov.au.
Every system that stays needs an owner, a person and not a team, and few organisations have enough people for everything they run, so the estate has to come down to what they can own.
Agents need the same subtraction. I have been running them since Claude Code was released, and the first thing you learn is that an agent has to be sandboxed or it wanders. Whatever it can reach ends up in its context window, and from there in its work. That is all that happened in June: someone set a goal, and the agent executed. The boundary has to be enforced outside the model, because one the agent is only asked to respect is a suggestion.
What we can do today
On 10 November the Prime Minister announced that frontier AI companies operating in Australia will have to meet mandatory safety and security obligations under the Australian AI Standards legislation, enforced by a new regulator working with the Australian AI Safety Institute. The companies will have to keep security controls across development and deployment, and notify the Government of incidents rapidly. That is the right place for a boundary outside the model, and the notification rule answers June directly: OpenAI told Services Australia 84 days after its agent got in.
A notification arrives after the call has been served, and only from companies within reach of an Australian regulator. As the Minister for Home Affairs put it in the same release, “Those who want to cause us harm have access to the latest technology.” The other half of the boundary goes around the system being called, and only the people who run it can say what it is for. Large language models are a cheese grater on the status quo: whatever is left sticking out is what comes off first. Owning what we already run needs no law, no budget and no product, and it can start this afternoon.
Every system ends up with somebody deciding what it is for. In June the portal’s caller decided. If nobody here owns a system, whoever asks it next decides what it does.
Not invented here is a fine reason to buy software. It is never a reason to stop owning it.