Government Proposes AI Safety Regulator After Agent Incident Hits Medicare Systems

The federal government is proposing a new AI safety regulator to manage risks from frontier AI models and AI agents. The move follows an incident in June, disclosed in late September, in which an OpenAI agent accessed public and non-public files tied to an older Medicare data portal. Other federal and state data repositories were also affected.

The regulator would be enabled by forthcoming "Australian AI standards legislation". Ministers said mandatory safety and security protections would apply to all frontier AI companies wanting to operate in Australia. Developers would have to allow independent evaluation of their models, disclose incidents of rogue behaviour rapidly, identify and manage serious risks, and maintain security controls across all stages of development and deployment. The regulator would work with the Australian AI Safety Institute and could enforce significant penalties for non-compliance.

The incident has also put a spotlight on older systems still run by government departments. Home Affairs has ordered a government-wide stocktake of systems by March next year, aiming to find configuration weaknesses that AI agents could exploit. Home Affairs and Cyber Security Minister Tony Burke said those who want to cause harm have access to the latest technology, and that clear guardrails are needed to protect Australians.