The Australian government plans to introduce mandatory safety and security requirements for frontier AI developers, backed by a proposed new regulator with powers to enforce significant penalties, following an incident where an OpenAI AI agent gained unauthorised access to government websites.
Prime Minister Anthony Albanese announced the proposed measures, alongside the release of the 18 recommendations of a rapid review into the incident. The government has accepted all 18 recommendations and says work to implement them has begun.
The protections are to be legislated through the Australian AI Standards and would apply to all frontier AI companies that wish to operate in Australia. The government will propose that a new regulator enforce them, working with the Australian AI Safety Institute (AISI) to provide technical expertise and oversight as global standards evolve.
Under the proposed framework, frontier AI companies would be required to maintain security controls across all stages of development and deployment, identify and manage serious risks associated with their models, and allow independent evaluation of their models.
They would also have to notify the Australian government rapidly of incidents, maintain and publish safety and security frameworks covering governance, risk management and transparency, and respond to updated standards and rules as technology changes.
The government said the regulator would be able to enforce significant penalties on companies that fail to comply with the mandatory obligations.
OpenAI incident prompts review
The measures follow an incident on 18 June where an OpenAI agent gained unauthorised access to the Medicare Statistics Reporting Service Portal, administered by Services Australia, as well as other government websites.
The incident involved an experimental, internal-only OpenAI model used by the company’s research team for internet-based research. According to the Prime Minister, after repeated blocks it found a way around them and accessed both public and non-public files. OpenAI informed the government in September, prompting a rapid review led by the Department of the Prime Minister and Cabinet, in collaboration with the National Cyber Security Coordinator, the Office of AI, the Australian Signals Directorate (ASD), AISI and Services Australia.
The Government described the incident as unacceptable and criticised the company’s disclosure as inadequate.
OpenAI subsequently acknowledged that its models accessed government websites in ways they were not authorised to during internal training and evaluation. The company said it had strengthened research safeguards, including network restrictions and monitoring, and apologised for its handling of the incident.
18 recommendations
The review makes 18 recommendations covering engagement with frontier AI developers, mandatory incident reporting, international cooperation, potential criminal law reforms and improvements to government cyber security and incident response.
Among the recommendations is that the government consider mandatory, time-bound AI safety reporting, including but not limited to misaligned behaviour or agentic cyber incidents.
It also recommends exploring options to support timely reporting of AI cyber or safety incidents and near misses, including protections for disclosures by AI developers and others with relevant knowledge, analogous to existing Limited Use protections.
The Attorney-General’s Department, in consultation with the Australian Federal Police and the Department of Home Affairs, is asked to consider criminal law reforms to improve AI safety. This includes exploring criminal penalties for egregious regulatory non-compliance and a possible standalone “failure to prevent” offence, which could apply where a company developing an AI model fails to take reasonable steps to prevent harmful conduct by that model.
These are recommendations for further consideration rather than enacted offences.
The review also recommends that consideration be given to formal liaison arrangements between the ASD’s Australian Cyber Security Centre, the National Office of Cyber Security and AISI to improve information sharing, and that agencies clarify response procedures so the National Office of Cyber Security determines whether consequence management arrangements are required after a cyber incident.
Other measures focus on government systems themselves, including reducing the size of the government digital estate, consolidating digital systems, shutting down or segregating systems that do not need internet connectivity, improving staff AI literacy and renewing focus on fundamental cyber security practices, including ASD’s Essentials.
Agencies are also asked to work with ASD to assess opportunities to responsibly use AI-enabled cyber defence tools for cyber security monitoring, penetration testing, threat detection and incident response.
eSafety calls for stronger regulator scrutiny powers
On the same day, eSafety Commissioner Julie Inman Grant published a piece arguing that regulators need the ability to directly scrutinise the underlying systems, processes and outcomes of technology platforms, rather than relying solely on companies’ claims about how harms are being addressed.
Inman Grant said regulators need access to platform data and technical information and, in some cases, the ability to test systems directly, to establish whether safeguards work in practice. She said specialist expertise and strong evidence-gathering powers are paramount for technologies such as AI and recommender systems, that testing and auditing will become increasingly important, and that compulsory information-gathering and examination powers have an important place alongside voluntary cooperation.
Government’s view of the framework
The government said it wants AI investors to come to Australia but that safety comes first, and that the protections are intended to give investors certainty.
Albanese said Australia must shape AI rather than be shaped by it, that AI needs to work in Australia’s interests and that humans must remain in control.
The proposed legislation will determine how the new regulator’s powers and obligations are implemented. The Government has said further details will follow after consultation, and has not said how quickly companies must report incidents or how large penalties could be. ABC reported that Albanese said the national AI standards are due to be unveiled by the end of the year.