Leading AI companies are discussing slower development, independent evaluators and shared safety standards. But the emerging system still leaves a fundamental question unresolved: who has the authority to stop a frontier model judged too dangerous to proceed?
THE UNIVERSAL RECORD
Sourced reporting. No opinions.
Brad Socha | September 16, 2026 | 5:29 AM EST
The debate over AI safety changed significantly this week. Leaders of some of the companies building the world’s most capable systems are no longer discussing only how to make their own models safer. They are publicly debating whether development itself sometimes needs to slow, and who should have the authority to make that happen.
Anthropic CEO Dario Amodei called on September 12 for frontier developers to deliberately pace improvements so safety research can keep up. His proposal would place independent evaluators inside AI companies with unusually deep access to models, training processes and safety practices. It would then seek common standards among leading developers and, eventually, international agreements covering the most dangerous capabilities.
OpenAI CEO Sam Altman subsequently supported deeper third-party evaluation, while Google DeepMind CEO Demis Hassabis has advocated an independent standards organization capable of assessing advanced models. OpenAI, Anthropic and Google DeepMind have also been holding discussions about coordinating on safety.
Yet agreement ends quickly when the question becomes one of authority.
Nvidia CEO Jensen Huang argued this week that new laws are unnecessary because companies already have commercial and legal incentives to develop safe products. Altman said companies should be prepared to slow or stop when they cannot proceed safely, while pointing toward independent oversight resembling systems used in aviation. The Trump administration has resisted creating a new federal AI regulator, emphasizing technological competition, particularly with China.
The result is an unusual governance problem: companies possess much of the technical expertise and direct access required to understand frontier systems, but governments possess the legal authority to impose rules. Independent evaluators may be better positioned to test company claims, yet generally lack the power to stop development themselves.
Who Can Actually Stop an AI Model?
Today, much of the immediate control remains with the developers.
OpenAI’s Preparedness Framework and Frontier Governance Framework establish internal processes for assessing advanced capabilities and applying safeguards. Anthropic operates its Responsible Scaling Policy, which links increasingly capable models to progressively stronger security and safety requirements. These systems can influence whether a company trains or releases a model, but they remain company-created governance mechanisms.
Independent testing offers another layer.
Amodei’s new proposal would give external evaluators continuing, employee-like access rather than bringing them in only to test a finished model. Anthropic says it will adopt that approach voluntarily and wants governments to require comparable arrangements across frontier laboratories.
The limitation is authority. An evaluator can discover a dangerous capability and recommend additional safeguards, but unless law or a binding agreement gives that organization enforcement powers, the final decision can remain with the developer.
That distinction has become more consequential as models acquire capabilities relevant to cybersecurity and other high-risk areas. OpenAI said this month that its Astra model had crossed its internal “Critical” cybersecurity threshold, meaning company evaluations found that, with appropriate tools and access, it could discover previously unknown vulnerabilities and develop exploits against well-protected systems without continuous human direction. OpenAI said the classification requires stronger safeguards during development and before release. That is the company’s own assessment under its Preparedness Framework, not a government certification of the model’s safety.
Anthropic separately reported on September 10 that its evaluations found frontier models capable of performing some military and intelligence-related tasks historically requiring scarce expert knowledge. The company said the findings support stronger safeguards against misuse.
Neither finding establishes that current AI systems are uncontrollable. They do, however, illustrate why testing increasingly involves capabilities with consequences extending beyond ordinary product quality.
Governments Are Building Different Forms of AI Oversight
The European Union has moved further toward legally enforceable oversight.
Under the EU AI Act, providers of general-purpose models classified as presenting systemic risk face requirements including model evaluations, risk assessment and mitigation, serious-incident reporting and cybersecurity protections. European Commission enforcement powers over general-purpose AI obligations took effect on August 2, 2026, including the ability to impose fines.
California has taken a different approach. Its Transparency in Frontier Artificial Intelligence Act requires large frontier developers to publish safety frameworks describing how they identify and mitigate catastrophic risks. It also establishes reporting requirements and whistleblower protections, with enforcement available through the state attorney general.
A more extensive federal proposal is now before the U.S. Congress.
The bipartisan FRONTIER Act, introduced in July, would establish federal oversight of advanced AI development and create requirements that scale with developer size, including risk-management frameworks, incident reporting and independent audits. The proposal would establish licensed independent verification organizations to examine whether major developers comply with their safety frameworks. It remains proposed legislation rather than existing federal law.
OpenAI this week publicly backed the bill’s independent-verification provisions, according to statements from its global affairs chief Chris Lehane.
These approaches expose another problem: AI models are developed and distributed internationally, while most enforceable laws stop at national or regional borders.
AI Safety Becomes an International Problem
The United Nations has begun building a global structure, but it does not function as an international AI regulator.
Its Independent International Scientific Panel on AI provides scientific assessments, while the Global Dialogue on AI Governance brings governments and other stakeholders together. The first dialogue was held in Geneva in July with delegations from 163 countries. The UN explicitly states that the dialogue is not a regulatory or enforcement body.
That leaves no global organization currently able to order every frontier developer to stop training a model.
Amodei has proposed progressively stronger international arrangements, beginning with agreements against particularly dangerous uses and common testing for cybersecurity, biological and alignment risks. More ambitious possibilities include limits on the speed of AI self-improvement or broader development pauses. He also acknowledges the central obstacle: such restrictions would require credible verification that participating countries were not secretly developing unrestricted systems.
Geopolitical competition makes that problem harder. A government that restricts domestic developers may fear that companies or states elsewhere will continue advancing. Conversely, rules that exist only inside individual companies depend on those companies maintaining restrictions while facing intense commercial competition.
The evidence therefore does not point to a single institution presently controlling the frontier.
Developers control their models and infrastructure. Independent evaluators can provide technical scrutiny. Governments can create enforceable domestic rules. International institutions can coordinate standards and scientific evidence. Each possesses something the others lack, and none alone currently provides comprehensive global oversight.
That is what makes the latest debate consequential. The question is shifting from whether powerful AI should be tested for dangerous capabilities to what should happen when those tests find them.
The technical ability to detect a risk and the legal authority to act on it are not the same thing. As frontier systems become more capable, resolving that gap may become one of the central questions of AI governance.
Sources:
Dario Amodei — We Must Pace the Frontier
https://darioamodei.com/post/we-must-pace-the-frontier
OpenAI — Frontier Governance Framework
https://openai.com/index/openai-frontier-governance-framework/
OpenAI — Path to Astra: Critical Capabilities and Frontier Safeguards
https://openai.com/index/path-to-astra/
Anthropic — Frontier Safety Roadmap
https://www.anthropic.com/responsible-scaling-policy/roadmap
Anthropic — Measuring Tactical Intelligence Targeting and Conventional Weapons Capabilities of AI Models
https://www.anthropic.com/research/intelligence-targeting-conventional-weapons-capabilities
European Commission — General-Purpose AI Obligations Under the AI Act
https://digital-strategy.ec.europa.eu/en/factpages/general-purpose-ai-obligations-under-ai-act
European Commission — Guidelines for Providers of General-Purpose AI Models
https://digital-strategy.ec.europa.eu/en/policies/guidelines-gpai-providers
European Commission — Enforcement Framework of the AI Act
https://digital-strategy.ec.europa.eu/en/policies/enforcement-ai-act
California Governor — Transparency in Frontier Artificial Intelligence Act
https://www.gov.ca.gov/2025/09/29/governor-newsom-signs-sb-53-advancing-californias-world-leading-artificial-intelligence-industry/
California Attorney General — Catastrophic Risks in Artificial Intelligence Foundation Models
https://oag.ca.gov/sb53
U.S. Representative Lori Trahan — FRONTIER Act
https://trahan.house.gov/news/documentsingle.aspx?DocumentID=3823
United Nations — Global Dialogue on AI Governance
https://www.un.org/global-dialogue-ai-governance/en
United Nations — Independent International Scientific Panel on AI
https://www.un.org/independent-international-scientific-panel-ai/en/preliminary-report
Reuters — Everyone Wants Safer AI. But Who Will Rein It In?
https://www.reuters.com/technology/artificial-intelligence/everyone-wants-safer-ai-who-will-rein-it-2026-09-16/
Reuters — Tech Leaders and Governments Split Over AI Risk
https://www.reuters.com/business/what-amodei-altman-musk-have-said-about-ai-risks-stoking-doom-fears-2026-09-14/
Reuters — Anthropic CEO Urges AI Companies to Slow Model Development
https://www.reuters.com/business/anthropic-ceo-urges-ai-companies-slow-model-development-2026-09-12/
TechCrunch — OpenAI, Anthropic and Google Have Been in AI Safety Talks
https://techcrunch.com/2026/09/15/openai-anthropic-google-have-been-in-talks-on-ai-safety-for-weeks/
CBS News — OpenAI Backs Measure Requiring Independent AI Audits
https://www.cbsnews.com/news/openai-sam-altman-frontier-act/
About the Author
Brad Socha is the founder of The Universal Record, focused on sourced, factual global reporting. Coverage includes international news, geopolitics, technology, and major developments.
