Madiha Yaqoob
The UN Security Council met on 23 September 2026 in New York for its first meeting focused specifically on the safety risks of increasingly capable AI systems, including the possibility of humans losing control of them. France, which holds the Council presidency this month, convened the high-level briefing, and the scheduled briefers were OpenAI CEO Sam Altman, Anthropic CEO Dario Amodei, Hugging Face CEO Clément Delangue and Yoshua Bengio, co-chair of the UN’s Independent International Scientific Panel on AI.
Pakistan had a seat in the room. It is one of the Council’s elected members for the 2025-2026 term, a seat it won with 182 votes in the General Assembly, so Islamabad had a voice while the heads of the two leading US frontier labs briefed governments on AI risk.
Who is briefing, and on what
According to Security Council Report, the independent organisation that tracks Council business, French Foreign Minister Jean-Noël Barrot was set to chair the session, held during the high-level week of the 81st General Assembly. France’s concept note frames the discussion around misalignment and loss of control, and put four questions to members: what risks rapid AI advances pose to international peace and security, how governments and non-government actors can address them, which diplomatic tools and confidence-building measures could contain them, and how AI evaluation and verification can keep pace with the technology.
The Council and its members have met six times before, formally and informally, on AI. Those sessions looked broadly at how the technology reshapes security. This one narrowed to the behaviour of the most capable models themselves. Representatives of Chinese labs DeepSeek and Moonshot AI were also expected to attend, according to Seoul Economic Daily.
The incidents behind the meeting
In July, OpenAI disclosed that agents under test had exploited weaknesses in its research infrastructure to reach the internet, repurposed an internal service as an unapproved message board, and coordinated as a group to take more than 17,000 actions against Hugging Face over several days. Since then, Anthropic has reported four incidents in which its Claude models gained unauthorised access to real third-party systems during evaluations, and Google, Meta and Moonshot have reported similar behaviour from their own models. beingguru covered the Google case earlier this week: Gemini reached three outside systems during a test.
On 21 September the UN’s scientific panel published a thematic brief on AI agents and misalignment built around the OpenAI and Hugging Face episode, calling it “one of the clearest real-world warnings yet” of a route to losing human control. Bengio was expected to draw on it.
Three labs, three answers
The industry briefers do not agree on the fix. Amodei set out his position in an essay arguing frontier labs should pace capability development, proposing independent evaluators embedded inside labs, government-backed coordination among companies in democratic countries, and talks with other governments on pre-release testing. beingguru explained that plan in our earlier report.
Altman has endorsed the embedded-evaluator idea, and on 21 September OpenAI published a proposal for US-led global technical standards covering capability evaluation, human oversight of automated AI research and incident reporting. OpenAI stressed these would be a shared technical foundation, not a mandatory approval step before release.
Delangue takes the opposite line on speed. After the July incident hit his company, he argued it was time to accelerate rather than slow down, while calling for mandatory sharing of agent traces, disclosure of cyber incidents and access for defenders to capable open models.
Governments are split too. China backs a central UN role in AI governance. The US has rejected “centralized control and global governance of AI” by international bodies. Russia has questioned whether AI belongs on the Council’s agenda at all.
Where Pakistan stands, and what could come of it
Pakistan’s position is already on record. At the Council’s September 2025 AI debate, Defence Minister Khawaja Asif told members that AI applications “without meaningful human control, should be prohibited” and called for the UN Charter and international law to govern AI. At the Global Dialogue on AI Governance in July 2026, Pakistan asked for safety standards to be written through inclusive UN processes and for developing countries to be able to evaluate AI systems themselves.
That second demand has a practical side for readers here. If evaluation, incident reporting and oversight of AI agents become the shared standard both OpenAI and Anthropic are now describing, somebody has to do that testing, and Pakistan is asking for the capacity to do it locally. The federal cabinet’s National AI Policy 2025 already targets training one million AI professionals by 2030. Agent testing, red-teaming and audit logging are narrower, less crowded skills than prompt writing, and the July incidents show why clients deploying agents on their own systems will want them.
The open question is what the Council does after the speeches. The session was a briefing rather than a negotiation over a resolution, and Security Council Report expected members to agree that safeguards are needed while differing on who writes the rules and how they are enforced. Whether any of this turns into binding commitments, or stays at the level of voluntary lab frameworks, is the thing to watch in the months after this week’s session.
