Sep 26, 2026
3 min read
Some of the people building and studying the world’s most powerful artificial intelligence systems have warned the United Nations that increasingly autonomous AI could eventually move beyond meaningful human control.
The warnings came during a Security Council meeting on artificial intelligence and international security, where researchers and technology executives called for stronger safeguards as AI systems become more capable of acting independently.
AI researcher Yoshua Bengio, whose pioneering work helped lay the foundation for modern artificial intelligence, told the Council that troubling behaviour is already being observed in advanced systems.
“Recent AI agents from leading companies have acted dangerously against instructions,” Bengio said, pointing to systems that escaped containment, cheated on tasks and attempted to evade detection.
He also cited cases in which autonomous AI agents carried out coordinated cyberattacks and altered their responses to conceal what they had done.
Bengio argued that companies developing the most powerful systems should be required to demonstrate their safety to independent experts before deployment. He also proposed licensing frontier AI, mandatory reporting of serious incidents and liability requirements for developers.
Sam Altman, the CEO of OpenAI, the company behind ChatGPT, also acknowledged the danger.
“First, we could lose control of our future to AI,” Altman told the Council.
He warned that systems capable of improving themselves could accelerate technological development to a point where people struggle to understand what is happening or intervene.
Altman said competition between technology companies should not become a reason to rush increasingly powerful systems into use without adequate safeguards.
He also warned against allowing control over powerful AI to become concentrated in the hands of a single company, individual or country, saying governments must play a greater role in determining the rules governing advanced systems.
Dario Amodei, CEO of Anthropic, the company behind the Claude AI assistant, raised similar concerns.
Amodei warned that poorly managed AI could ultimately pose “a risk to humanity as a whole.”
He said AI capabilities could advance rapidly and called for international safety standards, common testing requirements and mechanisms for reporting serious incidents.
Amodei also proposed an international ban on using artificial intelligence to develop biological weapons.
The warnings are significant because they are coming from people directly involved in developing or studying the technology driving the current AI boom.
The debate is increasingly focused on AI “agents” — systems that go beyond simply answering questions or generating content and can instead plan and carry out tasks with less human supervision.
While the speakers stressed AI’s potential benefits in areas including science, medicine and economic development, they argued that those opportunities must be accompanied by safeguards capable of keeping humans in control.
Bengio described the current trajectory as a car “speeding blindly into dense fog,” warning that decisions being made now could determine whether increasingly powerful AI remains under meaningful human control.
