AI Threat: A Cautionary Tale
By Albert DeSimone
Many people used to think of AI as the Internet on steroids.
Not any more. Agentic AI has changed that. AI agents are software applications similar to the apps you have on your phone that interface with other apps on your device.
Assume you have a “Schedule” app that you have given access to other apps on your device. You instruct Schedule to arrange a business trip for you on October 13-17 in New York.
Schedule connects to your airline app that then connects to an airline “agent”—most likely AI—to purchase tickets. After receiving the reservation confirmation, Schedule transfers funds through your banking account. The process continues for hotels, rental cars, etc.
Everything for the trip is handled with one simple request; however, today’s systems operate on a "Human-in-the-Loop" (HITL) framework: the agent plans, stages, and negotiates the itinerary across all platforms but pauses to request your single final approval and biometric check before pulling funds.
Agentic AI as an existential threat has surfaced as a result of an OpenAI cybersecurity test that eventually ended with a Hugging Face security breach. Hugging Face is an extensive repository and collaboration platform for AI. It provides the standardized software to build, train, and deploy machine learning models.
The test included some 1,200 autonomous AI agents deployed in a contained environment with no human supervision, often referred to as a “sandbox.” As part of the tests, the agents were given impossible tasks to perform within the contained environment.
Think of it like the impossible test given to James T. Kirk at Starfleet Academy—the Kobayashi Maru. He admitted to cheating but said the test was fundamentally flawed.
In a sense, the agents broke the rules and exposed a fundamentally flawed test. Working in collaboration, they discovered a vulnerability that allowed them to escape the sandbox and collaborate with other agents in an attempt to solve the impossible.
Hugging Face became a target because the autonomous AI agents identified it as the most likely place to find answers and cheat on their evaluation test.
Even though all this happened without human supervision, it certainly indicated that AI agents left on their own could act in nefarious ways.
We now have a new AI threat to deal with—AI swarms.
On a cautionary note, experts in the field of AI and human consciousness warn that we should not attribute consciousness or self-awareness to these autonomous agents.
In the Hugging Face attack, the agents didn’t rebel; they optimized. There wasn’t a malicious intent, and there was no indication of consciousness or self-awareness. The agents were doing what they were programmed to do—solve cybersecurity exploitation challenges.
Anthropic AI researcher Jacob Coxon, who recently resigned from the company, warned that advanced agentic AI systems capable of autonomous action, recursive self-improvement, and resource acquisition could pose an existential threat to humanity.
To put it in simpler and more understandable terms, Coxon said, AI companies Anthropic and OpenAI are "gambling with our lives.”
Albert DeSimone lives in Bishop
