Interview · AI Security · 19 minutes
When AI Agents Outrun Their Guardrails
NIST researcher Apostol Vassilev on why no guardrail removes all of the risk, who should own AI risk, and why “human in the loop” gets harder at the speed and scale of agents.
Megan Dahlgren in conversation with Apostol Vassilev, National Institute of Standards and Technology (NIST)
In this conversation
- Why securing AI is fundamentally different from securing cryptographic systems
- How AI is changing vulnerability patching and the balance between attackers and defenders
- Who should own AI risk across an organization
- The accountability questions that arise when agents take unexpected actions
- Why “human in the loop” becomes harder at agent speed and scale
- Where startups can help build the tools for real-time AI governance
About the conversation
What happens when AI agents move faster than the systems, and the people, meant to keep them under control?
Apostol Vassilev is a senior researcher at NIST, where his team maintains a taxonomy of attacks and mitigations in machine learning. Drawing on his work in adversarial AI, cryptography and security standards, he explains why guardrails leave risk behind, and why managing that risk takes changes in both technology and organizational oversight.
For executives, security leaders, founders and anyone deploying AI agents, the central question is how to capture the benefits of AI while preparing for failures we can’t fully predict.
Read the essays
Why AI Guardrails Don’t Eliminate Business Risk
A NIST researcher’s “insecurity proof” shows AI systems stay vulnerable whatever guardrails you add. The useful question for leaders is how much risk remains — and whether it’s worth carrying.
AI Agents Are Changing the Economics of Cybersecurity
AI makes old vulnerabilities exploitable and gives unskilled attackers nation-state reach, while defenders carry costs attackers ignore. Nobody has a finished model of the new economics yet. Here are the questions it has to answer.
Who Owns AI Risk? Moving from Periodic Reviews to Real-Time Governance
When agents act at machine speed, quarterly reviews and a human approving every step both stop working. NIST’s Apostol Vassilev on accountability, continuous monitoring, escalation, and the staffing it will take.
What to do now
Assume the guardrails will fail.
Treat any claim of complete protection with skepticism. Estimate the risk that remains and decide whether the benefit justifies it.
Give one function the full picture.
Everyone reports AI risk; one function combines it into a single enterprise view the board can act on.
Re-rank your patch backlog.
Vulnerabilities once deferred as too hard to exploit may no longer be safe to postpone.
Watch agents in real time.
Replace quarterly reviews with continuous monitoring of what agents are doing and what they can reach.
Design the escalation points.
Decide in advance which agent actions go to a person, and let automated monitoring handle the rest.
Put numbers on it.
Start pricing the cost of attack and defense, even roughly, so AI risk can enter the business case.
Join the research
What are you seeing break?
What Breaks Next interviews enterprise technology leaders, security researchers and founders about where AI fails in production. If you’re deploying agents, governing them or building the tools to control them, we’d like to hear from you.