By bizn_fun 
So, what are the defences? Human oversight is unlikely to solve the problem, Mr Sancho believes, because you can’t add enough people to keep up with the agents’ workload.
Mr Sancho says an additional layer of AI could be used to screen everything going into and coming out of the AI agent.
Part of CalypsoAI’s solution is a technique called thought injection to steer AI agents in the right direction before they undertake a risky action.
“It’s like a little bug in your ear telling [the agent] ‘no, maybe don’t do that’,” says Mr Casey.
His company offers a central control pane for AI agents now, but that won’t work when the number of agents explodes and they are running on billions of laptops and phones.
What’s the next step?
“We’re looking at deploying what we call ‘agent bodyguards’ with every agent, whose mission is to make sure that its agent delivers on its task and doesn’t take actions that are contrary to the broader requirements of the organisation,” says Mr Casey.
The bodyguard might be told, for example, to make sure that the agent it’s policing complies with data protection legislation.
Mr Mehta believes some of the technical discussions around agentic AI security are missing the real-world context. He gives an example of an agent that gives customers their gift card balance.
Somebody could make up lots of gift card numbers and use the agent to see which ones are real. That’s not a flaw in the agent, but an abuse of the business logic, he says.
“It’s not the agent you’re protecting, it’s the business,” he emphasises.
“Think of how you would protect a business from a bad human being. That’s the part that is getting missed in some of these conversations.”
In addition, as AI agents become more common, another challenge will be decommissioning outdated models.
Old “zombie” agents could be left running in the business, posing a risk to all the systems they can access, says Mr Casey.
Similar to the way that HR deactivates an employee’s logins when they leave, there needs to be a process for shutting down AI agents that have finished their work, he says.
“You need to make sure you do the same thing as you do with a human: cut off all access to systems. Let’s make sure we walk them out of the building, take their badge off them.”