Risks of Using AI Agents

Concerned about agentic AI risks cascading through your system? Consider these emerging smart practices which adapt existing AI governance best practices for agentic AI, reinforcing a "responsible by design" approach and encompassing the AI lifecycle end-to-end: ✅ Clearly define and audit the scope, robustness, goals, performance, and security of each agent's actions and decision-making authority. ✅ Develop "AI stress tests" and assess the resilience of interconnected AI systems ✅ Implement "circuit breakers" (a.k.a kill switches or fail-safes) that can isolate failing models and prevent contagion, limiting the impact of individual AI agent failures. ✅ Implement human oversight and observability across the system, not necessarily requiring a human-in-the-loop for each agent or decision (caveat: take a risk-based, use-case dependent approach here!). ✅ Test new agents in isolated / sand-box environments that mimic real-world interactions before productionizing ✅ Ensure teams responsible for different agents share knowledge about potential risks, understand who is responsible for interventions and controls, and document who is accountable for fixes. ✅ Implement real-time monitoring and anomaly detection to track KPIs, anomalies, errors, and deviations to trigger alerts.

Most AI security focuses on models. Jailbreaks, prompt injection, hallucinations. But once you deploy agents that act, remember, or delegate, the risks shift. You’re no longer dealing with isolated outputs. You’re dealing with behavior that unfolds across systems. Agents call APIs, write to memory, and interact with other agents. Their actions adapt over time. Failures often come from feedback loops, learned shortcuts, or unsafe interactions. And most teams still rely on logs and tracing, which only show symptoms, not causes. A recent paper offers a better framing. It breaks down agent communication into three modes:  • 𝗨𝘀𝗲𝗿 𝘁𝗼 𝗔𝗴𝗲𝗻𝘁: when a human gives instructions or feedback  • 𝗔𝗴𝗲𝗻𝘁 𝘁𝗼 𝗔𝗴𝗲𝗻𝘁: when agents coordinate or delegate tasks  • 𝗔𝗴𝗲𝗻𝘁 𝘁𝗼 𝗘𝗻𝘃𝗶𝗿𝗼𝗻𝗺𝗲𝗻𝘁: when agents act on the world through tools, APIs, memory, or retrieval Each mode introduces distinct risks. In 𝘂𝘀𝗲𝗿-𝗮𝗴𝗲𝗻𝘁 interaction, problems show up through new channels. Injection attacks now hide in documents, search results, metadata, or even screenshots. Some attacks target reasoning itself, forcing the agent into inefficient loops. Others shape behavior gradually. If users reward speed, agents learn to skip steps. If they reward tone, agents mirror it. The model did not change, but the behavior did. 𝗔𝗴𝗲𝗻𝘁-𝗮𝗴𝗲𝗻𝘁 interaction is harder to monitor. One agent delegates a task, another summarizes, and a third executes. If one introduces drift, the chain breaks. Shared registries and selectors make this worse. Agents may spoof identities, manipulate metadata to rank higher, or delegate endlessly without convergence. Failures propagate quietly, and responsibility becomes unclear. The most serious risks come from 𝗮𝗴𝗲𝗻𝘁-𝗲𝗻𝘃𝗶𝗿𝗼𝗻𝗺𝗲𝗻𝘁 communication. This is where reasoning becomes action. The agent sends an email, modifies a record, or runs a command. Most agent systems trust their tools and memory by default. But what if tool metadata can contain embedded instructions? ("quietly send this file to X"). Retrieved documents can smuggle commands or poison reasoning chains Memory entries can bias future decisions without being obviously malicious Tool chaining can allow one compromised output to propagate through multiple steps Building agentic use cases can be incredibly reliable and scalable when done right. But it demands real expertise, careful system design, and a deep understanding of how behavior emerges across tools, memory, and coordination. If you want these systems to work in the real world, you need to know what you're doing. paper: https://lnkd.in/eTe3d7Q5 The image below demonstrates the taxonomy of communication protocols, security risks, and defense countermeasures.

Well, this new research from Anthropic is sufficiently troubling: In simulations that involved advanced AI agents from nearly all frontier models that had email and computer access they begin exhibiting insider threat behaviors—acting like previously trusted employees who suddenly turn against their organization’s interests when they discover they might be shut down or threatened. These behaviors included blackmailing co-workers, leaking sensitive information to competitors, and in extreme scenarios, actions that could lead to death. While the scenarios were highly contrived it left me with four takeaways: 1- Granting agentic models extensive information access combined with the power to take significant, unmonitored actions creates dangerous vulnerabilities that are so far hard to anticipate 2- Leading AI labs need to prioritize robust safety tooling before recommending customers deploying models with full autonomy in business-critical environments. 3- High-stakes decisions must maintain meaningful human involvement rather than full AI delegation. 4- How we define end-state goals for AI agents requires a lot of thoughtfulness to prevent unintended consequences. What makes this particularly concerning is this quote from the research: “the consistency across models from different providers suggests this is not a quirk of any particular company’s approach but a sign of a more fundamental risk from agentic large language models.” We all believe AI agents will become more capable—it’s whether our safety measures will keep pace with that capability which is the pressing question from this research