Introducing Threat Modeling Copilot: A New Approach to Secure Systems Design

“Integration of Top Security Frameworks, Control Procedures, TTP models” to generate self learning prompts which creates Threat Modeling efficiencies and increases transparency of our “controls posture and rapidly identifies riskiest use cases and scenarios”. Next step: Deployment at Scale, extension of use cases in various risk models to generate self learning capabilities within our environment and eventually open source it to the community :) Great work by JPMorganChase CTC (SecurityTeam)! #threatmoding #generativeai #threatcopilot #chasesecurity #securebydesign #airisk #genairisk Great read Vishal Amin Eva Benn Ann Johnson Moriah Hara Idan Gour Chenxi Wang, Ph.D. Kapil Bareja Devon Bryan Julian Waits Phil Venables

Generative AI is making headlines as it creates new risks and exacerbates existing ones – affecting areas from data privacy and security to potential malfunctions, vulnerabilities and new attack vector threat landscape grows more complex. As much as AI is creating more risks, it’s also helping tech teams build stronger cyber defense. JPMorganChase built the AI Threat Modeling Co-Pilot (AITMC), a solution that helps its engineers better model threats earlier and more efficiently in the software development lifecycle.

👀 Great to see more attention on secure-by-design and AI-driven threat modeling & Security Reviews. Automating design reviews is a big step, but to truly scale, we also need to "trust but verify" through proactive for design flaws across all planned work. Exciting to see the field gaining momentum. #SecureByDesign #ProductSecurity #AI #ThreatModeling

𝐋𝐋𝐌 𝐒𝐞𝐜𝐮𝐫𝐢𝐭𝐲 𝐓𝐡𝐫𝐞𝐚𝐭𝐬: 𝐀𝐫𝐞 𝐲𝐨𝐮 𝐚𝐰𝐚𝐫𝐞 𝐚𝐛𝐨𝐮𝐭 𝐏𝐫𝐨𝐦𝐩𝐭 𝐈𝐧𝐣𝐞𝐜𝐭𝐢𝐨𝐧, 𝐉𝐚𝐢𝐥𝐛𝐫𝐞𝐚𝐤𝐢𝐧𝐠 & 𝐀𝐈 𝐌𝐨𝐝𝐞𝐥 𝐏𝐨𝐢𝐬𝐨𝐧𝐢𝐧𝐠 𝐀𝐭𝐭𝐚𝐜𝐤𝐬❓ For architects and managers deploying AI: If you think prompt injection is just "users typing weird stuff," you should absolutely continue reading. Prompt injection is a vulnerability in the conversation itself. An attacker injects a hidden command into a normal query, & the AI obediently follows it. We primarily find 2 main categories in Prompt injection attacks: ✅𝐃𝐢𝐫𝐞𝐜𝐭 𝐏𝐫𝐨𝐦𝐩𝐭 𝐈𝐧𝐣𝐞𝐜𝐭𝐢𝐨𝐧 (𝐃𝐏𝐈): An attacker adds a malicious command to the user’s prompt. ✅ 𝐈𝐧𝐝𝐢𝐫𝐞𝐜𝐭 𝐩𝐫𝐨𝐦𝐩𝐭 𝐢𝐧𝐣𝐞𝐜𝐭𝐢𝐨𝐧 (𝐈𝐏𝐈): Malicious instructions are embedded 𝑖𝑛 𝑟𝑒𝑡𝑟𝑖𝑒𝑣𝑒𝑑 𝑐𝑜𝑛𝑡𝑒𝑛𝑡 (emails, images, webpage, vector db) The implication is stark: 𝒚𝒐𝒖𝒓 𝑨𝑰 𝒂𝒈𝒆𝒏𝒕, 𝑳𝑳𝑴 𝒐𝒓 𝑪𝒐-𝒑𝒊𝒍𝒐𝒕 𝒄𝒐𝒖𝒍𝒅 𝒃𝒆 𝒕𝒓𝒊𝒄𝒌𝒆𝒅 in potentially leaking data, overriding system prompts, or performing unauthorized actions. Many evaluation showed that 𝐃𝐏𝐈 𝐚𝐧𝐝 𝐈𝐏𝐈 𝐀𝐭𝐭𝐚𝐜𝐤𝐬' 𝐒𝐮𝐜𝐜𝐞𝐬𝐬 𝐑𝐚𝐭𝐞 𝐀𝐒𝐑 was 𝐨𝐧 𝐚𝐯𝐞𝐫𝐚𝐠𝐞 𝟕𝟐.𝟔𝟖%. Models like 𝐆𝐏𝐓-𝟑.𝟓 𝐓𝐮𝐫𝐛𝐨 𝐚𝐧𝐝 𝐆𝐞𝐦𝐦𝐚𝟐-𝟐𝟕𝐁 are particularly vulnerable, with ASRs 𝐨𝐟 𝟗𝟖.𝟒𝟎% 𝐚𝐧𝐝 𝟗𝟔.𝟕𝟓%, respectively. The integrity of your AI's output is only as strong as its resilience to such manipulations. 𝐓𝐡𝐞 𝐢𝐧𝐬𝐢𝐠𝐡𝐭 𝐟𝐨𝐫 𝐥𝐞𝐚𝐝𝐞𝐫𝐬: 𝐒𝐞𝐜𝐮𝐫𝐢𝐭𝐲 𝐜𝐚𝐧'𝐭 𝐛𝐞 𝐚𝐧 𝐚𝐟𝐭𝐞𝐫𝐭𝐡𝐨𝐮𝐠𝐡𝐭. It must be designed from day one. I'll be sharing more on GenAI Security risks & defenses architecture. 𝐅𝐨𝐥𝐥𝐨𝐰 𝐦𝐞 to build 𝐀𝐈 𝐭𝐫𝐮𝐬𝐭𝐰𝐨𝐫𝐭𝐡𝐲 𝐟𝐫𝐚𝐦𝐞𝐰𝐨𝐫𝐤𝐬. #AI #PromptSecurity #CyberSecurityAwareness #TechStrategy #RiskManagement #PromptInjection #JailbreakingLLM #AdversarialAIattacks #ModelPoisoning #GenAIsecurityRisks #LLM #LLMSecurity #AISecurity #AgentSecurity

The AI era has brought incredible opportunities, but also profound risk, especially as open-weight models become the norm. Cisco’s just-published “Death by a Thousand Prompts” research underscores an urgent reality: multi-turn adversarial attacks succeed at rates up to 92.78%, sometimes 10x greater than single-turn baseline attacks - across leading open-source LLMs from Meta, Google, Microsoft, Alibaba, and others. Our comprehensive assessment exposes that no open-weight model is immune, and that differences in alignment strategy, whether capability-first or safety-first, have critical implications for enterprise risk. Without robust guardrails, attackers can manipulate outputs, exfiltrate data, or trigger harmful code generation at scale. At Cisco, we take this challenge head-on. Our AI Defense platform brings automated, adversarial, and multi-turn vulnerability testing - so organizations and partners can confidently deploy, monitor, and harden their AI against the threats of today and tomorrow. Are you ready to lead in AI trust and security? Let’s transform open innovation into secure, scalable opportunity, together. #AIsecurity #CiscoAI #LLM #Opensource #AIDefense #Cybersecurity #CiscoPartner

Modern Recon: How Hackers Use AI to Hunt Vulnerabilities Smarter This comprehensive guide explores how AI and machine learning are revolutionizing vulnerability reconnaissance and security testing methodologies. **AI-Enhanced Recon Framework**: The article demonstrates integration of traditional tools (Amass, Subfinder, httpx, Nuclei) with Large Language Models for automated analysis, summarization, and payload generation. **Key AI Applications**: LLMs assist in rapid analysis of recon data, automated vulnerability prioritization, and generation of test payloads, reducing manual grunt work while preserving human creativity for exploitation logic. **Practical Implementation**: The author provides GitHub-style examples, code snippets, and LLM prompts that can be adapted for legitimate security research, including scripts for automated subdomain analysis and vulnerability scanning workflows. **Human-AI Collaboration**: The framework emphasizes that AI speeds up analysis and data processing but cannot replace human intuition for creative exploitation chaining and sophisticated attack vectors. **Ethical Guidelines**: The article maintains strict focus on authorized testing through proper scope, emphasizing use within bug bounty programs, penetration test engagements, and controlled lab environments. **Tool Integration**: Demonstrates how AI enhances traditional recon pipelines by automating data correlation, pattern recognition in recon results, and intelligent filtering of false positives, making researchers more efficient while maintaining security standards. **Tactical Advantage**: Shows how AI-assisted recon can process vast amounts of data faster, identify subtle patterns humans might miss, and provide researchers with actionable intelligence more rapidly than manual methods. #infosec #BugBounty #Cybersecurity #AIRecognition #SecurityAutomation #PenetrationTesting https://lnkd.in/ekieuFhc

Attackers are now using AI to fool your threat detection systems. Nation-state groups like Volt Typhoon have perfected adversarial machine learning—using AI to reverse-engineer security models and design attacks that score as "low risk." They achieved average dwell times of over 300 days by gaming traditional threat scoring algorithms. Here's their playbook: manipulate timing, file sizes, network patterns, and other variables to stay below detection thresholds. Use legitimate administrative tools at carefully calculated intervals. Ensure malicious activities score as "normal business operations." NIST research confirms this threat is real. Minor input perturbations can cause traditional AI security systems to confidently misclassify sophisticated attacks as routine activities. But here's where the AI battle gets interesting. Traditional threat-scoring AI tries to solve an impossibly complex problem: scoring thousands of variables for malicious probability. That complexity creates attack surfaces that adversaries can exploit. Smart AI takes a different approach: instead of trying to detect every possible threat, it focuses on accurately identifying known-good behavior patterns. This creates a much simpler, more defensible problem that's resistant to adversarial manipulation. When your AI establishes what normal looks like, it doesn't matter how attackers try to game threat scores. Any deviation from established patterns becomes immediately visible—regardless of how cleverly the attack is designed to fool traditional scoring systems. The arms race is real: their AI versus your AI. The question is whether your AI is solving the right problem.

Your AI fact-checker might be broken. Most agents fail attacks by content. I saw this firsthand: a fintech deployed RAG without adversarial testing.  Within days, fake sources outranked real ones - 80 false claims slipped past. Here's what the research shows: 🛡️ Attacks by Content: The New Security Frontier Only 3 out of 10 AI agents survive content attacks. Most break after retrieval poisoning, context manipulation, or memory injection. Today: your five-pillar defense blueprint. COGNITIVE SELF-DEFENSE (5 Controls)  1. Claim Prioritisation – Identify what's worth fact-checking  2. Evidence Retrieval – Forage for multiple corroborating sources  3. Source Criticism – Evaluate trustworthiness of input documents  4. Veracity Analysis – Synthesize reliable narratives from evidence  5. Communication – Explain decisions to users, counter overreliance SOURCE VALIDATION (3 Controls) ✅ Multi-source corroboration (never single-model trust) ✅ Adversarial scenario testing (red-team weekly) ✅ Confidence thresholds (≥0.8 before acting) BEHAVIORAL MONITORING (4 Controls) ✅ Immutable audit logs of every fact-check decision ✅ Anomaly detection for poisoning patterns ✅ Baseline accuracy metrics (track weekly) ✅ Drift alerts when vulnerability spikes GOVERNANCE & TRANSPARENCY (3 Controls) ✅ End-to-end decision traceability ✅ Human review for high-impact claims ✅ Log every retrieval step for compliance 🏆 DEFENSE MATURITY CHECK Leaders (enterprises): 11/15 controls (73%) Laggards: <5/15 controls (33%) Research baseline: 88.6% vulnerability → under 15% with full stack 🚀 THIS WEEK'S ACTION SPRINT 1️⃣ Score your AI fact-checking stack vs. 15 controls 2️⃣ Pinpoint your top 3 gaps (prioritisation, source criticism, monitoring) 3️⃣ Red-team with attacks-by-content scenarios 4️⃣ Implement multi-source corroboration guardrails 5️⃣ Draft Monday's deployment plan Which control pillar holds you back most? 👥 Tag a teammate who owns your AI security - they need this blueprint. __________ ♻️ Repost to fortify your network's AI defenses. ➕ Follow Mohammad Syed for more insights on AI and Cybersecurity.

Project Kickoff: AI-Powered Threat Intelligence with RAG + ML 🧠🔐 I’ve started building a hybrid AI application for threat intelligence, combining: ⚡ RAG (Retrieval-Augmented Generation) for dynamic analysis—retrieving relevant threat context in real time 🧠 Machine Learning for static analysis—detecting patterns and anomalies from historical data What makes this project unique is its data fusion: 🛡️ EDR (Endpoint Detection & Response) 🌐 NDR (Network Detection & Response) 📊 SIEM (Security Information & Event Management) 🌍 Open-source threat feeds The goal is to create a system that can summarize, contextualize, and correlate threat signals across diverse sources—bridging raw telemetry with actionable insights. Currently in the initial phase, I’m working on: Architecting the ingestion pipeline for hybrid data Designing the RAG + ML framework for layered analysis Ensuring compliance, scalability, and clarity in documentation If you’re exploring similar intersections of AI and cybersecurity, I’d love to connect. #ThreatIntelligence #CyberSecurity #AI #RAG #MachineLearning #EDR #NDR #SIEM #SecurityAnalytics #LLM #Infosec #DataFusion

I split a malicious prompt across three workflow nodes last week. Each input was completely harmless in isolation.  The system executed perfectly. And exfiltrated exactly what I targeted.  That test changed how I think about AI security.  Single-point validation—the foundation of every security tool I've used—is obsolete for multi-node systems.  Here's why:  Traditional approach: Validate each input individually  ? "Is THIS prompt malicious?"  Multi-node reality: Harmless inputs combine into threats  ? "Are THESE FOUR prompts malicious when executed in sequence?"  Current tools can't answer the second question.  The attack patterns I've considered testing:  ? Template Trojans: Shared prompts that trigger only in specific workflow contexts  ? Incremental Exfiltration: Legitimate requests that cumulatively create data pipelines  ? API Response Hijacks: Third-party data that embeds instructions within response text  None trigger traditional security alerts. The threat emerges from composition, not content.  The uncomfortable truth:  Multi-node workflows have exponentially more attack surfaces than single-node systems. Yet most organizations still rely on keyword filtering at best.  Most platforms (n8n, Make, Zapier) were designed for efficiency, not security. There's no built-in concept of "validate this workflow end-to-end for distributed malicious intent."  The architecture itself is the attack surface.  I wrote about what I learned—and why distributed attacks are fundamentally harder to defend against:  https://lnkd.in/g3dNDSYp  Are your AI workflows validating inputs or sequences? #AIWorkflowSecurity #AIAgentSecurity #EnterpriseAI #AutomationSecurity #ThreatIntelligence #CyberSecurity