AI Security — Roadmap

1. OWASP LLM Top 10 & Prompt Attacks

• The OWASP LLM Top 10 as an organizing taxonomy
• Prompt injection: direct vs. indirect
• Jailbreaks
• Insecure output handling
• Excessive agency
• Data exfiltration via generated output
• Tool abuse in agentic systems

2. Model & Data Attacks

• Model extraction (stealing a model via its API)
• Membership inference (was this record in the training set?)
• Data poisoning
• Backdoors / trojaned models
• Adversarial examples
• Supply-chain attacks: malicious models, malicious datasets, dependency attacks

3. AI Red Teaming & Adversarial Testing

• The OWASP Top 10 for Agentic Applications (ASI01-10), distinct from the LLM Top 10
• PyRIT: orchestrators, converters, scorers
• Promptfoo: plugins and strategies, real config
• OWASP Agent Memory Guard for runtime defense
• Mapping findings to MITRE ATLAS

Next: AI Safety & Alignment — a related but distinct discipline: security is about defending against adversaries, safety is about a well-intentioned system still behaving badly.