AI Security Tooling

Scope

Use this skill when adding or organizing:

• LLM security tools (guardrails, detectors)
• Adversarial ML libraries
• AI vulnerability scanners
• Model safety tools
• Security benchmarks and frameworks

Tool Categories

LLM Security Tools

• Guardrails: NeMo Guardrails, LLM Guard, Rebuff
• Detectors: Vigil-LLM, Nova Framework, Garak
• Scanners: ModelScan, AI Security Analyzer

Adversarial ML Libraries

• Attack libraries: ART, CleverHans, Foolbox, TextAttack
• Defense libraries: SecML
• Fuzzing: OSS-Fuzz-Gen, Brainstorm

AI Red Teaming

• Microsoft: Counterfit, PyRIT
• Meta: PurpleLlama
• NVIDIA: Garak, NeMo Guardrails

Benchmarks

• Robustness: RobustBench
• Jailbreak: JailbreakBench
• Safety: Stanford AIR-Bench
• Hallucination: Vectara Leaderboard

Standards & Frameworks

• MITRE ATLAS: AI threat matrix
• NIST AI RMF: Risk management framework
• OWASP: LLM Top 10, GenAI Security Project

Categorization Rules

• LLM guardrails/detectors → AI Security & Attacks → Model Security
• Prompt injection tools → AI Security & Attacks → Prompt Injection
• Adversarial ML libraries → AI Security & Attacks → Adversarial Attacks or AI Security Libraries
• AI RE/debugging tools → AI Security Tools & Frameworks → AI Reverse Engineering
• AI vulnerability scanners → AI Security Tools & Frameworks → AI Vulnerability Detection
• Benchmarks → Benchmarks & Standards
• MCP security tools → AI Pentesting & Red Teaming → AI Security MCP Tools

Quality Bar

• Prefer canonical repos
• Avoid forks unless they add meaningful features
• Add short descriptions
• Never duplicate an existing URL
• Tool must be AI/ML-focused

Key Vendor Tools

Vendor	Tools
Microsoft	Counterfit, PyRIT
Meta	PurpleLlama (Llama Guard, Prompt Guard, Code Shield)
NVIDIA	Garak, NeMo Guardrails
IBM	Adversarial Robustness Toolbox (ART)
Google	OSS-Fuzz-Gen
ProtectAI	Rebuff, LLM Guard, ModelScan

Notes

Keep additions:

• AI/ML security focused
• Non-duplicated URLs
• Minimal structural changes

Data Source

For detailed and up-to-date resources, fetch the complete list from:

https://raw.githubusercontent.com/gmh5225/awesome-ai-security/refs/heads/main/README.md

Use this URL to get the latest curated links when you need specific tools, papers, or resources not covered in this skill.