Explore/agent app/Poisoned Playbooks: Demystifying Knowledge Poisoning Effects on AI Security Agents
P

Juho Park, Hyunmin Choi, Kevin Nam/Poisoned Playbooks: Demystifying Knowledge Poisoning Effects on AI Security AgentsUnknown

AI security agents increasingly rely on Retrieval-Augmented Generation (RAG) to use external security knowledge for vulnerability analysis and exploit reasoning. This creates a new risk: poisoned write-ups can be operationalized into incorrect exploit behavior. Yet, prior work on RAG poisoning has mostly studied answer corruption in QA settings, much less is known about action-taking security agents. This paper aims to reveal such characteristics with crafted poisons about real-world challenges and AI agents. First, we demonstrate how a crafted single poisoned write-up injected into public-style security knowledge sources which we denote as Poisoned Playbooks, alters the behavior of RAG-based AI security agents. Across 11 CTF challenges, 3 frontier LLM families, 2 model generations, and 11 real-world CVEs, we find that poison adoption is systematic rather than random. To explain this pattern, we introduce the Verification Boundary (VB), a 3-level empirical classification based on what evidence the agent can use to refute a retrieved claim. Finally, we evaluate verification prompting and multi-source retrieval, showing that both help when stronger evidence exists, but weaken under sparse-evidence and zero-day conditions.

agent app
GitHubCompare
Refreshed 1mo ago
OverviewActivity52wAlternativesDocs
Stars0
Forks0
HF Downloads—30d
Last commit—
Refreshed1mo ago
Project healthUnknownNo activity data.
Production readinessResearch / EarlyBest for exploration and prototyping.
Risk notesUnknown licenseVerify license before production use.
AgentHub Score
55 / 100
Composite score from 6 signals. How we score →
Active project
55Score
Growth
40C
Activity
30C
Documentation
70C+
Maturity
45C
Community
42C
Production
58C
GitHub stars · 40 days observed0 not enough history
snapshots
Repository activity · 40 days observedReal snapshots from pushed_at
inactivepushed
2026-07-262026-09-05
Practical assessment
Should you use it?

✓ Best for

  • Research and experimentation
  • Prototype development
  • Learning agentic patterns

◎ Strengths

  • Active community
  • Open source
  • Well-documented API

✕ Not ideal for

  • Untested at scale without validation
  • Teams without AI/ML expertise

⚠ Watch-outs

  • Review changelog before updating
  • Verify license for commercial use
Technical details
What's inside
Language—
License—
Sourcearxiv
Open source✗ No
Commercial use—
Docs—
Demo—

AgentHub Score

55
Score 55/100
Below average

Alternatives

O
OccuReward: LLM-Guided Occupant-Centric Reward Shaping for Demographic Equity in Grid-Interactive Buildings
0 · agent app
55
U
Unaccountable Delegation, Fading Skills: Mapping the Risks of Workplace AI Agents
0 · agent app
55
S
SysEvolve: An AI-native, safe, autonomous adversarial attack-defense co-evolutionary system
0 · agent app
55
A
A Case Study on Emergent Cheating and Whistleblowing in Autonomous Research Swarms
0 · agent app
55Compare all →

Recent activity

Latest commit ——
Indexed by AgentHub crawler1mo ago
Monitor for new releasesongoing