Explore/observability/Efficient Auditing of Adversarial AI Agent Behavior from Agent Traces
E

Eugene Zhang, Cheng-Yun King Yang, Dongyan Xu/Efficient Auditing of Adversarial AI Agent Behavior from Agent TracesUnknown

AI agents powered by large language models (LLMs) can perform complex tasks but may harm the systems they operate in, either intentionally or unintentionally. Existing agent monitoring approaches rely on rule-based guardrails or LLM-based trace auditing. However, rule-based guardrails can be bypassed through obfuscation and may miss harmful actions beyond their predefined rules, whereas applying an LLM to audit every action is costly. We present a two-stage agent trace auditing framework. The first stage uses single-event and trace-sequence rules to select pending actions for inspection; the second uses an LLM audit agent to examine each selected action in the context of the agent's preceding trace before execution. We jointly refine the gate rules and audit instructions using training data, allowing the framework to adapt to complex agent behaviors rather than relying solely on predefined rules. On the public benchmark OpenAgentSafety, our framework reduces the average number of LLM audits from 8.15 to 2.33 per run and token usage from 47.8k to 14.6k, with a detection rate of 72.8\% compared with 81.5\% when every action is audited. In two simulated multi-agent case studies, the framework flags all malicious traces while reducing audit token usage by more than 80\%.

observability
GitHubCompare
Refreshed 6h ago
OverviewActivity52wAlternativesDocs
Stars0
Forks0
HF Downloads—30d
Last commit—
Refreshed6h ago
Project healthUnknownNo activity data.
Production readinessResearch / EarlyBest for exploration and prototyping.
Risk notesUnknown licenseVerify license before production use.
AgentHub Score
55 / 100
Composite score from 6 signals. How we score →
Active project
55Score
Growth
40C
Activity
30C
Documentation
70C+
Maturity
45C
Community
42C
Production
58C
GitHub stars · 0 days observed0 not enough history
snapshots
not enough history
Repository activity · 0 days observednot enough history from pushed_at
inactivepushed
not enough history
not enough history
Practical assessment
Should you use it?

✓ Best for

  • Agent tracing in production
  • LLM cost tracking
  • Debugging agent failures

◎ Strengths

  • Integrates with major frameworks
  • OpenTelemetry compatible

✕ Not ideal for

  • Small prototype projects
  • Teams without existing observability culture

⚠ Watch-outs

  • Data retention costs at scale
  • Privacy implications of storing LLM traces
Technical details
What's inside
Language—
License—
Sourcearxiv
Open source✗ No
Commercial use—
Docs—
Demo—

AgentHub Score

55
Score 55/100
Below average

Alternatives

A
AgentSpy: Making AI Agent Behavior Observable
0 · observability
55
B
BOHM: Zero-Cost Hierarchical Attribution for Compound AI Systems
0 · observability
55
T
Traccia: An OpenTelemetry-Based Governance Platform for AI Systems
0 · observability
55
F
Formal Methods Meet LLMs: Auditing, Monitoring, and Intervention for Compliance of Advanced AI Systems
0 · observability
55Compare all →

Recent activity

Latest commit ——
Indexed by AgentHub crawler6h ago
Monitor for new releasesongoing