Explore/observability/Formal Methods Meet LLMs: Auditing, Monitoring, and Intervention for Compliance of Advanced AI Systems
F

Parand A. Alamdari, Toryn Q. Klassen, Sheila A. McIlraith/Formal Methods Meet LLMs: Auditing, Monitoring, and Intervention for Compliance of Advanced AI SystemsUnknown

We examine one particular dimension of AI governance: how to monitor and audit AI-enabled products and services throughout the AI development lifecycle, from pre-deployment testing to post-deployment auditing. Combining principles from formal methods with SoTA machine learning, we propose techniques that enable AI-enabled product and service developers, as well as third party AI developers and evaluators, to perform offline auditing and online (runtime) monitoring of product-specific (temporally extended) behavioral constraints such as safety constraints, norms, rules and regulations with respect to black-box advanced AI systems, notably LLMs. We further provide practical techniques for predictive monitoring, such as sampling-based methods, and we introduce intervening monitors that act at runtime to preempt and potentially mitigate predicted violations. Experimental results show that by exploiting the formal syntax and semantics of Linear Temporal Logic (LTL), our proposed auditing and monitoring techniques are superior to LLM baseline methods in detecting violations of temporally extended behavioral constraints; with our approach, even small-model labelers match or exceed frontier LLM judges. Our predictive and intervening monitors significantly reduce the violation rates of LLM-based agents while largely preserving task performance. We further show through controlled experiments that LLMs' temporal reasoning shows a pronounced degradation in accuracy with increasing event distance, number of constraints, and number of propositions.

observability
GitHubCompare
Refreshed 4mo ago
OverviewActivity52wAlternativesDocs
Stars0
Forks0
HF Downloads—30d
Last commit—
Refreshed4mo ago
Project healthUnknownNo activity data.
Production readinessResearch / EarlyBest for exploration and prototyping.
Risk notesUnknown licenseVerify license before production use.
AgentHub Score
55 / 100
Composite score from 6 signals. How we score →
Active project
55Score
Growth
40C
Activity
30C
Documentation
70C+
Maturity
45C
Community
42C
Production
58C
GitHub stars · 72 days observed0 not enough history
snapshots
Repository activity · 72 days observedReal snapshots from pushed_at
inactivepushed
2026-07-262026-10-07
Practical assessment
Should you use it?

✓ Best for

  • Agent tracing in production
  • LLM cost tracking
  • Debugging agent failures

◎ Strengths

  • Integrates with major frameworks
  • OpenTelemetry compatible

✕ Not ideal for

  • Small prototype projects
  • Teams without existing observability culture

⚠ Watch-outs

  • Data retention costs at scale
  • Privacy implications of storing LLM traces
Technical details
What's inside
Language—
License—
Sourcearxiv
Open source✗ No
Commercial use—
Docs—
Demo—

AgentHub Score

55
Score 55/100
Below average

Alternatives

A
AgentSpy: Making AI Agent Behavior Observable
0 · observability
55
B
BOHM: Zero-Cost Hierarchical Attribution for Compound AI Systems
0 · observability
55
T
Traccia: An OpenTelemetry-Based Governance Platform for AI Systems
0 · observability
55
I
In-IDE Toolkit for Developers of AI-Based Features
0 · observability
55Compare all →

Recent activity

Latest commit ——
Indexed by AgentHub crawler4mo ago
Monitor for new releasesongoing