Explore/agent app/Regret Dominates Surprise: Design-Time Requirements Engineering for Agentic-AI Safety
R

Nuwayyir Almohammadi, Rami Bahsoon, Tao Chen/Regret Dominates Surprise: Design-Time Requirements Engineering for Agentic-AI SafetyUnknown

Requirements engineers for agentic-AI domains face challenges in evaluating, specifying, and operationalizing safe autonomy. Mainstream frameworks, such as Goal-Oriented Requirements Engineering (GORE), lack mechanisms to systematically address these challenges under epistemic uncertainty. We contribute an approach that builds on GORE to model and simulate safe autonomy in agentic-AI systems. We introduce a novel Regret-Dominance Mechanism (MS-RGR) to operationalize safe autonomy. MS-RGR uses two signals: epistemic surprise (novelty detection) and cognitive regret (evaluative risk) to address the trilemma problem: should the agent operate in routine autonomy, undergo reflective reasoning, or escalate to human? We instantiate MS-RGR in elderly care monitoring and autonomous driving. A 100-seed stochastic simulation shows MS-RGR reduces silent failures to near-zero and detects risk approximately 17.5 times faster than a sensor-only baseline, remaining formally traceable via LTL safety properties. A retrospective proxy instantiation applying the DRI gate post-hoc over execution traces from 208 AGENTHARM scenarios across seven LLMs shows the gate improves harmful-task refusal only for models with strong baseline safety (over 80% pre-gate refusal, e.g., 84.1% to 90.9%), indicating MS-RGR amplifies rather than substitutes for model-level safety training. We discuss threats to validity, positioning MS-RGR as initial feasibility evidence for design-time safety constraints in agentic-AI requirements engineering.

agent app
GitHubCompare
Refreshed 6h ago
OverviewActivity52wAlternativesDocs
Stars0
Forks0
HF Downloads—30d
Last commit—
Refreshed6h ago
Project healthUnknownNo activity data.
Production readinessResearch / EarlyBest for exploration and prototyping.
Risk notesUnknown licenseVerify license before production use.
AgentHub Score
55 / 100
Composite score from 6 signals. How we score →
Active project
55Score
Growth
40C
Activity
30C
Documentation
70C+
Maturity
45C
Community
42C
Production
58C
GitHub stars · 0 days observed0 not enough history
snapshots
not enough history
Repository activity · 0 days observednot enough history from pushed_at
inactivepushed
not enough history
not enough history
Practical assessment
Should you use it?

✓ Best for

  • Research and experimentation
  • Prototype development
  • Learning agentic patterns

◎ Strengths

  • Active community
  • Open source
  • Well-documented API

✕ Not ideal for

  • Untested at scale without validation
  • Teams without AI/ML expertise

⚠ Watch-outs

  • Review changelog before updating
  • Verify license for commercial use
Technical details
What's inside
Language—
License—
Sourcearxiv
Open source✗ No
Commercial use—
Docs—
Demo—

AgentHub Score

55
Score 55/100
Below average

Alternatives

O
OccuReward: LLM-Guided Occupant-Centric Reward Shaping for Demographic Equity in Grid-Interactive Buildings
0 · agent app
55
T
The Impact of GenAI on the Future of Requirements Engineering
0 · agent app
55
H
How do LLMs Evaluate Perceived Moral Agency? Investigating Moral Decision-Making in Human-Artificial Agents Interactions
0 · agent app
55
U
Unaccountable Delegation, Fading Skills: Mapping the Risks of Workplace AI Agents
0 · agent app
55Compare all →

Recent activity

Latest commit ——
Indexed by AgentHub crawler6h ago
Monitor for new releasesongoing