PM-Bench: Evaluating Prospective Memory in LLM Agents — AI benchmark

Genglin Liu, Saadia Gabriel/PM-Bench: Evaluating Prospective Memory in LLM AgentsUnknown

A significant challenge in agentic AI is prospective memory: the ability to execute an intention at a specific future cue or state while other activities are ongoing. We introduce PM-Bench, a text-based benchmark for measuring prospective memory capabilities in modern LLM agents. Inspired by the Virtual Week paradigm from cognitive science, PM-Bench evaluates how well LLM agents maintain user intentions, execute delayed intentions, and monitor latent environment changes. Over the course of a simulated seven-day week, agents must continue an ongoing activity while deciding whether any deferred task is due. We compare eight state-of-the-art LLMs on PM-Bench under eight different agent configurations. PM-Bench proves challenging across all settings: the best method, a GPT-5.4 agent, reaches only 65.1\% F1 score under our evaluation. Furthermore, no single strategy for improving prospective memory dominates across models. We release PM-Bench as a controlled testbed for diagnosing these failures and developing training or inference-time interventions that support reliable prospective behavior.

benchmark

Stars0

Forks0

HF Downloads—30d

Last commit—

Refreshed4h ago

Project healthUnknownNo activity data.

Production readinessResearch / EarlyBest for exploration and prototyping.

Risk notesUnknown licenseVerify license before production use.

AgentHub Score

48 / 100

Composite score from 6 signals. How we score →

Active project

48Score

Growth

40C

Activity

30C

Documentation

70C+

Maturity

45C

Community

42C

Production

58C

GitHub stars · 90 days0 +0.0%

30d90d1y

Commit activity · 52 weeksActive contributor activity

LowHigh

JunSepDecMarNow

Practical assessment

Should you use it?

✓ Best for

Research and experimentation
Prototype development
Learning agentic patterns

◎ Strengths

Active community
Open source
Well-documented API

✕ Not ideal for

Untested at scale without validation
Teams without AI/ML expertise

⚠ Watch-outs

Review changelog before updating
Verify license for commercial use

Technical details

What's inside

Language—

License—

Sourcearxiv

Open source✗ No

Commercial use—

Docs—

Demo—

PaperarXiv ↗

AgentHub Score

Score 48/100

Below average

Alternatives

crewai

26.1k · Multi-Agent

autogen

42.7k · Multi-Agent

smolagents

11.2k · Coding

openai-agents-python

9.4k · Multi-Agent

Compare all →

Recent activity

Latest commit ——

Indexed by AgentHub crawler4h ago

Monitor for new releasesongoing