Tag: AI observability
The Tool Timed Out. Did the Agent Just Do It Twice?
A tool timeout does not prove that a write failed. The action may already have happened. This practical AI agent testing example shows how naive retries create duplicate side effects, and how idempotency keys, reconciliation, explicit uncertain states, deterministic assertions, and failure injection make state-changing agent tools safer.
The AI Agent Test Manual Is Now Available
The AI Agent Test Manual is a practical guide to evaluating, testing, monitoring, and trusting production AI agents. Learn how to test prompts, tools, memory, retrieval, multi-agent workflows, LLM judges, release pipelines, observability, incident response, and the complete system surrounding an agent.
