Calibrate Llm Evaluators From Pinned Human Judgments On Traces

Kitaru records or imports production AI agent traces as sessions, then replays the real agent code against changes in model, prompt, or tools so you see what improved or regressed before shipping. Self-hosted, Apache 2.0, with Python/TypeScript SDKs, framework adapters, MCP server, and agent skills.

AI Tools | Python · agent-evals · observability