Evaluating LLM Agents in CI

Agent evaluations are flaky, expensive and easy to fool. This talk covers the harness we run on every pull request: deterministic seeds, judge calibration, cost budgets, and the tripwires that catch a regression before it reaches a customer.