Evals & Quality

Everything the radar has featured in this area, newest first.

• 2026-10-20 · AQuA: an ambient quality agent that diagnoses your production agent — Silent regressions that still return HTTP 200 are the defining failure mode of production agents. 'Agent watches agent, human approves fix' is becoming the standard outer loop, and steering it with a plain-English goal.md puts it within a PM's reach.
• 2026-10-20 · How To Use Decision Models for Evals — Judge cost is becoming the long pole of continuous evaluation. Decision models cut it by an order of magnitude, but default 0.5 thresholds and uncalibrated scores can quietly lie to you. Choosing between them is a product decision about false alarms versus missed failures.
• 2026-10-20 · Evaluating AI agents: moving past the vibe check — This is the clearest big-vendor endorsement yet of the practitioner consensus: error analysis first, pass/fail judges instead of 1–5 scales, and evals as a release gate rather than a dashboard. It gives you a vendor-neutral way to argue for eval investment internally.