AI Enthusiast Log

Sign in Subscribe

Topic

evaluation

A collection of 3 issues

Waymo's eval-centric AI development: useful framework, unverified safety claims

Waymo presented eval-centric AI development as its core engineering philosophy at VB Transform 2026, and the approach deserves attention from teams building production AI systems. The company treats evaluation not as a final gate before deployment but as a continuous design constraint that shapes how projects mature. That framing

Intuit overhauls AI agents to fix compounding errors with new architecture

At VB Transform 2026, Intuit VP of AI Nhung Ho described a compounding-error problem that pushed Intuit to rebuild its agent architecture twice in about four months: agents passing results to each other in natural language, with each handoff losing context the next agent needed to act on. "

Eval‑first: Bounding Claude Sonnet 4.5 Breaks with Testing

A reporting pipeline that translated natural language into structured API calls crashed after upgrading to Claude Sonnet 4.5. The model began folding the post_body payload into the description field and, when uncertain, asked clarifying questions. Because the downstream code expected a JSON object with a clear description, endpoint