AI Enthusiast Log

Sign in Subscribe

Topic

ai-evaluation

A collection of 3 issues

Evals are the new PRD, Expedia's AI chief says, while agent incidents climb

Expedia Group's first chief AI and data officer Xavi Amatriain used his VB Transform 2026 appearance to argue that product requirements documents for AI systems should be replaced by evaluation suites. "The new PRD are the evals," he told the audience, framing evaluation design as the

Enterprise AI agent evaluation gap leaves half of passed agents failing

A VentureBeat Pulse survey of 157 enterprises found that half have shipped an AI agent or LLM feature that cleared their internal evaluations and then failed a customer in production. Only 5% say they fully trust automated evaluation. Yet two-thirds already allow, or are actively engineering toward, allowing agents

Amazon AGI lab says AI agent pilot to production gap is measurement failure

Amazon's AGI autonomy lead used a VB Transform 2026 stage to argue that the enterprise agent deployment gap is a measurement failure, not a capability one. Bryan Silverthorn, who joined Amazon through the Adept AI acquisition, said teams are still scoring agents on benchmarks that mask the dimensions