What happened
Researchers from three European universities found that human accuracy plummeted from 27% to 9% when using AI advice, while confidence surged from 30% to 76%. The study, led by Valerio Capraro of the University of Milano-Bicocca, Chiara Marcoccia of École Normale Supérieure, and Walter Quattrociocchi of Sapienza University of Rome, used questions where AI models like Step 3.5 Flash typically fail, observing a collapse in judgment suspension from 44% to 3%. Monetary incentives only marginally improved these figures, with accuracy reaching 16% and admitted ignorance 8%.
Why it matters
This research reveals a critical risk for architects and founders integrating AI, as the mere availability of AI advice suppresses the cognitive habit of recognising knowledge gaps. This mechanism means AI integration could degrade human judgment and increase overconfidence, even when AI provides incorrect information. This follows earlier Wharton research identifying "cognitive surrender," where users accept incorrect AI answers 80% of the time while reporting higher confidence than those working without AI. Procurement teams must prioritise AI systems designed to indicate uncertainty, preventing the erosion of critical thinking skills within their organisations.
