List’s new book, The Voltage Effect, draws on his experience as a member of President Bush Jr’s Council of Economic Advisers, as well as stints as chief economist at the ride-hailing apps Uber and Lyft, to provide five “vital signs” to ponder before scaling ideas. Both Tory leadership contenders should heed them. List exposes the pitfalls of much “evidence-based policy” while inoculating readers against the unthinking tendency to extrapolate from small trials to full-blown national programmes.
First, beware false positives when scaling ideas. Small experiments often imply something works, but, without replication, noisy data can conceal an absence of impact.
Next, know your audience. Smart thermostats are supposed to slash CO2 emissions, yet List’s study of a randomised introduction of them in California found meagre energy savings. Engineers had assumed households had the rationality of Spock from Star Trek. They instead got energy users who were more akin to Homer Simpson. Businesses and governments must make sure their concepts work with the actual relevant groups.