Thanks to Tyler Cowen for pointing me to a great book by Philip Tetlock called "Expert Political Judgement: How Good Is It? How Can We Know?"
The answer is that experts' predictions are no better than those made by regular people. In fact, many experts have a worse track record than would be achieved by a toss of the dice. (The worst performing experts are "hedgehogs: thinkers who 'know one big thing.'" By contrast, "foxes: thinkers who 'know many small things'" tend to do somewhat better.)
Louis Menand's review in the New Yorker notes that this phenomemon holds across a variety of disciplines and over one hundred studies. College counsellors who were provided detailed information about students and even allowed to interview them did worse in predicting their freshman grades than a simple mathematical algorithm. Psychologists were unable to outperform their secretaries in a test to diagnose brain damage.
As a former "expert" at the World Bank for fourteen years, I can say that this resonates with me. I cannot speak highly enough of the technical expertise of my colleagues, and I have many friends still there. Yet, it was a highly sealed and self-referential environment that admitted little input from other experts, much less from the average guy on the street in Jakarta or Nairobi.
Although I became a reasonably senior executive and was successful by most measures, the truth was that I found the place intimidating. So many decisions and predictions had to be made about so many complex things, and I often found myself wondering how people could walk around the halls looking so self-assured.
This all changed when Mari and I did the Development Marketplace, which opened the front doors of the Bank to anyone in the world with a good idea. Over one thousand groups --many of them grassroots -- applied to the first marketplace. The result was extraordinary - the quality of the ideas, the energy, the commitment, and the resources that "regular" people brought to the Marketplace were in aggregate far superior to what most Bank experts like me could think up.
The amazing potential unleashed at that Marketplace was enough to give me the courage (some say temerity) to leave the World Bank and help launch GlobalGiving. I have never looked back.
[GlobalGiving]
