Monday, July 27, 2009

What's wrong with experts?

What's wrong with experts?

Nothing, as long as they don't monopolize control over decisions, resources, and information. Experts - people with special skills, a lot of experience, and/or who have thought a lot about an issue - should be an important part of any decision-making or resource-allocation process.

Experts are especially critical in technical fields, where there is a clear link between their expertise and outcomes. For example, if you are going in for heart surgery, you want the best expert you can find. Ditto if you want to fly in a jet, build a high-rise building or dam. Even in these areas, however, you want competition among experts, because that competition drives innovation and efficiency over time. Competition also allows best practices in these fields to adapt to changing technologies and socio-economic circumstances.

But there are many areas where there is no clear right answer or no clear best practice. And it is precisely in these fields that we must be careful of a "tyranny of the experts." Economic and social development pose many challenges for which there are no clearly demonstrated productive solutions. The low returns to the $2 trillion spent on development aid to date are testimony to this.

The relatively small number of aid agencies controlling the bulk of official aid means that a small number of experts control decisions over the allocation of most of the resources. This raises multiple problems. Let me demonstrate by describing a time in my career most people don't know about.

In the late 1980s and early 1990s, I worked in the World Bank's Jakarta office, where a group of two or three of us were in charge of hundreds of millions of dollars of rubber, palm oil, and coconut projects. For a period, we had more knowledge than any of our Bank colleagues about the financing, planting, and processing of these tree crops in Indonesia. As a result, any decisions about the design or implementation of projects had to come through us. Looking back, there were several problems with this:

* We knew a lot, but we didn't know everything - in fact I realize now that we were wrong about some fundamental things.

* Bandwidth: Managing projects in the Bank bureaucracy was exceptionally challenging, so we had to work many, many hours per day just to keep our own ideas and projects moving; there were not enough hours in the day for us to listen to a lot of new or opposing ideas.

* We had little incentive to listen to different views about what should be done, because we had all the power.

* Our rewards came from pleasing our colleagues and boss, not from the on-the-ground success of our projects. We would therefore respond to our colleagues' concerns first, and those of beneficiaries second (and in any case, we had to spend much more time at headquarters than in the field.)

* In an expert-driven culture, we was expected to come up with the right answer, and then implement it - not to launch of bunch of experiments with different approaches to see what worked best.

The bottom line is that, for what we were doing, there was no single right answer. But our opinions, biases and time constraints prohibited us from considering, allowing, or evaluation a variety of approaches to see what worked best. I also realize now that what might have worked best in 1985, before I began working on these projects, was not the same thing that would work best in 1990. But our various constraints (and power) inhibited our ability to adapt our thinking to the ongoing technical, social and financial changes in the environment.

In retrospect, I still believe that our expertise was valuable. But we did not leverage that expertise in the right way. Instead of our centrally planned, top-down design and administration of these massive projects, we should have been convening a conversation among all the stakeholders, promoting a series of different approaches, and helping everyone learn from the different outcomes so they could iterate toward ever better results.

The secret is that few "experts" like the current system. They know deep down that it is inefficient and ineffective. When given the opportunity to play more of a convening and coaching role, most of my colleagues at the Bank loved it because they knew they were adding more value. The challenge for the future is for us to enable experts to play this role to a much greater degree in the future.