Holding Beliefs Lightly

"But I’m trying to hold my beliefs lightly, in the long view that almost all of them, I’m sure, will one day be seen as very naive or even completely misguided.

The purpose of holding them lightly is not to drop them completely, but just make it more difficult to beat other people over the head with them, more difficult to hold on to them when they’re clearly not being helpful, and easier to swap for other ideas, when those new ideas appear promising."

That is from a very nice post about "smart aid" by David Week.   Any serious aid worker is constantly trying to infer principles about what works and what doesn't, and then to have those principles guide his/her actions.  Naturally, many of us want to tell others what we have discovered about what is effective and what is a waste of time and resources.  But we should realize, says Week, that we will often be wrong, and that should be very humbling.

Free the aid bloggers!

"In general I fully support transparency, but these people could lose their jobs."

That is what Saundra Schimmelpfennig told me when I asked her why some of the best aid bloggers out there were anonymous. She is right, of course. But it is also a shame. If there is a common thread running through our understanding of effective aid, it is the need to experiment, learn, and adapt. This means admitting to - rather than hiding - things that don't work, so that we can learn from them. The anonymous bloggers I was referring to talk about the reality of aid work, warts and all. They have a following because their readers know that they are speaking the truth. But their employers could not tolerate the truth, so these bloggers have to remain in the closet.

Good aid projects come from good conversations

If we could directly observe the results of aid projects, we could avoid a lot of the heavy bureaucratic processes and reporting that makes life miserable for so many aid workers and that often impedes and obscures quality work.  Like Owen Barder, I believe the time is coming when new approaches and technologies will help us clear away a lot of this bureaucratic brush so we can focus on results.

In the meantime, a blogger called Tales from the Hood has a nice post on practices considered correlated with "good aid" - i.e., aid that has the desired impact.  I particularly like his/her first point:

1) Starts and ends with the needs of those affected by poverty, disaster, and conflict (a.k.a. “the poor”, “aid recipients”, “program participants”, “beneficiaries”…).  ...[I]f we’re to do it right, if we’re to plan and implement good aid, our starting point needs to be those whom we seek to serve. If that starting point is anything else (for example, the needs of a particular donor, surplus of something…) then a recipe for bad aid has already been started.

This requires listening to what communities themselves want. And then listening to how they feel that projects are being implemented.  And then listening afterwards to what they learned.

Listening is not easy. Communities are full of diverse interests and unequal distributions of political power and voice, and it is usually tough to sort out these various influences.  Many aid workers and organizations still lack the capacity to listen systematically in such contexts. But dealing with ambiguity and conflict by not listening is usually a poor strategy.

And listening should not be one way. The best aid projects I have seen have been born from a conversation between community members and aid workers. All involved in the conversation bring unique information, perspectives, and desires. A constructive outcome - leading to a good aid aid project - requires give and take on both sides. I love this piece by David Gaus where he describes how he learned to listen - and be heard - while working on health care in Ecuador.

How much can we rely on medical experts?

"He charges that as much as 90 percent of the published medical information that doctors rely on is flawed."

That is from a recent article by David H. Freedman in the Atlantic Monthly sent to me by my friend April Harding.   The "he" is Professor John Ioannidis, who has done as much research on this topic as anyone in the world.   Here are some other key quotes from the article:

... he was struck by how many findings of all types were refuted by later findings.

“I realized even our gold-standard research had a lot of problems,” he says. Baffled, he started looking for the specific ways in which studies were going wrong. And before long he discovered that the range of errors being committed was astonishing: from what questions researchers posed, to how they set up the studies, to which patients they recruited for the studies, to which measurements they took, to how they analyzed the data, to how they presented their results, to how particular studies came to be published in medical journals."

... 80 percent of non-randomized studies (by far the most common type) turn out to be wrong, as do 25 percent of supposedly gold-standard randomized trials, and as much as 10 percent of the platinum-standard large randomized trials...

 “You can question some of the details of John’s calculations, but it’s hard to argue that the essential ideas aren’t absolutely correct,” says Doug Altman, an Oxford University researcher who directs the Centre for Statistics in Medicine.

If between a third and a half of the most acclaimed research in medicine was proving untrustworthy, the scope and impact of the problem were undeniable. That article was published in the Journal of the American Medical Association.

Sometimes people contact me to say they think I should emphasize more the importance of experts in development.  I often respond by saying that the issue is in development is finding what works rather than relying on people who have credentials based on their degree or who they work for (see Saundra Schimmelpfennig's excellent post on this).  By contrast ( I used to say) relying on credentials made sense in certain fields where there was a clear link between credentials, knowledge, and outcomes such as structural engineering and medicine.

This article makes me think that I should re-think my reference to medicine - a field that needs to re-examine its own standards of proof and good practice.  It does not mean that I will stop going to the doctor when I need to.  It does mean that I will ask a lot of questions (something fortunately encouraged by my current doctor).

Does this have anything to do with philanthropy and development?  I am not sure, but I did like this quote from another researcher, Athina Tatsioni:

“Usually what happens is that the doctor will ask for a suite of biochemical tests—liver fat, pancreas function, and so on,” she tells me. “The tests could turn up something, but they’re probably irrelevant. Just having a good talk with the patient and getting a close history is much more likely to tell me what’s wrong.”

Professional is not a Title

"Professionalism may have less to do with your job title/organization and whether you are paid staff or a volunteer, and more to do with how you approach aid/development."

That is from Saundra Schimmelpfennig's excellent post over at Good Intentions are Not Enough.   There has been a lot of commentary on Nick Kristof's recent NYT Magazine article about "DIY" foreign aid.  The heat:light ratio of that commentary has been high.  Saundra's common sense in summarizing the apparently diverging views on this topic is most welcome, because it shows that there is more consensus than might be apparent. Here is more:

"There is a need for fresh perspectives and a variety of ideas and approaches. However this must be tempered with knowledge of the factors that led to success and failures in the past so the same mistakes are not constantly repeated."

I recommend the whole post.

Transparency: How to Get from Consensus to Impact?

Here is a nice paper from Homi Kharas of Brookings on how transparency can transform accountability in the official aid sector.  The paper goes beyond treating transparency as a slogan and instead asks "what is it good for?"

Here are some thoughts that came to mind as I read the paper:

1. Key questions: a) Who is doing what (b) to whom (c) with whom (d) why, and (e) how is it going?
2. An emphasis on beneficiary voice is key.  Kharas says "In many ways, the call for aid transparency is a hand maiden of the call for greater ownership by recipients."
3. Transparency should be mostly about learning rather than punishment or blame - see this.  
4. This will only work if we get the incentives right - incentives for donors to put out data, and incentives for people to provide feedback and input.  Top-down mandates will not work; my own experience is they will sink under their own weight. As a corollary, simpler may be better than complex.  Witness Yelp and TripAdvisor.  Both have flaws, but they are sustained and influence behavior in ways more complex and heavy systems do not.
5. For these initiatives to work, they require a combination of factors - the right information, gathered from the right sources, displayed through the right user interface, to the right people, at the right time.
6. Lant Pritchett has a very nice article about the disincentives that donors face for honest evaluation.  Bottom line is that there is all downside, and little upside. 
7. Devesh Kapur and I predict that allowing each donor to run its own transparency system will lead to obfuscation and lack of network effects.  The biggest effect would come from a single, independent system that donors can't massage. 

What's the Bottom Line?

Aid is made less effective by the incentives which aid agencies face, which they in turn transmit to their staff.  In large part, these unhelpful incentives are a consequence of lack of information about results.  If we can measure results better, and if we can use this to simplify the management of aid (and not simply bolt additional reporting on to existing bureaucratic processes),  this will enable more decentralised decision-making, respect country ownership, make the jobs of aid workers and government officials more rewarding, improve the effectiveness of aid, and so reduce poverty faster.

That is from Owen Barder's post Incentives, Results, and Bureaucracy in Foreign Aid. If you have never worked in a large aid organization, this piece will help you understand the pressures that even the best aid workers face.  And if you have worked in one of these organizations, you will find that Owen offers a ray of hope that might (just might) allow you to stop spending so much time on internal process and start spending more time on what we all care about: results.