A New Kitchen Gadget or a LifeSaver?

It needs to get as much input as possible from the people who will actually use the stoves. The stoves will need to be as much like existing stoves as possible, to minimize the change in cooking style required to use them. In particular, women need to be able to cook traditional foods that are appealing to their families. Listening to the women who’ll cook on them is the best way to do that.

That is from a nice guest post by Alanna Shaikh, commenting on Hilary Clinton's recent speech at the UN promoting improved cookstoves. Many people are unaware that breathing poorly ventilated cookstove fumes kills an estimated two million people a year.

Improved stoves can reduce the fumes problem while also reducing fuel costs.  But as any of us who have bought a fancy new gadget for our cooking spouses can attest, "great" ideas aren't always appreciated - much less used - if they don't meet the needs of the user. 

Alanna points out the needs for careful consumer research - and even broad consumer marketing, which is rare in the aid and development field.  Even with both of those, it is no guarantee the cookstove initiative will succeed.  In fact as I have written before, it takes an average of 58 ideas for each initiative that succeeds.  But while improved cookstoves are not a silver bullet, they are well worth promoting.

Read the whole post here; it is very good.

Whatever Donor Wants?

But before we rush to the toolkit and assume that better data is all that donors want and need, it’s important to take a step back and remember that while metrics are critical and have their place, they’re only one piece of the puzzle. As studies indicate, there are other equally-important things to consider, among them, personal relationships, family dynamics, social networks, values, and commitment to particular causes or issues.

That is from a nice article by Cynthia Gibson in the Non-Profit Quarterly.  As someone who studied economics and cost-benefits analysis in grad school, I probably appreciate and rely on data more than most.  But at the same time, I also realize that data is not what determines most decisions - even at so-called expert aid agencies and foundations.

For many years, I was annoyed by this and felt that harder arguments and more facts would sway people. But changing human nature, as desirable as that may be, is something rarely achieved.  So perhaps, as I noted earlier, the best approach is to try to work with - rather than reform - human nature to improve decisions about philanthropy and aid.

For marketplaces like GlobalGiving, this does not mean giving up emphasis on data and impact.  To the contrary.  One of the key functions of a good marketplace is to use metrics in the background to drive the average quality of projects higher and higher over time - and to do this while enabling donors to find good projects that resonate with their personal relationships, family dynamics, social networks, values, and favorite causes.

Developing Communities - Not Countries

Guest post by Felipe Cabezas.

"There is no such thing as the Western world and the developing world." – Hans Rosling

The speakers at Monday’s TEDxChange stressed countries’ progress toward achieving the Millennium Development Goals. Even though countries have not universally met them, they have made impressive strides in a relatively short period of time – so much so that classical divisions between the “developed” and “developing” worlds are now muddied.

Yet we still refer to developed/developing countries, North/South and First/Third Worlds in our discourse.

But data implies that development functions on a smaller scale rather than on a larger one. When Hans Rosling dissects country bubbles in his visualizations, he illustrates that average information hides vast differences between regions’ development achievements. He argues that, because “there’s such a lot of difference within countries, it’s not relevant to have [average data] on a regional level. We must be much more detailed.” Bill Easterly takes that extra step and, by zooming into New York City, reveals that significant socioeconomic differences exist even in neighborhoods consisting merely of city blocks.

We already know this. So why do we still refer to developed and developing countries?

This linguistic habit blurs details and positions communities in need to disappear from view. Take Bennett County, South Dakota. Life expectancy in the United States is 78.11 years but in Bennett County is 66.6 years – on par with Azerbaijan, considered a developing country in the 2009 Human Development Report. But by using a developing-country framework, funders will invest in health initiatives in Azerbaijan – not in the United States. What about Bennett County? Doesn’t it warrant assistance, too?

This is not to say that organizations do not assist communities. They do. So then let’s reflect that in the way we speak!

Let’s refer to developing communities instead of developing countries.

This may seem like an insignificant change, but it alters the underlying cognitive framework that serves as the basis from which aid organizations operate. Incorporating developing communities into our lexicon portrays the world as a patchwork of variably developed communities that does not conform to national boundaries – a framework that more accurately reflects the reality of development.

Urban Agriculture Challenge: Communities Helping Themselves (With Delicious Results!)

GlobalGiving nurtures bottom-up, community-based solutions to pressing social problems. We believe in the power of small over large, local over centrally planned and grassroots over top-down. This is why we jumped at the chance to partner with Bonterra Vineyards and Growing Power to support urban agriculture.

Urban farms help low-income communities access fresh food, generate employment, enhance food security, and improve quality of life. Rather than relying on fast food chains or large supermarkets, urban residents with access to a local farm can eat fresh fruits and vegetables grown right in their communities.

I grew up in Kentucky. It’s a great state. But parts of Louisville have been labeled “food deserts” due to the lack of accessibility to fresh food. Through its urban farms, Breaking New Grounds not only brings fresh produce to these underserved neighborhoods, but also provides agricultural training to local residents, and creates new, environmentally-friendly jobs.

In Denver, while fresh food is available in summer, winter months often mean relying on food grown and processed thousands of miles away. Feed Denver catalyzes urban farms that can be operated year-round, giving urban dwellers access to high-quality food from January through December.

Until October 7, these urban agriculture programs – and several others - are participating in an online fundraising challenge on GlobalGiving, with the chance to win up to $20,000 in contributions provided by Bonterra Vineyards.

To further highlight the power of communities working towards a common goal, the Bonterra-Growing Power-GlobalGiving challenge features a collective group incentive. If each participant raises at least $2,000 from 25 or more unique donors, all will receive a $1,100 bonus from Bonterra Vineyards. As on a community farm, each participant’s individual effort will contribute to the larger good. I like the taste of that!

Does the Truth Set Us Free?

Facts, they found, were not curing misinformation. Like an underpowered antibiotic, facts could actually make misinformation even stronger.

That is from an article by Joe Keohane summarizing a number of recent studies about human nature.

I spent the first half of my career working at aid agencies gathering facts to inform policy analysis and recommendations to governments.  This describes the mental model I followed most of those years:

If people are furnished with the facts, they will be clearer thinkers and better citizens. If they are ignorant, facts will enlighten them. If they are mistaken, facts will set them straight.

Alas, that does not seem to be the case:

Rather than facts driving beliefs, our beliefs can dictate the facts we chose to accept. They can cause us to twist facts so they fit better with our preconceived notions. Worst of all, they can lead us to uncritically accept bad information just because it reinforces our beliefs.

When someone does not agree with us, we are often tempted to redouble our arguments to convince them we are right.  Aid agencies often offer carrots and sticks as well.  The research points to why that often does not work either:

The more threatened people feel, the less likely they are to listen to dissenting opinions, and the more easily controlled they are.

Is it time for us to let go of the idea that facts and good analysis can convince people?  I hope not.  But on the other hand, the Buddha realized that he could only be enlightened if he faced reality rather than ignoring it.  The question is how best to combine human nature with facts and analysis to help society progress.

"They even told me so, but I ignored it at first"

I have learned that the community had the answer. They even told me so, though I ignored it at first. But they left it up to me to figure out the 'how.'

That is from a nice paper that Jim Hennigan sent me by David Gaus, titled The Rural Hospital in Ecuador.  It's about how a highly trained doctor and public health expert came to Ecuador with his own well developed sense of what should be done. 


The quote above hints at a successful combination of bottom up and top down: too often experts come in thinking they know what to do as well as how to do it.  But communities generally know what they want; they just don't know how best to get it.  And that is where expertise brought in from the outside can make a real difference.

Innovation Challenges - surprising sources, surprising approaches

Teams from Virginia, North Carolina and Winterthur, Switzerland, with roots in the world of auto racing have won the first Progressive Insurance Automotive X Prize, the $10 million competition aimed at advancing the technology for more fuel-efficient vehicles.

That is from today's New York Times.   One of the benefits of challenges like the X Prize is that anyone can participate, and often the winners come from non-traditional places.  In this case, who would have thought that the winners would come from the auto racing industry, which is known for sacrificing fuel consumption in favor of speed.  And while the second and third place winners used electric engines, the first-place team relied on a not-so-new approach called internal combustion.

The timing of this is appropriate, since I am speaking today at the NCOC's Civic Innovator's Forum, co-sponsored by the Case Foundation, which has done what may be the leading analysis of promoting innovation through challenges and competitions.  Check it out - and while you are at it, check out GlobalGiving's own Global Open Challenge, which is now underway.

"Don't be seduced by visualization..."

"Don't be seduced by visualization.  It can be cool for data wonks.  But you need to focus on real users who are trying to solve a problem."

That was Tim O'Reilly at the World Bank last week, speaking at the opening of a session discussing the Apps for Development competition.  

Tim told me he had never been to the World Bank before.  But many of his words of wisdom apply more broadly to aid initiatives.  Try replacing "apps" with "projects" in the following quotes:

"It may be the case that you invent apps that no one uses."
"Look at other apps to see if you can become part of their ecosystem."
"It may well be that you don't have the right data for your cool new app."
"Think of your app as a component rather than a platform."
"Don't assume your app will be used in isolation."

The purpose of the session I attended was to prime the ideas pump for apps.  Someone at my table suggested an app that allows users to tell developers what kind of app users would find useful.  In light of this previous post, you can imagine that I seconded that person's suggestion.

Listening and Learning

A recent post "If you can flip a coin, can you be an expert?" got a mostly favorable response, but I want to elucidate and emphasize a few things in this post and the next.

First, listening to communities should be the foundation of any aid initiative.  What community members want for their lives should be the starting point.  Though aid agencies may not be equipped to address a problem such as security, the fact that security is of great concern to the community should inform what projects get funded and how they are designed.  For example, if security for women is a big issue, then the design and placement of wells or standpipes is critical.  Or if social tensions are high between certain groups, then great care needs to be taken to design projects that don't exacerbate these tensions.

Second, though there are some outstanding exceptions, as a rule we don't yet do enough listening.  A recent study by Alex Jacobs and Robyn Wilford concluded that "most NGOs do not manage 'participation' or 'downward accountability' in a systematic way."  It noted that a "number of pioneering innovations are emerging...[but] most are still in the experimental stage..."  It cites the Humanitarian Accountability Partnership's 2007 Standard as offering a constructive approach, along with pilots being conducted by the NGO Keystone.  GreatNonProfits is also piloting an approach that listens to a broader set of stakeholders.  NGOs are not alone in not listening enough.  I can speak from long years of experience that listening is an even greater challenge for official aid agencies such as the World Bank, ADB, and USAID.

Third, most aid workers are trying to do the right thing, but they usually have to spend a lot of time and energy managing upwards within their own bureaucracies.  The Jacobs and Wilford study cited above discusses these dynamics, which will be familiar to staff of NGOs and official agencies alike.  In short, the incentives for listening to communities are attenuated at best.

Fourth, listening is hard.  Power dynamics sometimes mean that community members don't say exactly what is on their mind.  In response to a question, they may say what they think the donor or implementing agency (or local official) wants to hear.   With support from the Rockefeller Foundation, we are working with Cognitive Edge on an indirect approach that relies more on storytelling to infer what people really think.  More details on that will follow.  Though this approach is by no means the whole answer, the initial results are encouraging.

Fifth, listening is messy.   There is no single "community."  Community members usually differ on what is most important to them, and the question arises about how to decide to whom you should listen. Should initiatives be decided on majority vote?  In addition, sometimes donors or implementers with a lot of experience feel strongly that a certain approach desired by community members won't work, and that there is a better way.  Should the donor or implementer over-ride the wishes of the community, or should they err on the side of accepting the community's wishes so that learning can take place if the project does not work?  Again, there is no easy answer to this - there needs to be a balance.  On this topic, I recommend David Ellerman's Helping People Help Themselves.

I suspect that the ability to listen is one of the most important factors determining whether aid workers can have a positive - and lasting - impact on a community.  That is why we are trying to help the project organizations on GlobalGiving listen better to the communities they serve.   As always, I welcome comments on this topic.
 

Why Peer Review Works in Science, and Why We Need It for Aid

Guest post by Marc Maxson.

"Calls for aid reform assert that better evidence will lead to better policy."

What if this isn’t true?

What if policymakers don’t really care about whether their policies align best with the evidence? Are we screwed?

No. Good systems can still achieve progress and coordination in spite of the people within that system. The example I know best is scientific peer review.

You might think that the peer review process is a system to weed out garbage and improve publications – but that’s a pleasant side effect. The actual “system” is built around these three components:

1) One “cannon” – Scientists within a field implicitly agree that there will be one shared body of knowledge, to which everyone will contribute. This knowledge can be spread across hundreds of journals because (a) all use the same peer review mechanism and (b) specialized search engines (PubMed, Web of Knowledge, and Scopus) index virtually everything – reducing the odds that an important paper will go unnoticed. Grantmakers also consult the one cannon before allocating.

2) Forced confrontations – Scientists must face their critics and respond to them. Most neuroscience papers are submitted at least 3 times, meaning you get to read about your professional inadequacies a hundred times over a typical career. Peer review also alerts your most successful competitors ahead of others, further pressuring you to address the weaknesses in your paper and resubmit. Dialogue results.

3) A reputation system for scientists (see the H-index) – This system fairly reflects your breadth and depth, ignores non-peer-reviewed work, and requires that your work not only pass peer review but also be valuable to others (frequently cited).

Science is simpler than international development. Knowledge is the only measurable output, and the system outlined above reinforces quality. Moreover, the relationship between scientists and grantmakers is driven by the quality of that knowledge, which is determined by one's peers - not the grantmakers. The NIH program officer doesn't need the knowledge he is funding; he wants to know how many people cited it and what the author's cumulative impact (H-index).

What happens when people try to game the system?

Let's assume for the sake of argument that scientists are only concerned with their own reputations. Scientific facts become a means to an end: prestige.

A scientist could try to publish a bunch of “facts” to vault his career, but only peer reviewed “facts” affect his prestige. Four or more quasi publications equal a peer-reviewed one; that's a lot of wasted effort.

A scientist who promotes his “facts” outside of journals won’t get cited and could get “scooped” by a competitor. These non-canonical publications win the media and public but lose in grant competition. The “herd” protects itself because the H-index never lies.

Groups of scientists colluding to publish each others’ papers and move their reputations forward also fail, for a number of reasons I explain in my longer post.

Dialogue

The best part of the system is that scientists are forced to confront the other viewpoint in order to publish and be heard by the larger community. This is something that is badly needed in aid, because it is currently an afterthought, and disorganized. Some discussion questions:

Q: What incentive do aid practitioners have to discuss work with their peers?

Q: How can grantmakers in international development work from a common set of knowledge, as science grantmakers do?

Q: What compels people to consider different viewpoints before acting?

Q: What is the basis of personal reputation in international development?

Q: What happens when we replace "experts" in the peer review model with "crowds" of beneficiaries?

Q: Could a system that guides grantmakers in this way work in international development?

If you can flip a coin, can you be an expert?

With support from the Rockefeller Foundation, we recently ran an experiment at GlobalGiving that had shocking results.  We asked people in four communities in Kenya to tell stories* about the development issues most important to them.  This was their answer:

In parallel, we asked experts (both local and foreign) with experience in those communities to predict what they thought the story would be about.  Here is the shocker:  Only 1 of the 65 experts and implementers correctly predicted the most common theme.

Personally, I found it surprising that 42% of the stories were about social relations, but on the other hand, I don't work in those communities so how should I know?  But I would expect the experts to know, wouldn't you?  Alas, the experts predicted only three of the top six concerns of the community.  In other words, they did about as well as flipping a coin.** 

The old style of aid is for experts to study the situation and decide what people need.  It is tempting to say that we should simply reverse this and let the people decide.  Exciting new technologies will enable beneficiaries to have a far greater voice in the coming years, and that is long overdue.  But the best system will likely  provide a balance of the two.  It will create an ongoing, iterative conversation between beneficiaries and experts about what is needed, what works and what doesn't, and what that implies about priorities and initiatives in subsequent rounds.

* The story telling part of this experiment was done with the help of Cognitive Edge and Irene Gujit. They were introduced to us by the Rockefeller Foundation, which is leading some of the most innovative work in the field I have seen in years.

** There were a total of twelve themes to choose from.