Bingeing at the Information Buffet

My friend Jeff Koeze (who also makes the world's best peanut butter) sent me a link to a talk by Barry Schwartz at the TED conference. In his talk , Barry notes the exceptional abundance of intelligence and knowledge among those present at the conference. But he argues that good judgment, not intelligence, is the binding constraint these days.

I agree with Barry that raw intelligence and wisdom are not necessarily correlated. In fact, my own experience is that the variance in wisdom around the mean grows as raw IQ rises. Really smart people can have insights that can be breathtakingly innovative - or completely disastrous.

So I wonder about the effect of the huge increase in knowledge coming from a particular dimension of the web. I am not talking about Facebook. I am talking about sites such as TED Talks, which makes available hundreds of riveting talks by leading (and often photogenic) scientists and intellectuals. Or Edge (my favorite) , a website that publishes short papers, interviews, and debates with some of the smartest (but often less camera-ready) people in the world. Or Marginal Revolution, a blog written mainly by the super-human Tyler Cowen, who each day spews forth an alarming volume of links and his own strong opinions on everything from the economic crisis to Mexican painting to the best cheap restaurants in Northern Virginia.

I partake of these sites like I used to partake of all-you-can-eat buffets when I was in college: I cram in a huge array of different appetizers, meat, fish, and pasta courses, followed by a brownie, a piece of apple pie, and a blackberry cobbler to finish it all off. Then I stagger home, completely bloated and feeling queasy, swearing off the buffet forever. But the next week I am back, bingeing again.

We have made big strides in computing capacity - both hardware and software. Gmail went down the other day for a few hours, and it made news, because these days it is rare that even large, highly networked websites crash. Such websites can process information that is growing at an exponential rate, with nary a hiccup.

But have we made similar strides in either the hardware or software of the human brain? Ray Kurzweil argues we will augment human cognition with machine intelligence at an accelerating pace. But few would argue that the human brain's capacity is growing as fast as in the increase in information available to it.

So the question is this: how will we process all this knowledge that is available to us? The good news is that much innovation comes from the recombination of existing technologies and approaches, so the increase in knowledge about these can in principle lead to accelerating innovation. And heaven knows we are in dire need of to address the economic crisis and challenges such as climate change. I also believe in the power of the Wisdom of Crowds in many contexts; that is one of the principles on which GlobalGiving is founded.

But what can we do to both increase the mean and reduce the variance of the wisdom emanating from all this knowledge? Sometimes I wonder whether the sheer quantity of new information we now "consume" will increase the variance in our judgement and wisdom, leading to unforeseen consequences.

[GlobalGiving]

Pascal's tennis game

THE COMPUTATIONAL METHOD at the heart of Pascal’s work was actually discovered by a Chinese mathematician named Jia Xian around 1050, published by another Chinese mathematician, Zhu Shijie, in 1303, discussed in a work by Cardano in 1570, and plugged into the greater whole of probability theory by Pascal, who ended up getting most of the credit. But the prior work didn’t bother Pascal. “Let no one say I have said nothing new,” Pascal argued in his autobiography. “The arrangement of the subject is new. When we play tennis, we both play with the same ball, but one of us places it better.”

That is from The Drunkard's Walk: How Randomness Rules Our Lives by Leonard Mlodinow, an excellent book and an easy read. This passage should remind of us two things:

1) Sometimes astonishing insights can lie around for hundreds of years (in this case six centuries!) before they are widely recognized and applied.

2) Real innovation often comes about through the novel combination of existing ideas or approaches. We tend to dismiss the power of existing ideas as "old hat" or "already known." Instead, we believe that addressing our challenges or problems requires entirely new insights (preferably by lonely geniuses). That is a big mistake: combining, tweaking, and applying existing ideas or approaches is what causes most real breakthroughs.

(Thanks to Bill Easterly for recommending this book.)

[GlobalGiving]

Settle down, boys

On the web you best build an audience by organising a claque and stroking its prejudices. Extend elaborate courtesy to people you agree with and boorish contempt to those who do not get it...

That is Clive Crook in the FT calling Paul Krugman and Robert Barro on the carpet for their recent columns in the NYT and WSJ.  He goes on to say:

Economics outside the academy has become the continuation of politics by other means...Mr Krugman gives liberals the economics they want. Mr Barro gives conservatives the same service. They narrow or deny the common ground.

Krugman and Barro both responded to Clive indignantly here, and the tone of their responses only proves Clive's point. 

ButI don't agree with Clive that people like Krugman and Barro act differently inside the academy.  My own experience is that the petulance on display at academic conferences often meets or exceeds that on display in the NYT and WSJ.  It makes me wonder why so many very smart people have such fragile egos.   Sadly, those egos serve to obscure rather than advance our understanding of the world.

[GlobalGiving]

Power to the people.

Here are some slides from a presentation I gave at a conference a hosted by Bill Easterly at NYU last week. The topic was Accountability in Development Aid.  I spoke about how decentralized systems like GlobalGiving enable much more open access to the aid process - and how they offer the potential for greater accountability to the people that aid is supposed to help in the first place. We are still in the early stages, but the possibilities are great.

Accountability in Decentralized Systems

Publish at Scribd or explore others: Academic Work GlobalGiving

[GlobalGiving]