What's so challenging about all of this is the idea that you can get very accurate predictions about geopolitical events without access to secret information. In addition, access to classified information doesn't automatically and necessarily give you an edge over a smart group of average citizens doing Google searches from their kitchen tables.That is from a recent NPR piece about how a group of regular citizens have been able to make more accurate forecasts than CIA experts. To its credit, the CIA actually sponsored this work, in collaboration with Philip Tetlock, whose work I have blogged about before. It also gives further weight to the importance of taking a dose of humility along with your own perceived expertise.
The CIA official who helped sponsor the work remains hopeful that experts still have a role, however:
Matheny doesn't think there's any risk that it will replace intelligence services as they exist.
"I think it's a complement to methods rather than a substitute," he said.
Matheny said that though Good Judgment predictions have been extremely accurate on the questions they've asked so far, it's not clear that this process will work in every situation.
"There are likely to be other types of questions for which open source information isn't likely to be enough," he added.
Share on Facebook
