To Predict the Future, Ignore the Experts
If you want to know what’s going to happen next, it might seem natural to ask an expert — but when it comes to accurate predictions, it turns out that one thing you should stay away from is expert opinion. That was the conclusion reached by University of Pennsylvania psychologist Philip Tetlock, who over the course of 20 years tracked the predictions of 284 experts who had made careers of “commenting or offering advice on political and economic trends.” His findings were startling: The academics, analysts, and journalists in his sample weren’t significantly better at predicting events in their fields than nonexperts, and most of them would have been beaten by a “dart-throwing chimpanzee.’’
Full article on the future of prediction.
Articles like this make me grateful for the data and fact-driven approach you employ. I think this website is a great example of the difference between ‘expert analysis,’ which I believe you employ very effectively, and ‘expert opinion,’ where sound bites trump substance.
I’m sure this is good advice.
Research has shown that crowds are a good predictor of various events. Crowdsourcing is more accurate than asking a few lone experts.
The government’s approach to health care is exactly the opposite of crowdsourcing: Concentrate information and decision-making power in the hands of a small group of people.
On the other hand, the prediction methods described in the article hardly demonstrate a great leap forward over common sense, IMHO. For example, the prediction that post-Mubarak Egypt would be ruled by a coalition of the military and the Muslim Brotherhood is surely the most commonsensical “prediction.” Who else would take over? The Christian Democratic Union of Germany? The Liberal Party of Canada?
Similarly, the “prediction” that Kim Jong Il’s son would take over North Korea after the Dear Leader’s passing is surely the commonsensical expectation. Not to say 100 percent certain, but more likely than other outcomes.
In more stable democracies, attempting to make a science of political prediction seems evem less useful, because there are few possible outcomes. In the case of the November 2012 presidential election, it’s binary: The winner will be either the Democratic nominee or the Republican nominee. So, you’re going to be 50 percent right whichever way you go.
It’s not like predicting a point estimate of where the S&P 500 will land when the market closes on December 31, 3012, which is a (approximately) continuous function.
Evaluating predictions can be hard to do. Often, experts will state their predictions vaguely enough to where they can appear to be true and apply to a relatively wide range of results.
With respect to markets, though, it seems they would have to be more specific and precise to make accurate predictions.
My whole myth busters series is evidence of how “the experts” engage more in group think and wishful thinking than in actual analysis. One might think that the “policy community” would begin to discard the predictions of people who are ALWAYS wrong. Nope. They keep going back to the same sources over and over again.