My friend, Louise, is a world-class forecaster. I’m trying to figure out how she does it.
Louise and I both volunteered for the Good Judgment Project, a crowd-sourced forecasting tournament for world events. Here’s a sample of the questions we’re forecasting:
When will China next conduct naval exercises in the Pacific Ocean beyond the first islands chain?
When will SWIFT next restrict any Russian banks from accessing its services?
When will Ethiopia experience an episode of sustained domestic armed conflict?
For each question, we get 100 points and a calendar. We distribute the 100 points based on when we expect an event to happen. If we don’t expect the event to happen, we simply place all 100 points beyond the calendar.
Philip Tetlock, who wrote the landmark study, Expert Political Judgment: How Good Is It? How Can We Know?, started the Good Judgment Project to improve forecasting of political events worldwide. Why would we need to improve our forecasting? Because experts are really lousy at it.
In his book, Tetlock studied 28,000 projections made by 284 experts. The results were little better than chance. Computers could do better. Tetlock surmised that crowds might do even better and started the Good Judgment tournament.
The tournament starts anew every year with several hundred volunteers. Louise and I participate in a tournament that started last December. We’ve been forecasting for close to three months. I looked up the standings last week. Louise was number one worldwide. I was number 48.
How does Louise do it? It could be that she’s a fox as opposed to a hedgehog. According to the Greek poet Archilochus, “The fox knows many things but the hedgehog knows one big thing.” Isaiah Berlin popularized the idea in the 1950s with his study of Tolstoy, titled The Hedgehog and The Fox.
So, which is better: a hedgehog or a fox? As with so many things, it depends on what you’re aiming to do. In Good To Great, Jim Collins argues that you need a hedgehog mentality to build a great company. You need to know one big thing and stick to it.
But what if you’re trying to forecast the future? Tetlock argues persuasively that foxes are better than hedgehogs. Why? Here’s how Stewart Brand explains it: “…hedgehogs have one grand theory (Marxist, Libertarian, whatever) which they are happy to extend into many domains, relishing its parsimony, and expressing their views with great confidence. Foxes, on the other hand are skeptical about grand theories, diffident in their forecasts, and ready to adjust their ideas based on actual events.”
It’s probably fair to guess that Louise is a fox. She doesn’t have one grand theory that explains everything from SWIFT financial transactions to Chinese naval maneuvers. She’s also flexible in her thinking. She’s willing to pick ideas from different sources and change her position if the evidence warrants. She gathers feedback and uses it to adapt and adjust.
In Tetlock’s studies, foxes outperform hedgehogs by a wide margin – in forecasting, though perhaps not in building great companies. Hedgehogs on the other hand, “…even fare worse than normal attention-paying dilettantes.”
Louise and I – and other volunteers in the Good Judgment Project – are probably normal attention-paying dilettantes. We follow current events but we don’t have grand theories that explain everything. In some cases, I suspect that Louise doesn’t know what to think. But she does know how to think. Like a fox.