Predicting the Future

By Philip Tetlock
The National Interest, September/October 2009

Edited by Andy Ross

The Fat Tail: The Power of Political Knowledge for Strategic Investing
By Ian Bremmer and Preston Keat
Oxford University Press, 272 pages

The Predictioneer's Game: Using the Logic of Brazen Self-Interest to See and Shape the Future
By Bruce Bueno de Mesquita
Random House, 272 pages

The Next 100 Years: A Forecast for the 21st Century
By George Friedman
Doubleday, 272 pages

Karl Marx quipped that, when the train of history hits a curve, the intellectuals fall off. There is great potential for mischief in the political forecasting business. The demand for accurate predictions is insatiable. Reliable suppliers are few and far between.

Some ways of thinking translate into more correct forecasts. Borrowing from philosopher Isaiah Berlin, I call them foxes — experts who know many things and are not finicky about where they get good ideas. Contrast this with what I call hedgehogs — experts who know one big thing from which likely future trends can be more or less directly deduced.

What experts think is a weak predictor of accuracy. But how experts think is a consistent predictor. Relative to foxes who are less encumbered by loyalties to an all-encompassing worldview, hedgehogs offer bolder forecasts and, although they hit occasional grand slams, they strike out a lot and wind up with decidedly poorer batting averages.

The authors of the books under review are all entrepreneurial futurists, but each offers a strikingly distinctive approach to prediction. I organize them under three headings: the super-pundit model, the technocratic-pluralism model, and the scientific-reductionist model.

The super-pundit model

George Friedman is the founder and CEO of a private intelligence and forecasting company. Of the three books, his was the least persuasive. Friedman leaves the impression that he is one of the select few today who can see deeply into the twenty-first century. He relies heavily on plate-tectonics metaphors.

Friedman admits that, in 2007, he completely missed the economic implosions of 2008. His defenders may argue that although we cannot predict a major earthquake in California in any given year between now and 2050, we can predict with substantial confidence that a major earthquake will occur. Readers must judge the merits of these metaphors for themselves.

If you average the predictions of many pundits, that average will typically outperform the individual predictions of the pundits from whom the averages were derived. This works when (1) the experts are mostly wrong, but they are wrong in different ways that tend to cancel out when you average; (2) the experts are right about some things, but they are right in partly overlapping ways that are amplified by averaging.

The technocratic pluralist model

Ian Bremmer and Preston Keat eschew bold forecasts. They encourage their readers to look closely at how to think about the future. They deploy a wide range of tools to assist them in this task and warn readers against common pitfalls. They use provocative thought experiments, scenarios, and historical analogies to alert readers to the potential for abrupt discontinuities.

We expect high-impact events to happen very rarely, but in fact they happen surprisingly often. One never knows when a revolution of some sort is just around the corner. Be prepared to be shocked by Rumsfeld's unknown unknowns.

Bremmer and Keat emphasize that probability distributions can suddenly take on shapes that a few years earlier were quite unthinkable. Economists call these distribution-altering events exogenous shocks. The authors point out too that just because something like geopolitical risk is hard to quantify does not give you license to ignore it. Rough approximations are better than tacitly treating the risk as zero.

The authors suggest we might consider scenario planning. This requires policy advocates to envision worlds in which things don't work out as planned. But for all their open-mindedness and sensitivity to state-of-the-art tools for analysis, the authors remain timid foxes.

The scientific-reductionist model

Bueno de Mesquita is an unapologetic hedgehog. He is a world-class political scientist who appreciates the pivotal role of transparent scientific methodology in advancing human knowledge. He makes strong claims about how far he can see into the future with the aid of the right expert inputs and game-theoretic algorithms.

Bueno de Mesquita's prediction machinery has a straightforward logic. Once we have mapped the option space, we get our experts to follow his four-step formula: (1) identify every individual or group with an interest in the decision, (2) estimate what each of the identified players wants, (3) estimate how big an issue this is for each of the players, and (4) estimate the influence of each player.

This formula is a variant of expected-utility theory: for each policy option and context, multiply each player's preferences by each player's motivation-to-prevail by each player's political clout, and then predict that the other side will choose the option with the most points.

Bueno de Mesquita offers an explanation for his success: the experts are wonderful sources for case-specific inputs, but they are not experts on how people make choices, which is the value added from the model.

I am not persuaded that Bueno de Mesquita is working with the right model of human choice. We need to know how rational the other side is, and how rational the other side assumes us to be. Once we admit these complexities of human nature back into the equation, we lose the simplicity of the model.

Tournaments?

Forecasters know they need to sound as though they are offering bold insights into the future. And they know they cannot afford to be linked to mistakes. Accordingly, they have to appear to be going out on a limb without actually going out on one. That is why they so uniformly appear to dislike affixing probability estimates to possible outcomes. It is much safer to retreat into the vague language of possibilities and plausibilities.

We may be stuck in a suboptimal equilibrium. But there is a way out. Players high up in the political system who want the best-possible forecasts could invest in a series of long-term forecasting tournaments designed to distinguish the more from the less promising forecasting approaches.


Philip Tetlock is the Mitchell Endowed Professor at the University of California, Berkeley.
 

AR  I guess I'd better take all this to heart as kindly advice for my next book, which presents an attempt to outline how technology will shape global developments in the next few decades.