The problem with “economists-failed-to-predict-the-2008-crisis” macrodeath articles

This week has delivered one more interesting batch of economics soul-searching posts. On Monday,  the Bloomberg View editorial board has outlined its plans to make economics more of a science (by “tossing out” models that are “refuted by the observable world” and relying “on experiments, data and replication to test theories and understand how people and companies really behave.” You know, things  economists have probably never tried…). John Lanchester then reflected on recent macro smackdown by Bank of England’s Andy Haldane and World Bank’s Paul Romer. And INET has launched a timely “Experts on Trial” series. In the first of these essays, Sheila Dows outlined how economists could forecast better (by emulating physics less and relying on a greater variety of approaches) and why economists should make peace with the  inescapable moral dimension of their discipline. In the second piece, Alessandro Roncaglia argued that considering economists as princes or servants of power is authoritarian, and that giving them such an asymmetric role within society is dangerous.

Rich and thoughtful as this macrodeath literature is, it leaves me, again, frustrated.  A common feature of virtually all articles  dealing with the crisis in economics is that they are built around economists’ failure to predict the 2008 financial crisis. And yet, they hardly dig into the sources, meaning and consequences of this failure (note: in this post, I’ll consider that a forecast is a specific quantitative and probabilistic type of prediction, and I’ll use the two terms interchangeably. Shoot, philosopher). The failure to forecast is usually construed as a failure to model, leading to suggestion to improve modeling either by upgrading existing ones with frictions, search and matching, financial markets, new transmission mechanism, more variables, ect., or going back to older models, or changing paradigms altogether. Yet,  economists’  approach to forecasting rely on much more than modeling strategies, history whispers.

Agreeing to forecast, disagreeing on how and why

Macroeconomics is born out of finance fortune-tellers’ early efforts to predict changes in stock prices and economists’ efforts to explain and tame agricultural and business cycles. In 1932, Alfred Cowles expressed his frustration in a paper entitled “Can Stock Market Forecasters Forecast?” No, he concludes:

A review of the various statistical tests, applied to the records for this period, of these 24 forecasters, indicates that the most successful records are little, if any, better than war might be expected to result from pure chance. There is some evidence, on the other hand, to indicate that the least successful records are worse than what could reasonably be attributed to chance.

Two years after Ragnar Frisch, Charles Roos and Irving Fisher had laid the foundations of the Econometric Society, Cowles liaised with the 3 men and established a Cowles Commission in Colorado Springs. It is not clear to me how pervasive a goal forecasting was in the first decades of macroeconomics and econometrics, how much it drove theoretical thinking, which role it had in the import of a probabilistic framework into economics. Historical works on Frisch and Haavelmo, for instance, suggest it is difficult to disentangle conditional forecasting from explaining and policy-making. Predicting was one of the 5 “mental activities” Frisch thought the economist should perform, alongside describing, understanding, deciding and (social) engineering (see Dupont and Bjerkholt’s paper). Forecasting wasn’t always associated with identifying causal relationships, as exemplified by the longstanding debate between chartists and fundamentalists in finance, but for early macroeconometricians, the two went hand in hand. That explaining, forecasting and planning were inextricably interwoven in Lawrence Klein’s mind is well-documented by Erich Pinzon Fuchs in his dissertation.  He quotes Klein saying his

“main objective [was] to construct a model that [would] predict, in the [broader] sense of the term. At the national level, this means that practical policies aimed at controlling inflationary of reflationary gaps will be served. A good model should be one that [could] eventually enable us to forecast, within five percent error margins roughly eighty percent of the time, such things as national production, employment, the price level…”

The notion that economics is about predicting is however not usually associated primarily with Klein’s name, but with Milton Friedman’s. In his much discussed 1953 methodological essay, Friedman proposed that the “task [of positive economics] is to provide a system of generalization that can be used to make correct predictions about the consequences of any change in circumstances. Its performance is to be judged by the precision, scope and conformity with experience of the predictions it yields.” These predictions “need not be forecast of future events,” he continued; “they may be about phenomena that have occurred but observations on which have not yet been made or are not know to the person making the prediction. And this is what makes economics policy-relevant, he concluded: “any policy conclusions necessarily rests on a prediction.” Klein and Friedman’s shared statement that the purpose of economic modeling is to predict has come to be widely accepted, yet it is not clear how many competing views of what the purpose of economics should be circulated in these years.

Most important, their longstanding dispute on statistical illusions reveals that they neither agreed on the purpose nor on the proper method to forecast, nor even on what a “good” forecast was. Klein believed macro econometric models should be as exhaustive as possible, Pinzon Fuchs documents, that they should accurately depict reality. This belief was tied to his desire to conceive engines for social planning, models that could provide guidance as to which exogenous variable the government should alter to achieve full-employment. In the NBER tradition, Friedman rather endorsed simpler models with few equations. He considered Klein’s complex machinery as a failure and endorsed Carl Christ’s idea that these models should be tested through out-of-sample prediction. Erich argues that Friedman was merely trying to understand how the economic system works. I rather interpret his work as an attempt to identify stable behaviors and self-stabilizing mechanisms. As Friedman believed government intervention was inefficient, he did not need the endogeneity or exogeneity of his variables to be precisely specified, which infuriated his Keynesian opponents. “The Friedman and Meiselman game of testing a one-equation one-variable model….. cannot be expected to throw any light on such basic issue as how to our economic systems work, or how it can be stabilized,” Albert Ando and Franco Modigliani complained in the 1960s. More fundamentally, Friedman doubted that statistical testing was fit for evaluating economic models. The true test was history, he often said, which might explain why, to Klein’s astonishment, he switched to advocating goodness-of-fit kind of testing with Becker in the late 1950s. Methodological pragmatism, or opportunism, as you want to see it.

What is the failure-to-predict about: statistical methods? Models? Institutions? Epistemology? 

As this historical exemple suggests, claiming that macro is in crisis because of economists’ failure to predict the financial crisis is too vague a diagnosis to point to possible remedy. For what is this “failure-to-predict” about? Is is a statistical issue? For instance, a failure to estimate models with fat-tailed variable distributions, or to handle a sudden unseen switch in the mean of that distribution (what Hendry calls “location shifts”). Or is it a theoretical issue? For instance, failing to explain why stock market returns are fat-tailed, to model firms and households‘ exposure to financial risk and its systemic consequences  into macro models, to take shadow banking into account, to identify the drivers of productivity. A bigger failure to model institutions, complexity, heterogeneity?  Improving theoretical modeling is the bulk of what is discussed in the macro death literature.

On the contrary, that changes in economic structures or in perceived ways for government to intervene in the economy (for instance through macro prudential regulations, QE, etc) have made  economists’ regular predictions  irrelevant, useless or less accurate is less discussed. Keynesian macroeconometricians have built models aimed at conditional forecasting (aka dealing wit questions such as : what happens to the economy if the government raises the interest rates?), though central bankers have sometimes used these for unconditional forecasts (to get next year’s GDP figures). But the “failure-to-predict” criticism deals with unconditional forecast, as was the case with part of the Phillips curve debate during the 1970s. Finance economists have also traditionally been mostly concerned with unconditional forecast. I’m thus left wondering whether the rise of financial dimensions in public intervention have led to misusing DSGE models, or have fostered the development of macro models aimed at hybrid forecasting.

Finally, this “failure-to-predict” literature might point to a deeper epistemological shift. It is, of course, one seen in some economists’ rejection of DSGE modeling and endorsement of alternative models (agent-based, evolutionary, or complexity), interdisciplinary frames, in their call to go back to Minsky or Kindleberger, or even to good old IS/LM. But among those economists who have traditionally endorsed DSGE macro, there also seems to be a shift away from forecasting (unconditional and conditional) as the main goal of macroeconomics or economics at large. In 2009, for instance, Mark Thoma has commented that “our most successful use of models has been in cleaning up after shocks rather than predicting, preventing or insulating against them through pre-crisis preparation,” and the blogosphere and newspaper opens are ripe with similar statements. These can be interpreted as rhetorical gestures, defensive moves, or early symptoms of an epistemological turn. Itzhak Gilboa, Andrew Postelwaite, Larry Samuelson and David Schmeidler have , for instance, recently worked out a formal model in which they suggest that economic theory  is not merely useful through providing predictions, but also as a guide and a critique to economic reasoning, as a decision-aid .(This ties in with their broader call for case-based reasoning).

Granted, this is all very muddled. I am probably making artificial distinctions (for instance, I couldn’t decide whether Gabaix’s work on power laws and granularity belongs to statistical or theoretical analysis), and I am certainly misunderstanding key concepts and models. But my point is, it should be the purpose of the macrodeath literature to un-muddle my thoughts.  What I’m asking for is two types of articles:

(1) articles on economists’ failure to predict the crisis that are explicit about what their target is and how their championed substitute approaches will yield better conditional/unconditional predictions. Or, if their alternative paradigms reject prediction as the key purpose of economic analysis, why, and what’s next.

(2) histories of how economists have theorized and practiced forecasting since World War II. A full-fledged history of forecasting in economics and finance is a little too ambitious to begin with. What I’m interesting in is why and when empirical macroeconomists, in particular macroeconometricians,  have endorsed (conditional?) prediction as their key objective, what resistances did they encounter, what were the debates over how to produce , evaluate and use forecasts, whether models built for conditional forecasting were used by central bankers and by their  own producers for unconditional forecasting (think Wharton Inc and DRI), whether it shaped their relationships with finance and banking specialists, and how they reacted to the first salvo of public criticisms (70s economic crisis, Phillips curve breaking down, ect.). Additionally, recasting public and governmental anxiety about forecasting in the wider context of changing conceptions and uses of “the future” may help understand the challenges postwar economists faced.

Note: These great fantastic TERRIFIC pictures are based on suggestions by @Arzoglou, @dvdbllrd, @1EquationShort, and @t0nyyates. One of these pictures, and only one, features real economists whose band name is an insider’s pun.

Advertisement