Friday, June 29, 2012

Macro, science, and falsification

Noah Smith has an interesting post up on the subject. It's one of those posts where on all practical counts I agree with him but I would not agree with a lot of the more abstract points he makes (namely, that macro is not scientific... or to be more fair what he actually said is that it's "barely a science at all"). The reason he says this (and the reason I disagree) is, of course, falsification. I've never been particularly impressed by Popperianism, as readers know. Science is as much about compiling convincing evidence as it is about a strict falsification regime. It's about deriving the best explanations we can about what we observe (which does not require falsification - although of course it never hurts), not about saying something about an ultimate reality. Science accepts that its explanations are imperfect and is happy with broader laws that explain processes well. It doesn't require that we provide an explanation so precise that it can be definitively falsified (although again, that's always nice). Of course someone who worries about being strict on falsification can live in the community of scientists fairly comfortably. Instead of theorizing that "all swans are white", you could always theorize that "a whole lot of swans are white". The latter theory works just fine. "A whole lot of recessions are caused by demand shocks, and this is how we think the ones that are work" is a fine macroeconomic scientific statement too. Most science is about this sort of stuff. Developing good descriptions, from observation, of one process out of many potential processes that may or may not be operating at a particular time. Falsification is never a bad thing when you do this sort of science, but when you realize that in complex systems sometimes you've got lots of processes going on that deserve careful research, the ideal-type of the falsification contest starts to seem a little beside the point of it all.

But like I said, as a practical matter I agree with a lot that Noah has to say. He concludes with: "The solution is for macroeconomists to A) admit their ignorance more often (see this Mankiw article and this Cochrane article for good examples of how to do this), and B) search for better ways to falsify macro theories in a convincing way." Both are very good suggestions, and I'm particularly interested in B.


Some reasons why people get so down on macro (inappropriately, in my view) is:

1. They have this idealized Popperian physics in their head (is this even how physics works?) of crisp laws describing phenomena and clear cut experiments, and that's just not what defines science nor is it representative of all science. Economics is more like biology: (a.) complex processes all at work at once (so it's not clear there has to be one "right" macroeconomic theory (b.) explanation is more important than prediction, precisely because it is a complex system, and (c.) there's a whole lot of good scientific work to do that is largely descriptive or taxonomical. Physics-envy is not good for economics. We're studying the social behavior of highly evolved apes. Economists are very specialized primatologists. That should give you a hint that you should probably look to biology rather than physics for a template.

2. We expect models to explain a lot more than they were designed to explain. This problem with how people view macroeconomics is well illustrated in a recent post (quoting an older article) by Mark Thoma. He writes: "Macroeconomic models have not fared well in recent years – the models didn’t predict the financial crisis". What a fascinating sentence! Why would anyone expect a macroeconomic model to predict a financial crisis? Who says this is a determinant of whether macroeconomic models have "fared well"? Can I say that household bargaining models haven't "fared well" because they didn't predict the financial crisis? Of course not. That's silly. The financial crisis was not a macroeconomic crisis. It was not driven by macroeconomic processes. The financial crisis was caused by the vagaries of the finance industry. You judge finance models by this, not macroeconomic models. Now, how have macroeconomic models done in predicting how the macroeconomy would respond to the financial crisis? How have macroeconomic models done in explaining the path of output, interest rates, inflation, and employment? Some have certainly done more than others. None predicted a financial crisis because, well, that's what finance guys do. Should we look into macro-finance linkages? Definitely. Definitely, definitely, definitely. But the crisis was caused by the structure of CDOs, by certain risk models, by expectations about house prices, and by the exposure of different financial firms. None of these are really issues that macroeconomists are best placed to speak on.

3. Prediction may be impossible. This kind of expands on point #1. We don't say that seismologists aren't scientists because they can't predict the epicenter and magnitude earthquakes a couple years in advance. We don't say that climatologists aren't scientists because they can't predict right now when and where the hurricanes will hit next hurricane season. We don't say that evolutionary biologists aren't scientists because they can't sketch out the food chain 100 million years from now. And these failures at prediction also don't cause us to abandon the models used by seismologists, climatologists, or evolutionary biologists. Even if you get caught up in this falsification project, falsification doesn't necessarily imply prediction. Prediction of complex systems is very hard. Sometimes we can just explain the behavior of complex systems. Scientists aren't miracle-workers.


  1. Daniel, I am always a bit confused about why, exactly, Popper's view is always quoted as self-evident and the ultimate truth. I think it's exactly true that Popper was mostly concerned about what may be called "exact" sciences, especially physics. For some time, at least, he called the theory of evolution a metaphysical construct (he later reversed this opinion, but it shows that his knowledge of scientific research was very narrow, even within the natural sciences). I don't see where he actually got into social sciences in the same way and believe that a convincing demonstration that his reasoning applies there is completely lacking.

    Furthermore, it's not as if his work were undisputed, there have been fierce critics like his initial student and disciple Paul Feyerabend who demonstrated that, even if the question if Popper's reasoning makes sense, it is not in any sense a path scientists are actually following. Though I think this is, strictly speaking, not a problem to Popper's theory - as falsification relates to a demarcation problem, not to a methodology - I don't see how the whole thing can be important if actual scientists ignore it. (One example is this: Feyerabend shows that, historically, Mach had the better arguments when he refused Boltzman's atomistic view and that, at the time, the atomistic view was NOT falsifiable as a theory, but merely served as a model, there were no means to actually show or disprove atoms - that is, that the atomistic worldview was accepted, was due to "propaganda", in Feyerabend's terms, and became a falsifiable theory later. Feyerabend suggests this episode, and lots of others, as evidence for his stance that the validity of theory should not, a priori, be judged by its falsifiability, and that one would lose lots of insights from discounting them on this basis). Also, I still do not understand how one can reconcile Kuhn and Popper (but perhaps that's just me).

    There are contemporary examples, too: take Noam Chomsky. His "Minimalist Program" is explicitly meant as such: a scientific program - which refers to Imre Lakatos' philosophy. I guess Chomsky is perfectly indifferent to anybody believing that his approach is not scientific...

    I guess the bottom line is: there is absolutely no reason to think that simply saying "Popper" and "falsifiability" is a basis to judge a theory - neither has Popper covered all of science, nor is he undisputed, nor does everybody care (and why should they?). My hunch is that if you ask people about what exactly they mean when they refer to Popper and/or falsifiability, the answer will be that for a theory to be scientific one has to be able to check it against facts. But, as Feyerabend says, this is not Popper, this is simply common sense.

    1. And there we go, Noah Smith in the comments:

      "You know, I've never read Popper?

      "Falsification" is shorthand. A term of convenience because it's too annoying to write something longer every time. What it really means is just "deciding that some ideas don't work by comparing them with reality." If you can't do that you've got nothing. One doesn't Popper to see the truth in that."

      I can't say for English, but German has a couple of words to say what Smith means - "falsif..." is a very specific term that carries a lot of epistemological connotations. Is there really no English word that is less bound to a specific body of philosophical work ?

  2. If people are down on macroeconomics, a few reasons come to mind:

    1) Macroeconomists claim to know more than they do. Example: The Charter Cities people claim, "We know the right rules."

    2) Related to the above, the failure of macroeconomists to recommend experimental policies. During the Great Depression, FDR tried a number of things, some of which did not work. Even Hoover could not stand by and do nothing, despite Mellon's advice to "purge the rot".

    3) Macroeconomists, on the whole, seem to take the viewpoint of creditors and the wealthy. Examples: The policies of the Troika in Europe, the American bailout of Wall Street but not Main Street, the claim that fiscal stimulus would not work, because "people" would save against future taxes, completely ignoring the existence of poor people.

    BTW, I am not down on macroeconomics, myself. :)

  3. On falsifiability:

    The extreme claim is that confirmatory evidence is no evidence at all. Hypotheses can only be disconfirmed. That is the basis for standard (non-Bayesian) statistical tests.

    But even if we admit confirmatory evidence, it is very weak, weaker than most people think. The observation of a white swan is confirmatory evidence for the hypothesis that something is a swan only if it is white (a logical formulation of the hypothesis that all swans are white, which is ambiguous in Engish). But so is the observation of a red Ford, and to the same degree.

  4. 1.) The econophysicists do cite Popper as having put into words what physicists have long done in terms of practice.

    2.) I think you separate finance and economics a bit too much on this part. There's a reason financial economics exists.

    3.) Prediction may be possible in terms of financial markets, actually. According to Dr. Michael Emmett Brady, Benoit Mandelbrot's research provides evidence that the financial markets ARE ergodic. They demonstrate ergodicity in their spikes and crashes. Therefore, it is perfectly possible to predict the rise and fall of a speculative bubble.

    1. I'm not separating finance and economics. I'm separating financial economics from macroeconomics. I'm saying don't expect a macro model to predict a financial crisis - that's not what they're designed for. How could they? Do we say that other non-finance models haven't fared well because they failed to predict the financial crisis?

    2. Daniel,
      isn't the state of financial markets deeply tied to macrophenomena? If so, why separate macro- and financial economics then?

      Also, what is your opinion on Keen's models of debt acceleration? Don't get me wrong, I am not intending to start a flame war on it, I am just interested in your outlook as a professional economist. Do you read that sort of heterodox theories?

  5. Daniel Kuehn: "I'm separating financial economics from macroeconomics."

    Aren't financial crises macroeconomic phenomena?

    D. K. : "I'm saying don't expect a macro model to predict a financial crisis - that's not what they're designed for. How could they?"

    Ah! The problem in a nutshell! :)

  6. If Friedman lived to see the falsification of monetarism of the day, it is not entirely non falsifiable.


All anonymous comments will be deleted. Consistent pseudonyms are fine.