Thursday, June 27, 2013

Questions from Russ Roberts, Part 1

He poses three questions to Krugman and me (and those of you who agree with me should feel free to jump in). Some I think take more thinking than others, and I've got a few other things to do today so I'll probably trickle the answers out. I'll start with the last one, the easiest one, and in a lot of ways the most frustrating one to see Russ asking:
"Third, does the failure of the Keynesian models using multipliers from past economic experience to accurately predict the effect of the 2009 stimulus package give you any uncertainty about your claims?"
I could just say "I reject the premise" and move on, but if someone with a platform like Russ is repeating the claim it's worth walking through what's wrong with it again (we've discussed this many times here and actually I discussed it a lot when I used to comment at Russ's blog too). My short answer is "I am always uncertain about my claims, but nothing in the data since 2009 has made me more uncertain about my fiscal policy claims (I'm not sure whether I should revise my priors on monetary policy)" - for the long answer, read on.

Russ is not being particularly forthright about what he's saying here. What he's actually asking is "why did the Romer-Bernstein prediction get it so wrong?". That version of the question is an excellent question but by skipping from that question to his question about "the Keynesian models" Russ is sneaking in a claim that people need to be aware of.

He is implicitly claiming that the problem with the Romer-Bernstein prediction was the multiplier and not the forecast, because forecasts have very little to do with Keynesianism. What Romer and Bernstein did was estimate a forecast of how the economy would behave with no change in fiscal policy (in their case, how the unemployment rate would behave) and then they subtracted off how much unemployment would have been eliminated by the stimulus package (I think it was a little higher than what was passed, but roughly the same).

I find Russ's implicit claim (and it would be nice if he confirmed that he thinks that so everyone is clear on where the disagreement is and where it isn't) to be entirely implausible. The economy is not a car... the economy is organic. It's a complex system. Forecasting a complex system is extremely challenging - particularly when you're doing it so far into the future (like a meteorologist, our short-term forecasts are decent enough). On top of the normally challenging task of forecasting you have to remember that Romer and Bernstein were forecasting at a time when credit markets were in turmoil and they had no idea how that would be sorted out. So in arbitrating between whether I trust the multiplier half of the prediction which has been carefully worked on and argued over for decades or the forecast half of the prediction which is weak by its nature and done quickly and largely done in the dark I unequivocally trust the multiplier half and am leery of the forecast.

Aside from the normal problems posed by forecasting a complex system, we have very good reason to believe that the forecasts in this particular crisis weren't up to the job because of the problems in the financial sector. I assume something like a DSGE was used to generate the forecast although it might have been even simpler than that, but even DSGE models don't take finance into account. Noah Smith had an excellent post on this issue a few weeks back summarizing research at the New York Fed (Mark Thoma also has thoughts here) that concludes that when you incorporate finance into DSGE forecast models (even without calibration - a critical point), the model does a lot better. Nobody used to use DSGE models in this way, though, because financial crises haven't played such a big role in macroeconomic fluctuations lately.

So trusting that forecast seems like a bad bet to me, but it's critical to understand that if Russ wants to cast doubt on the multipliers it's a bet that Russ is making. It's up to him to explain why - I couldn't tell you that.

So what about multipliers?

We don't actually have real-time estimates of the multiplier associated with the Recovery Act or the changes in state and local spending during that time. We need to "identify the model" to get that, which basically means we need find a way to deal with the fact that causality runs in both directions*. Fiscal policy has some effect on output, but output also determines fiscal policy. For certain components of fiscal policy that vary automatically with changes in the economy like unemployment insurance, taxes, and food stamps (we call these "automatic stabilizers") the causal link from output to fiscal policy is very tight. This means that if we just look at the raw data there is a natural tendency (because of the causality running from the state of the economy to fiscal policy) to observe a negative relationship between fiscal policy and output or at least to underestimate any positive causality running from fiscal policy to output.

I won't go into the details here about how we "identify the model" here (search my archives - we talk about this stuff a lot), but suffice it to say that that is the principal obsession of empirical economists, whether they're doing micro or macro work. These are estimated and argued about in the literature, and if Russ wants me to rethink my views he needs to dig up that literature and tell me why I'm wrong in my assessment of it.

As it stands if I had to choose between (1.) the Romer-Bernstein consensus on multipliers, and (2.) the DSGE forecasts that Romer-Bernstein presumably use, I have considerably more faith in (1.).

Why am I wrong?


  1. Answering part of question 1. Krugman used a similar transformation a few years ago but I can't find the original post but it shows that we have a balance sheet recession where savings is much larger than investment. Something that historically is unusual and reflects that rises in government spending will not crowd out investment or raise interest rates. It also lends credit to the idea that we're in a liquidity trap.

  2. also notice how he says the Krugman thinks that debt to GDP ratio, supply side, factors, crowding out are not relevant right now, when Krugman never stated that nor do I think he has ever done so. I think you are right remember that Krugman disagreed with the Romer-Bernstein forecast.

  3. There are two states of the world. In one, the projections based on Romer-Bernstein were shown to be more or less correct. In the other, they are shown to be very incorrect. Both the DSGE model and the fiscal multiplier are more likely to be true in the first world than in the second world. Therefore, since we are living in the second world, you should adjust your priors downward, if only very slightly.

    1. This assumes my prior was that the DSGE forecast and the multipliers were more or less correct to begin with. If I had assumed that DSGE probably wouldn't do well forecasting this crisis (and I did have doubts about this when Romer-Bernstein came out), it's not entirely clear why I ought to adjust my priors.

      If the first scenario turned out to be true I probably should adjust my priors in favor of forecasting, but as you say that's not what happened.

      If we are talking about minor-prior tweaking I might concede points similar to this (although I've learned more about multipliers and fiscal policy in a depression since 2008 which push those priors the other way more significantly). But we're at the point where Russ is writing things like "the failure of Keynesian models" as if that's something that's actually happened. So in that sense the argument here at least is very different.

      An argument on Increasing Marginal Utility, I agree, would probably be on different issues and run very differently from an argument on Cafe Hayek.

  4. Also, I reject the characterization of the Romer-Bernstein as a consensus forecast. It is the high end of reasonable; see Ramey's lit review in JEL. There are many, many high quality studies using SVAR and other modern methods of identification that find the multiplier to be less than 1.

    1. The consensus between them, I mean. They are considering a bunch of different studies and they came to the conclusion of what's the best one. I agree that's not everyone's assessment (and we've discussed Ramey and others on here before).

    2. Aren't most of the studies in Ramey's lit review that have a multiplier below 1.0, not entirely relevant to our current situation (the great recession) because most of them do not include liquidity trap time periods? My question was simply from a cursory overview of their summaries, so it's possible I missed something.

  5. "He is implicitly claiming that the problem with the Romer-Bernstein prediction was the multiplier and not the forecast, because forecasts have very little to do with Keynesianism."

    Except when they are successful forecasts about price inflation made by Paul Krugman (and David R. Henderson). Then they are incredibly relevant to the policy debate. (I'm more criticizing Krugman than you, Daniel, FYI.)

    1. The empirical content of the prediction comes from the DSGE model.

      The empirical content of the price inflation forecast comes from whatever informed your mental model.

      As I implied in the other content, we should all update against both while trying to be as honest about our priors as possible.

    2. Krugman also just said inflation is gonna be low for a while. He's not really "forecasting inflation". And I think Ryan is right - having a mental model that says that given we're in a depression interest rates and inflation are gonna be low for a while is different from forecasting their precise values.

      And even if we were to forecast their precise values, the zero lower bound helps a little, doesn't it! But success on that count has very little to do with the stability of long-run forecasting.

      Think of it this way - if there were no zero lower bound he probably wouldn't have "forecast" interest rates so well (I use quotations because again - this really isn't forecasting that he's doing at all).

    3. Anyway on inflation Krugman was wrong - didn't he say we'd have deflation for a much longer period of time?

  6. Maybe im just uneducated, but this reads like you're saying:

    "the models fail, but that doesn't mean that the models are wrong, just that the forecasts are wrong due to things we can't be sure about. I am an empirical economist, even though I seem to make theory first and work out the empiricism later."

    Maybe the problem is that economics just can't be an empirical science without godlike knowledge?

    1. I didn't say anything like that - let me know if there's something specific you're caught up on.

      We're in the same predicament with forecasting as geologists with earthquakes or climatologists and the weather or evolutionary biologists and future species or astronomers and solar activity.

      The presence of absence of a crystal ball is not really a relevant determinant of good empirical science.


All anonymous comments will be deleted. Consistent pseudonyms are fine.