Bob Murphy had asked me to read his new paper with Phil Magness a little while ago to get reactions before it was released, and I was swamped and didn't get a chance to. Still semi-swamped, I read it tonight, but I don't want Bob to take this as ambushing. I genuinely hadn't had a chance when he initially sent it. I do want to give some thoughts about how I think people should approach the paper.
1. Accept all the botched history stuff and don't get hung up on it. A lot of people have focused on this and it is understandably frustrating Bob. It's not the main event - it's a little context of sloppiness on Piketty's part that sets the stage for the rest of the paper. A hook, if you will. And it has hooked some major news outlets.
2. So moving on from the botched history my biggest reservation is that despite the fact that they went into the excel files they really only take a superficial look at the massive empirical undertaking here and then just raise a bunch of red flags when things aren't immediately obvious from the technical appendix. This technical appendix is quite big actually. I get the impression that some people don't realize how sparse these things sometimes are. Behind a technical appendix is of course all the data and code itself and you are never going to be able to reproduce that exactly from the appendix. That is not a problem - sometimes you have to actually make the effort of emailing the author and tracking that stuff down. Replicating studies is extremely tedious work. The one time I've done it (not even for anything we were publishing - just to get a sense of what another author was doing) it required extended email conversations with the author and looking over their code and data. It's a big undertaking. That's not what Magness and Murphy seem to have done. They seem to have just shrugged their shoulders when they weren't sure what was going on in the technical appendix and then started hoisting up a bunch of red flags.
3. So a running theme in the paper is that Piketty makes all these data decisions and they all always seem to work in his favor. Of course when you're splicing together data there are a bunch of ways to do it and you're going to make a lot of data decisions for all sorts of reasons that are likely not all going to be enumerated in the technical appendix. So I took a closer look at the data that went into Table 10.5 (again - I did this exercise a couple months ago just to understand what he was doing but didn't really jot it down), curious if Magness and Murphy were really right that at every stage of data decision, where Piketty could have gone another way, it always helped his narrative. It didn't take very long to find at least three cases in the one figure.
3a. First, Piketty splices the K&S series with the SCF series using a multiplier. It shifts the level of the K&S series up but preserves the trend. The multiplier is of course based on years where both SCF and K&S data are available. However, Piketty uses a stable multiplier of 1.25 for the 1930s through the 1950s - the period when capital took a big hit from depression and war. The stable multiplier is an odd choice because the SCF/K&S ratio is actually declining as you go back in time. If you project the trend in the multiplier rather than using a stable multiplier you'd get a multiplier that's lower than 1.25. But Piketty, for whatever reason, didn't do that. Failing to project the multiplier back and instead using a fixed multiplier of 1.25 results in a higher top 1% and 10% wealth share for the 30s, 40s, and 50s than there otherwise would have been making the reduction from depression and war less dramatic and therefore the rise in wealth inequality less dramatic than it would have been otherwise.
3b. Magness and Murphy complain that Piketty uses SCF for the 1960s before jumping back to K&S in the 70s. The reason of course is that there are no SCF data points in the 70s. But what is the effect of using the SCF in the 60s - of making this a "Frankenstein" graph, in Magness and Murphy's words? Well if instead they had continued with the way they were adjusting the K&S series and waited until the 1980s to introduce the SCF, the 1960s point would have actually showed lower inequality. So by using the SCF in 1960 they are making the mid-century Golden Age a little less Golden, working against his narrative.
3c. Moving on from the 1960s, Magness and Murphy then complain that the K&S data is used in the 1970s to make a distinct trough in the U-shape for wealth inequality. (This is actually a slight mistake on Magness and Murphy's part. The data comes from the 1960 data point and only the growth rate from 1960s to 1970s is borrowed from K&S.) So what's the alternative? Well instead of using the 1960 SCF figure and projecting forward using K&S growth rates the other thing he could have done is use the K&S inequality data and then harmonize it the same way he harmonized the 30s-50s. Which multiplier would you use? The stable 1.25 multiplier or the 1.29 estimated multiplier from 1960? It turns out it doesn't matter - no matter how you calculate that alternative it's actually lower than what Piketty ended up doing. So once again, Piketty's data choice for the 1970s - taking the 1960 SCF and applying K&S growth rates - made the trough less deep than it would have been if he used the K&S data itself.
So there's three examples that I just saw and thought were worth sharing. I'm sure there are lots of others. And the point isn't to suggest that Piketty is being overly conservative. I'm sure some data choices "helped" him. My point is simply that there are a bunch of decisions to be made and picking out a few that helped him doesn't make much of a case against him. All of these decisions seem fairly reasonable whatever ultimate decision is made, and it only has minor effects anyway.
4. So one last point - again I am worried that Magness and Murphy rushed through this (it's only been six or seven months since he was first blogging on it) when they write that "Figure 10.5 continues from there as he extrapolates a wealth distribution for the top 10% by simply appending fixed percentages without further explanation". So this is really worrisome. He actually does give explanations. The fixed percentages come from the SCF studies. Now the Kennickell study is a little confusing to track down because it's posted as Kennickell 2000 at the Federal Reserve but then there's a 2001 updated version that Piketty cites. But all of it's quite easy to track down and find the "fixed percentages" that are appended. Like Magness and Murphy I wasn't quite sure what was going on everywhere, but the idea that it's there "without further explanation" is plain nonsense. So is this as really as deep as they went? I don't know - it seems like they were stumped as soon as they came across those fixed percentages. But it reinforces my impression that this big hole they've allegedly blasted in Piketty simply amounts to giving up any time they ran into a head scratcher in the technical appendix.
I am now curious about a couple things:
1. Did Magness and Murphy notice the three points I made here where Piketty's data decisions work against his narrative? And if they did why didn't they include them?
2. Did Magness and Murphy even try to email Piketty and ask for do-files, etc.? (This is a little less relevant for Figure 10.5 but more relevant for some of the other wealth series, particularly the UK).
3. What did the referee reports at the Journal of Private Enterprise say? I'm really curious about where the referees pushed back.
It looks as if M&M have not fallen as badly on their faces as the ludicrous Giles at FT. But they pretty clearly have wildly oversold their case and should cool it quite a bit on all the chest thumping and self glorifying.
ReplyDelete