Monday, February 4, 2013

What's wrong with this research?

Here.

I have no doubt prison has these sorts of effects but I have a strong sense the reported effects are exaggerated. What do you think about the design of the analysis (click through the link in the NPR piece to the journal article - there's info to glean even fom the abstract).

I've spent a lot of time in the NLSY. There are about 9,000 youth in it, so when they refer to "tens of thousands" I think they mean across multiple waves. Only a small portion of these youth are ever incarcerated - it was so small that my co-author and I couldn't do much with it when we were looking at employment rates for young adults. So that's an issue, but what else concerns you here?

UPDATE: I should add - the author may very well have listed all these limitations in his article. That's not unusual at all and in my mind it's perfectly appropriate: dig up what you can and qualify the hell out of it. But this NPR report does not seem to have the same caution.

7 comments:

  1. I gave it a cursory glance, and one of the things I noticed they do is keep in zero earners. They say that "doing so highlights the distinction between criminals and noncriminals," but that's a weird distinction to make. Obviously criminals have higher illegal income than noncriminals--that's true by definition! What is really of interest here is whether incarceration increases the illegal income _of criminals_.

    The more interesting analysis here is to compare kids who got incarcerated against kids who were engaged in the same illegal activity but didn't get incarcerated. That teases out the actual effect of incarceration from the effect of just not being a criminal in the first place. His approach is virtually designed to overstate the effects of incarceration.

    ReplyDelete
    Replies
    1. So the Tobit regression that they mention in the abstract should take care of the zero earners (although I'm not sure how that behaves at small tails... sometimes these models can be less reliable, which is why I raised the point that there are very few incarcerated youth in the NLSY).

      Your second paragraph was my big concern. I see no evidence that they're grappling with the really serious problems of endogeneity here at all. Kids who get incarcerated are different from kids that don't. Kids that get incarcerated for long periods are different from kids who get out early. Comparisons with the same background is a good start but there's likely a lot that's unobserved.

      I would have preferred some sort of quasi-experimental set-up to pull these issues apart - perhaps using different treatment of youth in the criminal justice system across different states (although that's problematic too because youth engaging in crime will know that sort of thing and adjust their behavior accordingly). That would be the center of an economics paper on this question, but I don't see any evidence that they've considered it here (I am, of course, going off an NPR piece and an abstract... but an economist would have put their identification strategy in the abstract!).

      Delete
    2. Also, the $11,000 quote is from the coefficient on current incarceration. How does he expect prisoners to be making legal earnings? It's a weird population to include in your regressions.

      Delete
    3. I would have at least preferred them to run a model with the dependent variable being changes in illegal income rather than actual absolute illegal income. The interesting thing here is that the coefficients on previous illegal income are so low. They're around 0.01--is previous illegal income really that weak a predictor of future illegal income? It's something like that which makes me disbelieve the results.

      Delete
  2. Is prison a school for crime?

    Wow, talk about a couple of hundred year old question.

    My take on this has been for some time that (a) prisons aren't a tool for the reduction of criminal behavior in society for a bunch of reasons; (b) they come with lots of unappreciated or underappreciated costs; and (c) that it is hard to learn to be free while you are in the unfreedom of a prison setting.

    ReplyDelete
  3. "Spending time in prison leads to increased criminal earnings," Hutcherson says. "On average, a person can make roughly $11,000 more [illegally] from spending time in prison versus a person who does not spend time in prison."

    This sounds like he is comparing convicts vs. non-convicts. He should at least compare criminals who go to prison with criminals who do not go to prison. Sure, in prison you may learn about committing crime and make connections. (Not unlike going to Harvard or Yale. ;) ) But if you spent the same amount of time out of prison but committing crimes and making connections, your criminal income would probably increase, as well. And maybe more than by $11,000 (per year, I suppose). ;)

    ReplyDelete
  4. Also, in prison you almost have to join a gang. That's what game theory tells us. That is one reason for the spread of gangs across the country. California had a good idea, to segregate prison populations so that you did not have racially based gangs competing within prisons. OC, that did not mean that rival factions would not form, but that the factions would not be fueled by racism, and that, outside prison, gang divisions would exist within racial groups.

    Unfortunately, racially segregated prisons were deemed unconstitutional.

    ReplyDelete

All anonymous comments will be deleted. Consistent pseudonyms are fine.