tag:blogger.com,1999:blog-1740670447258719504.post5725269246526856081..comments2024-03-27T03:00:27.024-04:00Comments on Facts & other stubborn things: Thinking about the specifications of the Dube, Lester, and Reich minimum wage study: Part 2 - controlling for county level trendsEvanhttp://www.blogger.com/profile/12259004160963531720noreply@blogger.comBlogger14125tag:blogger.com,1999:blog-1740670447258719504.post-90969248231759530812014-01-20T10:52:13.902-05:002014-01-20T10:52:13.902-05:00This is GREAT stuff Daniel, thanks. If you have ti...This is GREAT stuff Daniel, thanks. If you have time, I'd love to hear you weigh in on the Ozimak article that "Lord" posted. He is claiming that if min. wage led to a permanently lower rate of growth, then the corrections introduced by Dube would mask it. If you could spell that out, it would be sweet.Bob Murphyhttps://www.blogger.com/profile/04001108408649311528noreply@blogger.comtag:blogger.com,1999:blog-1740670447258719504.post-10541686880320915632014-01-19T14:32:16.201-05:002014-01-19T14:32:16.201-05:00Thanks.
This is somewhat off topic but do you kn...Thanks. <br /><br />This is somewhat off topic but do you know of any studies that do a good job of controlling for non-wage benefits? As a former restaurant employee, one of the biggest perks was 50% off meals. At one restaurant, I got free meals! Anyways, the point is that this type of benefit pervades restaurants (and many low wage businesses) so I'd be curious to know whether those benefits - or those kinds of benefits in general - take a hit with higher minimum wages. <br /><br />From an identification standpoint it's often hard to measure non-wage benefits. And it appears to me that not controlling for these benefits would bias the results as the lower benefits would be a unit specific simultaneous treatment effect. <br />Ednoreply@blogger.comtag:blogger.com,1999:blog-1740670447258719504.post-82740613234343707652014-01-19T06:53:38.197-05:002014-01-19T06:53:38.197-05:00The way I think about it at least is that all DIDs...The way I think about it at least is that all DIDs are FEs but not all FEs are DIDs, insofar as in a general FE without something like DLR's contiguous county pairs you are comparing a treatment case to all counties that didn't get treatment. The mechanics I suppose are still quite similar, but the whole identifiication strategy behind the DID is that you have a matched comparison group. The geographic fixed effects they added showed that that difference was a big one.<br /><br />A lot of the concerns carry over. I have an article out in JEBO now that relies on more naive variation in policies across states and we don't rely on this matching - but we still had to make corrections for things like serial correlation that come up in the panel DID models (pointed out by Bertrand, Duflo, and Mullainathan). So a lot is indeed the same, but you can see how the major factor identifying a DID model is missing in the FE model. Daniel Kuehnhttp://www.factsandotherstubbornthings.blogspot.comnoreply@blogger.comtag:blogger.com,1999:blog-1740670447258719504.post-34748029083546139612014-01-19T03:38:37.318-05:002014-01-19T03:38:37.318-05:00I'm a bit new to the experimental econometrics...I'm a bit new to the experimental econometrics literature so this is just a question for clarification. I'm confused by your wording in the beginning where you seem to say that a FE model is different from a DID model. My understanding is that a FE model with two periods is the same as the corresponding DID model. And that in general a FE model is like a generalization of DIDs. Is this wrong? Or am I just confusing words?<br /><br /><br />Ednoreply@blogger.comtag:blogger.com,1999:blog-1740670447258719504.post-71837885658781961622014-01-18T21:01:40.589-05:002014-01-18T21:01:40.589-05:00Nobody said just adding variables reduces bias.
I...Nobody said just adding variables reduces bias.<br /><br />It matters what is and isn't controlled for depending on the identification strategy of the model. Adding a county-pair dummy does a lot to remove the bias. Adding some random thing might not (although it's difficult to understand what would INCREASE the bias, but you could imagine adding variables that significantly change the interpretation such that it's not something you care about).<br /><br />I'm not sure you understand what is going on in the models - I'm happy to explain more if you want.Daniel Kuehnhttp://www.factsandotherstubbornthings.blogspot.comnoreply@blogger.comtag:blogger.com,1999:blog-1740670447258719504.post-23008516844851237402014-01-18T18:23:21.691-05:002014-01-18T18:23:21.691-05:00ehhh... not exactly, but I don't want to expla...ehhh... not exactly, but I don't want to explain it anymore. You can't just add variables and claim a reduction in bias. That's not statistically valid. And yes, models that depart from expected theory should not inspire confidence. They should be checked and checked and checked again, over and over, until there is sufficient evidence to overrule theory.<br /><br />But anyway.RP Longhttps://www.blogger.com/profile/15028013805248797978noreply@blogger.comtag:blogger.com,1999:blog-1740670447258719504.post-14085685781378735272014-01-18T15:47:58.988-05:002014-01-18T15:47:58.988-05:00Hadn't seen this - I'll take a look at it ...Hadn't seen this - I'll take a look at it and try to get to it in the next couple days.<br /><br />I have not looked at the Meer and West paper yet, although I've come across it in reading up on DLR. So this actually may take more of an investment before I can respond.<br /><br />The Wolfers paper Ozimek cites is very good. He is responding to Liz Peters, who I know from the Urban Institute. She became center director of my center maybe six months before I left. So I didn't know her well, but a little. Both (Wolfers and Peters) are smart cookies.Daniel Kuehnhttp://www.factsandotherstubbornthings.blogspot.comnoreply@blogger.comtag:blogger.com,1999:blog-1740670447258719504.post-43523378969476411852014-01-18T15:40:33.291-05:002014-01-18T15:40:33.291-05:00re: "To that model, we add variables that con...re: "To that model, we add variables that control for variation by county. Without knowing anything else about anything, I can already predict with certainty that the significance of the minimum wage variable will decrease and the value of its parameter will change. "<br /><br />Well I don't think you know the first one at all, but the latter seems pretty likely.<br /><br />re: "Without knowing anything else about anything, I can already predict with certainty that the more counties you add to your model, the more pronounced this effect will be."<br /><br />What do you mean by "pronounced"? Are we talking about standard errors or changes of the point estimates? The standard errors should get smaller (i.e. - more significant) and there's no reason at all the point estimates themselves should change (they will, just as a consequence of an alternative sample, but the expected value of the effect of increasing the sample size should be zero).<br /><br />You should be more comfortable with DLR's results, not less. You seem less comfortable and if these are the reasons I don't think it's justified. Precision should improve, not be reduced. The added variables and controls should reduce bias, not increase.<br /><br />You seem to want to say that because they depart from the result you were comfortable with from theory before the departure is something that needs to be justified. I strongly disagree. That's why we do empirical testing - to confirm theory. So what I've done in these posts is walk through all the controls and changes: contiguous county sample, geographic fixed effects, time fixed effects, and time trends - and shown how each of them reduces bias in the estimate. Do you disagree with one of these arguments? Because if none of these arguments sound wrong to you you should feel more comfortable with DLR's results than the naive regression. If one of the arguments seems wrong to you then that's something to talk about.<br />Daniel Kuehnhttp://www.factsandotherstubbornthings.blogspot.comnoreply@blogger.comtag:blogger.com,1999:blog-1740670447258719504.post-71936438196764823612014-01-18T14:47:59.476-05:002014-01-18T14:47:59.476-05:00Any comment on Adam Ozimek ?Any comment on <a href="http://www.forbes.com/sites/modeledbehavior/2014/01/13/why-do-economists-disagree-so-much-about-the-minimum-wage/" rel="nofollow">Adam Ozimek</a> ?Lordnoreply@blogger.comtag:blogger.com,1999:blog-1740670447258719504.post-30859663501353157902014-01-18T14:34:56.087-05:002014-01-18T14:34:56.087-05:00Correction, that last comment of mine should begin...Correction, that last comment of mine should begin with "I'm **NOT ** arguing about what you personally..." My apologies.RP Longhttps://www.blogger.com/profile/15028013805248797978noreply@blogger.comtag:blogger.com,1999:blog-1740670447258719504.post-51651025695586202812014-01-18T14:34:06.190-05:002014-01-18T14:34:06.190-05:00I'm arguing about what you, personally, "...I'm arguing about what you, personally, "should" do. My point is a relatively simple one. <br /><br />The simpler model - the one that finds that increases to the minimum wage increase unemployment - is consistent with a simple, intuitive, and long-standing theory.<br /><br />To that model, we add variables that control for variation by county. Without knowing anything else about anything, I can already predict with certainty that the significance of the minimum wage variable will decrease and the value of its parameter will change. Without knowing anything else about anything, I can already predict with certainty that the more counties you add to your model, the more pronounced this effect will be.<br /><br />Considering that this is little more than a mathematical fact about statistical models, we should all be very comfortable with the idea that the standard of proof for this new model - the one that includes counties - should be much more stringent. We should anticipate the effects I've just described, and run through a full range of tests to isolate them and minimize them. <br /><br />There are many such statistical tests and variable inclusion algorithms and criteria for doing so. Now, I'm just "some dude." I don't do this for a living, and I don't read a lot of economic literature. But I do know a thing or two about statistics and a teeny, tiny bit of economic theory. When I see a model that presents statistical results I fully expect to observe based purely on the design of the model - and when that model is subsequently used as evidence against a well-established, long-standing, fully intuitive theory, the homunculus in my head starts asking me to rule out all mathematical problems prior to considering the model's conclusions.<br /><br />This is not a demand on you, personally. It is not a description of something I want you do to, for yourself or for my benefit. This is a comment I am making on your blog post. It may not be the comment you expected. It may not be a comment you know how to respond to, or whether to respond to it at all. It may not be the kind of comment you enjoy, and you may or may not have anything to say by way of reply. It could be that it is just what entered into my head upon reading your blog post. That is what most of my comments are, anyway.RP Longhttps://www.blogger.com/profile/15028013805248797978noreply@blogger.comtag:blogger.com,1999:blog-1740670447258719504.post-510659994701763272014-01-18T12:49:08.528-05:002014-01-18T12:49:08.528-05:00Bob has an actual issue with the specification and...Bob has an actual issue with the specification and I respond with a discussion of the appropriateness of the specification. I don't know what your concerns are except that you don't think it fits the theory you think it should.Daniel Kuehnhttp://www.factsandotherstubbornthings.blogspot.comnoreply@blogger.comtag:blogger.com,1999:blog-1740670447258719504.post-69347617248885920282014-01-18T12:46:32.922-05:002014-01-18T12:46:32.922-05:00Nothing would change my mind on county time trends...Nothing would change my mind on county time trends. Trends that occur both pre and post cannot be attributable to the minimum wage as far as I know and they HAVE to be eliminated for an unbiased estimator. So I guess another way of saying that is that to convince me you'd have to show that something very funny is going on with causality here or that I don't understand the properties of the DID estimator. I don't think you'll do the latter and anything you could come up with on the former won't make anywhere near as much sense as the spatial heterogeneity in time trends.<br /><br />What model am I supposed to be questioning? The DLR empirical model? I've commented at length now on why I think their approach is right and I'm probably not done yet. Give me a convincing reason to question it. Every query I make about their modeling choices comes out with them making the right modeling choices so far. I've asked several times "why would you do this?" and "how what is the correct way to deal with this in the data or that in the data" and every single time the DLR way seems the best from what I know about this estimator. So I think you're simply wrong when you say I don't question the model.<br /><br />On the second half of that paragraph I've talked at length about what I think the empirics say for theory too. Search "minimum wage" in my search function if you're curious.<br /><br />I don't understand the last paragraph at all. It has to be explained? What do you think we spend so much time thinking about identifying the model for?<br /><br />If you want me to throw anything out you have to give me a good reason to throw it out. I've described in great detail here why I think they are right.Daniel Kuehnhttp://www.factsandotherstubbornthings.blogspot.comnoreply@blogger.comtag:blogger.com,1999:blog-1740670447258719504.post-85944474131352525502014-01-18T12:23:39.290-05:002014-01-18T12:23:39.290-05:00"So Bob's issue is bigger than the easily..."So Bob's issue is bigger than the easily dispatched with concern that we matched on counties and then decided that wasn't good enough (I didn't quote that part). The concern is that somehow we are absorbing the effect of the minimum wage."<br /><br />This is my concern, reflected in my comment in your previous post. I don't think you've managed to convince me here. The problem I have with your above explanation is that it reflects over-confidence in the model. You write, "But I don't see any clear evidence that controlling for county time trends doesn't improve the estimate." <br /><br />What would change your mind? Again, you write: "So, in order for Bob's fear to be a problem you can't just have county or state specific differences in trend between treatment and control cases. You'd need to have trends that change at the implementation of the minimum wage, and in a way that biases (rather than just adds noise) to the estimate."<br /><br />At no point do you question the theoretical validity of the model. A model that reaches a conclusion that stands in stark contrast to expectations driven by economic theory is a model that provides at least *SOME* "evidence that controlling for county time trends doesn't improve the estimate." That's because data is not supposed to diverge from economic "law" without good reason.<br /><br />That "good reason" might be a revised theory, or an exogenous shock, or a data anomaly, or something else. It has to be explained. That's the whole rub. It has to be explained. And, moreover, the explanation has to be better, clearer, and more conclusive than the theory that served us so well for so long.<br />RP Longhttps://www.blogger.com/profile/15028013805248797978noreply@blogger.com