Tuesday, January 21, 2014

Fool me once...

So I got fooled once by David Henderson's post where I thought he was going to say that everybody agrees that the question is about the role of labor supply incentives in the ultimate impact of UI on unemployment.

Shame on him, so the saying goes... but I'm not sure that's entirely fair - so let's say maybe shame on me for expecting that.

Now I see a post from Russ Roberts titled simply "Paul Krugman is not a hypocrite".

FANTASTIC!, I honestly thought.

We are making progress!

We are not making outrageous personal attacks and we're going to talk about economics, even if we disagree very strongly about the scientific reasonableness of a claim!

Well this is fool me twice, so it's definitely "shame on me" at this point.

If we are going to make personal commentaries rather than talk about economics in the economic blogosphere, you should tell me how ridiculously cute my almost-four-month-old daughter is.




Krugman on UI... and another great big sigh of disappointment for the economics blogosphere

Krugman derangement system can be pretty impressive sometimes, but the recent spat has to win some sort of prize. I've commented in a few places on it, but I thought this blog might need a little break from the more serious minimum wage wage posting I've been doing.

Russ Roberts recently accused Krugman of lacking intellectual credibility because he mentions a micro labor supply effect of unemployment insurance in his textbook but in a recent blog post he criticizes Robert Barro for inferring that because of these micro incentives, the idea that UI can reduce unemployment is (to quote Krugman's paraphrase) "self-evidently absurd".

Of course it's not "self-evidently absurd" at all. You can have negative incentive effects and a reduction in unemployment quite plausibly if the unemployment rate is high because of demand problems. There is no contradiction whatsoever. And Russ Roberts, with a PhD from the University of Chicago, should be able to understand this point.

Then David Henderson jumped in and I thought things would get a little more sane when I read: "The issue--and everyone on both sides agrees that this is the issue--"... and I was SURE the next line would say "is whether or not you can acknowledge negative incentive effects and still argue that UI reduces unemployment".

Because THAT is what the issue is, and David is usually mild-mannered and to the point and I honestly expected that's what I would read next. But no, it continues: "is whether Krugman is being hypocritical in his discussion of unemployment insurance."

A good alternative to this is Chris Dillow's post. He starts by referencing Bob and Russ, but he doesn't say anything like "The issue--and everyone on both sides agrees that this is the issue--is whether Bob and Russ are being jackasses to Krugman"

Because there's a point where the Krugman derangement syndrome gets old and we need to focus on the economics if we really want to be an economics blogosphere.

For what it's worth Barro clearly understands there's nothing even passingly hypocritical in Krugman's case. Barro writes:
"Yet Keynesian economics argues that incentives and other forces in regular economics are overwhelmed, at least in recessions, by effects involving "aggregate demand." Recipients of food stamps use their transfers to consume more. Compared to this urge, the negative effects on consumption and investment by taxpayers are viewed as weaker in magnitude, particularly when the transfers are deficit-financed. Thus, the aggregate demand for goods rises, and businesses respond by selling more goods and then by raising production and employment. The additional wage and profit income leads to further expansions of demand and, hence, to more production and employment. As per Mr. Vilsack, the administration believes that the cumulative effect is a multiplier around two."
Of course, he goes on to dispute the argument, and that's fine. As I said above - THAT should be the real question here: who is right about the effect of UI. But despite disagreeing, Barro knows full well that nobody says that the incentive effects aren't there, they say that they are overwhelmed by other effects when the economy is demand-constrained.

Krugman is obviously not a hypocrite. That is NOT the question at hand. Krugman does not lack intellectual credibility. Krugman is not the embodiment of Orwellianism (a comment on Russ's blog). Let's get back to economics, people.

Monday, January 20, 2014

Brief, and entirely unsatisfying post on Ozimek and the minimum wage

Commenters "Lord" and Bob Murphy both suggest I look at Ozimek's post on DLR here. It's very good, although I don't know how much it resolves. It goes over a lot of the time-trend issues we've been over here. Wolfer's identification of bias, for example, is similar to what identified as the case that Bob was talking about, which could be a possibility (although one I doubt).

The flows may help to solve these sorts of questions - there are papers on that from Dube and from Meer and West. I'd have to read them. But I can't see how that definitively resolves anything. There can be no rate-of-change change without a change in the flows so the same ambiguities in the changes or lack of changes in the levels is going to be there in the flows too. The difficult task isn't identifying a trend change (which requires a flow change and results in a level change), the difficulty is identifying the right counterfactual trend.

It ultimately boils down to whether we think the time-trends are appropriate or not and if there's an obvious econometric test for it I'm not sure what it is. Time-tends may be wrong, but Occam's razor seems to suggest we should include them (as in DLR). Spatially heterogeneous time trends seem more reasonable than just the right circumstances that would actually introduce bias by including time trends.

Neumark and Wascher suggest we might want non-linear time trends instead of linear ones. One reasonable way to test this is to do an out of sample specification test using the comparison cases. So use a couple specifications of the time trend for periods -12, -11, -10,..., -2, -1, 0, and then figure out which specification best predicts the trend in 1, 2, 3, 4,...,10, 11, 12. Since these cases don't have any dynamic effects of the minimum wage, it should give you a better sense of the non-linearity of time trends. Now, you have to argue that that specification of the time-trend (linear, non-linear, etc.) is also true in the treatment case. But since we're not using the same slopes or parameters itself that seems defensible.

Really I'd need to read Meer and West and the responses but I feel like many of the same points are made here that I made the other day, namely: (1.) time trends should help to reduce bias in most cases, but (2.) you can imagine specific scenarios where the opposite would be the case.

Entirely unsatisfying, eh?

I still think DLR offers the most sensible default - just at first appearances. That doesn't mean there isn't something else going on, but I think it needs to be demonstrated.

Saturday, January 18, 2014

Ryan Long on the econometrics of the minimum wage: a big picture explanation

I've been a little concerned that my last couple posts have been confusing for some people based on some comments from Ryan Long about all the variables DLR are putting in, so I want to zoom out to the big picture a little. First Ryan expressed concern that we're adding too many variables in a fixed effects model and that that is losing us significance*. Recently he expressed concern that we were just adding variables to reduce bias on the idea that adding more variables reduces bias.

This one concerns me a lot more and now I'm worried more people have missed the whole point of these posts. We are not just chucking things in the model and waiting to lose the significance. We have a treatment effect we're trying to estimate but we have non-experimental data so we need to figure out a way to mimic an experiment and get at least a good sense of what the treatment effect is. DLR have chosen to do that with what is at its core a DID set-up.

But once you do that, the comparison group you have can still run into certain problems that can bias the result. We've worked a lot with these models, though, so we know ways around those problems and usually that involves adding other variables. We're not just adding them for the hell of it - we're adding them because when you add a variable it changes which bit of variance in the data you are using to estimate the effect.

That bolded sentence is the key here.

And that's been the point of my last several posts. Bob Murphy raised concerns (not Ryan's concerns - I think Bob understands the big picture about non-experimental estimation I'm laying out here) about certain variables that were added. My view is that all of these were essential to get unbiased results and represent an improvement on earlier estimates.

So that has been the point. I've been trying to explain why changing the model in X way gets you a better estimate than refraining from changing it in X way. It's not just a matter of adding any ol' variable.

* Don't worry - it's not the case - there's a tremendous amount of degrees of freedom so there is no concern about that. In fact DLR's models should have (I'd have to double-check) many orders of magnitude more degrees of freedom than Neumark and Wascher's, which was a state study. Moreover, only the significance of the minimum wage variable in the employment model dropped, not in the others. If it were a df problem they'd all be mush - there'd be no reason for one model to be unaffected and one to lose significance if that were the problem. Finally, Ryan can easily look at the standard errors - they haven't exploded or anything like that. It's just a regular old insignificant effect - no funny business. That would not have gotten past the editors and the referees of RESTAT.

More thoughtful than you might think at first on the minimum wage...

I saw this on facebook the other day.

It sounds like your usual Bill Maher complaint that some of you may be quick to dismiss, and ultimately I don't know if I entirely agree with it either. But there's more to it than first meets the eye, and it hinges on questions of wage bargaining, monopsony, etc. in a lot of ways.

So what is the economic science behind intuition like this? What's your take on it?

Photo: Bill Maher, nailed it.

Thinking about the specifications of the Dube, Lester, and Reich minimum wage study: Part 2 - controlling for county level trends

Q: When is a fixed effects model not a fixed effects model?
A: When it's a difference-in-differences model.

The most important question in any impact analysis is "how do they identify their model"? Sometimes its buried in the math, but there are a few canonical forms of how to identify a model (often very closely related) that in my opinion at least help to think about model specification and exactly what kind of assumptions and variation the authors are relying on.

I'm sure a lot of you know that a fixed effects model is just a model you run on panel data with dummy variables for each cross-sectional unit to soak up all the time-invariant non-observed characteristics, and dummy variables for each time period to soak up all the common time trends. The big thing you don't get automatically in a fixed effects model is control of time-variant variables that are cross-sectional unit specific.

Turning a fixed effects model into what is essentially a difference-in-differences (DID) model is pretty straightforward. In fact we discussed it in the last post on Dube, Lester, and Reich (DLR): you just include county-pair fixed effects in their case. These fixed effects capture any variation across pairs, so the only variation left to estimate the minimum wage coefficient on is variation within a pair, between the counties in that pair over time. DLR have intermediate versions of this, restricting the estimation of the effect to pairs within regions, states, and metropolitan areas. But what they're narrowing in on is essentially the DID. The logic of the DID is straightforward and I want to walk through it before getting to more of Bob Murphy's thoughts.

You have panel data so you've got data before and after a treatment. You also have two cases: a treatment case (on the left, below), and the comparison case (on the right). The treatment case may be changing over time anyway without the treatment, so to isolate the treatment effect any changes in your comparison case (the paired county in DLR), is subtracted out of the treatment effect. Why? The changes in the comparison cannot (or should not... there's a different literature on that issue) be affected by the treatment because it didn't get the treatment. So that small gap on the right is the counterfactual of what would have changed in the absence of treatment, and therefore cannot be attributed to the treatment effect on the left. Notice also that it doesn't matter if the comparison case is a little different from the treatment case (see how I've drawn it a little lower?). What matters is the differential response to treatment, because the DID estimator is:

[(Post-Treatment) - (Pre-Treatment)] - [(Post-Comparison) - (Pre-Comparison)]

[UPDATE: I had the terms switched above before - this version is correct. You take the raw change in the treatment case, but then you want to subtract out the change in the comparison case from that]

So if there's something about the comparison group that's time-invariant that makes it a little different from the treatment group, that's OK. That's why we have county dummy variables. What's more problematic is differences in the counties over time (which I'll discuss below).


The profile over time in the diagram above is flat, but we could easily imagine a common time trend (imagine the slopes in the figure below are the same!). This doesn't matter for the simple DID case at all for two reasons. First, if the portion of these time trends common to all counties is already absorbed by the time period dummy variables I mentioned above. Any other time trend that is common between the paired counties will be subtracted out of the treatment effect by the exact same logic of the case without the time trend: we are removing the change in the comparison group from the


As I alluded to above, the big trouble comes in when you have time trends in the treatment and comparison group that are different. That would look something like this:


If you implement the DID estimator here it will make the treatment effect a lot smaller because there was a big change in the comparison group over time relative to the treatment group (in other words, the match might have been good, but it wasn't perfect). Looking at what's actually going on, though, you can tell that the true treatment effect should be exactly the same - we're just conflating the rate of change that has nothing to do with the treatment effect with the treatment effect itself.

What you want to do in this case is control for the trend rate of change by county so that any increase in the comparison group in the post period that follows that rate of change is not used to penalize the treatment effect. You could just as easily imagine a scenario where you'd want to do this because it would over-estimate the treatment effect. I draw it this way because this is what DLR came across. Once you control for that time trend, you're back to the situation of the first picture (common time trends will be swept up in the county-specific time trends, which is just fine - we don't care about common trends), and you've got an unbiased DID again.

So as best as I can tell, Bob Murphy has two related concerns. First, he's concerned that we're including other controls when we were supposed to be dealing with all that by matching counties. That, I hope, is clear from both this post and the last post: even good comparison groups can be improved upon. You never have a perfect comparison group until you have random assignment.

But there's another issue he has with this. About a year ago, Bob wrote:
"What Dube, Lester, and Reich are really saying here, is that maybe for some reason minimum wage hikes happen to be concentrated in regions that have lower than average employment growth. Hence, just because we find that teenage employment grows more slowly in regions with higher minimum wages, doesn’t mean we can blame it on the relatively higher minimum wage. But hang on a second. Minimum wage hikes aren’t randomly distributed around the country, such that we might happen to get an outcome where they tend to be concentrated in slow-growth regions. On the contrary, minimum wage hikes are implemented by “progressive” legislatures, who also (given my economic worldview) implement other laws that retard adult employment growth.

For example, suppose that if a state legislature jacks up the minimum wage, then it is also likely to pass “pro-labor” stuff like laws giving unions more organizing power, laws allowing unfairly terminated employees to receive years of back pay, and laws granting extra perks for maternity leave. Now, these last three items I listed: Would they reduce the employers’ incentives to hire teenagers or adults, more? On the margin, they would make it costlier to hire adults, because if penalties are expressed in years of back pay, or have to do with paid leave, or strengthen unions who traditionally are going to organize adults…You get the picture. Adults make more than teenagers, and so these rules will penalize adult employment more than teenage employment.

Thus, if my model here is correct, it would produce the pattern we actually see: Looking narrowly at minimum wage laws, they seem to retard teenage employment. But then when you ask if states with high minimum wage laws have a bigger slowdown in teen employment versus adult employment, the signal becomes much weaker. It looks like, by dumb luck, for some reason all the minimum wage hikes happen in states that also have slower-than-average employment growth among adults."
So Bob's issue is bigger than the easily dispatched with concern that we matched on counties and then decided that wasn't good enough (I didn't quote that part). The concern is that somehow we are absorbing the effect of the minimum wage.

This may happen under very special circumstances, but generally it's not a problem. Bob is - I think - forgetting the panel element to the data. We are subtracting out the pre-period from the post-period for both the treatment and the comparison, and then comparing those two differences. We know the post period minus the pre period for the comparison group should have no effect at all of the minimum wage so that is the appropriate counterfactual. When we are controlling for a county specific trend we are saying "those secular trends that were going on before anyone adopted a minimum wage would have gone on if the minimum wage hadn't been adopted, so we want to clean that out of the treatment effect". If they are common between counties, my diagram two shows why that's not a concern. If they're different between counties (maybe because one has a progressive legislature), it needs to be accounted for. You are not weakening the signal you are making the signal more accurate because the only impact attributable to the minimum wage is what changes after its implementation. A time trend that continues on the same after as it did before does not change after the implementation of the minimum wage.

What special circumstances might justify Bob's fear? Time trends that are not the same before and after the minimum wage and that are not related to the minimum wage. That might look something like the following:


Let's say the true impact of the minimum wage does not increase the rate of growth of Y in the post period. In other words, let's say the slopes in both these cases would have happened without the minimum wage. If we control for county time trends using pre-period data in this case, we would find that the minimum wage had the effect of:

#1. A one time, persistent, positive shock to Y, and
#2. An increase in the rate of growth of Y

Why? Because we're differencing out the county trend in the comparison case, but we're only differencing out part of the county trend in the treatment case. The rest of the county trend in the treatment case is going to be attributed to the treatment. The #2 effect is false and introduces bias into the estimate of the treatment.

So, in order for Bob's fear to be a problem you can't just have county or state specific differences in trend between treatment and control cases. You'd need to have trends that change at the implementation of the minimum wage, and in a way that biases (rather than just adds noise) to the estimate.

It's not a completely crazy fear. You could have, for instance, a very liberal state legislature that implements a bunch of stuff at once, including a minimum wage law. That's possible, but the effect isn't clear to me. If Bob thinks the liberal legislature will tend to make employment for teens worse, and all these reforms were clustered, that would understate the effect of the minimum wage. Of course the opposite could also be true if a bevy of liberal reforms were to help. If it hurt adults more than it hurt youth that only seems like it would impact the minimum wage estimate if the effect of the minimum wage were estimated relative to the impact on all adults, and I don't think it is. I'm not really sure where those concerns about adult employment relative to youth employment come from.

So I will concede that because lots of different changes may happen together in a state legislature, it would be nice to account for that (of course if the increase is coming from a federal increase, whatever is going on in the state legislature shouldn't matter). These could certainly improve the estimate further. But I don't see any clear evidence that controlling for county time trends doesn't improve the estimate. Remember, the trends as calculated vary across counties, not the difference between the pre- and post- trends. And as my last figure illustrates you'd need the difference between pre- and post- trends for this to mess up the DID estimator.

There may be a Part 3 to this series. DLR also include placebo effects as a sensitivity analysis on these time trends. But I don't have a good sense yet of how all that works right now. If I have time to figure that out and put that together I will.

Friday, January 17, 2014

The minimum wage and turnover

I've been spending a little time poking through Arin Dube's publications and working papers today, and one interesting working paper I found was some research on the impact of the minimum wage on employment flows (also with Lester and Reich). Aside from being interesting in its own right, I'm sharing it here because it speaks to another suggestion that Bob Murphy had in his recent post. Bob writes:
"Again, even taking the new generation of studies at face value, they overlook a major drawback to the progressive goal: The studies look at the absolute growth in employment, rather than the unemployment rate, among low-skill workers. So even if it’s true that, say, a Burger King franchise will hire roughly the same number of teenagers between now and 2020 as it otherwise would have, it might not be the same group of teenagers getting jobs. Rather, at the $7.25 level there will be lower-skilled applicants cycling through, with a high turnover rate as the store manager tries to find the few decent workers in the bunch. At the higher rate of $10.10 per hour, higher-skilled kids (perhaps those from affluent families who are home from college) will enter the mix in greater numbers. The manager will be pickier on the front end in giving somebody a bite at the apple, and there will be less turnover. (Note that this isn’t merely hypothetical; the studies finding “no effect” often cite “lower job turnover” as an explanation for how the firm responds.) Thus, even taking the studies at face value, it is entirely possible that there are a bunch of people with low skills who now can’t get a job, who otherwise would have been able to. They are merely being displaced by higher skilled workers who otherwise would not have been interested in a position paying so little."
Dube's work suggests that at least part of this story is right - the lower turnover part. Their work doesn't seem to speak to the second half of Bob's point about low skill workers.

[I had initially misunderstood this point from Bob - for some reason I read "at the $7.25 level there will be lower-skilled applicants cycling through, with a high turnover rate", and I thought he was referring to the last increase. My comment on this point was therefore a little confused, but I share the skepticism about increased turnover - we apparently agree on more than I thought! You could imagine arguments either way -I don't think it makes sense to increase turnover, but you could imagine ex post learning about productivity at least counter-acting any decline in turnover from more careful screening. However, Dube's research seems to suggest that the forces that act to reduce turnover are stronger.]

Minimum wage theory: a commenter's question

I am not a very good theorist. To some extent I think I probably grasp macro theory better than micro theory, but the point is I'm not a theorist in any case. Day in and day out I am an empirical economist with varying degrees of sophistication depending on the project I'm working on. Out of personal interest I sometimes moonlight as a third-rate dealer in second-hand theories of first-order importance (in other words, I like to do a little history of economic thought).

I don't pretend to be a theorist, though, and in the interest of not pretending to be one I wanted to just share commenter YouNotSneaky!'s questions and thoughts about the theory behind the nil effect of the minimum wage in the data:
"Can you give a link to a paper that makes the monopsony argument? It's been awhile since I looked at it. I did look up the Burdette & Mortensen matching model recently which I gather is what this argument is based on. However, in that model (or ye basic search model, like say in Romer's Adv Macro) minimum wage still decreases employment. In B&M a minimum wage *can* increase social welfare but that's different (the increase in utility of those who retain higher paying jobs is greater than those who experience longer unemployment spells). You can use the B&M monopsony model to argue for minimum wage, but you can't use it to explain why the empirical work does not detect employment effects.

Personally I'm pretty sure something else is going on (probably the data just isn't good enough, not enough variation, close to equilibrium min wages, adjustments in hours rather than persons etc)

This is one version of the B&M http://econ.tau.ac.il/papers/macro/postmatch.pdf"
I don't know Burdette and Mortensen specifically although it is this sort of model that Manning uses and refers to when he talks about monopsony and the minimum wage. I've read a little of him, and I've read some of Pissarides on equilibrium unemployment. I rely on much simpler expositions of fixed costs and turnover to motivate my understanding of the connection to the minimum wage - namely, the Oi paper on quasi-fixed factors. And I agree (and have stated here) that this flavor of models is not entirely reassuring on the employment effects, however that should be more apparent in the long run than in the short run I think. I don't know - let me know what you think about that argument.

I think the data is getting better and the identification strategies are getting better and the result is not going away, so I would not blame the data.

I do think the other two issues raised are relevant: that these are often "modest" increases, not departing far from the equilibrium wage, and that there are adjustments on other margins potentially. Dube, Lester, and Reich do provide an upper bound on the hours adjustment. It's not immediately obvious to me why it would make sense to make your adjustment on hours rather than employment (in the pre-Obamacare era at least!). The only reason to do that, it seems to me, is to deliberately fool economists.

Thinking about the specifications of the Dube, Lester, and Reich minimum wage study: Part 1

Bob Murphy recently probed some important questions about the Dube, Lester, and Reich paper on the minimum wage (hereafter DLR). Ultimately I think Bob's concerns are misplaced, but these are exactly the issues we should be thinking about and arguing about with the minimum wage literature - not petty assertions that the other side doesn't understand the law of demand. So let's jump right in - Bob writes:
"I admit that I have not studied them in depth, but if you look at the discussion in the survey article I mentioned above, here’s what it sounds like: You can take all of the adjacent counties in the country for which you have continuous data over a long period (such as 16 years), where the applicable minimum wage is occasionally different in each county (because they fall in different states). If you “naively” run a regression on this dataset, then the classical consensus emerges: It does indeed seem that a higher minimum wage is associated with a slowdown in the growth of employment."
I think this is an entirely fair read of DLR. Their paper has two samples - a full county sample and a contiguous county sample that is a collection of county-pairs (with some counties repeated if they are paired with multiple border counties). They find this classical result in both samples if they just run a regression akin to Neumark and Wascher. This is a good thing - we want confirmation of results across datasets (Neumark and Wascher use states). Bob continues:
"However, this could be a spurious result, because states with high population growth might just so happen to also match the federal minimum wage, rather than setting a higher state level. To correct for this, the newer studies introduced a regional dummy variable into the regression analysis, at which point the negative effect of the minimum wage almost disappears.

If indeed what I just described is what’s going on, then that seems ludicrous. The point of matching contiguous counties is to isolate all other relevant variables, except for the applicable minimum wage. You can’t use the weather (one of the explanations given in the survey article to explain the flaw in the original studies, which did not correct for geography) to explain why people would flock to one county versus the adjacent one
."
This is where I think Bob starts to get things wrong. In the first place I don't think he's understanding what DLR are doing. The regional dummies and the matching are not being done together. See DLR's description here (specification 1 is the naïve regression described above):


Remember that even when you use the second sample of contiguous counties they aren't automatically matched yet. It's just a whole bunch of counties thrown in together that happened to be on state borders. When you start introducing geographic controls into the fully county sample (specifications 1 through 4), you very quickly lose the negative correlation. In other words, spatial heterogeneity matters crucially for these results. The finding that even regional time specific dummies in the full sample will lose you the negative effect is the whole reason why they are justified in going further and "matching" (as Bob phrases it) the counties. But there is no matching until specification 6. Specification 5 runs the naïve regression on the raw contiguous counties file, and specification 6 finally introduces county-pair dummies that "match" the counties (it would be more accurate to say that they eliminate the between-pair variation and only rely on the within-pair variation to identify the minimum wage effect). You might have to go to table 2 in the paper to see it clearly enough, but only the regressions where the red outlined contiguous county dummies are included are "matched":


Now Bob also raises concerns about why they're tossing other stuff in when they've already got a county match. I hope it's clear that they are not "matching" counties until the sixth specification, but this point is still worth addressing because it is relevant for other issues that Bob raised with the paper almost a year ago. Finding a good comparison group is the principal task of the microeconometrician. It's identical to the issue of "identifying your model". But when you find a good comparison group it doesn't mean there's no room left for improvement. So you'll often include other controls or matching procedures after finding a good comparison group (in this case, contiguous counties).

When I was at the Urban Institute, one project I was on was to evaluate a job training program for high-growth industries (often advanced manufacturing and ironically now, construction). We had individual level data, but we paired cases that showed up in contiguous localities (in this case a WIA designated area, not necessarily a county). One county's workforce investment board would direct people to the program we were looking at, the other would just have the training options that were generally available. That was a big step forward compared to what we're calling here the "naive" approach of comparing our treatment cases to anyone in the country that ever sought out job training. By focusing on contiguous localities we got a much better comparison sample. But there was still a lot to be improved on. So after that, we used propensity score matching to further match the characteristics of the group, and then on top of that we included other control variables in the ultimate model specification.

The point being that you never have a perfect comparison group. Using contiguous counties in DLR is a big improvement on Neumark and Wascher but that hardly means that everything is controlled for.

This gets us into the controls they add - specifically, time trends. I'll get that up in Part 2, hopefully soon. that deals with the concerns raised by Bob about a year ago. I wanted to get it in here, but I realized there's a lot to say to clarify the basic specification issues raised in his more recent post.

To sum up - even very simple controls for spatial heterogeneity like regional dummies strongly imply that you need to control for geography, motivating DLR to introduce their contiguous counties strategy.