Thinking about the specifications of the Dube, Lester, and Reich minimum wage study: Part 2 - controlling for county level trends

Q: When is a fixed effects model not a fixed effects model?
A: When it's a difference-in-differences model.

The most important question in any impact analysis is "how do they identify their model"? Sometimes its buried in the math, but there are a few canonical forms of how to identify a model (often very closely related) that in my opinion at least help to think about model specification and exactly what kind of assumptions and variation the authors are relying on.

I'm sure a lot of you know that a fixed effects model is just a model you run on panel data with dummy variables for each cross-sectional unit to soak up all the time-invariant non-observed characteristics, and dummy variables for each time period to soak up all the common time trends. The big thing you don't get automatically in a fixed effects model is control of time-variant variables that are cross-sectional unit specific.

Turning a fixed effects model into what is essentially a difference-in-differences (DID) model is pretty straightforward. In fact we discussed it in the last post on Dube, Lester, and Reich (DLR): you just include county-pair fixed effects in their case. These fixed effects capture any variation across pairs, so the only variation left to estimate the minimum wage coefficient on is variation within a pair, between the counties in that pair over time. DLR have intermediate versions of this, restricting the estimation of the effect to pairs within regions, states, and metropolitan areas. But what they're narrowing in on is essentially the DID. The logic of the DID is straightforward and I want to walk through it before getting to more of Bob Murphy's thoughts.

You have panel data so you've got data before and after a treatment. You also have two cases: a treatment case (on the left, below), and the comparison case (on the right). The treatment case may be changing over time anyway without the treatment, so to isolate the treatment effect any changes in your comparison case (the paired county in DLR), is subtracted out of the treatment effect. Why? The changes in the comparison cannot (or should not... there's a different literature on that issue) be affected by the treatment because it didn't get the treatment. So that small gap on the right is the counterfactual of what would have changed in the absence of treatment, and therefore cannot be attributed to the treatment effect on the left. Notice also that it doesn't matter if the comparison case is a little different from the treatment case (see how I've drawn it a little lower?). What matters is the differential response to treatment, because the DID estimator is:

[(Post-Treatment) - (Pre-Treatment)] - [(Post-Comparison) - (Pre-Comparison)]

[UPDATE: I had the terms switched above before - this version is correct. You take the raw change in the treatment case, but then you want to subtract out the change in the comparison case from that]

So if there's something about the comparison group that's time-invariant that makes it a little different from the treatment group, that's OK. That's why we have county dummy variables. What's more problematic is differences in the counties over time (which I'll discuss below).

The profile over time in the diagram above is flat, but we could easily imagine a common time trend (imagine the slopes in the figure below are the same!). This doesn't matter for the simple DID case at all for two reasons. First, if the portion of these time trends common to all counties is already absorbed by the time period dummy variables I mentioned above. Any other time trend that is common between the paired counties will be subtracted out of the treatment effect by the exact same logic of the case without the time trend: we are removing the change in the comparison group from the

As I alluded to above, the big trouble comes in when you have time trends in the treatment and comparison group that are different. That would look something like this:

If you implement the DID estimator here it will make the treatment effect a lot smaller because there was a big change in the comparison group over time relative to the treatment group (in other words, the match might have been good, but it wasn't perfect). Looking at what's actually going on, though, you can tell that the true treatment effect should be exactly the same - we're just conflating the rate of change that has nothing to do with the treatment effect with the treatment effect itself.

What you want to do in this case is control for the trend rate of change by county so that any increase in the comparison group in the post period that follows that rate of change is not used to penalize the treatment effect. You could just as easily imagine a scenario where you'd want to do this because it would over-estimate the treatment effect. I draw it this way because this is what DLR came across. Once you control for that time trend, you're back to the situation of the first picture (common time trends will be swept up in the county-specific time trends, which is just fine - we don't care about common trends), and you've got an unbiased DID again.

So as best as I can tell, Bob Murphy has two related concerns. First, he's concerned that we're including other controls when we were supposed to be dealing with all that by matching counties. That, I hope, is clear from both this post and the last post: even good comparison groups can be improved upon. You never have a perfect comparison group until you have random assignment.

But there's another issue he has with this. About a year ago, Bob wrote:

"What Dube, Lester, and Reich are really saying here, is that maybe for some reason minimum wage hikes happen to be concentrated in regions that have lower than average employment growth. Hence, just because we find that teenage employment grows more slowly in regions with higher minimum wages, doesn’t mean we can blame it on the relatively higher minimum wage. But hang on a second. Minimum wage hikes aren’t randomly distributed around the country, such that we might happen to get an outcome where they tend to be concentrated in slow-growth regions. On the contrary, minimum wage hikes are implemented by “progressive” legislatures, who also (given my economic worldview) implement other laws that retard adult employment growth.

For example, suppose that if a state legislature jacks up the minimum wage, then it is also likely to pass “pro-labor” stuff like laws giving unions more organizing power, laws allowing unfairly terminated employees to receive years of back pay, and laws granting extra perks for maternity leave. Now, these last three items I listed: Would they reduce the employers’ incentives to hire teenagers or adults, more? On the margin, they would make it costlier to hire adults, because if penalties are expressed in years of back pay, or have to do with paid leave, or strengthen unions who traditionally are going to organize adults…You get the picture. Adults make more than teenagers, and so these rules will penalize adult employment more than teenage employment.

Thus, if my model here is correct, it would produce the pattern we actually see: Looking narrowly at minimum wage laws, they seem to retard teenage employment. But then when you ask if states with high minimum wage laws have a bigger slowdown in teen employment versus adult employment, the signal becomes much weaker. It looks like, by dumb luck, for some reason all the minimum wage hikes happen in states that also have slower-than-average employment growth among adults."

So Bob's issue is bigger than the easily dispatched with concern that we matched on counties and then decided that wasn't good enough (I didn't quote that part). The concern is that somehow we are absorbing the effect of the minimum wage.

This may happen under very special circumstances, but generally it's not a problem. Bob is - I think - forgetting the panel element to the data. We are subtracting out the pre-period from the post-period for both the treatment and the comparison, and then comparing those two differences. We know the post period minus the pre period for the comparison group should have no effect at all of the minimum wage so that is the appropriate counterfactual. When we are controlling for a county specific trend we are saying "those secular trends that were going on before anyone adopted a minimum wage would have gone on if the minimum wage hadn't been adopted, so we want to clean that out of the treatment effect". If they are common between counties, my diagram two shows why that's not a concern. If they're different between counties (maybe because one has a progressive legislature), it needs to be accounted for. You are not weakening the signal you are making the signal more accurate because the only impact attributable to the minimum wage is what changes after its implementation. A time trend that continues on the same after as it did before does not change after the implementation of the minimum wage.

What special circumstances might justify Bob's fear? Time trends that are not the same before and after the minimum wage and that are not related to the minimum wage. That might look something like the following:

Let's say the true impact of the minimum wage does not increase the rate of growth of Y in the post period. In other words, let's say the slopes in both these cases would have happened without the minimum wage. If we control for county time trends using pre-period data in this case, we would find that the minimum wage had the effect of:

#1. A one time, persistent, positive shock to Y, and
#2. An increase in the rate of growth of Y

Why? Because we're differencing out the county trend in the comparison case, but we're only differencing out part of the county trend in the treatment case. The rest of the county trend in the treatment case is going to be attributed to the treatment. The #2 effect is false and introduces bias into the estimate of the treatment.

So, in order for Bob's fear to be a problem you can't just have county or state specific differences in trend between treatment and control cases. You'd need to have trends that change at the implementation of the minimum wage, and in a way that biases (rather than just adds noise) to the estimate.

It's not a completely crazy fear. You could have, for instance, a very liberal state legislature that implements a bunch of stuff at once, including a minimum wage law. That's possible, but the effect isn't clear to me. If Bob thinks the liberal legislature will tend to make employment for teens worse, and all these reforms were clustered, that would understate the effect of the minimum wage. Of course the opposite could also be true if a bevy of liberal reforms were to help. If it hurt adults more than it hurt youth that only seems like it would impact the minimum wage estimate if the effect of the minimum wage were estimated relative to the impact on all adults, and I don't think it is. I'm not really sure where those concerns about adult employment relative to youth employment come from.

So I will concede that because lots of different changes may happen together in a state legislature, it would be nice to account for that (of course if the increase is coming from a federal increase, whatever is going on in the state legislature shouldn't matter). These could certainly improve the estimate further. But I don't see any clear evidence that controlling for county time trends doesn't improve the estimate. Remember, the trends as calculated vary across counties, not the difference between the pre- and post- trends. And as my last figure illustrates you'd need the difference between pre- and post- trends for this to mess up the DID estimator.

There may be a Part 3 to this series. DLR also include placebo effects as a sensitivity analysis on these time trends. But I don't have a good sense yet of how all that works right now. If I have time to figure that out and put that together I will.