Don Boudreaux on empirical social science

I don't want to give the impression that this post is one that I largely disagree with. Actually a lot of it is strong, particularly his discussion on the role of theory (I also liked the prior post where there seemed to be some recognition that reductio ad absurdum is often a terrible way to evaluate an economic claim). As Cafe Hayek posts go, this one is quite good. But I did want to highlight this passage, not to trash Don but just to raise concerns about an attitude that bothers me a lot:

"...there is no shortage of empirical studies that document the minimum-wage’s detrimental impact on low-wage workers. There are also, of course, other empirical studies that find the opposite effect, and yet others that find no significant effect.

That’s the way it is with social-science (including economic) data: they almost never speak for themselves clearly and without significant exceptions. Someone can always challenge even the most consensus-supported studies as having left out important variables, gotten causality backwards, employed inappropriate aggregation, examined inappropriate time periods, and on and on."

I don't like this attitude that empirical analysis is all over the map and you can  criticize all studies. Of course as in any science we're faced with a distribution of point estimates, which we usually take to be around a "true" point estimate. We are usually most confident about this if the studies at the center of mass seem to be the best designed and come out as the most precise. If you find some studies with a positive minimum wage effect and some with a negative, and some with no effect there's a good chance that there's actually no effect of the minimum wage.

Here is an excellent graphic from a new CEPR publication on the employment effects of the minimum wage. The estimated elasticities are on the x-axis, but what I really like is that the reciprocal of the standard error is on the y-axis. That means lower variability (not necessarily more accurate - we'll get to that) estimates have a higher value on the y-axis.



This is really not the picture Don is painting of the empirical literature for the economic impact of the minimum wage. This reaffirms my view from early meta-analytical work that the minimum wage essentially has no employment impact. Now there's a substantial cluster of slightly negative results in this graph, which is the origin of the Neumark and Wascher point that we do have a consensus for a slightly negative impact of the minimum wage. I'm actually willing to embrace that "negative but small" conclusion for essentially the same reason that Don points out - it sure has a theoretical basis. But embracing that conclusion as a practical matter doesn't change the fact that the zero-effect findings look very strong , and anyway the empirical evidence is not wishy-washy at all as Don seemed to suggest. You either have a "no effect" option or you have a "maybe a teeny effect" if you're being objective.

Claims like Jeff Tucker's assertion that one's position on the minimum wage is something of an intelligence test (in his case implying that you are not intelligent if you don't oppose it) fares very poorly.

Now I also alluded to the difference between precision and accuracy above - or what econometricians usually refer to as efficiency and bias. Bias - that your point estimate may not be centered on the "true" estimate - is the primary concern.

One of the biggest sources of variability in any set of studies is the inherent bias of a particular identification strategy. Economists almost never have a true experiment, so we have to use a variety of tricks to identify the relations we're interested in in a system that is simultaneously determined. Often these tricks will influence the point estimate itself. In immigration studies, for example, you have a lot of natural experiment studies and a lot of what I guess you'd call "shift-share" studies. The former tends to find positive effects and the latter tends to find negative effects (none substantial unless you're looking at very close labor market competitors).

Presumably there's something systematically different about these approaches. The natural experiment approach usually involves looking at a large influx of immigrants at a single point in time. The problem with that is that there is still a non-random aspect to (1.) which immigrants go, and (2.) where they go. Immigrants are going to be attracted to local economies where they have a shot at succeeding and where they might have a comparative advantage. You're going to naturally pick up lower negative effects or even positive effects as a result. The strategic behavior on the part of immigrants is good to know and may even reassure us, but it does not present an unbiased estimate of the impact of a truly exogenous migration shock. The shift-share analyses associated with Borjas apparently have some negative biases just like the natural experiment studies have some positive biases, although I'm personally less well versed in those criticisms.

The point is, when you consider these points the meaningful variability between empirical estimates starts to decline a lot. When you really understand why one set of studies is higher and one set of studies is lower - and if the methods of both studies have implicit biases, then you start to see them as upper and lower bounds - not as widely varying estimates of the same underlying parameter.

This is true of fiscal multiplier studies too. War spending studies give low estimators (I have a research note on this that will hopefully see the light of day at some point), studies where there is spill-over produce low multipliers, studies that look at state or county level spending by the federal government produce high multipliers because all jurisdictions pay federal taxes (so the counterfactual gets a negative hit).

A good empirical economists should be able to do what you might call mental meta-analyses like this. To really know an empirical literature you need to have a sense of the different strategies that people use for doing empirical work and the strengths and weaknesses of each strategy because a lot of the variability of the results are baked in by the differences in the strategies. When you start to think of the literature in these terms you realize there really isn't as wide of a range of results as you first thought. Often you have one group of studies using a strategy that introduces a negative bias with lower results and another group of studies using a strategy that introduces a positive bias with higher results. Your reaction to that should not be "wow there's a wide range of results" - it should be "we can be pretty sure that the real answer lies somewhere in between".

Empirical economics is not a free-for-all, but it does require effort. Most questions are actually clearer than they appear at first glance, but you have to really understand what the studies are doing that makes them different.