I strongly concur with this. Here is some of it:
"A researcher has a theory, call it X1, that can be expressed as a model of how some portion of the world works. Among other things, this theory predicts an outcome Y1 under a specified set of circumstances. There is a dataset that enables you to ascertain that these circumstances apply and to identify whether or not Y1 has arisen. How should this test be interpreted?
My proposal is simply this: the researcher should be expected to consider how many other plausible theories, X2, X3 and so on, also predict Y1. This should take the form of a section in the writeup: “How Unique Is this Prediction?” or something like that. If X1 is the only plausible theory that predicts, or better permits, Y1—if Y1 is inconsistent with all X except X1—the empirical test is critical: it decisively scrutinizes whether X1 is correct. If, however, there are other X’s that also yield Y1, the test is much weaker. It will accurately determine if X1 is false only if all the other X’s are false as well."
We like to talk about Popper, but the fact is I think falsification is so elusive that it's dangerous to talk about empiricism as falsification at all. Falsification is better thought of as something to always keep in the back of our minds rather than something we "do".
It has occurred to me before that a lot of what I've worked on revolves around pointing out precisely what Dorman is saying here. I have:
- Explained in the RAE how Woods, Murphy, and Powell forget other equally plausible explanations of the effect of fiscal policy in 1920-1921,
- Explained in Econ Journal Watch how Buturovic and Klein ignore other more plausible explanations of observed relations between ideology and "enlightenment" (they pushed the data a little harder and ended up more or less agreeing with me),
- Explained in my QJAE article under review that observed wage aggregate cyclicality obscured other alternative explanations for the impact that Hoover had on wages.
Considering these things is important, particularly in such a hard-to-statistically-identify field such as macroeconomics. Indeed, I think these things bother me so much precisely because I came up through labor economics and econometrics, where people obsess over model specification and identification.
4 hours ago