Peter Dorman on Economics as a Science

He writes this:

"Congratulations (maybe) to the Opera (Oscillation Project with Emulsion-Tracking Apparatus) team for their (possibly) revolutionary finding that a few neutrinos were able to defy Einstein and travel from Geneva to central Italy faster than the speed of light. If true, it will require a revision of basic physics that borders on science fiction.

I heard through the grapevine, however, that some senior scientists with this project did not give permission for their names to be on the article setting out the results, including one of the individuals who helped conceive and organize Opera from the start. They are passing up the opportunity to be connected to a historic breakthrough in their field. Why?

The answer is that, with such an extraordinary anomaly, there is a risk of error. Mismeasuring the distance within the apparatus by 12 meters, for instance, would reverse the results. Above all—and this is why economists should be interested—a physicist would suffer a huge, possibly irreparable blow to his or her career by being attached to a claim that is later found to be wrong. Type I error (false positives) are taken very seriously. The logic of this extreme asymmetry, so much weight on Type I, so much less on Type II, is explained in this earlier EconoSpeak post.

It’s rather different in economics, isn’t it? If someone shows you have made a false claim in a published article, you can write a gracious response thanking the critic and go on. More likely, you will double down and spin out more studies defending your original argument. Either way, if you’re wrong it’s no big deal. Lots of the top economists in the professional firmament have been wrong at one time or another (or even all the time), and it hasn’t set them back. Meanwhile, physics evolves over time toward ever-closer approximation of the real universe, while economics accumulates error along with insight."

I would say it somewhat differently. I don't think the problem here is a Type 1 error/Type 2 error emphasis problem. Everyone that thinks in terms of faux-Popperian hypothesis testing is primarily concerned with Type 1 error. The real difference is that the specification of economists' empirical work is a lot less certain than the specification of the empirical work in a lot of physics (I imagine string theory types are more comparable to economics). The fact is, economics isn't physics. Our theoretical work consists of models to demonstrate processes that go on amidst lots of other processes. You can bring those models to an empirical test, but the interpretation of the test is never quite as clear as "what is the speed of a neutrino".

A better comparison for what Dorman is talking about here is measuring something like GDP or some other economic variable. If the director of the BEA or the Census bureau came out with truly bizarre results he or she would be concerned as well and they would be criticized for mistakes.

While a lot of physicists measure values like this with exciting new equipment, that's not the work of most economists. Measuring values in economics is fairly perfunctory for most of the profession. We can always improve our measures, but we don't need something like CERN to generate a GDP statistic. Our work is different from the work of many physicists - our work is interpreting this data knowing that the economy is a very complex system.

So I'm not quite sure the analogy holds, personally.