Friday, September 23, 2011

Peter Dorman on Economics as a Science

He writes this:

"Congratulations (maybe) to the Opera (Oscillation Project with Emulsion-Tracking Apparatus) team for their (possibly) revolutionary finding that a few neutrinos were able to defy Einstein and travel from Geneva to central Italy faster than the speed of light. If true, it will require a revision of basic physics that borders on science fiction.

I heard through the grapevine, however, that some senior scientists with this project did not give permission for their names to be on the article setting out the results, including one of the individuals who helped conceive and organize Opera from the start. They are passing up the opportunity to be connected to a historic breakthrough in their field. Why?

The answer is that, with such an extraordinary anomaly, there is a risk of error. Mismeasuring the distance within the apparatus by 12 meters, for instance, would reverse the results. Above all—and this is why economists should be interested—a physicist would suffer a huge, possibly irreparable blow to his or her career by being attached to a claim that is later found to be wrong. Type I error (false positives) are taken very seriously. The logic of this extreme asymmetry, so much weight on Type I, so much less on Type II, is explained in this earlier EconoSpeak post.

It’s rather different in economics, isn’t it? If someone shows you have made a false claim in a published article, you can write a gracious response thanking the critic and go on. More likely, you will double down and spin out more studies defending your original argument. Either way, if you’re wrong it’s no big deal. Lots of the top economists in the professional firmament have been wrong at one time or another (or even all the time), and it hasn’t set them back. Meanwhile, physics evolves over time toward ever-closer approximation of the real universe, while economics accumulates error along with insight."

I would say it somewhat differently. I don't think the problem here is a Type 1 error/Type 2 error emphasis problem. Everyone that thinks in terms of faux-Popperian hypothesis testing is primarily concerned with Type 1 error. The real difference is that the specification of economists' empirical work is a lot less certain than the specification of the empirical work in a lot of physics (I imagine string theory types are more comparable to economics). The fact is, economics isn't physics. Our theoretical work consists of models to demonstrate processes that go on amidst lots of other processes. You can bring those models to an empirical test, but the interpretation of the test is never quite as clear as "what is the speed of a neutrino".

A better comparison for what Dorman is talking about here is measuring something like GDP or some other economic variable. If the director of the BEA or the Census bureau came out with truly bizarre results he or she would be concerned as well and they would be criticized for mistakes.

While a lot of physicists measure values like this with exciting new equipment, that's not the work of most economists. Measuring values in economics is fairly perfunctory for most of the profession. We can always improve our measures, but we don't need something like CERN to generate a GDP statistic. Our work is different from the work of many physicists - our work is interpreting this data knowing that the economy is a very complex system.

So I'm not quite sure the analogy holds, personally.

2 comments:

  1. Since I've yet to get to the graduate level of economics, I can't help but ask this question: do mainstream economists use Exploratory Data Analysis? That's what the econophysicists (and other scholars) who wish to publish in Physica A have to use.

    Also, what are your impressions of statistical mechanics, from what you know of that particular branch of physics?

    But speaking of statisical testing, here's a book review of Didier Sornette's "Why Stock Markets Crash" on a blog dedicated to econophysics. I own a copy, and it's a good read so far. It's critical of the misuse and abuse of statistics in economic research...

    http://econophysics.blogspot.com/2007/12/book-review-why-stock-markets-crash.html

    ReplyDelete
  2. It should be noted that the physicists in question are not claiming that neutrinos travelled faster than the speed of light--they are reporting that they obtained this result, a result which they are skeptical of, and openly seeking assistance in explaining it. It may be, as you indicate, something as simple as a measurement error, or a slight variation in the rotation of the Earth, or any number of other factors which could skew the results. My prediction is that a) the source of error is found, and no neutrinos need be hauled into traffic court for breaking the speed limit, and b) no scientific careers are ruined.

    Generally, the things that WILL get scientific careers ruined are things like a) fraud or dishonesty, b) extreme shoddiness; c) extreme self-promotion.

    The reason that Pons and Fleischmann now have reputations in the toilet is not that they got cold fusion wrong, its that they both got it wrong and flaunted protocol in the process, turning the scientific method into a three-ring circus.

    But still--would be nice if fraudsters and hucksters in economics were effectively shown the door, instead of being so able to find sympathetic consumers of their stuff in the political sphere.....

    ReplyDelete

All anonymous comments will be deleted. Consistent pseudonyms are fine.