"Scientism" accusations need meat on them - math helps us understand the social and natural world

One of the things that frustrates me to no end about accusations of "scientism" and "physics envy" is that they are always hopelessly vague and wishy washy. What precisely do you have concerns about? It's rarely made explicit because in my opinion the critique is rarely carefully considered by those making it. Constrained optimization is occassionally cited specifically for censure. It's a convenient case because Samuelson explicitly mentioned thermodynamics in his early work on constrained optimization, which painted a bulls-eye on him for people who were more interested in painting him as a physics-envier than they were in good economic science.

Outside of that, few specific critiques of why a technique is appropriate or not are offered.

I thought of that this morning when I was looking at this new NBER working paper titled "Quantile Regression with Censoring and Endogeneity". This is precisely the sort of thing a lot of the "scientism/physics envy" people would talk about: fancy math for fancy math's sake according to them. That's really unfortunate. Here's a summary of what this paper does:

"In this paper, we develop a new censored quantile instrumental variable (CQIV) estimator and describe its properties and computation. The CQIV estimator combines Powell (1986) censored quantile regression (CQR) to deal semiparametrically with censoring, with a control variable approach to incorporate endogenous regressors. The CQIV estimator is obtained in two stages that are nonadditive in the unobservables. The first stage estimates a nonadditive model with infinite dimensional parameters for the control variable, such as a quantile or distribution regression model. The second stage estimates a nonadditive censored quantile regression model for the response variable of interest, including the estimated control variable to deal with endogeneity. For computation, we extend the algorithm for CQR developed by Chernozhukov and Hong (2002) to incorporate the estimation of the control variable. We give generic regularity conditions for asymptotic normality of the CQIV estimator and for the validity of resampling methods to approximate its asymptotic distribution. We verify these conditions for quantile and distribution regression estimation of the control variable. We illustrate the computation and applicability of the CQIV estimator with numerical examples and an empirical application on estimation of Engel curves for alcohol."

I wish they came out with this five years ago, because this actually sounds like a very useful technique. We could have made good use of this in an evaluation of child welfare reform that I helped perform for the Department of Health and Human Services a couple years back (the evaluation itself is still working its way through the DHHS bureaucracy, but background is available online and we've presented preliminary results). We used quantile regressions for the analysis, but one thing we didn't have was an IV version of a quantile regression. This would have helped tremendously because we were investigating a series of highly endogenous child welfare outcomes - but we did have a quasi-random treatment which would have worked with the CQIV presented here. Some of our variables were also censored because of institutional time constraints within the child welfare system. A quantile regression approach that deals with both censoring and endogeneity in such a clean way is great stuff.

The trouble is, if you open up that working paper you see a lot of math. I would have had to spend a fair amount of time sitting down and reading it very closely to understand the gist of it. Then I'd probably have to take the time of walking down the hall to Doug Wissoker's office - a senior econometrician here, whose door I knock on often - and humbly profess my remaining ignorance and ask him to help me. And then he'd have to sit and look at for a while to figure out what they were getting that. And then we'd also share what we eventually did with it in meticulous detail to our technical working group and then they'd make sure we were doing something reasonable. It would have been quite a process - but it would have considerably improved our understanding of how abused and neglected children fare in the child welfare system in this country. And that is a very good thing.

These are the sorts of papers, though, that often just get dismissed as "scientism" with no details on exactly what is wrong with investing the time to learn and use it. That bothers me a great deal.

If Biology Had Schools of Thought Like Economics Does

In this post I discuss how silly arguing from the perspective of "schools of thought" is in economics. Let me clarify, I don't think all economists do this. It's most problematic among macroeconomists, and particularly among macroeconomists that can't distinguish between their politics and their economics. This isn't an issue at all among microeconomists. Microeconomists differ over specific findings, but there are no great over-arching "schools of thought".

Anyway, by means of illustration I wanted to sketch out how silly it would be for evolutionary biologists to do what macroeconomists do. Imagine these schools of thought:

The Foodians: The Foodians strongly maintain that evolution is primarily driven by access to food sources. When climate changes, the kinds of food growing in an area changes and organisms who are best suited to survive eating the new collection of food available are the ones that survive and propagate successfully. Food drives evolution.

The Climatists: The climatists regularly accuse the Foodians of ignoring underlying causes (despite regular references in Foodian articles to things like changing climates). Climatists argue that it's wrong to say that food supply drives evolution - ultimately changes in the climate cause evolution. Organisms that are more prepared to survive in colder weather, or wetter climates, or what have you are going to survive and propagate successfully. Climate drives evolution.

The Dispersionists: The Dispersionists, on the other hand, think that both the Foodians and the Climatists are barking up the wrong tree. Evolution depends on the availability of lots of genetic variation in a population and this is going to depend on how dispersed or concentrated, large or small the population of a species is. If there are just a bunch of small colonies of an organism scattered widely across the planet with little contact between each other, genetic variation is not going to be as wide as if a lot of organisms of the same species congregated together and could easily travel through the population and mate with each other. Population dispersion is what drives evolution. Moreover, the other schools have a fatal flaw: they consider population variation to be homogeneous! They aggregate too much! Despite the regular mention of variations in the preparedness of organisms to eat certain foods or deal with certain climates, the Dispersionists still accuse Foodians and Climatists of genetic aggregationism... no one quite understands why they accuse them of this. Presumably its because Dispersionists are smitten with the way they talk about genetic disaggregation and like to feel special.

The Asteroidists: The Asteroidists argue that the biggest episode that life has to deal with on a planet is surviving being hit by an asteroid or other incoming rock. Large rocks flying into Earth drive evolution because the impact is so violent and the resulting climatic changes are so all-encompassing that only organisms designed to deal with the extreme cold after the impact blocks out the sun, or those who can burrow deep underground, or even those who are simply lucky enough not to live directly in the impact zone can survive. Impact events drive evolution.

*****

Can you imagine how goofy that would be? The disagreements I outlined above are largely based on caricatures of the other positions. There's very little that is obviously mutually exclusive in any of this, and whatever is mutually exclusive can be explained by acknowledging "yes, X is a process that operates sometimes and excludes the prospect of Y process occuring, but at other times Y may occur without X occuring".

The reason why economists do this is probably mostly because of politics and ideology, but it's allowed to perpetuate itself because we also have a very weak commitment to the idea that (1.) we are scientists - primatologists, in fact, and (2.) we are studying a complex phenomenon that involves multiple simultaneously operating processes.

I think it would be legitimate to say "I study this particular extinction episode a lot, which was caused by an asteroid strike so I emphasize the role of asteroids in my own research but I recognize all the other forces". That's fine. It is also legitimate to have a preference for what explains a large portion of the subject at hand in general, without thinking of other forces as illegitimate or mutually exclusive. What's wrong is to pretend that what we're dealing with should be treated as warring schools of thought. This isn't to say there can never be warring schools of thought. I'm not sure what a good example in biology would be - punctuated equilibrium vs. gradualism? That seems like it might fit the bill. But they should not be as common as they are in economics.

Macroeconomics, Science, and Engineering

Robert Johnson and I have been discussing the extent to which economics is a science (or a "soft science") in the comment section of this post. The very term "soft science" is like nails on a chalk-board to me. I find it completely vacuous. I'll loudly proclaim that there are varying complexities of the systems that various scientists study and that this needs to be taken seriously - but this doesn't really speak to the scientific quality of that field of study. Social science is not the half-way point between the "sciences" and the "humanities". "Social science" is the name we give to certain sciences because our self-absorption and self-aggrandizement revolts against the idea of classifying economics as a sub-branch of primatology.

Anyway - I can't really comment much longer on that post, but at the end I was getting the sense that a lot of the difference between my views and Robert's views might be emerging from the fact that I separate questions of science, engineering, and forecasting. To me they are very different things. One is the pursuit of understanding through the scientific method. Another is the application of that understanding to problem solving. The third is the application of that understanding to piercing through the "dark forces of time and ignorance that envelope our future". All are noble pursuits. All are rightfully done by economists with varying degrees of success. All are quite distinct, though. I think we are quite good at economic science, and somewhat less good at economic engineering and forecasting (we are probably better at engineering and forecasting than meteorologists, worse at engineering but better at forecasting than geneticists, and worse at both engineering and forecasting than astronomers).

Anyway - all I intended to do here was to quickly point readers to an essay that Greg Mankiw wrote a while back on the macroeconomist as an engineer and the macroeconomist as a scientist. Here's a good selection from the beginning:

"To avoid any confusion, I should say at the outset that the story I tell is not one of good guys and bad guys. Neither scientists nor engineers have a claim to greater virtue. The story is also not one of deep thinkers and simple-minded plumbers. Science professors are typically no better at solving engineering problems than engineering professors are at solving scientific problems. In both fields, cutting-edge problems are hard problems, as well as intellectually challenging ones. Just as the world needs both scientists and engineers, it needs macroeconomists of both mindsets. But I believe that the discipline would advance more smoothly and fruitfully if macroeconomists always kept in mind that their field has a dual role."

"Thinking like an economist" on climate change

Bjorn Lomborg, a famous "climate change skeptic", is publishing a new book where he identifies climate change as one of the biggest problems facing the human race, which should be addressed by a carbon tax and billions invested in addressing the problem (HT - Tyler Cowen). All very shocking stuff, right? This pronouncement by Lomborg is going to electrify the environmentalist community and it will be presented as a major coup.

The problem is, as far as I can tell Lomborg isn't really saying anything new. He's always asserted that climate change is real and a problem, and he has always (like a huge portion of economists) identified a carbon tax as the solution.

I think the public's view of economists is driven by the weird way that economists think and interact (relative to everyone else at least). For example, economists are very good at compartmentalizing different parts of a problem, and they are also good at thinking speculatively. Thus, in Superfreakonomics, you have the authors speculating about the relationship between time preference and the mitigation of climate change. They set aside the question of whether we should do anything about the problem (let's say, for the sake of argument, that we should) and ask "what should we do?". It may be more efficient to wait until climate change is actually a problem and cheaply pump sulfur aerosols into the atmosphere to cool down the Earth, compared to the impoverishing impact of reducing our carbon consumption. Like it or not, this is a serious thing to consider: do we do painful remedies now when we are relatively technologically unadvanced, or do we do cheap remedies later when we'll have greater technical know-how anyway. Normal people don't think in terms of these time preferences and trade-offs... but economists are not normal people.

Lomborg is the same way - he's severely criticized Al Gore and other alarmists for hyping extreme scenarios and misleading the public about the reality of climate change. Normal people assume this means that Lomborg is somehow unconcerned about climate change. After all, Al Gore seems to have good intentions. Anyone criticizing Al Gore must not take climate change seriously. This is not the case at all.

Generally speaking, natural scientists approach natural science objectively and social issues subjectively and emotionally. The general public often approaches both natural and social science issues somewhat subjectively and emotionally. Economists tend to approach social questions quite objectively. You can't assume that because an economist opposes cap-and-trade he is unconcerned about climate change. You can't assume that because an economist opposes the minimum wage he is unconcerned about low-income families. You can't assume that because an economist supports large budget deficits they are unconcerned about fiscal responsibility. They may be unconcerned about those things, but not necessarily. They view these questions objectively, as scientific questions, whereas the general public sees them as moral questions. If you come out against any proposed climate change policy it is assumed that you don't care about climate change. If you come out against any policy purported by elected officials to help the poor or to bring responsibility to Washington, it is assumed that you don't care about those things.

Of course, part of "thinking like an economists" is laughing it off when people assume you're a monster... in retrospect this probably isn't very conducive to straightening things out.

Another important element of "thinking like an economists" is amicable arguing. Criticism among economists is famously vicious, but professional. This catches a lot of non-economists off guard, which is why I think Lomborg has been branded as an opponent of dealing with climate change (and actually - Lomborg is not an economist, although he is a social scientist and he does "think like an economist"). I've run into concern about this "amicable arguing" at Cafe Hayek a lot. People there have accused me of "disrespecting" Don Boudreaux and Russ Roberts. That's because most people argue with people they don't like and try to smooth out disagreements with people they like. Economists aren't like that, as anyone who has been to an academic workshop in economics can tell you. This is what I tried on several occasions to tell people commenting on Cafe Hayek - I'm just raising what I see as concerns with Don and Russ's argument. Patting them on the back for good points seems unnecessary and unproductive. Beyond a shadow of a doubt, Don and Russ both get the same kind of scrutiny in their own workshops at GMU (just take a look at the Youtube debate between Bryan Caplan and Peter Boettke - amicable argument is the norm within economics departments).

Normal people don't generally act like this. Vigorous disagreement and disputation are taken as signs of opposition. Normal people try to cushion criticisms of friends, and let themselves go in criticisms of enemies. I've never seen that tendancy in economists. You don't do your friends any favors by indulging inaccuracy, and as Keynes said "There is no harm in being sometimes wrong- especially if one is promptly found out".

When you add all these things together: an eagerness to speculate, an ability to compartmentalize, objectivity on questions of social import (we can also think of this as a well tuned ability to distinguish between "is" and "ought" or positive and normative points), and an argumentative nature (even with friends), what you get is a group of people who can easily be mistaken as supporting something they actually oppose or opposing something they actually support. But that's not because we're being unclear about it - it's because you normal people are.

Boettke on New Methods of Peer Review

I want to pass this post by Peter Boettke for reactions, although I don't have time to think in detail about it now. He mentions a New York Times article about a more open, blog-like "crowd-sourcing" of the peer review process. I didn't even get a chance to read the article in detail yet, but it sounds interesting.

I do have my doubts, though. I have very little experience with the peer review process, and the experience I have had doesn't entirely reaffirm the value of the status quo, but I worry that this is the wrong stage in the process for introducing a blog-like approach.

It seems to me peer-review is simply a way of ensuring quality and making sure important points in research were hit, right? I don't see how this blog-like approach helps with that. What blogging can offer is a way to formalize a much wider range of comments and critiques than the normal journal format for replies and comments. A good example of this is what Econ Journal Watch is doing to collect a series of critiques of the Buturovic and Klein article on economic ideology (which I participated in). We could have more of this sort of thing, and we could presumably have each of these critiques as blog posts with comment sections, methods of voting on the salience of different critiques, etc.

But is it really necessary as a gate-keeper? I don't think so.

This is not to say the current peer review process is perfect - it's not. But it seems to me the biggest problem with peer review are issues of (1.) elevating the most sexy, counter-intuitive findings and empirical tricks, regardless of whether they are the right findings, and (2.) having a bias against insignificant findings. I don't see how these would be solved with a blog-like approach.

I also imagine it would be a nightmare to respond to everybody in an equitable way. I had only two reviewers for my recent Review of Austrian Economics article, and on some issues they advocated exactly the opposite change! And this is RAE - presumably the reviewers of other journals are even less homogenous than the reviewers of this one.

So my initial reaction is to move forward with these possibilities and definitely experiment with them, but I'd hesitate to use them at the peer review stage.

Do others have thoughts?

Literature and Social Science

Mark Thoma links to an interesting blog post on the relationship between literature and social science - specifically, the extent to which we can use literature as data in social science.

This was very interesting because it's been on my mind a lot as I grapple with Lovecraft's political economy. He left a lot of letters and essays that lay out his ideas clearly in straightforward non-fiction. But what to do about the stories and poems? They have insights too, but I want to be very careful about how I use them. One of the most interesting divergences between his stories and his non-fiction is that his explanation of human fear of the unknown and of the future (very important in economics) is completely different in each. Read the first section of Call of Cthulu and the first section of his essay Supernatural Horror in Literature and you get two diametrically opposed understandings of our fear of the unknown. I've approached this under the assumption that the view in Supernatural Horror in Literature is Lovecraft's own view, and that the inversion of that argument in the Call of Cthulu was for dramatic effect - to make the fear he considers to essential to good literature more potent (precisely as he says a good author should in the essay!). But this shows some of the pitfalls. This is a straightforward case, but it demonstrates how careful you have to be when you use literature to make objective statements like this.

Here are some good selections from the blog post:

So, what about it? Can a work of fiction have realistic, referential content? Is a novel sometimes an empirical statement? Can a fictional character truthfully represent aspect of what it is like to be black in America, South Asian in Manchester, or gay in a suburban Illinois high school?...

...Here is one possible answer: fiction is always fiction, and normally does not have empirical validity. If we want to make empirical statements about social relations, class attitudes, or typical social values of specific groups, we need to do so based on valid methods of social research: surveys, focus groups, interviews, and observations of behavior...

...Another very different response goes along these lines. Novelists are sometimes skilled social observers, and some of these are also skilled "painters" or evokers of what they have seen. A great novelist can pull together his/her many insights and observations into a powerful description of a fictional world or experience that captures an important sociological truth about the society depicted. So these novelists do in fact gain knowledge of social life through observation, and they represent that knowledge through the fiction they produce. Both parts of this epistemic process are subject to criticism; but both are valid knowledge practices...

...Seen in this way, a novelist is somewhat akin to an ethnographer, trying to make sense of a complex system of behaviors and meanings and expressing his/her findings in a way that is truthful to the social reality observed. It seems that one would have to be a pretty narrow-minded epistemologist to hold that there is
only one kind of knowledge, and that literature necessarily falls outside its
bounds.

The author goes on to embrace the second option, and I think I do too. The comparison to ethnographers is apt, I think, and to make myself clear and not understate my skepticism I should probably note that I don't put a great deal of stock in ethnographers! But it's not a completely useless effort either. It's simply something that you need to approach with a great deal of caution. You need to be able to determine whether the fiction writer is saying something (1.) about the world that he observes, (2.) about his normative position on the world he observes, (3.) about a world he doesn't observe but wants to describe, or (4.) some mix of the three. Usually it's the last option - some mix of the first three. To have any hope of navigating this, I think it's very important to have a sense of the author's literary theory - to know what they think they are doing. Without that there's really no hope of getting useful information because you have no way of arbitrating between those four options. And even with that it's still highly speculative.

Anyway, these are interesting issues to think about at least. Ultimatley I think literature at best can provide a nice frame for social science. Framing is going to be a more fruitful contribution than the provision of any data.

Cosmology seems to make macroeconomics look decisive!

So I was watching the History channel last night, and they were showing an episode of The Universe called "Beyond the Big Bang", which covered cosmology, the Big Bang, and the formation of the universe immediately after the big bang. Neil deGrasse Tyson had a very interesting thing to say about cosmology:



"There were more theories running around than data"

Wow - that sounds familiar! It certainly doesn't seem to lead Tyson to call cosmology "fake science", as Russ Roberts likes to call macroeconomics.

Wikipedia confirms - check out the number of cosmological theories. You can even scroll down to the twentieth century, and it still puts macroeconomics to shame (Brad DeLong recently identified seven basic macroeconomic theories, compared to my count of twenty cosmological theories in the twentieth century alone).

I think another important point is that we fight over these macroeconomic theories, but ultimately that's kind of a weird thing to do. There's no good reason why most of them can't be integrated. Arnold Kling puts forward a mixed Minsky-Austrian story. I think Keynes, Minsky, and the Austrians all make sense and I'm perfectly comfortable with major tenets of monetary disequilibrium theory, and of course I believe real shocks can matter which means I don't think there's anything especially wrong with Real Business Cycle Theory. The problems come in when people point to a specific theory and say "this, and only this, is how macroeconomic fluctuations happen". Every recession is different, and presumably they're going to have a mix of causes. Macroeconomists have done a pretty decent job outlining these causes. What we need more of is understanding how these theories integrate together and what factors are important for what specific episodes. What we need less of is this attitude that there is one answer.

Are the twenty cosmologies as easily integrated and traded off? Well, they're explaining a single event and many of them contradict each other on the very nature of the universe, so probably not - but I'm sure to some extent they do.

I'm not trying to ridicule cosmology - as Tyson says, it's a matter of the data that's available - that's a limiting factor in science. But challenges don't make something non-scientific! Science is all about the method of gathering and testing knowledge.

Anyway, I just thought this was interesting. Phsyics is usually held up by the critics as some sort of gold standard. My response is "well, economics is really more like biology", but the fact is physics is struggling or has struggled with the same things that certain sub-disciplines of economics struggles with. That's life - science is hard.

Social Science as a Science

More debate has flaired up over whether social science is really a science. The source of it is this City Journal article by Jim Manzi. The article is actually quite good, and highlights a lot of the empirical limitations I talk about a lot on here. Some of the commentary is less persuasive in my mind.

Razib Khan and many of his commenters overstate their case against social science as a science in this post, with many of the commenters clearly ignorant of how social science does its work, and confusing political claims with the claims of social science.

Arnold Kling, not exactly a huge cheerleader for the "social science as science" crowd, at least focuses on the original Manzi article and some of the good points that it made.

One of the frustrating things about the people who don't think social science is a science is that they're so shifty as to why. It usually starts with a methodological point, criticizing our ability to establish counter-factuals (something that is very explicitly talked about among us - this isn't a huge secret or embarassment - it's a fact of life working with the data we work with), and our lack of experimentation. When you point out the plethora of other sciences that don't experiment it jumps to another of other justifications: (1.) predictive ability (the right response: we have better predictive ability than meteorologists, and why do you blame us for not predicting recessions exactly but you don't discount geology for not predicting earthquakes exactly), (2.) ideology (the right response: ummm... yes, ideological social science is not scientific - if you're going to claim all social science is ideologically determined I'd like to see some data to support that - since I'm a social scientists of course), (3.) multiple theories (the right response: sure, but again that's just a result of the complexity of the system under study and the lack of counter-factuals. Falsification is a lot harder. You see more theoretical positions arguing in paleontology than in the rest of biology, and you see more theoretical positions arguing in biology than in chemistry - it's a function of the data, not of the "scientificness" of what we're doing).

I think there are two primary reasons why people make the mistake of thinking that social science is not a science:

1. Sentimentality: People don't like to think of themselves as objects of study. The universe can obey laws, but humans can't. We're special. An evolutionary biologist won't come out and tell you that we're made in God's image, but the residual hubris of that cosmology remains for a lot of people. We're different and you just can't study us and our social system. You can scientifically study the social systems of lower organisms, but not humans.

2. Cargo Cultism: I'm going to turn Fenyman on his head here. A lot of natural scientists are huge practitioners of cargo-cultism. If you don't do science their way - if you don't reproduce the forms of their research, then you're not scientific. The best example of this is when people say that you need experimentation to be scientific - that's cargo cultism par excellance. They often don't think about how the nature of the subject determines the form of how you approach it. I've always said social science is most like biology - and probably the paleontology comparison is even better since for the most part we're looking at historical data. Physicists and chemists don't impose the forms of their science on paleontologists because it would be inappropriate - that would be cargo-cultism. Something goes out the window when humans are concerned. The Fenyman point was really unimpressive to begin with - I think people accepted what he said just because they respect his other work and it reinforced what they wanted to believe. What was his example? People who "sit at a typewriter" and make claims that "food grown with fertilizer that is organic is better for you than other food". What does this have to do with social science? This isn't even social science - it's an anatomical/medical example, and not even an example - it's something that Fenyman just made up off the top of his head. The idea that people would cite this as an authoritative point is more than a little disappointing. It was a poorly reasoned argument on Fenyman's part without any evidence and ultimatley it was probably symptomatic of the very cargo-cultism he thought he had identified in social science.

I have yet to see a good argument for why social science isn't a science. Everything I've ever read reduces to cargo cultism or sentimentality about human beings. None of it references the requirements for a science to be a science, and if it tries to it usually fails miserably.