Henderson and Gochenour on Presidential Greatness

David Henderson and Zac Gochenour have a new paper on the assessment of presidential "greatness" by historians. David discusses it here, and sociologist Fabio Rojas discusses it here. Zac if you have a blog and I'm missing your discussion, let me know. Since two of my favorite things to think about are American history and interpreting empirical work, I thought I'd discuss this.

Their conclusions are highly counter-intuitive (to me at least - I'm guessing many readers won't find them counter-intuitive). To use Rojas's phrase "presidents who kill people are popular". Henderson and Gochenour don't put it quite like that, but their view is very similar. They write: "Our data analysis suggests that wars in which a large percentage of the U.S. population is killed will, all other things equal, cause historians to judge as great a president on whose watch those wars occurred." They are not shy about making this a causal story. To me, this doesn't pass the smell test. All other things equal, we like presidents less when they have a lot of blood on their hands. Which of course raises the question of whether "all other things" really are equal here.

1. Military Deaths Per Capita

One of the nice things about the paper is that all the data is provided in an appendix, so you can reproduce their results and try out a few things (the formatting makes it a little hard to copy - if you want a Stata version of the data, my do-file, or my log file - since I'm not going to clutter this post with output - just email me). Their results seem to be robust to different specifications of military deaths per capita (MDPC, their explanatory variable of interest), which is always appealing. I was a little leery of a simple rank, so I did a natural log of the raw numbers and that gave me similar results. I'm going to use log MDPC going forward because it's easier for me to interpret.

The assignment of MDPC was a little troubling to me. Now, they do come out and admit that they were somewhat arbitrary in assigning MDPC when discussing the fact that they attributed all of the Revolutionary War deaths to Washington, even though that war happened years before his presidency. This decision was especially confusing to me because they do have a "war hero" variable in the regression already, which is presumably the whole reason why the Revolutionary War was significant for Washington. He was not a Lincoln or a Wilson that decided to send us to war. He simply served in a prominent position during that war. But that's what the "war hero" variable is for. This isn't a minor issue, of course. Washington happens to be the second highest ranked president and the Revolutionary War is very high on the MDPC scale. So deciding whether to treat Washington like other generals in exactly his position (Eisenhower, Grant, Jackson) or treating him uniquely can really swing the results. Making the decision in a way that makes your result stronger - if that is the only general you treated that way - is of course going to raise eyebrows.

But there's one more MDPC choice that bothered me, and that's Lincoln/Buchanan. Buchanan stood out in their scatterplot as one of the lowest ranked presidents, and the reason for Buchanan's consistently low ranking is unambiguous. I'll pass the mic to Wikipedia for this one, but anyone familiar with American history knows why: "However, his [Buchanan's] inability to impose peace on sharply divided partisans on the brink of the Civil War has led to his consistent ranking by historians as one of the worst Presidents." So here we have a guy that we regularly rank low in the scale of presidents because we lay the Civil War at his feet, and yet Civil War deaths get attributed to Lincoln who, most Americans would agree, did the right thing in a tough and inevitable situation. This seems odd to me. If we think the carnage of the Civil War matters in the ranking of presidents (and almost certainly it does) shouldn't the guy that we think holds a lot of blame for not solving the sectional strife before war have some of those deaths attributed to him?

If you reassign Washington's logged MDPC to zero and you set Buchanan's logged MDPC equal to Lincoln's (so I'm not even removing the deaths from Lincoln's count), the logged MDPC variable (which I've switched out for their ranked MDPC variable because I like variables I can interpret) goes from being significant and positive (i.e. - more death is associated with a higher presidential rank) to small and insignificant (no empirical relation).

2. Causality, endogeneity, or spurious finding?

I think the two decisions I made above are less arbitrary than the decisions made by Henderson and Gochenour, but let's say you disagree with me and we just keep their significant relationship between MDPC and rank. I'm still not sure this means what they're implying it means. A "great" president in my mind is going to be a president that does the right thing in a tough time when the stakes are really high. You could have lots of potentially great presidents out there, but of course if they never have the opportunity to do great things their presidency won't be "great". Now, when the stakes are high people are of course willing to pay a high cost, including going to their death. When the fate of the Union is at stake, when the enslavement of millions is at stake, when the freedom of Europe from fascism is at stake, when a nation or armed group attacks us unprovoked, etc. - we usually think it's appropriate to risk life to address the situation. So the very situations which are high-stakes enough for presidents to make great decisions are also situations where you're likely to see a lot of battle deaths - not because we think death is a great thing, but precisely because the stakes are so high.

This interpretation - which I think makes the most sense - suggests that the relationship between MDPC and presidential greatness is largely spurious, rather than being causal as the authors suggest.

Another interpretation is that they have the causality reversed. We consider presidents great if they step into great conflicts and make the best of a bad situation. Nobody likes the fact that we had a Civil War, but we think Lincoln did the right thing in it. Nobody likes the fact that we had a Revolution. It would have been nice if we just parted amicably with Britain. But Washington (and Jefferson and Adams for that matter) had the right approach to a bad situation, and we recognize that. The same goes for WWII and even WWI. Roosevelt didn't ask Hitler to invade Western Europe and threaten the future of Western civilization. Wilson didn't ask for that powder-keg to go off, nor did he invite Germany to attack our civilian vessels. But given the bad situation, these presidents responded appropriately.

If this is the right interpretation (and I think this interpretation makes sense too), then Henderson and Gochenour have an endogeneity problem. It's analogous to simple regressions of GDP on fiscal stimulus. If you just do a simple regression (like they do here) with no attempt to identify your model, you're going to get a negative relationship between fiscal stimulus and GDP. Why? Because we only do fiscal stimulus when the economy is bad! These presidents only make tough decisions when they are handed a rotten situation!

We can argue about whether WWI was a war of choice for Wilson, or whether German submarines firing on American ships and revealed plans to invade the country from the south provided a good reason to declare war on a country that was already ravaging our allies. If plans to invade you and actual attacks on your ship aren't cause for war, I'm not personally sure what is.

But the point is, I think both of these explanations - for spuriousness and bias due to endogeneity - make a lot more sense than the idea that we like presidents that preside over lots of war deaths. You would think with such a counter-intuitive conclusion Henderson and Gochenour would provide some explanation of why they think people love war death so much. I'm not sure what an explanation would be.

3. Other variables

I ran some other variables that I thought you all might be interested in. First, what struck me about the high-ranking presidents is that the reason why I thought we loved Lincoln and Washington so much is because they expanded the scope of liberty for the American people. Washington was instrumental in freeing America from the British, and Lincoln was instrumental in freeing millions of enslaved black Americans. If you had to pick one reason why we like those two, "liberty" would be that reason, right? Since Johnson played a major role in expanding civil rights for blacks in the 20th century too, I added a dummy variable for Washington, Lincoln, and Johnson called "liberty". When you add that to the regression with my revised (insignificant) logged MDPC variable, "liberty" is barely insignificant at the 10% level (worth reporting I think - since we only have 40 observations!). When you drop the log MDPC variable, "liberty" is significant at the 10% level and is associated with a 147 point increase in the C-Span ranking.

Then I thought about why we like the other guys, and the reasons are quite similar to why we like Lincoln and Washington. We like Franklin Roosevelt and Wilson, for example, because they advocated and fought for liberty abroad. Reagan is also associated with the end of the Cold War, so I added those three to Lincoln, Washington, and Johnson for a "liberty, broad" variable. This one was not significant.

I also tried a dummy variable for whether the president was a "founding father" (I identified that as Washington, John Adams, Jefferson, and Madison). I was surprised that wasn't significant. I also added a dummy for whether the president was a Virginian. Unfortunately, that doesn't matter either.

4. You can't measure a singularity with a regression

In my third section where I was adding other variables, I was really trying to think about what it is we look for in a president. We like presidents that are instrumental in freeing people who weren't previously free, for example. But when you think about the determinants of presidential greatness, you very quickly get into very specific reasons. Why do we like Johnson despite Vietnam? Civil Rights. Why do we not like Nixon despite ending Vietnam and a fair amount of other good policies? Watergate and general creepiness. Why do we like FDR besides WWII? The New Deal. Why do we like Jefferson? We could list things like the Louisiana Purchase, but really its because Jefferson crystallized the American idea - and he did it throughout his life, not just (and perhaps not even primarily) through his presidency. If we really want to measure why we like certain presidents and why we don't like others, the exercise would degenerate into a cross-section with forty fixed effects. Needless to say, that's a non-starter.