Essie Massoumi Lecture

I just wanted to share some quick thoughts from a great lecture today in my microeconometrics seminar with Dr. Essie Massoumi, of Emory University. He's in town for the week working with my professor so he came in to talk with us about some of the interesting growth areas in econometrics.

He started by laying out the fundamental problem a lot of econometricians feel like they are dealing with, and how they solve that problem: identification of some treatment effect. He was specifically discussing propensity score matching and describing the average treatment effects that come out the other end. There is not a lot of work to do elaborating on that method, he said. His concern is that this way of thinking about the problem eliminates a substantial amount of information.

Focusing on average treatment effects ignores how distributions change as a result of a treatment. This can make a big difference for certain populations if treatment effects are not constant over the distribution of the outcome variable. Consider the case of the gender wage gap, which is reported as an average gender effect, but is different along the income distribution (plausibly, it is lower for low income individuals and higher for high income individuals). Of course you have to know where your treatment group is on the distribution of outcomes as a counterfactual to assess this (that was the point of starting with a propensity score set-up). Probably the most common way of addressing these concerns is with quantile regressions.

Another line of research he talked about was on establishing stochastic dominance of a distribution - in other words, identifying whether one distribution can be said to be preferred to another according to a wide class of preferences. If stochastic dominance could be established, that would solve a lot of policy choices. He also pointed out that if one could not establish stochastic dominance that tells us something interesting too: that disagreements over policies have more to do with values and preferences than they do about disagreements over program impacts.

One of the points I raised - in the context of his discussion of propensity score matching - is that when we do propensity scores we're typically most concerned about unobservable characteristics. We wave our hands at that and say that the observables are a good proxy for the unobservables so its like we've matched on unobservables too. No one really believes that deep down, but people go along with it anyway. My point was that this becomes a much bigger deal when we do what he proposes - consider treatment effects separately across the distribution of outcomes, rather than just looking at average treatment effects. If the unobservable characteristic is correlated with the outcome (which it ought to be), then a lot of the heterogeneity in the effect size across the distribution of outcomes is going to be driven by unobservables. Taking an average treatment effect at least had the virtue of averaging that out too - you could imagine situations where looking across the outcome distribution would be a liability if unobservables were a substantial problem. He seemed to agree with me, although the professor pointed out that conditional treatment effects would also probably solve some of this.

I know this is all a little vague - I'm as much jotting this down so I can remember the issues that came up as anything else.

I have not really digested the tools he talked about (he didn't even get to talking about them in great detail). I think what's important is the principle that is motivating the use of those tools: how average treatment effects can sometimes be very misleading.