Always dig around in your data

I am much more excited about this restricted Dept. of Education data than I was when we first applied for it. I had no idea it had monthly longitudinal employment records for these people. I figured I'd be stuck with a couple employment variables every couple of years (for each wave of the survey).

Nope - monthly history.

The thing is this is the Dept. of Education, so they don't care about that. They used all that great data to create a couple summary variables and then stuck it all in a source file and buried it way down in a sub-folder. But I sniffed it out! If this were a Bureau of Labor Statistics dataset that would have been front and center (in fact it is front and center in things like the NLSY).

So the lesson is: always dig around in your data and get to know it. When you get started with some data treat it like a long-term relationship: a woman you're trying to woo, rather than a one night stand to satisfy your more immediate needs. I got that sort of relationship with the NLSY and have since repeatedly turned back to it because I knew the data like the back of my hand. Trying to develop that sort of relationship right now with the SIPP and the B&B.

Microdata is complicated and when people make investments in big surveys like this there's usually a lot of moving parts involved. There is always more to it than there first appears to be. Get to know it!