Even when you've got a sample of millions to draw on. This article describes how doctors are essentially making educated guesses when they offer prognoses, particularly as their ability to treat illnesses has improved. Doctors have a lot of data to work with, and a lot of variation in the data to extract estimates from. But it's still hard for them.
Macroeconomists, on the other hand, are working with very small samples which makes prediction harder and makes inaccurate predictions more glaring (everyone's only looking at one time series after all). None of this is surprising - we're dealing with complex systems in both cases. This isn't planetary motion, people.
And yet for some reason these experts are often judged by their predictions. But we shouldn't ever turn to experts on complex systems for prediction. Expertise isn't about prediction, it's about explanation. A doctor may be bad at prognosis, but they are quite good at explanation (and thus figuring out workable solutions). That's why we should keep them around. Not because we expect them to have a crystal ball.
Friday, January 20, 2012
Subscribe to:
Post Comments (Atom)
Posts
The problem with explanations is that they are often not falsifiable. So you have to demand at least one prediction: "What will show that your solution worked?"
ReplyDelete"Macroeconomists, on the other hand, are working with very small samples"
ReplyDeleteYeah, but economists have a cell's eye view. I bet doctors would kill for that.
It is not just hard, it is impossible.
ReplyDeleteMy theory is the set of smart, insightful, reasonable, doctors/economists who often make accurate predictions will contain a subset who guessed right most of the time. Is that end of the bell curve smarter, or luckier? I would argue that it is impossible to tell. With a large sample group, someone is going to flip six heads in a row. That smart person will have lots of evidence to support the idea that they can flip a coin to heads at will.