"Words ought to be a little wild, for they are the assault of thoughts on the unthinking" - JMK
- Corey Robin responds to his critics. I found the reaction to him quite disappointing. It'll be interesting to see the reaction to this.
- Articles like this (HT Ryan Murphy - who I've already argued extensively with about this on facebook) about how wine ratings are "random" really bug me. The randomness comes from the fact that, in a manner of speaking, the model is mis-specified. Unfortunately this stuff comes up a lot in the Journal of Wine Economics (which I used to follow but haven't lately), probably because the numbers are easily available and economists love using whatever data they can get their hands on. If you think wine ratings are a measure of some objective underlying measure of quality, then you're missing the point. There's nothing objective about this - wine is diverse and wine preferences vary considerably from person to person. Not only is it subjective, but it's also contextual (you are going to respond differently to wine X after tasting wine Y than if you tasted it before). That's why when you do the tastings they give you the whites first, then the reds, then the deserts. All a high wine rating tells you is "a guy that has a lot of experience drinking different wines liked this one time", and if it has a medal it means "a bunch of people that have a lot of experience drinking different wines liked this one time". Those are both very useful bits of information even if there is nothing objective about it, particularly for people who are new to wine. If you know what you like in wine, what's more useful of course is an actual review - a description by someone (that's not selling the wine!) of what characteristics to expect. Again, just like different authors of a book review of the same book you'll find some commonalities between reviews but because of the complexities of books and the different preferences of readers (and the literary and intellectual context of a given reader) book reviews can still vary widely between each other. If I were to say "well that means the book review isn't objective", I would be missing the point. If I were to say "the book review is random" I would be overstating my case. We still think these book reviews contain valuable information. So it is with wine rating, criticism, and reviews.
- I always figured fracking was more or less safe and an important way out of the dirty fuels dilemma, but this study nudges my priors just a wee bit on the drinking water pollution issue.
- "[Joan Robinson] wrote that Piero Sraffa used to tease her, saying that she ‘treated Marx as a little-known forerunner of Kalecki’ (J.Robinson 1966: vi)."
from Marcuzzo & Rosselli - Economists in Cambridge, p. 185.
Tuesday, June 25, 2013
Subscribe to:
Post Comments (Atom)
Posts
I always figured the side effects of fracking were unknown. :)
ReplyDeleteDaniel wrote: " If you think wine ratings are a measure of some objective underlying measure of quality, then you're missing the point. There's nothing objective about this - wine is diverse and wine preferences vary considerably from person to person. Not only is it subjective, but it's also contextual (you are going to respond differently to wine X after tasting wine Y than if you tasted it before). That's why when you do the tastings they give you the whites first, then the reds, then the deserts."
ReplyDeleteyes, this is all true. And I think people who blame wine tasting for not being scientific(ally objective) misunderstand the purpose and epistemology of wine tasting. But I'm not sure how many people actually fault wine tasting for not being scientific(cally objective). I think most people would readily admit that the mere fact that wine tasting is both contextual and subjective *in the ways that you describe above* is not enough reason to dismiss the opinions of experts. What *would* be a good reason to not take those opinions of experts very seriously is that if under the same (except for one factor) conditions they relatively consistently judge wines quite differently if the labels are different (if one is a very expensive wine and the other a cheap wine for example). Another good reason would be if there is hardly any consistency in what different experts or the same experts at different times (but under very similar conditions) say about the same wine.
And it's experiments that suggest / show that this sort of stuff is actually very common (in the sense that when a controlled or natural experiment takes place, these are the typical results) that undermine the credibility of expert wine judges.
"All a high wine rating tells you is "a guy that has a lot of experience drinking different wines liked this one time", and if it has a medal it means "a bunch of people that have a lot of experience drinking different wines liked this one time"."
So far, so good. Sort of by definition (if one replaces 'All of' by 'What') it is true that "All a high wine rating tells you is "a guy that has a lot of experience drinking different wines liked this one time", and if it has a medal it means "a bunch of people that have a lot of experience drinking different wines liked this one time." But what one assumes is that this bit of information allows you to infer other things about said wines. And this inferential power is exactly what is undermined by the results of the experiments described in the articles.
"Those are both very useful bits of information even if there is nothing objective about it, particularly for people who are new to wine.""
But these are only very useful bits of information if one in fact is reasonably allowed to infer more things from those facts, and this is exactly what is problematic as I described above.
re: "What *would* be a good reason to not take those opinions of experts very seriously is that if under the same (except for one factor) conditions they relatively consistently judge wines quite differently if the labels are different (if one is a very expensive wine and the other a cheap wine for example)."
DeleteYes but this problem even plagues scientists that are not questioned on this count.
Scientists will treat papers by a well known author vs. papers by a patent clerk differently just as surely as wine critics will judge bottles they are told are expensive or cheap differently.
That's why we do blind reviews and blind tastings.
re: "But what one assumes is that this bit of information allows you to infer other things about said wines. And this inferential power is exactly what is undermined by the results of the experiments described in the articles."
DeleteAgree with the first sentence, disagree with the second (unless by "is undermined" you mean "is maybe only partially undermined", but if you had an appropriate view of wine critics in the first place I think even that would probably be unnecessary).
Daniel wrote: "Yes but this problem even plagues scientists that are not questioned on this count.
DeleteScientists will treat papers by a well known author vs. papers by a patent clerk differently just as surely as wine critics will judge bottles they are told are expensive or cheap differently.
That's why we do blind reviews and blind tastings."
2 things:
1. blind reviews are, to my knowledge, way way more common than in wine tasting, and this may be indicative of the problems in wine tasting discussed above
2. although blind reviews in science typically do not lead to *uniform* conclusions (all blind reviewers having the exact same opinion of an article or other piece of scientific work) i think it's fair to say that the variety in views will be way way less than the variety in evaluations of wines tested blindly (but under essentially the same conditions). Under blind review conditions, different scientists are much more likely to agree / will agree to a much greater extent in their evaluations of an article than different wine tasters in their evaluations of a wine
My point is that I don't think one can use the argument that subjectivity and/or context also play a role in scientific judgements as an argument against concerns about the much much much greater role of context and/or subjectivity in wine tasting.
re: "although blind reviews in science typically do not lead to *uniform* conclusions (all blind reviewers having the exact same opinion of an article or other piece of scientific work) i think it's fair to say that the variety in views will be way way less than the variety in evaluations of wines tested blindly (but under essentially the same conditions)."
DeleteOf course! Because there are objective and timeless rules of good science (or at least we tell ourselves there are) where as "good wine" is subjective, varying, and contextual.
re: "My point is that I don't think one can use the argument that subjectivity and/or context also play a role in scientific judgements as an argument against concerns about the much much much greater role of context and/or subjectivity in wine tasting."
Wait, so now you are agreeing with the point I just made (before reading your last paragraph)?!?!? Well what's the problem if you agree it's subjective?
I don't understand what you mean here. Do you see a contradiction or other kind of inconsistency in what I wrote? If so, what is it?
Delete- I agree that wine tasting is subjective and context-dependent,
- I say that wine tasting is way way more subjective and context-dependent than science is
- (and I would add that I don't think subjectivity and context-dependency can or should be completely eliminated from science (let alone from wine tasting!)),
- that subjectivity and context-dependency in wine tasting is excessive (and of a more pernicious nature), so much so
- that blind reviews seem to show that there is little consistency (and note that consistency is wholly consistent with subjectivity and context dependence) between the evaluations by one and the same person at different times, and the evaluations of different experts at the same (or different) times,
- and that this means that the results from non-blind (or blind) wine tastings tell the public little if anything more about the features and/or quality of the wine than that the wine that won was judged more favorably by one or more judges than the other wines (the public does not know whether the fact that this wine won was the result of features of that wine or of chance or of some other aspect)
In sum, my objection to wine tasting is that there is a disproportionally large role for BS and a disproportionally small role for actual expertise (let alone actual expertise that may benefit the public) in the practice and profession. The profession claims way more expertise and knowledge than it actually possesses.
perhaos the following clarification is helpful: I think in science there is some ineliminable (and potentially useful epistemic) role for subjectivity and context-dependence. I think that although in wine tasting there is a much much greater role for subjectivity and context-dependence this does not necessarily make it impossible to gain any useful (for advicing wine consumers and producers) knowledge about certain taste-related features of wine. And I think that in practice the knowledge claims that are often / typically made by members of the wine tasting profession vastly exceed the actual knowledge that they can legitimately be said to have. In particular, they claim a much greater consensus in their evaluation than they actually have, and this discrepancy is demonstrated by controlled and natural experiments.
DeleteSubjectivity and context-dependence are potentially wholly compatible with widespread agreement. And there are good potential justifications for claiming that such widespread agreement actually amounts to (tacit or explicit) knowledge or expertise.
So in principal the subjectivity and context-dependence involved in wine tasting need not stand in the way of widespread agreement. But blind tests and other kinds of controlled or natural experiments demonstrate that there actually is very little agreement among wine experts, much much less agreement than what the profession actually claims. And at least to the extent that this supposed widespread agreement was thought to buttress the knowledge claims made by the profession, they actually have way less knowledge and expertise than they claim to have.