tag:blogger.com,1999:blog-1740670447258719504.post7427504678215840674..comments2024-03-27T03:00:27.024-04:00Comments on Facts & other stubborn things: Epic Megan McArdle failEvanhttp://www.blogger.com/profile/12259004160963531720noreply@blogger.comBlogger17125tag:blogger.com,1999:blog-1740670447258719504.post-32190819179179880532013-02-08T23:03:58.437-05:002013-02-08T23:03:58.437-05:00"It's a freaking Pareto distribution"..."It's a freaking Pareto distribution"<br /><br />No it isn't. It's a lognormal distribution, a gamma distribution or a Nakagami-M distribution, maybe even a Rayleigh distribution. But, not a Pareto distribution, though it may follow that model for a part of the curve.Currenthttps://www.blogger.com/profile/08645195276844244481noreply@blogger.comtag:blogger.com,1999:blog-1740670447258719504.post-48051435049310396672013-02-08T20:46:56.112-05:002013-02-08T20:46:56.112-05:00And... I just realized I've been taken in by m...And... I just realized I've been taken in by multi-correlation. These are the brackets: 15, 10, 10, 15, 25, 25, 50, 50. Notice anything? Yeap, they are negatively correlated with the income distribution.PrometheeFeuhttp://prometheefeu.wordpress.comnoreply@blogger.comtag:blogger.com,1999:blog-1740670447258719504.post-67970925809680442782013-02-08T20:41:45.142-05:002013-02-08T20:41:45.142-05:00The graphs are so misleading as to be useless. I c...The graphs are so misleading as to be useless. I calculated the Pearson coefficient of the size of the bar vs the width of the bracket. A full 50% of the variation in the height of the bar is explained by the bunching of the data. I don't know why the census bureau would publish that kind of stuff. This is really a GIGO sort of situation...PrometheeFeuhttp://prometheefeu.wordpress.comnoreply@blogger.comtag:blogger.com,1999:blog-1740670447258719504.post-80525711661767144502013-02-08T19:16:57.166-05:002013-02-08T19:16:57.166-05:00One should realize McArdle's career is based o...One should realize McArdle's career is based on being a knave and injecting foolishness into public discourse. She is an awful person. -- RobertAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-1740670447258719504.post-41231255485964789492013-02-08T17:10:05.986-05:002013-02-08T17:10:05.986-05:00You also may have been confused by my use fo the t...You also may have been confused by my use fo the term "skewed bell curve", which is an accurate visual description but of course could be taken to imply that I agree with his belief that income should somehow follow a normal distribution.Janehttps://www.blogger.com/profile/10085770123311419448noreply@blogger.comtag:blogger.com,1999:blog-1740670447258719504.post-59684295171476466832013-02-08T17:07:34.885-05:002013-02-08T17:07:34.885-05:00Ahh, your 4:57 pm post makes clear the source of o...Ahh, your 4:57 pm post makes clear the source of our misunderstanding.<br /><br />The bit about climbing back up above their leftward neighbors does not refer to the graph *I* made; it refers to the graph that *Jon Evans was looking at*. Here's what I wrote, to review, with the relevant qualifier set off with asterixes:<br /><br />"Jon Evans is right about one thing *in the chart he linked*: income does not, quite, follow the skewed bell curve that you'd expect. Look at the bars around nice, round income numbers like $50,000 or $100,000. Suddenly, instead of falling slightly as you'd expect, they climb back above their leftward neighbors. Someone is rounding off to 5's and 10's--either the bosses who set salaries, or the folks who report their incomes. Probably both. So there is an interesting statistical anomaly here. But it's not a result of rising American income equality. It's a product of an entirely different historical accident: the fact that human hands have five fingers."<br /><br />I'm aware that in the graph *I* made, the $50,000 bracket climbs back up above its leftward neighbor because it's bunched (and also because the brackets are generally wider and represent something slightly different from the data in the Wikipedia graph.) But that has nothing to do with the passage above. And the differing width of the income brackets doesn't matter for the purpose that I used those graphs for--which is to point out that *over time*, we do not see the substantial upward and leftward movement that Jon Evans seems to believe is happening, and evidenced in the chart of 2010 income inequality.<br /><br />The point about the clusters around $50,000 and so forth in his graph was sort of a throwaway--not a criticism of him, just the sort of oddity that I like to point out to my readers. At any rate, you missed the modifying clause and read me incorrectly, which explains why we have been so confused about each others' responses.Janehttps://www.blogger.com/profile/10085770123311419448noreply@blogger.comtag:blogger.com,1999:blog-1740670447258719504.post-70307934040383459012013-02-08T17:05:39.902-05:002013-02-08T17:05:39.902-05:00I know what he wrote, Megan. Why do you think it&#...I know what he wrote, Megan. Why do you think it's more likely that the graph was used to support "It’s moving up and to the left" instead of what I would have thought was the more obvious "Oh, how naïve my teachers were."<br /><br />That's my question. I know what he wrote.<br /><br />I would have thought it was obvious that "It’s moving up and to the left" is supported by the reams of research suggesting that that have been big news for years now.<br /><br />Certainly we can't just declare that every sentence in an article is derived from the one graph in the article!<br /><br />So do you have a reason to think it is? I don't see one. I think you're being a bit of a bully to Jon and making some big mistakes yourself.Daniel Kuehnhttp://www.factsandotherstubbornthings.blogspot.comnoreply@blogger.comtag:blogger.com,1999:blog-1740670447258719504.post-92094125796511751492013-02-08T16:59:38.075-05:002013-02-08T16:59:38.075-05:00As for your other question, I think that Jon Evans...As for your other question, I think that Jon Evans "is confused and thinks that a static graph is evidence of a middle class being hollowed out" and that he is citing this graph as evidence for this proposition because he featured the graph at the beginning of his post on how robots are stealing all the middle class jobs, and then wrote: <br /><br />"When I was growing up in Canada, I was taught that income distribution should and did look like a bell curve, with the middle class being the bulge in the middle. Oh, how naïve my teachers were. This is how income distribution looks in America today . . . <br /><br />That big bulge up above? <b>It’s moving up and to the left</b>. America is well on the way <b>towards having a small, highly skilled and/or highly fortunate elite, with lucrative jobs; a vast underclass with casual, occasional, minimum-wage service work, if they’re lucky; and very little in between.</b><br /><br />But it won’t be 19th century capitalism redux, there’ll be no place for neo-Marxism. That underclass won’t control the means of production. They’ll simply be irrelevant."<br /><br />The graph doesn't show anything of the sort. Now, maybe you think he's not taking the graph as evidence. But he doesn't offer any <i>other</i> evidence for that proposition, and if this graph is not supposed to be evidence, then what is it doing there at all? <br /><br />If Evans was using it simply as an illustration, it's incredibly misleading. That graph does not show the distortion of the normal bell curve that income "should" have; it shows a fairly standard income distribution, with some compression of the brackets at one end. As I think you yourself are saying. But at points it seems as if you're attributing Evans' erroneous belief to me, though it's not clear and I may be mistaken about that.Janehttps://www.blogger.com/profile/10085770123311419448noreply@blogger.comtag:blogger.com,1999:blog-1740670447258719504.post-89857753693411673412013-02-08T16:57:38.882-05:002013-02-08T16:57:38.882-05:00Jane -
No one has disputed your point that people ...Jane -<br />No one has disputed your point that people are more likely to report 50,000 as opposed to 49,500. I'm not sure why you keep coming back to that point.<br /><br />You want to know why your graph peaks in the middle - it's because the middle bar is for a bracket $25,000 wide and the bracket to the immediate left of that is $15,000 wide and the bracket to the left of that is $10,000 wide.<br /><br />And with brackets many multiples as wide as Jon's brackets, of course you're not going to pick up the shifts in income distribution that have been so carefully documented.Daniel Kuehnhttp://www.factsandotherstubbornthings.blogspot.comnoreply@blogger.comtag:blogger.com,1999:blog-1740670447258719504.post-47282703051941223682013-02-08T16:51:31.559-05:002013-02-08T16:51:31.559-05:00Jon doesn't have data. Jon took a graph off o...Jon doesn't have data. Jon took a graph off of wikipedia. That's the graph we're talking about. (I did get the underlying data, and the graph represents it accurately).<br /><br />The columns I'm talking about are the ones at $50,000; $60,000; $70,0000; $100,000; and $150,000. Knowing what I do about how the data is collected (household survey), I'm betting that even if we graphed a line, we would see discontinuities in those places. <br /><br />These are all bands of $5,000 worth of income, with the exception of the last two. I'm not sure why you're arguing that the increase in those lines--and not, say $120,000 or $30,000--is due to the size of the income brackets. They're all $5,000 bands, the same as the rest of the brackets, which mostly step downward in more-or less the way we'd expect. (It may help if you look at the original graph, which you can get by clicking on the graph in my post. Or by going here: http://en.wikipedia.org/wiki/File:Distribution_of_Annual_Household_Income_in_the_United_States.png<br /><br />Surely you must have come across this in your studies; it seems like every time I talk to people who work with these sorts of surveys, they chew my ear off about rounding in self-reported data.Janehttps://www.blogger.com/profile/10085770123311419448noreply@blogger.comtag:blogger.com,1999:blog-1740670447258719504.post-69632155196457215802013-02-08T16:39:05.747-05:002013-02-08T16:39:05.747-05:00Right, log income is of course going to have a dif...Right, log income is of course going to have a different distribution (and nice statistical properties too). But we live and trade in a world of levels :)Daniel Kuehnhttp://www.factsandotherstubbornthings.blogspot.comnoreply@blogger.comtag:blogger.com,1999:blog-1740670447258719504.post-81996691240269477652013-02-08T16:38:14.652-05:002013-02-08T16:38:14.652-05:00re: "The climb back up above their leftward n...re: <i>"The climb back up above their leftward neighbors is not an artifact of the fact that the brackets are wacky (except among the last two). It's presumably an artifact of the fact that people tend to round to numbers like $50,000 or $100,000, a phenomenon that I'm sure you're familiar with."</i><br /><br />No, it's EXACTLY because you're presenting bunched data. Get Jon's data, aggregate them into the bunched intervals you present, and you'll see the same effect.<br /><br />If you double the number of people in a bracket relative to another bracket, low and behold the bar is going to be twice as high and you're going to see a peak closer to the middle.Daniel Kuehnhttp://www.factsandotherstubbornthings.blogspot.comnoreply@blogger.comtag:blogger.com,1999:blog-1740670447258719504.post-56172277395926366952013-02-08T16:36:19.152-05:002013-02-08T16:36:19.152-05:00I am happy to do a post going into more detail in ...I am happy to do a post going into more detail in a little while.<br /><br />Could you let me know why you think he is confused and thinks that a static graph is evidence of a middle class being hollowed out? I don't see anywhere in his post where he cites that as evidence.<br /><br />My concern, in a nutshell, is that you complained about his presentation of a graph with bunching, asserting he drew misleading conclusions from it (it's not clear to me at all that he did - he seemed aware that those last two bars were bunched), and then you went ahead and presented an even more bunched graphic without letting your readers know that explicitly and drawing conclusions like "instead of falling slightly as you'd expect, they climb back above their leftward neighbors" that <i>depend on that misunderstanding.</i><br /><br />The brackets seem to me to be far too wide to detect a shift in inequality. The fact that most other indicators point in the direction of inequality makes me think his data was much better and much better interpreted (of course we've got growing inequality for lots of reasons - some benign, some not).Daniel Kuehnhttp://www.factsandotherstubbornthings.blogspot.comnoreply@blogger.comtag:blogger.com,1999:blog-1740670447258719504.post-61703084332888895782013-02-08T16:27:10.177-05:002013-02-08T16:27:10.177-05:00Gee, I never expected a non-skewed income distribu...Gee, I never expected a non-skewed income distribution. I don't know why anyone would expect one. The first graph looks familiar.<br /><br />And why aren't we looking at the log of income? The second set of graphs is closer to that, with unequal ranges. Something that I don't see mentioned in the article as quoted here.Minnoreply@blogger.comtag:blogger.com,1999:blog-1740670447258719504.post-36691837064225886832013-02-08T16:21:38.474-05:002013-02-08T16:21:38.474-05:00I am Megan--this is my old blogger account, which ...I am Megan--this is my old blogger account, which I still use to comment.<br /><br />Mr. Kuehn, even if the brackets are bunched at the right, that wouldn't cover up the "upward and leftward" movement which is the main subject of the chart I created. Unless we're talking about an "upward and leftward movement" occurring entirely within the top two brackets, in which case who cares?<br /><br />The climb back up above their leftward neighbors is not an artifact of the fact that the brackets are wacky (except among the last two). It's presumably an artifact of the fact that people tend to round to numbers like $50,000 or $100,000, a phenomenon that I'm sure you're familiar with. The brackets that I was referring to were the ones on the left hand side--the ones that are all the same size ($5,000 of income).<br /><br />I'm not sure exactly what your complaint is. Is it that the chart I created from census data is "misleading" in the way that I said Jon Evans' graph was? But I didn't say Evans' graph was somehow itself substantively misleading; I said that you couldn't infer the hollowing out of the middle class from it, as Jon Evans did. Other people may have slapped labels like "how to lie with statistics" on it, but I didn't say he was lying; I said that Evans was inferring too much.<br /><br />Is it that I was "unaware" of the bunching? But your evidence is that I didn't mention it in an argument where I don't see that it makes any difference.<br /><br />Is it that I bunched the income? In that case, your complaint is with the census bureau, not me (their data was also the source of the WIkipedia graph that Evans is looking at).<br /><br />If you could clarify, perhaps we'd have a more productive conversation. I suspect that you reacted to other peoples' glosses on my post as "Jon Evans is being misleading" and then were incensed when you thought you saw me being misleading in exactly the same way. But the actual message of my post was "Jon Evans is confused" and I'm afraid that I didn't make the same mistake he did--that of thinking that a static graph of one year's income inequality was evidence that the middle class is being hollowed out as all the jobs disappear and the income flows to capital.Janehttps://www.blogger.com/profile/10085770123311419448noreply@blogger.comtag:blogger.com,1999:blog-1740670447258719504.post-31264554275423671362013-02-08T16:08:30.761-05:002013-02-08T16:08:30.761-05:00Are you Megan or Jane?
Look, for one thing the au...Are you Megan or Jane?<br /><br />Look, for one thing the author of the piece mentioned nothing about those last two bars. I assume he was well aware of your point because it is labeled pretty clearly on the graphic itself.<br /><br />You, on the other hand, seemed completely unaware of the bunching in the graph you provided as a counter-argument.<br /><br />You would not have written this: <i>"Meanwhile, there's no evidence that the middle class is getting poorer. Here's the Census's Households by Total Money Income table, summarized for 1967 through 2011, which shows the percent of households in each bracket. (Figures are in inflation-adjusted dollars) As you can see, there's no big movement "up and to the left", other than a slight expected income downshift during the Great Recession.<br /> <br />However, Jon Evans is right about one thing in the chart he linked: income does not, quite, follow the skewed bell curve that you'd expect. Look at the bars around nice, round income numbers like $50,000 or $100,000. Suddenly, instead of falling slightly as you'd expect, they climb back above their leftward neighbors. Someone is rounding off to 5's and 10's--either the bosses who set salaries, or the folks who report their incomes. Probably both. So there is an interesting statistical anomaly here. But it's not a result of rising American income equality. It's a product of an entirely different historical accident: the fact that human hands have five fingers."</i><br /><br />If you were aware of the bunching in your own graph. The only reason why you see the "climb back up above their leftward neighbors" is that the income brackets are wacky.<br /><br />If you knew this all along, you ought to retract those last two paragraphs.Daniel Kuehnhttp://www.factsandotherstubbornthings.blogspot.comnoreply@blogger.comtag:blogger.com,1999:blog-1740670447258719504.post-6042602296530123172013-02-08T14:52:46.809-05:002013-02-08T14:52:46.809-05:00Mr. Kuehn, "Department of Awful Statistics&qu...Mr. Kuehn, "Department of Awful Statistics" is a long running feature on my blog, in which I point out statistical mangling.<br /><br />In this case, the mangling was the statement by Jon Evans of Techcrunch that the income distribution, normally a normal curve, was bunching up to the left while the rich get richer. As you yourself point out, income is not "supposed" to be a normal distribution. <br /><br />The reason that both his and my graph bunch as they do is that this is how the census presents the data. I don't think there is anything wrong with bunching the top of the distribution to the right--in fact, I think it's necessary, since the alternative is to have a bunch of barely visible bars trailing out about a mile to the right. The problem is with drawing conclusions from the bunching. Or thinking that this somehow represents a huge departure from historical precedent.<br /> <br />I thought I laid all this out pretty clearly in the post. I'm sorry that I failed to do so clearly enough for you to understand what I wrote.Janehttps://www.blogger.com/profile/10085770123311419448noreply@blogger.com