Thursday, May 23, 2013

Liars, Damn Liars... and People Who Even Lie About Statistics

So, courtesy of a friend this morning, I was linked to this little gem: Tattoos and Other Easy Ways to Ruin Your Body.

Where to begin? The mainsplaining lead-off?

Women, let me tell you something that your friends and many guys will not.

What follows is a lengthy tirade that manages to hit every square in misogynist twit bingo. The purity myth (in several iterations!). "Universal anecdotal evidence" (I kid you not). Generalizing for all men as if he speaks for his entire gender. Appealing to a book from 1933 as conclusive evidence that tattoos have "always" been the mark of prostitution (many, many cultural histories say otherwise, but when has that ever stopped someone strapped for evidence?). The appeal to conservatism as "the new hip", right down to using the terms "hip", and "with it". The use of the terms "alpha" and "beta", despite the fact that the purported roles are based on junk science.

This, however, was just the appetizer. The main course of ignorance came in the "purely statistical analysis" he posted later: Tattoos 2: Electric Boogaloo.

So, he leads off:

Those that apparently didn’t care for my “opinionating” will be happy to note this one is pure statistics. Here is graph one

Now, notwithstanding that the first included item is a chart, not a graph, there's nothing wrong with it. It's drawn from the Annals of Epidemiology, a respectable journal with an impact factor of 3.215. However. The caption in the journal for the chart in question contained the following:

... 3% of women who reported one sex partner or none reported being tattooed compared to 30% of women with 11 or more lifetime partners.
But our twit author follows that up with a caption of his own:
Statistic 1: Less than 3.3% of women with tattoos have had under 1 lifetime partner or less. On the other end of the spectrum a solid 33% have had over 11. This doesn’t look like an even distribution.

That 3% of women with 1 or fewer sex partners have tattoos does not mean that only 3% of women with tattoos have one or fewer sex partners! For the purposes of illustration, imagine a population where every woman has had one or fewer sex partners (probably not a very happy population!). 3% of these women have tattoos. In that situation, 3% of women with 1 or fewer sex partners have tattoos, but 100% of women with tattoos have 1 or fewer sex partners! This is not the case in the study cited, but should amply illustrate why the author's statement is a glaring statistical error. The same is true for his statement regarding the 11+ crowd.

There's a comparison chart posted for men and a false assumption that the reading audience shares his opinion of what deviancy is, but we'll skip over that in pursuit of the promised "pure statistics".

... well? Where are they?

Turns out, all the remaining "statistics" are drawn from graphs which, in turn, are drawn from these two blogposts.

The first post suggests that the graph has been included from a 1991 study, but given that the graph includes a misspelling, I am inclined to think it is of the blogger's own creation. I will track down the citation later (at least this one was well-cited), but while the language of the blog post is couched to suggest that the graph was drawn from the study, it is never explicitly stated to be. Somewhat suspect.

The other graphs, however, are extremely suspect - a guest post by a blogger known only as "INTREPID", who indicates that his data is drawn from the 2002 and 2006 US National Surveys of Family Growth, but not which datasets are being graphed, or any proper citation.

In both cases, our twit includes the graphics with a sense of "so there!", without any concrete citations to work from. These are the "pure statistics" from which he's developed his considered opinion.

(As an aside, I find it interesting that the twit doesn't include Chart 3 from the second blog post, which indicates that past a certain threshold, women with more premarital partners have _more_ stable marriages - I can only speculate that it wasn't included as part of his oh-so-careful statistical analysis because it didn't match his predetermined conclusion.)

After restating the statistical error I indicated above, our twit concludes (emphasis mine, for the sake of hilarity):

Any and all anecdotal objections are made in the face of these statistics and facts, all of which were gathered professionally in double-blinded and rigorously peer-reviewed studies. Ignore this work at your own peril.

I leave Mr. Karamazov with the words of the inestimable Terry McGinnis:

You make me laugh. But only 'cause I think you're kinda pathetic.

No comments:

Post a Comment