Book Rec: How To Lie With Statistics

[The Rabbi]

How To Lie with Statistics, by Darrel Huff, with illustrations.

I saw mention of this book in another book of investment classics. This is an old book, written in 1954, but the examples are just as good today.

In it, he talks about ways statistics are mangled and manipulated to make them say what the author wants. Among those methods are, self-selecting samples, comparisons of various sorts, shifting definitions of "average" and graphs.

The one on average was a real eye opener. There are actually three things called average:

-The Median: The halfway point between the two highest and lowest points.

-The Mean: Generally what we think of as average, the total number divided by the number of items, e.g. the average of 2,5, and 8 is 5.

-The Mode: the largest occurance of anything in the sample.

And conservatives arent above using this kind of stuff either. One article I saw on Somalia, making it out to be a Libertarian Paradise, cited the stat that the country has more doctors now per 1000 population than it had under a central gov't.

THis got me thinking. The country has no medical school, so it isnt graduating doctors. It isnt a vacation hot spot, so doctors arent moving there.

Two things could account for it: 1) The situation is so bad, doctors are coming to volunteer their services; 2) The population has shrunk so a steady number of doctors would actually look larger next to a smaller population. I havent delved into it to figure out which it is,probably a combination.

The debate on illegals is one where figures and statistics are so manipulated (on both sides) that they are probably meaningless by this point.

The book is fun to read an only about 150 pages. Go read it.

[Marswolf]

Mannnnnnnnnn, I read this book back in the 60s.

I think it is one of those that should be required reading for all students.

[fury2neon]

It is a very good book, I read it in college.

[TGO David]

Two things could account for it: 1) The situation is so bad, doctors are coming to volunteer their services; 2) The population has shrunk so a steady number of doctors would actually look larger next to a smaller population.

Or option 3) Since there are no medical schools, the "doctors" are just people with on the job training and no formal education because things are so incredibly bad over there that this is happening out of necessity.

[fury2neon]

Or option 3) Since there are no medical schools, the "doctors" are just people with on the job training and no formal education because things are so incredibly bad over there that this is happening out of necessity.

So they are kind of like Granny from the the Beverly Hillbillies...

She was a an MD, Mountain Doctor. So they would be DD's, Desert Doctors...

[The Rabbi]

Or option 3) Since there are no medical schools, the "doctors" are just people with on the job training and no formal education because things are so incredibly bad over there that this is happening out of necessity.

Yeah, you could redefine the term doctor to get what you wanted.

[Guest Disney]

Lawyer = Doctor in many places. So should we kill all the doctors?

[Guest Len]

Great book, highly recommended. I learned about this book in grad school and have used some of the examples in classes ever since. This sort of knowledge is exactly what is needed to improve information literacy, especially since usually the most "advanced" stuff the media will report is the margin of error.

In political polls, the following are of particular importance when looking at the numbers:

1) Sample size. Anything national should have a BARE minimum of 1000 full respondents. (Not 1000 calls, but 1000 people who actually answered all the questions.)

2) Sample selection. Several issues here:

a) is it a stratified random sample?

is it "registered voters," "citizens," "adults," or "likely voters?" (The latter is the most accurate.)

c) Is the sample derived nationally or locally or regionally, accounting for population density variations, voting patterns, etc. (This is where stratification is useful.)

d) How the sample is selected. Ie, via the phone book? Are cell phones included? (Cell phone users-those without landlines- tend to bias the sample toward the young & the Democrat) What about people without phones? Yada.

3) Method of administration. Avoid internet-based polls like the plague they are. Easy to manipulate and very biased towards 'net users (obviously.)

4) Survey questions. How the question is asked has an influence on the answers, and therefore could sway the stats in the end. I'm not just talking about the evil "push poll," but things much more subtle. Even the order of the answers (tendency to pick "A" since its first, etc.)

And this is only scratching the surface. Its quite possible the "four out of five dentists surveyed" actually work for whoever makes Trident...

[The Rabbi]

He had a great example of a toothpaste company that ran an ad saying their toothpaste showed 23% fewer cavities after use.

What happened was, in any group small enough you will find either more cavities, about the same or fewer. So they got groups of 12. Any group showing about the same or more cavities (since freakish accidents happen in samples that small) was ignored. When they finally got a group that showed improvement, that was the sample they advertised. Very clever.

btw, see today's news that voters on the internet went for Obama over Hillary? Talk about junk testing.

[Guest Len]

Exactly. Two great examples about the effects of sample size (toothpaste) and methodology (Obama).

He had a great example of a toothpaste company that ran an ad saying their toothpaste showed 23% fewer cavities after use.

What happened was, in any group small enough you will find either more cavities, about the same or fewer. So they got groups of 12. Any group showing about the same or more cavities (since freakish accidents happen in samples that small) was ignored. When they finally got a group that showed improvement, that was the sample they advertised. Very clever.

btw, see today's news that voters on the internet went for Obama over Hillary? Talk about junk testing.

[Guest Hyaloid]

I haven't read the book, but it is now on my list!

I run into this damned near daily. Any medical 'study' has to be picked apart with a fine tooth comb, and I would say the percentage in which there are fatal or highly questionable bias or study design flaws is very high, maybe 60-70%.

[The Rabbi]

Medical studies are very susceptible to this, especially the confusion between percentage and percentage points.

Let's say you do a study and in your study there is a link between coffee consumption and pancreatic cancer (this was actually done). So you can crow "drinking coffee makes you x% more likely to get pancreatic cancer."

Well, accoridng to something I saw somewhere, in order for this to be significant, it has to be considerably larger than what is usually recorded. But in the public mind the link: drink coffee----------->pancreatic cancer has already been made.

[Guest Len]

Very true. Another issue with medical studies (and many others as well) is the possible built-in bias of who is doing the study. (Ie, if a drug company is paying for the research that supports whether their drug works, or if the reserarchers have a stake in the results, beware.)

Its amazing what a little researcher bias can do for the results. Or maybe its just a remarkable coincidence...

[Guest Hyaloid]

Very true. Another issue with medical studies (and many others as well) is the possible built-in bias of who is doing the study. (Ie, if a drug company is paying for the research that supports whether their drug works, or if the reserarchers have a stake in the results, beware.)

Its amazing what a little researcher bias can do for the results. Or maybe its just a remarkable coincidence...

That's the biggie in medical research... dig deep enough and there's almost always a connection to whichever drug or treatment performed the best.

[Marswolf]

You really have to watch post hoc fallacies in research, especially it would seem in medical research.

99.9% of all Americans over the age of 20 who die of cancer have eaten mashed potatoes. Therefore, eating mashed potatoes causes cancer.

[The Rabbi]

Yup.

Also: "the mean age at which people die is 65. Therefore no one collects Social Security and its a useless program."

[analog_kidd]

I heard somewhere that 68% of all statistics are completely made up on the fly. No, wait, make that 54%.

[The Rabbi]

Actually he mentions in the book that anytime you see a decimal it is only there to add credibility. In reality statistics are never that precise.

[Guest Len]

I get various types of false cause errors in student persuasive speeches all the time. It is very hard to prove causality in anything but highly controlled situations or completely closed systems. Neither of these things is very common or very representative of the real world.

You really have to watch post hoc fallacies in research, especially it would seem in medical research.

99.9% of all Americans over the age of 20 who die of cancer have eaten mashed potatoes. Therefore, eating mashed potatoes causes cancer.

[Marswolf]

Len, back in my undergraduate days, between my running to classes trying to not get eaten by dinosaurs , I was asked to critique some research being done by one of the department chairmen at my university. I know...that's hard to believe, but actually is true. Long story.... Anyway, this one research project was a real mess. All sorts of dubious "conclusions" that didn't follow logically at all.

Gave me a healthy disrespect for scientific studies that I carry to this day.

[Guest Hyaloid]

Reminds me of one of my surgeon buddies who makes the following observation:

If you hear a doctor say, "In my experience", that means they have seen one case of a particular pathology or occurance... If they say, "in case after case", they have seen two cases... If they say, "In a series of patients", that means three cases.

Funny how many times healthcare providers use those terms, but I imagine it is applicable to just about anyone.

[jgrauman]

ETSU had a required class to be taken in your first two years of school. "Probability and Statistics" great class, they taught how to find, disect and conclude statistical information and the ways that information is manipulated

[Guest Old Chief]

I has been said that figures never lie; but liars figure. Manipulation of statistics fits into this category.

[Marswolf]

ETSU had a required class to be taken in your first two years of school. "Probability and Statistics" great class, they taught how to find, disect and conclude statistical information and the ways that information is manipulated

My wife and I think this should be taught in elementary school and followed up with an annual refresher course.

[fury2neon]

My wife and I think this should be taught in elementary school and followed up with an annual refresher course.

After seeing what they are passing off as math, science and history to my kids I think they start teaching basic math, science and history again. It seems the classes are more about cultural diversity than about the actual subjects. And don't get me started on this estimating stuff they teach kids before they teach them to add and subtract.