Health Studies Are Worthless to Those Who Care About Health (Blog)

[J1234]

Originally Posted by Chris Anthony

That'd be my job, actually. Full disclosure: I'm a data analyst working for a doctor at the School of Medicine at Johns Hopkins University. I'm the guy who runs the numbers for the studies you're dismissing, and it's in my personal and professional interest to make sure that those numbers are as accurate as they can possibly be, regardless of what conclusion they support. The funding for our studies comes from the National Institutes of Health, and we are not beholden to them to produce particular results or to study particular subjects.

Hey there, data analyst, Chris. I'm a data person as well. I agree with most of what you are saying, but have this to add:

Confirmation Bias: The numbers may be right, but they can be gathered to support a hypothesis, rather than test one. And you'd never know the difference.

Optimizing vs. Statisfying: Optimizing your health outcomes requires exposure to thousands of peer reviewed studies, integration and comparison of effect sizes, and shifting through various external and internal validity issues as well as issues surrounding statistical power, examining missing data patterns, construct validity of the independent and dependent variables, etc . . . and then, and only then, letting the cream rise to the top. This requires a level of critical thinking, detail orientation and methodological familiarity that most people either don't have, aren't capable of, don't have time for or aren't motivated to do. So the only option most people have is to statisfy: take what "seems good enough" and leave their particular outcomes to chance. They then build off this little bit of empiricism various inductive principles that define the workings of the world. The whole thing sort of goes like behavioral shaping via operant conditioning. But that's the best most people can do.

I mean, seriously, most people don't even read CR, before buying a car . . . they act on reputation or other such heuristics. Someone here, who's a self-proclaimed tech geek, didn't even do a head to head comparison of MP3 players before dumping money out for an ipod. Statisfying is expected, optimizing is exceptional--hence, the boldness of "Health Studies are Worthless to Those Who Care About Health."

Hawthorne Effect: Any change = a positive outcome. Therefore, everything works.

Black Box Syndrome: Put something in the box, another thing magically comes out of the box. Most people never ask what's inside the box that lets one thing go in and something else come out. Also most people don't understand how to find out what's in the box without opening the box. Some people would even argue that it's pointless thinking that there actually IS and inside of the box as it doesn't matter as long as the magical property keeps coming out.

Framing and schematics: There are 4892358729875... different ways to look at any given problem. Only 500 may actually shed light on relevant mediators and moderators and the rest may yield statistical significance due to their relationships with these mediators and moderators. From a theory POV, you can never really know what it is that you are looking at.


So yes, in the most roundabout way, for those who only care about outcomes "Health studies are worthless . . ."

[Inquisitive Raven]

Others have already addressed the accuracy of medical reporting in the mainstream media, so I'll address the issue of medical error.

I'm not going to argue with Wikipedia's figures since the estimates are all over the maps. I am going to point out that unless the study is about medical error, error is really irrelevant to the topic of the essay. Y'see the vast majority of medical error is traceable to one of two things: overextended medical personnel (e.g. doctors working excessively long shifts, nursing staff inadequate to cover the floor), and communication failures (e.g. misdispensed meds due to doctors' poor handwriting). Neither of these really has anything to with most health studies.

They do generally have simple fixes. (Note: simple doesn't necessarily mean easy.) As an example, some hospitals have implemented systems where the doctor types the prescription into an automated system which then dispenses the meds. While this doesn't eliminate error, it has reduced it drastically. Other fixes included putting doctors on shorter shifts so they can get sleep, and using automated diagnostic tools where accurate diagnosis can be reduced to a few factors.

You can find some good information on medical error in the book Complications: A Surgeon's Notes on an Imperfect Science by Atul Gawande.

[Brutha]

And, honestly, if two studies come out, one that says that Thing A is good for you and one that says that Thing A is bad for you, they're clearly not doing the same thing. In fact, the only similarities that can be assumed from that information are that a) they're both studies, and b) they're both looking at Thing A. Since they're drawing such wildly different conclusions, the reader has to assume that they're approaching the problem from different directions.

In addition most of the time you, as a customer approach the subject from a third perspective, "is this healthy for me"?
Even when the studies are made with the best intention they don't answer the question of the customer.
So why should you transfer the data of the study to make a conclusion about your problem?
On the other hand your body is built in a way that it can check whether something is healty for you and give you feedback through your intution.
Sure you should feed your brain with as many information as possible, but the information that you gain from you own body has much better quality then the information you get from studies over very small problem (every study needs to ignore a lot of factors that have an effect on your health, you don't know if you may get health benifit A but lose benifit B when the study is only about A).

So test different health factors and use your intution to judge them.

[del]

Hi, I was looking at this material, and thought I’d post an answer to my niece, who asked the following question.

Fluoride has been tied to bone cancer, lower IQs, and osteoporosis. So why is it still being added to your water?
By Timothy Gower, Prevention magazine

My answer to her query is both specific, and also general as to the scientific method.
I thought some of the posts were quite enlightening, like that of Akashic Librarian/Chris Anthony.

HI, (to my niece) I understand your curiosity. As a lifelong scientist I will try to give you a partial reply; but don’t have a lot of time to look up related articles, etc. at this time.

There is one study apparently showing an increase in bone cancer in male children; though another similar study did not show this propensity. There are 400 cases of osteosarcoma in children diagnosed each year; 250 are males. The author admits that her study isn't all that large, and has some limitations, and further research is needed to confirm such a connection. I never really believe a first test, until scientists from another unrelated laboratory or clinical patient group confirm it. It is probably worthwhile to run another series.

It is time-consuming to research these reports in the literature properly (reading the entire original article; reading the author's discussion, reading comments by other respected researchers in the field, looking at the statistics, and looking at the statistical analysis to see which methods of statistical analysis they used; and trying to understand which should have been used, and trying to understand the particular analysis, etc.).

Did they use the one-sample t-test; one-sample median, binomial, Chi-square goodness of-fit, chi- square test, Fisher's exact test, one-way ANOVA, Wilcoxon-Mann whitney test, Kruskal Wallis, McNemar, Friedman, repeated measures logistic regression, factorial ANOVA, simple linear regression, non-parametric correlation, analysis of covariance, discriminant analysis, multivariate multiple linear regression, etc. I never used all of these myself. Some are appropriate in one situation and ludicrous in another.

Sometimes the answer seems obvious without any arcane mathematics that if 200 fellows go aloft in airplanes and parachute; and all those who used Brand A parachutes survived, and all who used Brand X parachutes were killed when they impacted the ground; that Brand X was at fault.

But, what if there were other variables, perhaps inadvertently unreported?
What if all the people who used Brand X were untrained, had average IQ's of 44, had osteogenesis imperfecta, and went up with an idiotic daredevil pilot, and jumped during high winds when over a lake or mountain peaks?

Whereas the guys using Brand A were all reasonably intelligent (personally have always wondered about the intelligence of skydivers - but never mind) experienced skydivers who jumped over open prairie or a huge expanse of sand on a pleasant windless day.

Sometimes the repeated experiment by others simply never gives the results of the original group, due to some inadvertent mistake or change in the test environment, or due to stupidity or malfeasance, or whatever. Remember the cold fusion test/debacle?

To my knowledge Fluoridation is probably safe if the dose is reasonable. Very high doses found NATURALLY in the groundwater can cause brownish mottling of the teeth (like my classmate, Marguerite). Extreme doses will show endemic fluorosis of the bones, with areas of increased density, changes in the growth "arrest" lines, some bone formation in soft tissues, etc. Too high a dose of even distilled pure water ingested over too short a time will kill. Too much Vitamin A will make your skin slough, and more will kill you.

When our boys went to England to fight the second war; they were shocked to find that the English who didn't fluoridate their water, bother too much with good dental practices, and were fond of sweets: had mostly rotted teeth. Even fairly young people. But, you could bet your month's pay that your English girlfriend's parents had rotted away half their teeth.

People a couple centuries ago believed that tomatoes were poison, and would not eat them; and a very long time ago that cats were good luck; and later that they were bad luck (especially black ones crossing your path).

A number of people believe they are being tracked by black helicopters; that meat won't rot if you put it under a wire pyramid of a certain size and proportions, and believe all the scariest stories about the bermuda triangle and Area 57, that Aluminum cooking utensils will surely kill you, that Elvis is still alive, that vegetarianism is the only way to live, that high colonic enemas are necessary for all mammals to live a healthy life, etc. No one can change their minds by showing them the best, most exhaustive heaping up of facts to refute their beliefs.

Beliefs are beliefs. Some believe in Muhammad and Allah, some don't. Some believe wrestling on TV is all real and on the up and up. Some believe minds can bend spoons; some believe spoons can bend minds.

I have told vegetarians that I will help them set up large groups of hunters and safaris to kill all the meat-eating animals in the world, and so get rid of all the evil meat-eating animals to make the world a better place, and make all vegetarians happy at last.

I find it a pain in the neck to argue a complex scientific logical question with someone on a blog who doesn't know what a peer review is, or what a double - blind test is, what Koch's Postulates mean, or have any idea how to prove a scientific principle. Archimedes and Hippocrates would be brought to tears to read some of the blogs.

I will only make so much effort to change minds. Right now I have some pressing matters and can't devote a lot of time to this. If I were one of the many sham scam artists; I would simply state a bunch of unsupported "facts" as the absolute truth, and have done with it.

Must get at some checks, property tax, investments, etc.

As I occasionally ended my lectures:
“I have just 2 more articles to read, and then I will know it all.”

Del