Dr. Michael Eades has a post entitled “Baboon Business” that shows how to dissect a scientific study. Dr. Eades says:
I’ve read the paper that is the topic of today’s post from beginning to end five times. Not because it is a brilliant, enlightening paper, but because I found it so worthless I kept thinking there was something I was missing. If this paper had been published in the Journal of the American Dietetic Association or some other third tier journal I wouldn’t have thought so much about it. Were it published in a second tier journal such as Metabolism, I would have wondered a little more. But it was published in the October issue of the venerable American Journal of Clinical Nutrition, a first tier journal for sure, and, arguably, the most prestigious nutritional journal in the world. I’ve decided to use this paper entitled “Arterial endothelial dysfunction in baboons fed a high-cholesterol, high-fat diet” to demonstrate how totally meaningless research can find its way to the best of journals thanks to a built-in bias among the “peers” who are reviewing such research and to show how to interpret a scientific/medical article. Critically reading a scientific paper is a piece of detective work. One has to discover motives, obfuscations, biases, and sloppy work and put it all together to get the real picture, not just the picture the author of the paper wants to be seen. Just like a good detective who assumes everyone is lying until stories are corroborated, so it is with the scientific literature. One must always corroborate, probe, compare and dig deeply because almost nothing is as it appears on the surface. As Sherlock Holmes says, “These are very deep waters.” In the case of the study we will in due course explore, the waters are very deep indeed.This all applies to the recent NIH study.