How to study studies
At Kitsap Medical Weight Loss we’re frequently asked what we think about this study or that.
At Kitsap Medical Weight Loss we’re frequently asked what we think about this study or that. As a society we’re certainly bombarded with opinions, if not always information. So, we’re going to give you some tips for separating scientific research from “junk science.” The best thing you can do is to read the actual study, not an abstract or a report by someone who skimmed the abstract. Then two important things to do are; look at the type of study, and who paid for it.
When it comes to studies on health and human nutrition, virtually everything is an epidemiological study, and those are the worst kind. Epidemiological studies are not experiments designed to find cause and effect, rather they are a very limited look at correlations that may or may not exist between two things. All they are good for is developing a hypothesis, and that on the basis of flawed, limited evidence.
To do a true study of nutrition and health you would have to lock your study subjects up and carefully control all aspects of their diet, activity, and environment. It would be incredibly expensive to do so with a statistically significant number of people over a statistically significant period of time. It would also be very difficult to get people to sign up for such a study.
Instead, nutritional studies use data collected from surveys where the subjects self-report what they have eaten, sometimes over very long periods of time going well back into the past. Assuming everyone in the study is reporting as truthfully as they can (which they don’t – some people report what they think they should report), people are known to forget things. Or underestimate. Or overestimate. From these types of questionnaire surveys, there is no way to know a subject’s true dietary intake, the true nutritional content of the intake reported, or any way to look at dietary patterns. This takes all the numbers out of context.
For example, The Oxford Red Meat and Cancer Study. The news reports you get on this declare with authority that eating a serving of beef, a slice of ham, a couple of sausages or slices of bacon four or more times a week will raise your risk of colon cancer by 20 %. Meaning – eating red meat and/or processed meat will cause cancer.
Here’s the full story: Researchers at Oxford University got a grant from Cancer Research UK (a group with a long history of anti-ketogenic bias) to purposely look for links between eating red meat and cancer. They harvested numbers from the UK Biobank, where they get medical records and a very limited survey for nutrition. The nutrition information was recorded once every three or four months, and simply asked for averages for all weeks except for more detail on the last 24 hours. This included data on over 475,000 people and stretched over almost six years.
Besides the obvious problems with self-reporting in this way, these isolated numbers don’t tell us if the meat consumed was grilled with spices, or smothered in a honey BBQ sauce. We don’t know if it was a grass fed ribeye steak or a hot dog. Based on this highly unreliable data, they found a correlation between an increase in red meat and processed meat consumption and a corresponding increase in the incidence of bowel cancer. Again, this does not show a cause and effect relationship, but the slant would have you believe it does.
To be fair, we have to give the folks at Oxford some wiggle room, because to give them a little credit, buried more than 20 pages into the actual report they admit that eating red meat does have positive aspects as well, like, for example, being by far the most efficient way to get the macronutrients we all need. They also report that relatively heavy drinking raises the chances of bowel cancer by 24%. Smoking raises the rate by 200%. It doesn’t report on sugar. Remember, the study doesn’t link any of these groups together, so there’s no way of knowing how many of those diagnosed with bowel cancer had more than one of these factors in play.
Even so, let’s say the ratio holds true, what exactly does that mean? Well, with the numbers they provided just over one-half of one percent of the people in the study were diagnosed with bowel cancer during the course of the study. A 20% increase in your chance of developing cancer would push the rate up to six-tenths of one percent, meaning if 1,000 people all went carnivore crazy, six would get cancer as opposed to five in 1,000 vegans.
Remember, you should read the whole study yourself to get the whole story. Also remember to follow the money. The study was funded by a group with an agenda. Is it really any wonder the cherry-picked data shows what they want it to show? The report acknowledges the researchers rejected some of the data at the far ends of the Bell Curve, figuring they weren’t “reliable.” What isn’t reported is how those ends were gerrymandered.
Shenanigans go on with these types of studies all the time. There’s a recent report from Cambridge University saying Coca-Cola funded “academic research” where they reserved the right to “quash studies” or “pressure researchers using the threat of termination,” in at least five cases in 2015-2016 alone.
If a report says, “researchers say” or “data suggests,” we suggest you take it with a grain of salt.
*Individual results may vary; not a guarantee.
We make it easy! Call us at (360) 626-1166 or request an appointment below.