What do you do when beliefs you thought you’d researched to death years ago are suddenly contradicted by fiendishly convincing studies? Do you adjust your beliefs? Or do you investigate further to confirm them?
Recently I participated in a discussion on Facebook about dairy consumption. I questioned whether it was healthy for humans to consume dairy (at least beyond infancy). I then produced a link to “7 Reasons Milk Is Bad For You” (Bustle, Rachel Krantz, 1/25/2016). I was then challenged by my Facebook friend Kevin Smith who produced his link, “Milk and dairy products: good or bad for human health? An assessment of the totality of scientific evidence” (Taylor & Francis Group, Tanja Kongerslev Thorning, Anne Raben, Tine Tholstrup, Sabita S. Soedamah-Muthu, Ian Givens and Arne Astrup, 11/22/2016).
Technically, neither article is a “study” in the classic sense. They both just contain links or references to actual studies. Neither article knows about the other. Thus there’s no point-counterpoint discussion going on (although there is some overlap on several points).
Rachel Krantz’s article did not appear in a scientific journal but a popular one that also contains articles about veganism, lifestyle choices, relationships, celebrities, etc.; however, she stripes her text with multiple hyperlinks to supporting studies.
The Taylor & Francis Group article is three times as long and is top-heavy with references. It has 114 footnotes, compared with less than half for the Bustle article.
Contradictory Views of the Same Subject
These two contenders widely contradict one another. For example on the issue of dairy products causing or exacerbating osteoporosis, T&FG says:
Osteoporosis has been described as a 'paediatric disease with geriatric consequences’ as low milk, and hence, low mineral intake during childhood and adolescence has been associated with significantly increased risk of osteoporotic fractures in middle and older age, particularly in women (49, 50).
Krantz references this article by Dr. Neil Barnard:
A study published by the American Medical Association in the Archives of Pediatric and Adolescent Medicine this year showed that active children who consume the largest quantities of milk have more bone fractures than those who consume less. This was not surprising. Prior studies show that milk consumption does not improve bone health or reduce the risk of osteoporosis and actually creates other health risks." (Physicians Committee for Responsible Medicine)
The two articles go on like this, one hard-hitting – and convincing – reference after another from sources that contradict each other whenever they intersect. So where’s the truth? Alas, hard to find. It would take weeks of surfing just to track down all the studies referenced. Only then could you take apart the assertions, one by one, and try to spot any cherry-picking of results, confirmation biases, and null hypotheses. Neither article evaluates information in the studies they reference: the control groups used, critical theories employed, data mining, ethnography, deviation, population size, focus groups, life histories, and a score of other arcane factors. There are people who make their livings analyzing these things! In short, whenever a study’s results are reproduced in any article you read, there is no way to tell if they’re trustworthy or not. Except if you read the original study. And you’re trained to spot irregularities.
Another Strategy in Order?
Why not find out more about the studies themselves? Do either of them directly address the results from their adversaries? Was there anything unethical or sloppy from the data-gatherers? Who benefits from the conclusions and how?
There is no sparring with adversarial research in Krantz’s article. No pro-dairy article was analyzed and criticized. (Too bad, spirited debate is always entertaining.) Same for the T&FG study. Both sides obsessively presented their points of view. Neither Krantz nor her magazine got back to me with questions I had, which I found rather crass. I felt ignored, I really did.
A Yellow Flag Awavin’
T&FG produced this disclaimer in its “Conflict of interest and funding” section at the end of the study.
Anne Raben is recipient of research funding from the Dairy Research Institute, Rosemont, IL, USA and the Danish Agriculture & Food Council. Tine Tholstrup is recipient of research grants from the Danish Dairy Research Foundation and the Dairy Research Institute, Rosemont, IL.
Interesting. While it’s nice of them to admit this up front, can you imagine this disclaimer being mentioned when a dairy industry brochure quotes their favorable findings?
Then there are the travails of T&FG itself. According to Wikipedia:
In 2017 Taylor & Francis was strongly criticized for getting rid of a public health journal editor in chief who accepted articles critical of corporate interests. The company replaced the editor with a corporate consultant without consulting the editorial board. [Also], the publisher has been accused of processing articles in a speedy manner without proper peer review.
While neither of these “whoopsies” necessarily raise a red flag, they do raise yellow ones. Sometimes such controversies are just the tip of the iceberg. T&FG may have gotten nailed for these two indiscretions, but are there are others lurking the wings? We just don’t know. Also, to be fair, it doesn’t necessarily mean that all of the research in their study is flawed because of these events. However, it can’t help but muddy the waters.
Who Benefits?
Finally, who benefits and how? A study that exhaustively and eloquently refutes the claims of the pesky anti-dairy folks may be appreciated by the dairy industry. Things are not looking good for them.
A recent Mintel report says the US dairy category will see a continuous sales decline, in contrast to its strong growth in 2014 when there was a combination of high milk prices, increased international demand, and dairy milk repositioning itself to align with health trends.
Who benefits from the Bustle article? Well the Bustle webzine of course. Anything that can boost readership must be smiled upon by their advertisers. Also Rachel Krantz, the author who wrote it. (Hopefully she got paid for it.) And possibly companies like Silk, the folks who make that thinly flavored almond milk. Any darts thrown at the dairy industry surely must delight them.
Conclusion
Which of these two articles can you believe? With so many contradictory studies in tow, they end up confusing rather than helping us.
Truth is ever elusive. For me, it boils down to personal experience. Half my life ago I stopped consuming dairy and almost immediately felt the effects. I felt less sluggish at the end of the day, and lost weight and managed to keep it off. Others on the internet have reported similar (but unquantifiable) effects: more energy, decreased sugar urges, fewer and less severe colds. Try going off it yourself for a month and see what happens.
What also helps me decide is the bad state of my credulity. It was dealt a knockout blow a long time ago by a healthy skepticism toward powerful corporations and interest groups like the American Dairy Association. Remember them? Fifty years ago they flooded the airwaves with ads like this: “You never outgrow your need for milk.” These days I don’t know anyone who still drinks their recommended three glasses of milk a day. This skepticism has served me well during my 9-to-5 years working for big corporations, and today, whenever I watch the national news.
In the end, deciding what to believe and consume is up to you, not research articles. You’re going to have to do your legwork with a skeptical eye. I can’t do it for you because frankly, I don’t have the time.
And now researchers have shown that taking vitamin D and calcium supplements has no effect on preventing bone fractures. It’s a real dilemma as so many older women are in a quandary with conflicting evidence and advice about dealing with osteoporosis. We always thought taking more calcium and vitamin D was the answer!