JAMA Meta-analysis of Antioxidants - Flawed Data, Biased Analysis, and Inappropriate Conclusions

Recent news stories sensationalizing the results of a new controversial study reported in the February 28, 2007 issue of the Journal of the American Medical Association (JAMA) imply that antioxidants do not have health benefits, and in fact, may lead to increased mortality. In our analysis of this study, we see little or no evidence to support this conclusion.

It is important to understand that this current study is not a new clinical trial, but a statistical analysis determined from many studies (called a meta-analysis). Meta-analysis studies are actually designed to pool similar studies for statistical comparison. However for this meta-analysis, the authors combined studies that differed vastly in design, use rates, duration, and study population. After careful review, it appears that the authors simply analyzed data that fit a predetermined conclusion, which is an invalid use of a statistical method. This is a great example of improper statistical use in research methodology.

There is a large body of data including observational studies, prospective epidemiological studies and randomized clinical studies that have shown positive benefits of antioxidant supplementation (including reduced cardiovascular disease, some cancers, immune support and reduced progression of eye disease). Interestingly, these studies were excluded from the analysis. And while the initial analysis examined 1201 research papers from 815 trials, only 68 trials were actually used in the final analysis.

Furthermore, when the initial results from this data did not show any effect on death rates, the authors removed an additional 21 studies (called a sub analysis) to draw their conclusion that supplements increase risk of mortality. Interestingly, in these 47 remaining studies, the doses used greatly exceeded normal use rates, and in many cases, were well above tolerable upper intake limits (UL) (i.e. they used doses that may not be safe). In contrast, the studies eliminated from this study generally used doses that did not exceed the UL and were more in line with actual use rates.

Finally, the majority of the studies examined are secondary prevention studies. That is, the study populations had already been diagnosed with diseases such as heart disease and cancers. This is a very risky population to study, and conclusions from these studies should not be used to make recommendations for prevention in generally health populations.

We are not the only ones who have criticized these results. Several scientific organizations and other researchers have already published criticisms of this paper. Professor Balz Frei director of the Linus Pauling Institute at Oregon State University stated:

"This is a flawed analysis of flawed data, and it does little to help us understand the real health effects of antioxidants, whether beneficial or otherwise. Instead of causing harm, the totality of the evidence indicates that antioxidants from foods or supplements have many health benefits, including reduced risk for cardiovascular disease, some types of cancer, eye disease and neurodegenerative disease. In addition, they are a key to an enhanced immune system and resistance to infection. The "meta-analysis" published in JAMA, which is a statistical analysis of previously published data, looked at 815 antioxidant trials but included only 68 of them in its analysis. And two of the studies excluded " which were published in the Journal of the National Cancer Institute and the prominent British medical journal Lancet " found substantial benefits and reduced mortality from intake of antioxidant supplements."

"If these two large studies had been included, none of the reported effects on increased mortality would have been significant, with the exception of the effects of beta carotene. And the research showing a higher incidence of lung cancer in smokers who take supplements of beta carotene or vitamin A is old news, that's been known for many years. Very high doses of vitamin A are known to have multiple adverse health effects."

"All the new study really demonstrates, is a bias toward identifying studies or research that show harm caused by antioxidants, and selective removal of research that shows benefits."

We don't know why the authors chose to evaluate this small carefully selected group of studies, or why they ignored the larger body of evidence published on the benefits of antioxidants. But it appears to be an obvious attempt to sensationalize incomplete data that serves no purpose other than to alarm a large portion of the supplement using population. We are not fooled by this report.

The full text of the Linus Pauling Institute press release can be found at:
http://oregonstate.edu/dept/ncs/newsarch/2007/Feb07/vitaminstudy.html

Additional comments on the JAMA study can be found at the following links:

http://www.crnusa.org/PR07_JAMA_antioxidant_metaanalysis_022707.html

http://www.nutraingredients.com/news/ng.as p?n=74582&m=1NIE228&c=cyqrgoymvgyorow

http://www.naturalproductsassoc.org/site/DocServer/Response_to_JAMA _Antioxidant_Study_2-27-07.pdf?docID=3201