So much for peer review

Katherine Rich writes:

In February 2018, a University of Waikato study published in Public Health Nutrition about sugary drinks from Canada, United Kingdom, Australia and New Zealand was shopped to the media no doubt with great excitement about its shocking findings. New Zealand drinks were possibly “the most unhealthy in the world”.

It was a big story covered by most media here and the news that New Zealand drinks were higher in sugar than other countries went global. The paper announced that “NZ had the highest percentage of beverages with sugar added to them (52%), whilst the UK had the lowest (9%, p<0.001). Sugar tax campaigners latched onto the low UK figure and used it to amplify their calls for sugar taxes and regulation making inflammatory comments about the food industry like, “Don’t trust them. Just regulate them”.

At the New Zealand Food & Grocery Council we couldn’t understand the big gap between countries. Major volume ranges were similar and major producers in each country had been on a reformulation journey for many years (in some cases decades). It didn’t make sense based on our global knowledge of the food industry.

What stood out the most when so many brands are global and the same in terms of sugar content, was the extremely low 9% of beverages with sugar added attributed to the United Kingdom.

The extremely low UK figure was used by pro-sugar tax campaigners in New Zealand to claim that the UK sugar tax on beverages had already been a resounding success even before it had come into force, and to pressure the New Zealand government to implement a similar tax regime here.

A lot about the paper and its findings didn’t add up. So we fact-checked the paper and found our instincts were correct.

Indeed, the maths did not add up. The most glaring error was so simple a high school student would be embarrassed.

Rather than the level in the UK being 9% it was 39%. Somehow the ‘3’ in 39 had been left off and not one of the seven highly educated authors had caught the error. So much for peer review.

None of the authors caught on, let alone the peer reviewers.

So you had all these shock horror stories based on a total fallacy.

When we dug deeper we found other flaws in the paper that were equally concerning and went a long way to explain the remaining gap between countries. Mistakes were made about all sorts of things: UK sugar tax rates, the way Australia and New Zealand define fruit juices and drinks, Ministry of Health nutrition guidelines, product labels, water differences. The authors also appeared not to recognise the difference between pack sizes and serving sizes. Some of the paper’s references did not support bold claims made in the paper or were merely published opinions, not research.

Yet again this paper got through peer review and was published.