Science - December 14, 2017

Creative with data: What do you do if other people distort your results?

Text:
Rik Nijland

Lobbying organizations, institutes, government bodies and companies have all been known to highjack Wageningen research results. And that can lead to such oversimplified stories doing the rounds that researchers have to call a halt. ‘Of course you know that NGOs cherry-pick, but when does it become a travesty of the truth?’

Text Rik Nijland illustration Pascal Tieman

It takes guts to accuse the national statisticians at Statistics Netherlands (CBS) of manipulating data. But Ingrid Tulp, a researchers at Wageningen Marine Research, took her courage in both hands at the beginning of November and did just that.
Statistics Netherlands used research data produced by Tulp and her colleagues for the Living Planet Report that focuses on saltwater and brackish nature in the Netherlands, published by the Worldwide Fund for Nature WWF. It was the Statistic Netherlands press release in particular that went down badly with Tulp. The headline  ‘30 percent less wildlife in the North Sea’ was a real case of jumping to conclusions, she said in Resource. ‘If numbers of one species have gone up by 20 percent and numbers of another have gone down by 60 percent, you can’t take an overall average from that. The total animal population is nothing to do with these figures.’
The press releases also laid the blame disproportionately on bottom trawling, says the researcher. Wageningen has not done any research at all on the cause of the decline in fauna populations. Bottom trawling certainly has played  a role, says Tulp, but the accusation that was made is ‘arbitrary’.

Discreet

Statistics Netherlands withdrew its statement after the criticism. The press release now bears the headline: ‘Biodiversity of North Sea in decline’. And bottom trawling is named as ‘one of the main causes of the decline in seabed fauna in the North Sea’. This seems to have cleared the air, even though nothing has been changed in the WWF’s Living Planet report.
This is not the first time WUR has been in the firing line over the way its research is interpreted. But how common is it, actually, for NGOS, institutes, government bodies or companies to distort research results to suit their own purposes, or to massage or misinterpret them? No one is very keen to answer this question.
Preferably, you settle such matters discreetly, emails Erik Toussant, communication advisor at the Plant Sciences Group. ‘Nobody wants to get in the papers with this sort of thing, even if they were very offended at the time. Sometimes there has already been a furore about it, whether it got in the press or not. And even if not, it can still damage relations. So it’s not very sensible to give it publicity.’

Mega barns

But sometimes the researcher just gets drawn in. Edo Gies of Wageningen Environmental Research found himself at the centre of a comparable affair, characterized by the agricultural press as:  ‘Milieudefensie (an environmental lobbying organization) distorts figures on mega barns’.  In the spring of 2015, Trouw newspaper stated on the front page that the number of so-called ‘mega barns’ had tripled in the last eight years. Press agencies ANP and FD also used Milieudefensie’s press release, based on research Gies did for the organization. The interpretation was shaky, though, as the researcher confirmed when Resource sounded the alarm. Milieudefensie had confused large farms with large barns. The number of the latter had doubled, not tripled.
‘In Resource I could bring in the nuances,’ reflects Gies. ‘That had a negative impact for Mileudefensie: after that attention was focused on the idea that the organization was unreliable, rather than on upscaling in livestock farming. Since then, though, I’ve talked it all through with Mileudefensie. How can you prevent such things from happening? Above all, by making agreements about how you publicize results. Actually it goes well nearly all the time.’

An NGO makes a claim, and journalists often just write it down unquestioningly

‘Overstated case’

The bottom trawling issue will also lead to consultations between Statistics Netherlands and Wageningen Marine Research, says Statistics Netherlands spokesperson Cor Pierik. ‘We need to know more about what the other party measures and how they work. There was obviously not enough exchange on those points at the workshop that was held beforehand.’ But Pierik denies that Statistics Netherlands was influenced by the client. ‘We write those press releases ourselves. WWF was not involved at any point. We adjusted it later because we had overstated the case somewhat, and had claimed a causal relationship for which we had no evidence. Normally we are very precise about that, but it seems that not enough people took a look at this.’
However those consultations go, the publicity horse has already bolted. The fisheries industry is surrounded by political polarization, so the article in Resource prompted questions in parliament. Bureaucrats at the ministries of Agriculture and Economic Affairs now have to respond to the shared concerns of the fisheries spokespersons from the CDA, Christian Union and SGP parties. They asked questions about the integrity of Statistics Netherlands and the bad press given to bottom trawling. The animal rights party also asked questions about the phasing out of bottom trawling.

Stakeholders

Even if it did cause a commotion, not responding was not an option, says Antoinette Thijssen, communication manager at Wageningen Marine Research. ‘You’d rather keep that kind of discussion among yourselves. But when Statistics Netherlands and the WWF made this public, other stakeholders such as the fisheries sector confronted us about it. The impression had been created that Wageningen Marine Research was one of the sources. That was the painful thing about it. Fine for them to use our data. But if the researchers say you cannot draw these conclusions, and then it is suggested that you were one of those behind them, then you have no option but to comment.’

Cherry-picking

It is particularly important to stay alert, thinks Thijssen, when doing research for lobbying organizations. Companies are wary of using research data for their own ends in the media. ‘If a business makes claims on the basis of research, journalists are often suspicious and double-check it. Just one mistake, and they come down on the company like a ton of bricks. But an NGO makes a claim, and journalists often just write it down unquestioningly. And NGOs are often granted a bit more leeway; of course you know they do a bit of cherry-picking. But at what point is it a travesty of the truth? It is essential to agree clear rules on communication.’
But scientists are only human too. In April 2013, Imares, the forerunner of Wageningen Marine Research, had to rectify one of its own press releases on research on the cultivation and harvesting of mussel seed in the Wadden Sea, a bone of contention at the time. The press release said the damage caused to nature was limited. That had to be corrected. Research partner NIOZ said in the Volkskrant newspaper that the conclusion was ‘clearly biased’ and ‘an extreme oversimplification’.

Reactions 1

  • Jaap Graveland, een geïnteresseerde burger

    Ik heb het Living Planet Rapport Zoute en Zilte Natuur net gelezen. Ik vind het heel evenwichtig geschreven, compleet ken en bronvermelding. De nuances staan er allemaal in, ook staat aangegeven waarom visserij nog steeds een factor van betekenis kan zijn. Het enige waarin het rapport niet altijd consequent is, is dat er laatste tien jaar stabilisering bodemdieren is, na afname in jaren ervoor. Het staat netjes in de hoofdtekst, maar niet in de conclusies op p. 79. Er is overigens ook geen toename na stopzetten grootschalige boomkorvisserij. Dat is relevant en vraagt om een verklaring. Er wordt vervolgens op verdedigbare wijze aangegeven dat de visserij die nog wel plaatsvindt mogelijk toch effect kan hebben. En dat de boomkorvisserij tot een blijvend verstoorde bodemstructuur kan hebben geleid, die bodemdierherstel in de weg zit.
    Kortom, ik vind de reactie van WMR sterk overtrokken. Zo sterk dat het op mij overkomt als: de pot verwijt de ketel, nl. (schijnbare)vooringenomenheid vanwege belangen. En begrijpelijk dat de inhoud van het rapport niet wijzigt, en WNF bij zijn conclusies blijft. Zo nieuw is het ook allemaal niet.

    Voor geïnteresseerden: lees het, het is overzichtelijk en bondig opgeschreven rapport. In een uur weet je heel veel meer over veranderingen in biodiversiteit in 25 jaar in Noordzee, kustzone, Waddenzee en zoute/brakke Deltawateren.


Re:act