Science - April 1, 2010

The top three statistical errors

PhD researchers regularly make mistakes in their research because of their inadequate knowledge of statistics. Saskia Burgers tries to rectify this with her new book Statistics for Researchers. Here are her top three statistical errors.

 1)  Before starting their experiments, researchers do not always think about the size of the sample they need in order to show a difference - between treatments A and B, for instance. On completion of the experiments, they are then disappointed that a difference that is genuinely relevant is not found significant. A simple example: 'We want to compare two levels of nitrogen application. If you put equal amounts of nitrogen on the fields, you still get a variation in the harvest for all kinds of reasons. So you take perhaps ten fields, so that you can repeat each application level five times. Then you calculate the average yield for the five fields with a lower nitrogen application level, and compare it with the average yield gained with the higher level. It is not difficult to work out beforehand what sample size you need in order to reveal relevant differences.'
2)   Researchers sometimes think that they are taking independent measurements when these are really just psuedo-repeats.  'In human research, it is easy to imagine that happening. For example: I am measuring someone's health over time. This involves ten independent, related measurements, which are related. But take a cross-pollination test with thirty seedlings in a greenhouse. Ten of the plants are in a greenhouse where the temperature is 18 degees Celsius, ten of them are kept at 20 degrees and ten at 22 degrees. Many researchers then think they are getting thirty independent measurements, but they are actually getting three. If something goes wrong in one of the greenhouses, it affects all the plants in it. You don't apply the temperature to all the plants independently. This is a pseudo-repeat.'
3)       'Rubbish in, rubbish out.' Researchers nowadays have access to menu-driven statistical programmes, with a whole range of options you can click on. 'If you click on OK, you nearly always get a result, even if you haven't indicated the structure of your experiment properly. Some very strange things come out of this. You really need to know what you are doing.'
Three good reasons for statistical training, then. And we can suggest three options:
-         Follow a course as part of your degree programme. There is a lot on offer, from 'Modern statistics for the life sciences' to 'Statistics for ecologists'.
-         Take a course in Statistics at the Wageningen Business School (WBS).
-         Get the book Statistiek voor onderzoekers, met voorbeelden uit de landbouw- en milieuwetenschappen, by S.L.G.E. Burgers en J.H. Oude Voshaar, published by Wageningen Academic Publishers, ISBN: 978-90-8686-135-5, 49 euros.
Saskia Burgers works at Biometris, the Wageningen UR institute for apply mathematics and statistics.