Does western development aid do any good? Do all those projects result in more development and less hunger and poverty? This question is increasingly relevant as the pressure on development aid budgets increases. For this reason, according to development economist Erwin Bulte, it is time to start evaluating development aid more precisely with the aid of a scientific approach. To this end Bulte developed a new method inspired by an approach to testing drugs known as randomized controlled trials. A dice decides which patients receive the drugs and which a placebo: or in this case, which African villages get to participate in a development project.
After a few years, Bulte then measures whether the villages with the project are better off than those without it. This is a new approach in the world of development aid. In the past, aid organizations and donors decided in advance what would be good for African farmers. Evaluation and research only took place after the project was finished. But it is difficult to evaluate properly like that, says Bulte, because the selection criteria for farmers in the project (are they enterprising? Do they live near a main road?) influences the results. What is more, you do not know how the farming community would have fared without the aid. By selecting the farmers randomly and explicitly including in your survey farmers who did not get the ‘medicine’, you can measure its impact far more precisely, claims Bulte.
This sounds logical enough, but there have been criticisms of the Wageningen approach. Some NGOs see it as unethical. Bulter is treating the African population like test subjects, they say, just as plant scientists might carry out tests on heads of lettuce. He deliberately withholds aid from a section of the population. The Wageningen researcher contests that criticism. ‘People are always selected for projects because the budgets are limited,’ he rejoins. ‘Usually the choice is political, whereas we let the dice decide. Which is fairer?’
The chief’s cut
In the academic world at least, Bulte’s approach is widely applauded. In 2011 he received one and a half million euros from the Dutch research funding body NWO for research into which institutions contribute to agricultural development in Africa. He is not afraid to include unconventional research methods and started a project to investigate the link between corruption and agricultural development. Together with PhD student Gonne Beekman, he distributed crop seed and artificial fertilizer in several villages in Liberia. On a pretext, he stored these aid packages for a few days in the huts of the local chiefs. Later he checked which goods had disappeared, as a way of measuring corruption on the part of the chiefs. He also recorded the investments farmers made in their farms and he used a game to find out how much money the villagers were willing to invest. ‘A very clear conclusion emerged. In villages where the chief is corrupt, the villagers invest much less. They are afraid that the chief will cream off some of their investments.’
This conclusion was grist to the mill of international donors who are on the warpath against corrupt local government and have nowadays embraced community driven development, an approach in which the village council gets to manage the project. In recent years many billions of dollars have gone into projects of this kind, but Bulte questions the effectiveness of this approach too. ‘Studies have revealed that the impact of this kind of project is 0.0,’ he says.
Research by Maarten Voors, a post-doc working with Bulte, has come up with an explanation for the failure of these community-led projects. In Sierra Leone Voors and Bulte worked with an NGO which offered villages project funding in exchange for protection for birds in the neighbouring forest. The villages decided for themselves whether to spend the money they earned on a latrine, a drying oven or a meeting house. Voors selected the villages at random, alternately channeling the development funding through the chief or through the village council. Afterwards it transpired that although the chiefs took more of a cut of the money when they were in charge, they were also more likely to get the project off the ground. With the chief in charge the toilet or drying oven gets finished, where the local council often fails. The chief’s organizational capacity is greater, explains Voors. And the more power he has, the more he tries to undermine the village community’s project, ensuring that a bottom-up project does not get off the ground.
Behaviour games
A follow-up project should provide more insight into the decision-making dynamics. Bulte: ‘There are really three sets of rules in the villages. There are the formal rules that come from the capital, which often play very little role in remote villages. Then you have the traditional laws administered by the chief and the innovations brought in by NGOs and the World Bank, with an emphasis on empowerment and participation. In an experiment in more than 100 villages we are looking at which set of rules is used by the population. Do efforts to change things at the local level get anywhere, and under what conditions does the project succeed?’ These projects, too, include a control group that does not receive aid. PhD researcher Haki Pamuk, for instance, investigated the role of innovation platforms in improving food security in a number of African countries. In these innovation platforms, farmers, government, suppliers and clients collaborate in the food chain. A platform was established in a random selection of the villages, with another set of villages forming the control group. In both types of village the researcher counted how many villagers lived below the poverty line. ‘You see great variations in the success levels but on average the innovation platforms do have an effect. In villages with a platform you have 13 to 20 percent fewer poor people after a couple of years.’
Carrying out field research using randomized controlled trials is now hot among donors such as the World Bank and accepted by development organizations. ‘The NGOs were hesitant to start with,’ says the development economist, ‘but they are obliged to show the impact of their projects. It is difficult to demonstrate that impact because development is complex and everything is interconnected. By setting up a project in the form of an experiment, with a control group, you can remove the bias of the researchers or the wishful thinking of the donors. We don’t know what is good for the population of Africa; that is precisely what we are researching. That is why this approach generates more knowledge about which projects help and which don’t.
Photo: Ben Schaap