Tag Archives: experimental methods

Payment by results: research methods and disciplinary power

I was sitting in a meeting with a social development organisation listening to the kinds of requirements that have been placed upon it by a governmental body in order to trigger the full funding for a grant that they had succesfully bid for. 10% of the grant is ‘performance related’. In other words, and on a sliding scale of reward for performance, the social development organisation has to prove that it has helped educate a certain number of girls in a developing country to a predicted level of attainment, and that these girls will have stayed in school for the three year duration of the project and not dropped out. Additionally money is released against the achievement of pre-reflected project milestones. ‘Results’ are validated by ‘rigorous research methods’ which turned out to mean quasi-experimental methods. In other words, the rubric insists that the project sites be compared with communities where there has been no such intervention, and which are ‘similar in every way’. The organisation will only be fully rewarded if it achieves exactly what it said it would, and precisely to the timetable it set out in the proposal.

This particular social development organisation I am visiting is one amongst a dozen or so others which have received similar or much bigger grants, some of which amount to the low tens of millions. All of them have proposed highly complex interventions in very different developing countries involving the girls themselves, their families, teachers, head teachers, community groups, religious and community leaders, sometimes even boys. As with most social development these days the intervention is highly ambitious and leaves the impression that the organisation, working through a local social development organisation in the country concerned, will be intervening in particular communities at breakfast, lunch and dinner and in a variety of different and incalculable ways. This combination of interventions may be necessary, but the extent and range of them makes the question of causality extremely problematic, experimental methods or no.

The other thing that struck me is that the dozen or so social development organisations receiving this money all have to use the same project management tools and frameworks so that the government department can aggregate progress and results across all countries and all projects. Quantification and standardisation is necessary, then, in order to render the projects commensurable, and in order to make a claim that the government has made a quantifiable contribution to the Millennium Development Goals (MDGs) which they can ‘prove’. The kind of assertion that the government would like to make is that it has improved X tens of thousands of girls’ education to Y degree through its funding of a variety of organisations. These results, the claim will continue, will have been rigorously demonstrated through scientific methods and will therefore be uncontestable. Continue reading

Doubt as a form of enquiry

In the last post I discussed what the pragmatic philosopher John Dewey referred to as the quest for certainty. I have been arguing that the discomfort that people feel if something isn’t completely nailed down in advance often prevents them from dwelling long enough with experience to work experimentally. There is rush to define, to plan out in advance, to idealise and to make certain and this is likely to prevent innovative ways of working to which organisations aspire. I have been making an alternative argument that without improvisation, spontaneity and risk there can be no innovation.

Dewey was interested in experimentation and argued that traditions of thought, such as mainstream philosophy, have conventionally been suspicious of the bodily, the temporal and the experiential, instead preferring Plato’s fixed and pure forms. We are generally encouraged to discover pre-existing ‘truth’, rather than dwell in the messy reality of experience. However, he himself was much less interested in knowledge as a pure and static expression of truth, and more committed to knowing as a form of active enquiry, the idea of constantly opening up experience to further experience. I think this idea of constant doubt and enquiry is especially relevant to managers who are thinking about how to deal with the ever changing patterning of experience in organisations that they have to deal with on a daily basis. Continue reading