Tag Archives: theories of change

Complexity and evaluation

Here is the abstract of my latest article on complexity and evaluation, which you can find here:

This article offers a critical review of the way in which some scholars have taken up the complexity sciences in evaluation scholarship. I argue that there is a tendency either to over claim or under-claim their importance because scholars are not always careful about which of the manifestations of the complexity sciences they are appealing to, nor do they demonstrate how they understand them in social terms. The effect is to render ‘complexity’ just another volitional tool in the evaluator’s toolbox subsumed under the dominant understanding of evaluation, as a logical, rational activity based on systems thinking and design. As an alternative I argue for a radical interpretation of the complexity sciences, which understands human interaction as always complex and emergent. The interweaving of intentions in human activity will always bring about outcomes that no one has intended including in the activity of evaluation itself.

Advertisements

Complex, but not quite complex enough II

Evaluation scholars abstract to varying degrees from the social programmes they are invited to evaluate. Perhaps the highest degree of abstraction is demonstrated by those evaluators using experimental methods who are concerned to draw statistical distinctions between a ‘treatment group’ and a comparator group which is randomly selected. Experimentalists are generally disinterested in social theory and think of causality in terms of independent and dependent variables. Meanwhile, adherents of Theories of Change (ToCs) made popular by the Aspen Institute (1997), draw on propositional logic and represent social change in the form of entity-based logic models showing the linear development of social interventions towards their conclusions. Additionally, however, they will often point to the importance of participation and involvement of the target population of programmes to inspire motivation. In this sense TOCs are a hybrid of functionalism and emancipatory social theory, which encourages participants in social programme to be active in the change process.

Less abstract still are ‘realist’ evaluators who claim to be interested in ‘generative’ theories of causality, i.e. ones which open up the ‘black box’ of what people actually do to make social programmes work or not.  Realistic evaluation draws on Bhaskar’s critical realism (1978) as taken up and developed by Pawson and Tilley (1997) and Pawson (2006) and is the theory most often linked to the complexity sciences, particularly complex adaptive systems theory (CAS).  In trying to reconcile realistic evaluation and CAS they adopt a functionalist, systems-based understanding as a default position and argue that interactions between human beings take place as ‘mechanisms’ and have an effect at different ‘levels’ of reality.The conceptual link between  CAS and realistic evaluation is that they both have an understanding that stability and change does not arise because of ‘variables’, the staple of experimental methods, nor does it proceed with propositional logic as in ToC, but as a result of what people are doing in their local interactions with other people. CAS are relational models demonstrating how patterns emerge over time because of ensembles of interacting agents. So from a realistic perspective and in the words of Pawson and Tilley:

Realists do not conceive that programmes ‘work’, rather it is the action of stakeholders that makes them work, and the causal potential of an initiative takes the form of providing reasons and resources to enable programme participants to change. (1997: 215)

So both CAS and realist evaluators are most interested in local interaction as the basis for developing more general observations about the success or otherwise of social interventions. Realistic evaluators argue that interventions do or do not achieve what they set out to because of a combination of context, mechanism and outcomes (CMO). The perspective is concerned with finding what works for whom and in what circumstances and then extrapolating a detailed and evolving explanation to other contexts. In Pawson’s words it is predicated on the ‘steady accretion of explanation’ (2006: 176) about a reality which exists independent of the evaluators who are enquiring into it.  Continue reading

Management fads and the importance of critical thinking

One of the main themes of Mats Alvesson and Hugh Willmott’s new edition of their book Making Sense of Management is that management, and the ubiquitous tools and techniques that accompany the practice are widely taken for granted as neutral, technical and helpful. In detail, and at length, they call these assumptions into question. Further, in a forthcoming article in the Journal of Management Studies, Alvesson, with his co-author André Spicer go on to accuse organisations of practising both knowledge and stupidity management. By stupidity management they mean the way that many organisations rush into adopting the latest management fad that everyone else is taking up, simply because everyone else is taking it up. They point to an absence of critical reflection and questioning in many organisations.

It is this process, endlessly rushing towards the next big idea provoked by an anxiety about keeping up with ‘the latest thinking’, or perhaps because of (self-imposed) coercion from peers or scrutinising boards and other agencies, that keeps the management shelves of bookshops filled to overflowing, and management academics and popular writers busy (and sometimes rich). Continue reading

G20 and different theories of change

Watching news reports about the G20 summit and protests I was struck by how many analogies that they offered for thinking about  theories of change.

capitalism-isnt-working

A BBC news reporter was caught up in the protest crowd outside the RBS bank in the city just before the clashes started with the police. The crowd ebbed and flowed: ‘there’s nothing we can do about this,’ complained the reporter, ‘because we’re just caught up in the crowd and well have to go along with it.’ As the crowd surged forward some of the protesters began to stone the bank: others hung bank clearly embarrassed by what was happening. Gradually, over time, with the police responding cautiously minute by minute, the violence receded and the protest turned into a good natured street party outside the Bank of England.

In The Civilising Process Norbert Elias describes how a more and more sophisticated society produces longer and longer chains of inter-dependent people, thus constraining what it is possible for any one individual to do. It struck me that this crowd was a good analogy for why change is so difficult. We do have choices, but in general we are constrained by the ways of thinking and acting of those we depend upon, and who depend upon us. We are linked together as if by some giant elastic band. We are caught up in the habitus, mostly not even aware as the reporter was, of what we take for granted. Short term changes can manifest themselves quite quickly, and sometimes quite violently, but longer term changes take more time to show but are likely to be more enduring.

I was also struck by the difference in the thinking of Gordon Brown and Barack Obama. For Brown ‘global problems demand global solutions.’ This is a classically systemic response and understands the leader to be a systems designer who can simply reengineer, or ‘fix’ a ‘broken system’.  Change is considered to be  a wholesale phenomenon and  something which is possible to organise top down. Barack Obama, meanwhile was busy having one-to-one conversations with other world leaders directly and personally changing the nature of the conversations, or perhaps having conversations for the first time with leaders that Bush had ceased speaking to. In doing so he is creating the possibilityfor small differences, an improved relationship and therefore new ways of understanding and mutual adaptation, to be amplified over time. It is important not to be naive about what might happen with these conversations, however, as the news presenter pointed out: President Kennedy renewed personal contact with Khruschev, who was then convinced that this was a sign of weakness on Kennedy’s part which he began to exploit politically. We cannot be sure that in acting differently what we intend will definitely come about. The important point for me of the G20 summit is not the final communique, however symbolic, but the possibility for changed relationships and thus ways of seeing the world that arises out of the myriad small conversations that happen between the leaders and civil servants.

On a discussion panel a young climate change activist was arguing that it was important not to try and solve new problems with old ways of thinking, but then proceeded to propose a programme of wholesale change himself. If his comments were directed towards world leaders his ask is a big one: the very people who have struggled to the top with old ways of thinking, Gordon Brown for example, are now being asked to think very differently, or in effect to be different people. And even if this were possible all world leaders are constrained by the vast chains of interdependent people who will respond in a variety of different ways  to what they propose. The problems might be caused by a particular phase of turbo-capitalism, but we are all to a greater or lesser extent, caught up in its modalities.

This is not necessarily grounds for pessimism, however, because there may be more profound changes afoot which have yet to manifest themselves which grow out of people’s daily experience, and perhaps dissatisfaction with the way they are obliged to live their lives. As the Canadian philosopher Charles Taylor observed:

“True, the philosophies of atomism and instrumentalism have a head start in the world. But it is still the case that there are many points of resistance, and that these are constantly being generated … We don’t want to exaggerate our degrees of freedom. But they are not zero”