Complexity and evaluation

Here is the abstract of my latest article on complexity and evaluation, which you can find here:

This article offers a critical review of the way in which some scholars have taken up the complexity sciences in evaluation scholarship. I argue that there is a tendency either to over claim or under-claim their importance because scholars are not always careful about which of the manifestations of the complexity sciences they are appealing to, nor do they demonstrate how they understand them in social terms. The effect is to render ‘complexity’ just another volitional tool in the evaluator’s toolbox subsumed under the dominant understanding of evaluation, as a logical, rational activity based on systems thinking and design. As an alternative I argue for a radical interpretation of the complexity sciences, which understands human interaction as always complex and emergent. The interweaving of intentions in human activity will always bring about outcomes that no one has intended including in the activity of evaluation itself.

Visioning backwards

On Wednesday 16^th October Mary Ward and Jo Collins, the founders of the Chickenshed Theatre, were interviewed by BBC Radio 4 presenter Jenni Murray on Woman’s Hour. They were invited onto the programme to celebrate the forthcoming 40^th anniversary of a theatre which was set up to involve young people in theatre, irrespective of their abilities. Both founders had a shared belief that they could produce excellent theatre with young people if they could encourage everyone to accept what young people bring, rather than what they don’t bring. They argued, for example, that young people are often much less judgemental about other young people with disabilities than adults are: they simply accept the disability as a given and proceed from there, without fuss. They argued that discrimination is a learned, social behaviour. That commitment, and the continuous improvisational ability to involve other people in the undertaking, has created an institution which has lasted 40 years although it has never received Arts Council funding.

‘What was your vision for Chickenshed?’ Jenni Murray asked. ‘We didn’t have a vision as such, we didn’t sit down and say “this is our vision”’, Mary Ward answered arguing that they had both felt impelled to include as many young people as possible, ‘but we just did it, and as we did it we became more and more committed to this idea that everyone can contribute to the production, the final end.’

Covering over paradox II

In the previous post I wrote about how paradoxes and contradictions produce unresolvable tensions for people working in organisations and often provoke strong feelings. For example, it is impossible to have reorganisation without including some people in the changes and excluding others, without having winners and losers, those who are satisfied and those who are not. All reorganizations are disruptions to power relationships which can sometimes be experienced as threats to identity or lack of recognition.

Last time I rehearsed some of the ways in which orthodox theories of management reduce the paradoxes of organisational life by turning them into dualisms, double binds, or separating them into sequential phases between stable states. In this post I will consider two other ways of re-presenting paradoxes in the form of idealisations and logic models.

Complexity and sustainability

Elinor Ostrom, who in 2009 was the first woman to win the Nobel Prize for economics, is due to give one of the Oxford Amnesty lectures this year.  You can watch her Nobel lecture accepting her award here. Ostrom’s work over five decades has been to conduct a huge variety of studies of what she terms ‘common pool’ joint  economic undertakings, such as forests, farmland held in common, irrigation systems and even the provision of services in cities in the United States to better understand human behaviour. She has been concerned to develop better understanding of how common pool economic ‘systems’ might be managed more sustainably. Her conclusions are that complex economic interaction can more helpfully understood with painstaking empirical study, but that there are no simplistic answers. To echo the title of her Nobel lecture, complex economic interactions demand what she terms ‘polycentric governance’, which is beyond both state and markets.

For example, her response to earlier studies criticising the messiness of provision of government and other agencies in a city area led her to conclude that what mattered was not the consolidation of agencies to rationalise away the mess, but  overall performance. There is no one best way of organising ‘messy’ provision, which might be providing a good level of services for local citizens. For example, her in-depth studies of police departments in six metropolitan areas did not find a single instance where a large centralized police department outperformed smaller departments serving similar neighbourhoods in regard to multiple indicators. A mixed population of departments, both large and small, was what worked best, she concluded.

Her studies of common pool resources led her to develop a range of inter-disciplinary methods better to understand her objects of study, including agent-based models, which we have discussed in earlier posts.

Attempts to make the uncertain certain

I was rung up the other week by someone who worked in a management team in a development organisation, which wanted to try some new initiatives in three ‘fragile states’. It had become clear to them that traditional ways of working, adopting and following logical planning instruments, were inadequate in these particular dynamic and fast-moving contexts, and they were keen to try a different approach. I began to discuss the possibility of working experimentally: with the teams already working in-country, why not start with what they would like to do. Take the first steps, reflect on it, see how it had gone, and then take the next steps. Repeat the process over again. The programme would evolve as new possibilities emerged, although it would take a good deal of discussion and judgement. Programme coherence would build up with retrospective sense-making over time. ‘Yes, but can you prove that this way of working is effective?’, my co-respondent asked.

In a recent journal article I described the way in which staff in an organisation I had a great deal of experience with had tried over time to reflect systematically on the way they were working. This involved acting with intention, but regularly being open to puncturing and questioning these intentions through discussion, reflection and involving the subjects of their intentions by asking them what they thought of the work. It often involved taking two steps forward and one step back, and seeing the process of reflection and discussion not as an adjunct to the work, but as the work itself. The staff often had to work to tight deadlines, to cut short their deliberations to meet them, so were not in any way paralysed by talking rather than doing. Talking was a form of doing. One of the reviewers of the article commented that this was all very well, but what had I actually said about working differently? What would an ideal model of working actually look like?

I was supporting an organisation think about how they might assess work they were doing in East and West Africa where they had made an explicit commitment to their donor that they would focus on what they thought would be sustainable ways of working. That is to say, instead of providing services or materials as such, they would support local stakeholders, central and local government officers, local organisations, politicians and local councillors to work out what their problems were and what they wanted to do about them. The staff in the organisation I was supporting were clear that they had expertise to offer, but the problems were not theirs to ‘solve’. They would support, cajole, facilitate, discuss, offer training if necessary or seed initiatives. But since the inception of the programme the relations with the donor had changed, partly owing to a change in personnel in the donor. Now the donor required ‘objective evidence’ that this way of working produced results, and that these results would be transferable elsewhere. Exactly which kinds of ‘instruments’ were they using to encourage local discussion, and how could they be validated?

In each of these three examples I would argue that there is an illusory quest for certainty.

Narrative, creativity and emergence

In a lecture given to students on Columbia University’s creative writing programme the novelist Zadie Smith responded to an invitation to speak about her craft. In doing so she gives a very good description of the ways in which one might pay attention to micro-interactions from which the global pattern emerges. She describes a complex, adaptive relationship with the act of writing.

She draws a distinction between macro planners and micro managers,  counting herself amongst the latter (and it is interesting to note how the language of managerialism has permeated even novelists’ language). Macro planners organise everything in advance: the material, the plot, the structure, and may even write their novel from the middle. It is this tight structure that they use as their enabling constraint, which gives them freedom on the one hand, but hems them in on the other. As one choice forces another, sometimes they are impelled to change the choices they have made, moving a locale from London to Berlin, for example.

Strategy as co-created narrative

Despite the fact that the literature on strategic planning has diminished considerably in the last fifteen years or so, still most organisations do it. So argues a recent article in the Journal of Management Studies by Jarzabkowski and Balogun. It has become what GH Mead would term a social object, and in terms of the social game of organisational practice lots of people do it because lots of people do it. Strategic planning still has its academic adherents, but probably the scholar who has done most to drive a stake through its heart is the Canadian academic Henry Mintzberg. With his two books The Rise and Fall of Strategic Planning and  Strategy Safari the second written with two colleagues, he has done more than most to call the practice into question.

Equally Ralph Stacey, from a complexity perspective, has argued that strategic planning must serve some other purpose than being a means of predicting and controlling since they so signally fail to do so in an unpredictable world.  Most organisations seem to get by despite their strategic plans rather than because of them. At his most laconic Stacey has considered strategic plans to be like an organisational rain dance.

So what is going on in organisations when people are trying to plan strategically and what kind of thinking do they get caught up in?