Is there any evidence for evidence-based management?

In previous posts we have considered the appeal by a variety of scholars to be more evidence-based in management. The idea is that management practice should be grounded in a stable body of generalisable knowledge, which should then ensure that managers in organisations can take up ‘best practice’ and aspire to better outcomes for the staff and organisations they manage.

This is a noble aspiration, particularly if it works against the dominance of fads and fashion in management, where managers may adopt a particular practice mainly because managers in other organisations in their particular field are doing so. But what is the evidence for thinking that there is such a stable body of knowledge?In a thorough piece of research published in the Academy of Management Perspectives in 2009, Reay, Berta and Kohn attempt to make an assessment of the robustness of the evidence base for evidence-based management. They did so by retrieving and reading all scholarly management articles published electronically through to 2008, searching on the words ‘evidence-based management’, ‘evidence-based decision-making’, ‘best practice’, ‘best evidence’ and ten other phrases, and then sorting them according to the quality of the evidence found. In all 169 articles were assessed according to six grades of evidence with 1, the highest and six the lowest. As far as the authors were concerned the highest level of evidence is generated by randomised control trials (RCTs) with a large sample, while the lowest level of evidence is merely the respected opinion of expert committee or authorities.

There were no articles rated at level one, i.e. the highest level of evidence and only 19 at level 2. Nearly 70% of articles were judged to be at the two lowest levels of evidence, being based either on the author’s opinion backed up by anecdote, or evidence generated through self-report and descriptive studies.

In posing the question ‘is there evidence that employing evidence-based management will improve organisational performance?’ the authors give a short answer: ‘No’.

“There is really not sufficient evidence on which to base managerial changes, any more than we would encourage physicians to change their practice based on opinions and anecdotes.’

This is not to say that the authors have given up hope, however. They believe that a handful of the articles do give good local examples of management practices, and growing awareness amongst managers of the need for such evidence to inform practice in future.

Although the authors could in no way be considered critical of the managerialist project, their conclusions are very similar to scholars who are, such as Ralph Stacey and Rakesh Khurana. Within the terms of the debate which considers management a science,where the highest form of evidence is considered to be an RCT, and where the closest parallel is thought to be medicine, there is little evidential basis for the overwhelming majority of practices which are recommended by hundreds of management books and thousands of management journal articles.

One way of coming at this would be to conclude that it is just a matter of time before sufficient numbers of researchers generating the highest level of evidence will eventually produce a stable body of knowledge which would inform key areas of management and leadership so that the quality of management could improve exponentially. Another conclusion that one might draw is that, after so many decades and with so much research, it is unlikely that management is a science and will ever produce a sufficiently stable body of knowledge, if  by science we mean an approach governed by linear cause and effect and producing generalisable rules applicable in any context and at any time. The reason there are no large-scale RCTs to underpin management practice is because there are simply too many variables involved in the management of organisations to make them an appropriate research technique. The parallel with medicine is simply inapposite.

If we thought that management was best described by drawing on the non-linear complexity sciences, which would mean privileging both stability and instability, context and a specific history of interactions between engaged agents, then we might find ourselves looking for different sorts of evidence, as well as different things to generalise about. We would pursue different enquiries about the practice of management in organisations and would not be so disappointed in accepting that management is not medicine.

Advertisements

7 thoughts on “Is there any evidence for evidence-based management?

  1. Marion Briggs

    Hi Chris …

    In case your readers mistakenly believe that medicine, science, and evidence-based practice are in any way synonymous, I want to point out that in reviewing current ‘evidence-based’ medical practices, a 2007 review done by the prestigious Cochrane Collaboratives showed that only 44% of the “best practice” interventions demonstrated benefit; 7% of the interventions actually caused harm; and for 49% of the interventions, there was insufficient evidence to conclude an intervention was either beneficialor harmful.

    So, I want to suggest first, that the RCT is a scientific protocol, not a medical one; the precise purpose of RCT’s is to reduce (preferrably eliminate) natural human variation that interferes with “proofs” in linear models. The RCT really shines in animal models of human disease or physiology where, for example, use of “knock out” mice (genes are “knocked out” that interfere with “pure” conditions) create “perfect” experimental conditions with no pesky confounding factors. The animal’s internal and external environment are perfectly controlled. As the new knowledge generated in such experiments is brought closer to humans, conditions get more and more complicated and are less and less easily controlled, all of which explains why experiments that work perfectly in the initial conditions under which they are tested, often fail in later stages that involve a much greater degree of complexity.

    All this leads me to suggest that the methodological problems of management and medical research have more in common than is immediately obvious. In both cases, it is the individual / local encounter that matters most and the individual / local condition that must inform the application of more general concepts. Nevertheless, generalizations can be helpful starting points and all that traditionalresearch can give us to work with. But a particular management decision or encounter, or decisions about management of a particular patient at a particular point in time must be able to account for the local and immediate circumstances – the very conditions which methods aimed toward understanding generalization must ignore.

    I would never suggest that we should abandon the RCT in medicine – many advances start in this way – more importantly, many ideas are abandonded when they prove harmful, ineffective, or just the wrong pathway altogether. Perhaps we shouldn’t abandon (the hope of) general theory in management science either. We do however need to recognize what the value is of generalized knowledge and how we can use it – or not – in particular situations – whether managerial or medical.

    Cheers
    Marion

    Reply
    1. Chris Mowles Post author

      Thanks Marion. So I wonder if it is an irony that RCTs which are designed to eradicate natural human variation, are, in the case of management, supposed to be measuring something that humans are doing. To be generalisable, according to the discipine of the logic, some management practice or other would need to be as standardised as possible. This works against the ideas that I know that you have been developing, that in order for a practice to flourish, particularly when taken up with others who are practising similarly, contextual variation and improvisation are inevitable and necessary. Chris

      Reply
  2. Pingback: Tweets that mention Is there any evidence for evidence-based management? « Reflexivepractice -- Topsy.com

  3. Marion Briggs

    Thanks Chris – Ironic indeed – and I would argue no less so for medicine than management. I think positivist science understands the irony of reducing/eliminating variation, but accepts it as necessary in order to have a starting point that the scientist understands will ultimately become “messy”, precisely because the discipline of the logic cannot be directly applied in the context in which it ultimately has to prove itself (albeit a scientist would not concern herself with whether or not her work ultimately becomes useable or practical – knowledge for its own sake is more than sufficient). In accepting the contradictions, medical scientists often stand as a discipline apart from practicing clinicians and two worlds seem to emerge (not quite incompatible, but mutually disinterested – perhaps even disdainful) – one of ideas, theories, experimental conditions, the other of human interaction necessarily full, as you say, of contextual variation, improvisation and uncertainty. The Cartesian precipice looms large and I would prefer not to tumble headlong into it! Whether management or medicine, it seems to me that we can accept the context and value of science, including the (limited) possibility of RCT’s, and at the same time, understand that we form and are formed in local human interaction that is necessarily homogeneous and heterogeneous, conflicted and harmonious, certain and uncertain all at the same time. Truth and reason coexist – in most cases truth is not finite or a-contextual, nor is reason fully contextual. Townley argues (with Toulmin) that reason and rationality operate as power/knowledge and have direct effects arising from and contributing to power and knowledge – I suspect it is here that we could fruitfully explore disparate rationalities.

    Reply
  4. Rotkapchen

    Is this a case of throwing the baby out with the bathwater? Is the use of evidence the problem or the means by which the evidence employed the real issue.

    Evidences are relevant — but for the purpose of providing a context. We already suffer from context-less decisions made all the time. So I’m seeing a pattern emerging: the inability to leverage evidence as the means to form a meaningful context for a decision.

    And decisions, ALL decisions, are based on some sort of context. I’m pretty sure that’s where the problem begins.

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s