Date: 08/05/19 | Duration: 00:22:40
Evidence-based practice is about making better decisions, informing action that has the desired impact, and it's central to the CIPD’s new Profession Map. Whether gravitating to practices that appear the most cutting-edge or sticking to established traditions without ever questioning their relevance, important decisions can suffer from not being grounded in a firm evidence base.
This roundtable has Eric Barends (Managing Director for the Center for Evidence-Based Management), Jonny Gifford (Senior Adviser, CIPD) and Nichola Stallwood (Head of Organisational Development & Training at the Zoological Society of London), discussing what evidence-based practice is, why it matters, and how to apply it at work.
View the full podcast transcript
Philippa Lamb: Fads, anecdotes, fake news and gut instinct, not the most reliable tools for HR practitioners but still surprisingly popular. So in this episode we’re going to take a long, hard look at an alternative – evidence-based practice – what it is, how it works and why it’s a far better toolkit.
First some background from perhaps the best-known expert in the field Eric Barends, Managing Director of the Center for Evidence-Based Management.
Eric Barends: So evidence-based practice is not something that we invented or the Center for Evidence-Based Management invented, it was introduced in the early 90s in medicine because even in medicine a lot of claims were being made regarding treatments or health or whatsoever and they felt that they should make a more evidence-based decision meaning they should have a look at the research findings to figure out whether the evidence supports the claim or whether it contradicts the claim. That is actually how it started.
So the official definition of evidence-based practice is that it’s about making decisions through the conscientious, explicit and judicious use of the best available evidence and the aim is to increase the likelihood of a favourable outcome of your decision.
Now that's a bit technical explanation of what it is. Important here it’s about multiple sources, not only one, and it’s about the critically appraised evidence. So you need to figure out how trustworthy the evidence is that you use in your decisions. But maybe a better understandable explanation is the fact that in daily life claims are being made all the time, claims about what we should do or shouldn’t do, about health or lifestyle or what to buy or what to eat etc. So in management it’s more or less the same, in management a lot of claims are being made: you should engage your workforce because then they become more productive; you should introduce self-steering teams because that is good for performance and evidence-based practice helps you to figure out the trustworthiness of those claims, how does the evidence support or maybe contradict those claims? And based on that outcome, based on that finding you can make better decisions. So in short it’s a tool, it’s an approach to figure out whether claims are trustworthy or not.
PL: Joining me in the room to talk about how we can put evidence-based practice to work I have, Jonny Gifford, Senior Adviser at the CIPD and Nichola Stallwood, Head of Organisational Development & Training at the Zoological Society in London.
CIPD thinks this is the way forward doesn’t it?
Jonny Gifford: Absolutely. I think there's a recognition that we as an HR profession have not been evidence-based enough in the past. Many professionals to be fair aren’t that evidence-based, some do better, healthcare for example, policing would be another and there's a great deal that we can do to both bake better quality evidence into the body of knowledge and also improve our skills as professionals so that we’re drawing on better sources of evidence and making better decisions.
PL: So Nicola obviously you've put this into practice, we want to hear all about that in a minute, but first do you want to tell us a bit about the Zoological Society?
Nichola Stallwood: Sure ZSL, the Zoological Society, we are across two sites in the UK and we’ve got Regents Park in London and we also have a zoo at Whipsnade and we’re based across six international offices, also where we run conservation programmes.
PL: And your role there is?
NS: I'm head of organisational development and training.
PL: Okay. So how many people are we talking about?
NS: We’re talking roughly about 1,000 people in total, plus about 500 volunteers.
PL: So both of you can you give us a bit of a picture of what areas of HR this sort of approach can be used in?
JG: So there is a whole range of people management areas that are ripe for the picking for evidence-based practice. I mean if we’re talking about motivation what actually motivates people? If we’re talking about what the link between motivation and performance is or the link between performance and well-being what does the best quality evidence actually say about these things because we can have many assumptions about these areas? Any area of practice really, whether it’s recruitment practice, performance management, on-boarding people, different approaches to pay, pretty much all areas of HR are ripe for the picking.
NS: In my role I'm responsible for introducing, to a large degree, new interventions or improving existing ones and so you’re right that the environment is rich for the picking in terms of evidence-based. But it’s not always the case that you come into a blank sheet of paper and can start off on the right foot. Very often you’re picking up something that's been happening in the same way over many years and actually to change that you’re going through a whole theory of change process as well as maybe introducing an evidence-based approach. I've been at ZSL now for a year and where I have found it easiest to introduce, and I won't say that I've tried very hard in other areas, but a part of ZSL is the Institute of Zoology and that is largely made up of research scientists and they are very keen to develop leadership, not only in their technical science but also in terms of what they call the ‘softer side’ of leadership. And their director is very open to getting some support from my role and my team in developing that. And with that open door, because let’s face it it’s not always an open door with senior people with leadership interventions or leadership programmes but with that open door I was able to start from a blank sheet of paper and start off seeking to take an evidence-based approach. I won't say it was brilliant but it’s certainly a start.
PL: So that was a quite helpful place to start wasn’t it because obviously academic community they’re used to evidence-based they use it in their own work so presumably you weren’t having to explain it from the ground up?
NS: Absolutely and actually that was kind of an in because talking from an evidence-based approach was really quite, I guess, attractive to the director of science and his senior team and so I kind of had to put my money where my mouth was and make sure that the approach was followed.
PL: Okay so can I talk about steps then what is the starting point? I mean presumably whatever problem you have or whatever you want to address you need to turn that into a question because you’re looking for answers aren’t you, is that right?
NS: Yes and I think our starting point with the senior research fellows as a group, a group of people together with the director of science, to talk about what indeed is that question that we’re seeking to address. And it’s a massive comfort zone for these people because that's how they operate, that's how they work. So actually we had close to a half day dedicated to exploring and unpicking what leadership might mean and bringing it down to a research question which we agreed was how to create a research ecosystem.
PL: Nice, I mean as you say it’s not always that straightforward to define the question is it? It sounds easy.
JG: I think you’re right it’s not easy and it’s absolutely crucial. So it’s about critical thinking really. You talk to employers or HR professionals about what your problem is and they’ll say, ‘The problem is we don't have a mentoring system,’ for example. And I say, ‘Well that's not actually your problem,’ that's what one of our colleagues, Rob Briner, would call ‘solutioneering’, it’s like defining your problem as the lack of a particular solution. It’s like no what are you actually trying to achieve? And you need to do that, you need to drill down and you need to go, what is it you're trying to achieve and why? How do you know that that is actually a problem in the first place? We need to reduce our absence levels. Okay what are your absence levels? Because very often people don't actually know what their absence levels are and if they do they certainly don't know how their absence levels compare to similar organisations in the same industry for example. So we’ve got to understand, have you got a problem in the first place? What is your problem? And try and not make that sound too aggressive, sort of saying to me ‘what’s your problem?’ but what is the problem you’re trying to fix and therefore what is the researchable question that you want answered?
PL: Okay so step one actually ask yourself is the question you think you’re addressing the question you should be addressing? Step two in my mind the best way is to find and select evidence, and I know this is a big question.
NS: It’s a really big question and it’s one that I struggled with if I'm honest. I think gathering the internal data and validating the assumptions we had around people’s misconceptions or misunderstanding of what leadership meant, and that involved meeting with the senior research fellows one to one, it involved looking at past trends in terms of development reviews and also these guys had been through promotion processes. So we had a lot of rich data to look through. But on the industry research, and I think this is possibly more of a learning and development area for myself and for my colleagues but actually accessing external data and meta-analysis on topics like leadership interventions that work, possibly the wrong question I'm asking there, but that wasn’t easy. I mean Google Scholar can give you a little bit but I felt like I was just scrambling around in the dark.
JG: Yeah so a couple of things strike me about that, one is that there is in a way an art about evidence-based practice. It sounds all very scientific but there's an art to actually doing it because we’re talking about bringing together different sources of evidence, different types of evidence, it’s not about turning practitioners into academics, it’s about drawing on that scientific, academic, research and also, as you say, bringing that together with organisational data and our own professional expertise and also stakeholder concerns. So we’ve got really quite different sources of data, we’re trying to bring them together and that's not a straightforward process.
PL: Yes, as Nichola says it sounds straightforward but it isn’t is it? Where do you find evidence? I mean obviously it depends on your sector, it depends what your organisation is but are there any useful steers we can give people about where to start with that?
JG: Yes so the type of question that would lead to the type of evidence that you’re looking for and that's why we need to be so clear about the kind of questions that we’re asking because it will take you to different places, if your question is about what’s the extent of this problem? Have we got a problem in this, that or the other? Then you’re going to be looking at organisational data, you may be looking at industry level data, etc. If your question is about this particular HR practice, does it work? Then you will need to look at academic research that's tried and tested and evaluated those practices with independent data – that's really important too.
PL: And of course that brings us to the seriously important question of how do you evaluate the data you gather because the world is full of information and data isn’t it? There's a quality issue. We’re not academics all of us so what are the guiding points there?
NS: Well I have to say that my north star was the one to one interviews that I had with each of the senior research fellows, that really clarified for me what their interest and their appetite and their questions lay.
PL: Tell us a bit about what you have done on the ground, we’ve talked about the process what have you done?
NS: So they were all really keen to say, ‘We don't want anything too formal,’ which made me smile, but they did want an opportunity to get together and have discussion and of course they were open to learning more which is the whole part of this process. So I was really keen to start facilitating them and hopefully inspiring them to have a different type of conversation and I knew from the conversations I had that they had never completed any kind of psychometric or leadership strengths analysis or assessment. So I ran the Clifton Strengths Finder 2.0 with the group which they all found very Americanised and they found the questions irritating and the reason I went for that tool was because they made their technical reports really accessible and I knew that I wouldn’t be able to answer the questions around standard deviation, reliability and validity as much as they would possibly want. So I was able to print off the technical reports, provide them on the day and I think that helped get it off to a really good start. They still had issues with those but it helped them understand that there was a kind of foundation and basis of research and evidence to the instrument. So it then gave them permission to relax a little, trust me a bit more and just get into the conversation around the diversity of strengths that they each bring because although highly intelligent, clever people they still had typical issues around tensions within teams, not talking enough to each other etc. So this helped them have a different kind of conversation and consider the differences that they each bring.
PL: That's interesting so in terms of behaviours after this process? I mean evaluation’s the next step what did you get?
NS: So very self-driven, so I've had the least to do in terms of being the facilitator that I've ever had with a group because they agree the actions and I said, ‘I’ll write these up for you,’ helpfully and they were like, ‘We know what the actions are.’ So they set up a bi-monthly lunch where they were all going to get together and they were going to take turns in sharing the issues or the research questions they wanted to talk more about.
PL: Were you invited to these lunches?
NS: No, although they were very friendly when they did see me.
NS: But that was when we met the second time they each referenced, really informally, but, ‘Oh you remember when we spoke about that when we met before?’ and it was the third lunch before the last one that's when we had that conversation. So you could tell it had started to become part of how they were operating. They are self-driven and they have started to create the community that they want to operate in for themselves and their teams.
PL: So this sounds like a success?
NS: So far, she says with fingers and toes crossed.
PL: It sounds good that you've given them the tools, they’ve picked them up, they're using them, it’s happening, are you thinking you'll now take this process into other areas of your work?
PL: That sounds good, so Jonny it’s an interesting story isn’t it and presumably the sort of experience a lot of HRs go through when they’re just in the foothills of doing this?
JG: Yeah and realistically I think most are in the foothills of doing this, HR as a profession is not as evidence-based as other professions. I think we can hold our hands up and be honest about that. But I think you’re right it’s a really nice example of how you can do this in practice. In terms of a model we have an established model which we’ve development with the Centre for Evidence-based Management saying here are six steps that you can go through in applying evidence-based practice, first of all searching for high quality research, so scientific, academic research, not everyone has access to it like you say, you can get a lot of stuff on Google Scholar. Knowing how to do the searches to begin with, there's a real craft to that. Then being able to understand the technical aspects of research and meta-analyses, but what we need to do is critically appraise that evidence, make sure that we’ve done a really good search of the evidence so that we’ve got all the bases covered, that we’ve found all of the best evidence in the area. Then we need to aggregate that and then the alchemy process is then taking that and turning it into really practical, prosaic, understandable, day to day recommendations and action points. And as you were saying Philippa as well there’s then a key role for practitioners to play in evaluating their own practices. So make sure you evaluate it, you know you've got loads of researchers, academics, who’d love to be able to help with that. And then you can be contributing back into the body of knowledge, so you’re not only drawing on the best available evidence out there but you can be contributing to it.
PL: The sense I get is there's quite a lot of resistance to evidence-based practice and there do seem to be quite a lot of misconceptions about it so can I put some to you and I’d be really interested to hear your answers? So here’s one for you, there's the sense that evidence-based approach ignores practitioners’ professional experience and it’s just all about numbers and stats.
NS: That doesn’t surprise me because it’s the kind of turgid, get stuck in so much analysis that you don't actually do anything and a practitioner can often feel like, I know the lay of the land, I know what’s needed here, let me just get on with it! And to be honest that's actually the expectations of many organisations, we pay your salary to do stuff, to deliver things and if you’re just doing lots of research that's not what we asked you to do.
JG: Yeah it’s the attitude of we pay you to know so if we've promoted you, you must know.
JG: I think linked to that there's a sense people have of we are evidence-based, of course we’re all evidence-based, yes in some sense we’re all evidence-based, we all draw on evidence in one way or another but evidence-based practice as a thing is something different.
PL: Okay, here’s another, it’s too time consuming, people are busy.
NS: I’d agree with that. It feels time consuming because I'm not very good at all of it and it’s about understanding at what points you need to get help and asking for that early on enough.
JG: It doesn’t have to be time-consuming so if you look at the hardest end of evidence-based practice, conducting full scale systematic reviews and meta-analyses yes that's very time consuming, that can take a couple of years for a team of academics but you don't need to do that, you can dip into it in fairly reliable ways without having to go down that route, but it’s a much better approach to look at say for example existing systematic reviews, or if we’re even more time pressured look at summaries, like practitioner based summaries.
NS: And I think there is a gap in the market for knowing where to access and find those summaries, I think that would be really, really helpful.
PL: So you’re convinced it’s the direction of travel we should all be moving in. Can I just briefly explore some of the pitfalls because obviously Nichola you've done this now, what were the things you felt actually I won't be doing that again?
NS: Because I was very conscious about following the steps and applying the process looking back I wish I’d just believed in myself a bit more.
JG: You mention the steps, so the steps of making sure you’re asking the right questions.
JG: Finding the right evidence, assessing it, aggregating it, pulling it together and drawing out lessons from it.
PL: So if people listen to this and are thinking, this is interesting I would like to know more about it, there is a lot .
JG: Yeah we’ve got stacks of stuff on the CIPD website, in particular with the new HR profession map, so first of all we articulate what we mean by being evidence-based, we’ve also got really nice examples on our website of topics that we’ve done that we’ve applied evidence-based practice to. So for example we’ve done a review on what really works in performance management, so that was called ‘Could Do Better’. We’ve also done taking a look at diversity and inclusion practices, again saying what are the actual outcomes and what are the drivers of diversity practices. What we’ve done in the profession map is to say, ‘Look here’s what you can do at this level, here’s something that's a bit more advanced, here’s a bit more advanced than that,’ and at the top you’re kind of evaluating your own programmes, you’re feeding back fantastic evidence into the body of knowledge.
PL: My thanks to Jonny, Nichola and Eric. For more information, as Jonny says, on evidence-based practice go to the website cipd.co.uk. Next month we’ll be looking at ethics. Now we’ve looked at ethics before – the business case, this time we’re going to be looking at flashpoints in organisations, what they are, how to spot them and how to deal with them. Join us then.