The Kirkpatrick model
The seminal model for L&D evaluation, first published in the 1950s by US academic Don Kirkpatrick remains influential today. Research conducted by Thalheimer indicates this model was first introduced by Raymond Katzell.
It outlines four levels for evaluating training:
- Reactions – reaction to a training intervention.
- Learning - ‘principles, facts etc absorbed’.
- Behaviour - ‘using learning gained on the job’.
- Results - ‘increased production, reduced costs, etc’.
This was helpful guidance when launched. In the 1980s Alliger and Janak found that the relationships between the levels were weak because each level is not always linked positively to the next.
Various surveys from the Association for Talent Development and our 2021 survey show attention is focused on evaluation of learning at the reaction level due to the time costs of measuring the other three levels.
Brinkerhoff success case method
A criticism of Kirkpatrick’s evaluation model is that changes to performance cannot solely be linked to learning. The Brinkerhoff success case method (SCM) addresses this challenge by proposing a wider focus.
An SCM evaluation involves finding likely ‘success cases’ where individuals or teams have benefited from the learning. These typically come from a survey, performance reports, organisational data or the ‘information grapevine’. Those representing potential ‘success cases’ are interviewed and ‘screened’ to find out if they genuinely represent verifiable success with corroborating evidence from other parties. Factors that contribute to success beyond the learning intervention are also explored.
An SCM evaluation also looks at ‘non-success cases’ to discover those who have found little or no value from the learning. Exploring the reasons why can be very illuminating.
The approach asks four questions:
- How well is an organisation using learning to improve performance?
- What organisational processes/resources are in place to support performance improvement?
- What needs to be improved?
- What organisational barriers stand in the way of performance improvement?
Following analysis, the success and non-success ‘stories’ are shared.
SCM is not itself a comprehensive evaluation method due to the nature of the sampling, it does however offer a manageable, cost-effective approach to determine success and improvement insights.
Weinbauer-Heidel’s levers of transfer effectiveness
Weinbauer-Heidel has published her approach called 12 Levers of Transfer Effectiveness. She recommends asking learners ‘What is the likelihood of you applying this learning?’ and using a ‘Net Transfer Score’ to demonstrate the impact of L&D.
Thalheimer’s Learning Transfer Evaluation Model
The Learning Transfer Evaluation Model is divided into 8 tiers and colour-coded to work as a kind of barometer using a traffic light system: green shows which methods are most useful in validating learning results, while red shows those which are inadequate in measuring learning.
Philips' return on investment model
Philips and Philips built on the Kirkpatrick model by adding return on investment (ROI). However, much ROI evaluating is carried out post project and does not build from a baseline. The arithmetic of ROI means that when a small cost learning intervention is set against a big project cost, it can look superficially impressive.
Some commentators ask whether a financial model represents the best way to address the effectiveness of learning. Does stating an ROI of x% help an organisation address its performance gaps and allow the L&D team to communicate their impact?
The value of using models to approach evaluation
Each evaluation model demonstrates a specific approach and some were developed to assess the value of individual training programmes not an holistic approach to L&Ds impact on the organisation.
Commonly called a ‘happy sheet’, this reaction activity is an established approach to gain learner feedback. It looks at learner satisfaction levels of, for example, the facilitator, materials, venue etc. Katzell and Kirkpatrick advise in their first level of evaluation the learner’s reaction to the learning itself needs to be recorded.
