Like any professionals, L&D leaders are under pressure to deliver results, gain credibility and demonstrate the value their training initiatives are bringing to the business. So how do you evaluate the impact of a training course and make sure it is aligned to the organisation’s strategy? What should we be measuring and how can objectives be established upfront?
Learning impact: let’s stop focusing on ROI
The Kirkpatrick model is the standard framework for evaluating training effectiveness. Level 1 of the Kirkpatrick model measures the learner’s reaction to the training and level 2 evaluates their understanding of it, level 3 considers the impact the learning has had on the learner’s knowledge and behaviour. More broadly speaking, level 3 helps evaluate the impact of the course on how that person works.
Traditionally, when assessing the impact of training, we think of ROI but there has been a lot of debate around the correlation between level 3 and level 4, which does not necessarily imply causation. On the other hand, the evaluation of a course goes beyond tracking course completion. So how can L&D give an evidence-based answer to justify the learning strategy to the business and all stakeholders?
Towards a holistic approach
At Enovation, we have been looking into how to assess and quantify the effect formal learning has on a learner when they return to their day-to-day duties. This moves away from the idea of a course being an event that is taken at some point in time and then potentially forgotten about, to one where learning is a process in an organisation which results in changes to the learner’s on-the-job behaviour. From this we can see it is important that learning outcomes are tied to performance objectives. When defining a learning outcome L&D should be considering how will the learning translate to a better performing employee.
Course Impact is a pragmatic approach to this. The tool allows for targeted pulse surveys that assess specific aspects of performance triggered by the completion of a course. Surveys can be sent to learners and those monitoring the learner’s performance at defined intervals.
Our Course Impact tool
Course Impact sends defined questions to the relevant audience when it is required. To cut down on the number of emails send to users, emails will be sent in digest form as configured (e.g. one email once a month with all relevant mails). The email will contain a link back to the LMS to complete a survey.
1/ Find the right KPIs
When choosing your specific evaluation questions at this level you will need to consider the following:
-Which particular job behaviours and/or competencies are expected to change as a result of the training programme?
-When, and over what timescale, will changes to the job behaviours and/or competencies be measured?
-What factors other than the training programme might influence changes to each job behaviour and/or competency? For example, have managers and supervisors supported learners in applying their new skills, have there been changes to organisational structures, have performance incentive schemes been introduced, or have there been other external environmental influences?
2/ Communicate with the stakeholders
Each of the above questions should be discussed and, where relevant, agreed with key business stakeholders in advance of the training development. The conclusions can then be integrated within both training and evaluation objectives and plans. Stakeholders should also agree how success will be judged and how the evaluation results will be used.
3/ Measure the feedback
These actions will help ensure that the evaluation is aligned to the organisation’s strategic objectives, is feasible to carry out and can also help in gaining support and engagement from training participants and other stakeholders during the evaluation process. They will also help to motivate managers to support learners to transfer their learning to the workplace, which should in turn have a positive impact on the training outcomes.
