Donald Kirkpatrick gave us the most well-known process to assess the effectiveness of training. Yet there often exists an ongoing challenge to get our organizations to understand the importance of conducting the evaluation process as well as the integral role that management plays in evaluating the impact of training.
A Quick Review
Before I go any further, let’s briefly review Kirkpatrick’s 4-level evaluation model:
Level 1 evaluation (reaction) is what we do when, at the end of a training session, we ask for the learners’ feedback as to how they liked the content, the learning process, and all the other factors associated with delivering the training. The tools we use are primarily surveys to collect the learners’ impressions.
Level 2 evaluation (learning) is automatically built into the design of the workshops. They are the practice exercises and assessment opportunities the learners are given to examine what knowledge and skill they have learned.
Level 3 evaluation (performance) aims to measure how much of the knowledge and skill acquired by the learners is being applied on the job after training. In order to assess this we engage in on-the-job assessments, employee observation, and interviews to determine the impact of the training on the job.
Level 4 evaluation (results) looks to measure the return on investment the organization gained as a result of the training provided. To determine this we compare pre- and post-training indicators that are measured in numerous and various quantifiable ways.
The Buy-in Challenge
Reportedly, level 3 and 4 evaluations are the ones that fewer organizations engage in. Why? Because in order to determine the level 4 return on investment, there is a need to have the on-the-job performance improvement information from level 3 evaluations – and conducting these evaluations can also be the most disruptive to day-to-day operations.
To add to this fact, we must consider that level 3 on-the-job observations, assessments, and interviews are primarily the responsibility of the learners’ managers and supervisors as THEY are the ones who, on a daily basis, are in contact with the learners. But managers/supervisors may either not understand or embrace their role in this aspect of evaluation or may simply complain about not having the time to do it.
The Hope for Level 3
In order to achieve buy-in, support, and commitment from management to conduct their share of level 3 evaluations, we need to make it personal to them – the “what’s in it for me.” If you ask most managers or supervisors, “What is the ONE job responsibility you dislike the most?” the most common answer you will get is, “Annual performance evaluation.” The reason? It is time consuming and, quite honestly, most managers may not even remember what most of their employees have done beyond the previous two months, at best.
However, if some of the same tools used in training are made available to managers, they can assess how their employees are implementing the knowledge and skills they acquired in the training back on the job. By doing so, we are providing them with a process with which they can keep easy records of each employee’s monthly performance and areas for improvement. This makes the annual performance evaluation of employees a much quicker, easier, and less painful process for them as managers AND, at the same time, they are conducting level 3 evaluations for the previously provided training!
Many Langevin clients have implemented this approach with a rather good rate of success.
Tell us what you are doing in your organization to establish or improve level 3 on-the-job performance evaluations and the results and challenges that you are encountering. We'd love to read your comments in the section below!