Can Kirkpatrick Level 1 Data Give Level 3 Results?
by Steven B. Just

In the Kirkpatrick hierarchy Level 1 evaluations (smile sheets) are low man on the totem pole, while Level 3 results (Transfer of Training) approach the training holy grail of business impact/return on investment. So is it possible to use Level 1 evaluations in such a way that they may indicate “Transfer of Training?” The, perhaps somewhat surprising, answer is a qualified “yes.” The secret is to use Level 1 evaluations longitudinally -- and to ask the right questions.

Level 1 evaluations are generally given right after a training experience, which provides only an immediate impression of the value of the training. For many reasons immediate impressions are not necessarily valid. We might have been enamored of the presentation style of an instructor or the cool graphical interface of an online course, but, like cotton candy, perhaps there wasn’t much substance there, and an immediate, positive evaluation might turn into a negative evaluation in three or six months.

Consider the analogy of reading a book: We’ve all had the experience of reading a book that we immediately loved, then six months later we can’t remember the plot of the book, or perhaps that we’ve even read it. On the flip side most of us have also had the experience of reading a book that was difficult or challenged some of our dearly held beliefs, so our initial reaction to the book was negative. But six months later we realize the book had a profound impact on us. The same can be true for training experiences.

So in a Level 1 evaluation include some questions related to transfer (e.g. “What is the likelihood you will use fill-in-the-blank on the job?”). Then ask the same questions in follow-up evaluations at three and six-month intervals (e.g. “Rate the usefulness that fill-in-the-blank has been on the job”, “When was the last time you used fill-in-the-blank in your work?”). Analyzing this longitudinal data can help you understand what types of training experiences “stick” and what types just “melt away.”