How to Evaluate Learning: Kirkpatrick Model for the 21st Century—A Revision

Dashe & Thomson

I was asked by Wendy Kirkpatrick to remove the copyrighted Kirkpatrick diagrammatic model from my original blog post, How to Evaluate Learning: Kirkpatrick Model for the 21st Century. Then we need to identify specific metrics to demonstrate and deliver on those expectations.

To “Kirkpatrick” or not to “Kirkpatrick”, that is the Question (or is it?)

Learning Rebels

To “Kirkpatrick” or not to “Kirkpatrick”, that is the question. Many a person has debated the Kirkpatrick evaluation taxonomy. To name a few: Dan Pontefract: Dear Kirkpatrick’s: You Still Don’t Get It (a personal favorite).

Trending Sources

ROI and Metrics in eLearning

Tony Karrer

I'm working on an article around the use of ROI and metrics in eLearning. Kirkpatrick's Level 3: Improving the Evaluation of E-Learning Level 3 evaluations measure whether the there was an actual transfer of learning to the actual work setting. Support - ROI Calculator Training & Support Return on Investment Calculator Investing in Learning: Consider Value, Not Just ROI ROI, or return on investment, is king in today's business world.

MTA: Why the Kirkpatrick Model Works for Us

CLO Magazine

As he settled into his new job, Wiedecker read Jim and Wendy Kirkpatrick’s book, “Training on Trial,” which inspired him to implement the Kirkpatrick training evaluation model at the MTA. Implementing the Kirkpatrick Model.

Kirkpatrick Model Good or Bad? The Epic Mega Battle!

Will at Work Learning

In this third installment of the series, we''ve engaged in an epic battle about the worth of the 4-Level Kirkpatrick Model. Will: I want to pick on the second-most renowned model in instructional design, the 4-Level Kirkpatrick Model. A business person’s dream!

Alternative to the Kirkpatrick Model of Training Evaluation

LearnDash

If you have been in the elearning (or training) industry for any amount of time, then you are most likely aware of the Kirkpatrick model of learning evaluation. One could write an entire book on the Kirkpatrick model and the different levels, but I am not going to get into too much detail.

How to Evaluate Learning: The Kirkpatrick Model for the 21st Century

Dashe & Thomson

Kirkpatrick’s revised “Four Levels of Evaluation” model, what we need to do is find out what success looks like in the eyes of these senior managers and stakeholders and let them define their expectations for the training program.

What Are Your Training Metrics Actually Measuring?

Your Training Edge

Much has been written on the subject and many experts have weighed in on what they consider to be the most crucial training metrics ( here are my top 10 ). Other people have suggested implementation, application, business impact, and ROI. business impact).

Avoid and Correct Employee Evaluation Pitfalls

CLO Magazine

Unfortunately, Alan didn’t have data to link the revamped training program to those key sales metrics. Developing metrics that tie directly to desired business outcomes has been critical to not only our training but to our performance support success as well,” said Joanne S.

Why We Should Stop Talking About ROI in Training

Mindflash

For as long as Return on Investment, or ROI , has been prevalent concept in business, it’s also been a fixture of workplace learning and performance. Many people look at Don Kirkpatrick ’s work from as early as 1959 as the beginning of ROI in learning and development.

ROI 35

New Year L&D Resolution? Align L&D with the Business

CLO Magazine

One of the most difficult challenges facing CLOs is aligning learning and development with the organization’s business goals. Homepage Commentary & Opinion Strategy alignment business goalsTags: alignment , business goals. By Marina Theodotou.

How Employee Performance determines the Success of Your Training Program

eFront

It’s also an excellent way to gauge how employees are able to contribute to the business. And its best benefit – it is an avenue for rewards and recognition of the business’ top employee performance and thus, also an instrument for augmenting both employee morale and productivity.

How to Measure Online Course Effectiveness

CourseArc

Kirkpatrick’s Four-Level Approach to Assessing Training Outcomes. The first order of business is to assess how learners react to the course by asking… What did participants think about the course?

Train 31

Boost your L&D visibility & credibility – The Annual Learning Report

Learning Wire

This article enables you to create your own Annual Learning Report, offering an opportunity to boost your credibility and visibility for key business stakeholders. Critical remarks have been placed at commonly used models to capture the added value of L&D, as Kirkpatrick’s model.

More on Re-evaluating Evaluation – Jack Phillips and ROI

Dashe & Thomson

I have been blogging a lot about Training Evaluation this year—mostly Kirkpatrick , but also Brinkerhoff and Scriven. I just realized that I haven’t included a single word about Jack Phillips , who introduced Return on Investment (ROI) as Level 5 to Kirkpatrick’s Four Levels of Evaluation.

Measuring Learning Delivery

CLO Magazine

Most practitioners measure delivery and other aspects of a course using Kirkpatrick’s level 1, typically via a short survey asking about content, delivery and the environment, as well as relevance to the job. Kirkpatrick, who invented the four levels, recommends the survey be administered at the end of the class to get immediate feedback and to ensure a 100 percent response rate. The Business Of Learning learning delivery metrics and measurement

In Defense of the Four Levels

Integrated Learnings

Over the past year or so, I’ve noticed several comments about how Kirkpatrick’s model of four levels of evaluation is outdated. An initial gap analysis should identify specific business needs (level 4) and what is required to fulfill those needs (level 3). Though it’s common to track “butts in seats” and other attendance-related metrics not accounted for in the model, these measures seem more related to staffing and forecasting as opposed to training results. By Shelley A.

Measuring Training Program ROI

LearnDash

At McKinsey & Company, only 25% of managers surveyed believed that the training programs measurably improved business results. Associating these metrics to key business outcomes allows you to better measure the true impact of a training.

ROI 72

Conducting Post-Course Evaluations

CourseArc

The industry standard Kirkpatrick model measures training based on the four levels of analysis: Level 1: Did the learners enjoy training? Level 4: What business results can be attributed to the training? Determining the metrics of success before course development is the first step.

The Only Person Who Behaves Sensibly Is My Tailor

Performance Learning Productivity

Effective Metrics for Learning and Development I wonder what Shaw would think if he saw the way learning and development is predominantly measured in organisations today. These time-honoured metrics developed for an industrial age are not the answer. A parallel path to learning metrics.

Starting from the end

Clark Quinn

Week before last, Will Thalheimer and I had another one of our ‘debates’, this time on the Kirkpatrick model (read the comments, too!). The reason I like the Kirkpatrick model is it emphasizes one thing that I see the industry failing to do. The important point is starting with a business metric. The problems with Kirkpatrick are several. And the impact is what the Kirkpatrick model properly is about, as I opined in the blog debate.

50 Years of the Kirkpatrick Model

Upside Learning

In the fifty years since, his thoughts (Reaction, Learning, Behavior, and Results) have gone on to evolve into the legendary Kirkpatrick’s Four Level Evaluation Model and become the basis on which learning & development departments can show the value of training to the business.

Why Learning Measurement Matters

CLO Magazine

Without metrics, it is not clear that change is going in the right direction. In metrics, the “what” of our work is about what we measure. Well known in learning and development work are levels 1 through 4 by Donald Kirkpatrick. In the Bellevue University Human Capital Lab work with Verizon Wireless, the what consists of business outcomes. The operations management team’s alignment, for instance, is a critical component of business impact.

5 time-saving tips for your next learner survey

Sponge UK

On a basic level, you need to know their reaction to the experience (known as Level 1 in the Kirkpatrick model). Finding out whether the audience found the elearning user-friendly, relevant and engaging are useful metrics to add to the evaluation mix.

Level of ‘levels’

Clark Quinn

I was defending Kirkpatrick’s levels the other day, and after being excoriated by my ITA colleagues, I realized there was not only a discrepancy between principle and practice, but between my interpretation and as it’s espoused. Both of these fail to understand the intention: Kirkpatrick (rightly) said you have to start at level 4. As a standalone metric, it ranks right up there with measuring learning effectiveness by the pound of learners served.

Learning Measurement: A Work in Progress

CLO Magazine

Learning evaluation, metrics and measurement are not where they need to be, but some companies have made progress. Learning evaluation and measurement have come a long way since Donald Kirkpatrick, widely credited as a trailblazer in the field, first introduced his four levels of learning evaluation in 1959. But the industry’s pace in modernizing the practice to fit evolving business needs has lagged.

How to Measure the Impact of Training on Your Bottom Line

CourseArc

This is why business entities need a different way to justify their investments in training and continuing education. Each of the above benefits — and several others — are quantifiable metrics that can be used, separately or in tandem, as part of any ROI calculation.

Deeper eLearning Design: Part 1 – The Starting Point: Good Objectives

Learnnovators

It should be an organizational performance gap that is demonstrated by an available metric. Our successes should be business successes! Follow Kirkpatrick backwards from ‘business problem’ to ‘change in workplace behavior’ to ‘learning needed to accomplish that behavior’, and make sure that your learning objective is going to lead to new behaviors that will address the measure that’s lagging. This is the first post in a series of six that covers Deeper eLearning.

Looking Within: How To Gather And Analyze Actionable Learning Insights

TalentLMS

Learning insights are often referred to by other names, like learning analytics, or learning metrics. Data includes the raw figures, responses and statistics gathered from a Learning Management System’s (LMS) reporting and the metrics we set. Beware the vanity metric.

eLearning Process Survey results!

Clark Quinn

The first question was looking at how we align our efforts with business needs. Ideally, we start from a business metric we need to address and work backward.

The Elevator Pitch for Trainers

Training Industry

The conversation that began with Kirkpatrick and continues with Phillips takes place at virtually every training conference, is represented as an article in almost all training publications and is a topic of debate whenever training professionals meet. The dialog focuses on metrics, or the lack thereof; ROI and the challenge of measuring it; and the meaning of “business impact.”

Training Evaluation: Why You Can’t Ignore It

Growth Engineering

In our blog on the Kirkpatrick model we overview the four steps that make up a comprehensive evaluation. But, these metrics don’t always translate directly into the financial language that senior business people often use.

Pushing back

Clark Quinn

In a recent debate with my colleague on the Kirkpatrick model, our host/referee asked me whether I’d push back on a request for a course. You can help them here, doing some business consulting on ROI for them. And this is the path to start demonstrating that you are about business. You’ll have to start being about more than efficiency metrics (cost/seat/hour; “may as well weigh ’em”) and about how you’re actually impacting the business.

An Overview: Phillips' Model for Learning Evaluation

Watershed

When we originally published our Learning Evaluation eBook and blog series, we covered four learning evaluation models representing a range of approaches and perspectives: Kirkpatrick, Kaufman, Brinkerhoff, and Anderson. One of the most frequently quoted aspects of Phillips' model is the addition of a fifth level of evaluation to Kirkpatrick's Learning Evaluation Model , which is return on investment (ROI).

Calculating the ROI of training – ensuring the L&D budget

Matrix

The Kirkpatrick model of training evaluation has the fourth level that is aiming to measure the results but it only takes things as far as checking if the information taught is being transferred on the job.

ROI: It’s All About Proximity

CLO Magazine

What can I do to finally associate employees attending or consuming one of my training programs — be it instructor-led or virtual — with their ability to truly produce business results? I’m not saying training does not contribute to a learner’s on-the-job performance, but in the sense of measuring a Kirkpatrick or Phillips level 4/5 impact, we all know “contributing” is not a strong word. The next fair metric related to training is knowledge gain.

What is stopping companies from measuring learning: Skillsets, datasets, toolsets or mindsets?

Xyleme

In one of our recent posts, " Why you need to take a Google Analytics approach to measuring learning ," we lay the foundation of our perspective regarding L&D''s constant battle for greater relevance in business strategy and planning.