More on Re-evaluating Evaluation – Jack Phillips and ROI

Dashe & Thomson

I have been blogging a lot about Training Evaluation this year—mostly Kirkpatrick , but also Brinkerhoff and Scriven. I just realized that I haven’t included a single word about Jack Phillips , who introduced Return on Investment (ROI) as Level 5 to Kirkpatrick’s Four Levels of Evaluation. My first exposure to Phillips’ ROI—although I didn’t realize it at the time—was through a colleague who introduced me to Kirkpatrick’s Four Levels.

The Kirkpatrick-Phillips Model – Part 4

CommLab India

In the third part of this series, we looked at the advantages and disadvantages of the Kirkpatrick Model of Evaluating a training program. I mentioned the Kirkpatrick-Phillips Model of Evaluation in passing, and promised you more on this “ evolved ” model. I call this an “ evolved ” model because, the Kirkpatrick-Phillips Model of Evaluating a training program is based on the Kirkpatrick Model of Evaluation. Kirkpatrick Model.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Kirkpatrick’s Four Levels of Evaluation

Learnnovators

It was while writing his thesis in 1952 that Donald Kirkpatrick became interested in evaluating training programs. In a series of articles published in 1959, he prescribed a four-stage model for evaluating training programs, but it was not until 1994, that he published “ Evaluating Training Programs: The Four Levels “ According to Kirkpatrick, evaluating training programs is necessary for the following reasons: 1.

Training Evaluation Beyond Kirkpatrick

Training Industry

Today, when we discuss training evaluation, we likely reference Kirkpatrick’s model with the familiar four levels: reaction, learning, behavior and results. However, several other models have been developed throughout the past 50 years that delve into training evaluation. Also, some models have expanded the process by adding additional steps after Kirkpatrick’s results to capture the broader impact on and value creation for society.

KIRKPATRICK’S FOUR LEVELS OF EVALUATION

Learnnovators

It was while writing his thesis in 1952 that Donald Kirkpatrick became interested in evaluating training programs. In a series of articles published in 1959, he prescribed a four-stage model for evaluating training programs, but it was not until 1994, that he published “ Evaluating Training Programs: The Four Levels “. According to Kirkpatrick, evaluating training programs is necessary for the following reasons: 1.

An Overview: Phillips' Model for Learning Evaluation

Watershed

When we originally published our Learning Evaluation eBook and blog series, we covered four learning evaluation models representing a range of approaches and perspectives: Kirkpatrick, Kaufman, Brinkerhoff, and Anderson. While there are many other learning models we’ve yet to cover, the one we’re asked about most often that’s not mentioned in our original series is Phillips' Learning Evaluation Model. Meet the fifth level of learning evaluation.

Is Kirkpatrick’s Model of Evaluating a Training Program The Best? – Part 3

CommLab India

This is the third blog in the Kirkpatrick Model of Instruction series. In the first part of the series, I covered the need to evaluate any training program and the basics of the Kirkpatrick model of evaluating a training program. In the second part of this series, I delved into each level of the Kirkpatrick model. Here’s what we know about the benefits of the model: The model can be used to evaluate classroom training as well as eLearning.

Determining The ROI Of eLearning – Using Kirkpatrick’s Model Of Training Evaluation

EI Design

In this article, I outline how you can use the Kirkpatrick’s model of training evaluation to measure training effectiveness, its impact, and the ROI of eLearning. ROI determination methodology: One of the popular models used for ROI determination is the Kirkpatrick’s model of training evaluati on. What Is Kirkpatrick’s Model Of Training Evaluation? How Can You Practically Use The Kirkpatrick Model Of Training Evaluation To Determine The ROI?

Determining The ROI Of eLearning – Using Kirkpatrick’s Model Of Training Evaluation

Adobe Captivate

In this article, I outline how you can use the Kirkpatrick’s model of training evaluation to measure training effectiveness, its impact, and the ROI of eLearning. ROI determination methodology: One of the popular models used for ROI determination is the Kirkpatrick’s model of training evaluation. What Is Kirkpatrick’s Model Of Training Evaluation? How Can You Practically Use The Kirkpatrick Model Of Training Evaluation To Determine The ROI?

ROI 56

The Phillips ROI MethodologyTM – Measuring Data at All Levels – Part 5

CommLab India

This blog is the 5 th part of the Kirkpatrick series that I have been writing about over the last few weeks. Part 1 , Part 2 , and Part 3 of this series dealt with the Kirkpatrick Model of evaluating a training program. In the fourth part of the series, we saw how the Phillips ROI Methodology TM suggests that measurements be taken at every level.

Front-End Analysis: Backward Analysis and the Performance Gap

Dashe & Thomson

ADDIE Backward Analysis Cause Analysis Four Levels of Evaluation Front-end Analysis Instructional Design Jack Phillips Joe Harless Kirkpatrick Performance analysis Performance Gap Causal analysis Front-end analysis Performance Analysis performance improvement

Free eBook: Practical Approaches To Determine The ROI Of eLearning – Using Kirkpatrick’s Model Of Training Evaluation

EI Design

This free eBook, Free eBook: Practical Approaches To Determine The ROI Of eLearning – Using Kirkpatrick’s Model Of Training Evaluation, provides practical tips that you can use to measure the effectiveness of online training, leading to the ROI on your training spend. Practical Approaches To Determine The ROI Of eLearning – Using Kirkpatrick’s Model Of Training Evaluation. What is Kirkpatrick’s model of training evaluation?

Short Course on Evaluation of Training and Learning in Knowledge Economy

The Performance Improvement Blog

LAD Global , in partnership with the Singapore Training and Development Association , has made my short course on evaluation of training and learning available for free online. My emphasis in this course is on using measurement and evaluation for learning. Much of evaluation in organizations today is still focused on formal training programs and limited to Kirkpatrick’s “level one”.

Learning to Learn from Evaluation of Learning

The Performance Improvement Blog

The Kirkpatricks have four levels, the Phillips have ROI, and Brinkerhoff has the Success Case Method. Each approach to evaluation of training has something to contribute to assessing the impact of formal training on employee learning. However, the value of evaluation is not in the data. The real value is in organizational learning from evaluation. Learning from evaluation depends on asking the right questions.

Evaluating Training Effectiveness and ROI

Geenio

To evaluate the effectiveness of education and its practicability from the financial standpoint, it is necessary to calculate the expenses connected with creating and conducting the trainings, gauge the results achieved by the employees who have taken part in them, and decide whether the increase in the employees’ efficiency and company profits was sufficient to recoup the associated costs. The Donald Kirkpatrick’s Learning Evaluation Model consists of four levels: Level 1.

ROI 100

What Is Learning Evaluation?

Watershed

Knowing how to aggregate and evaluate all types of learning is a priority for many learning and development practitioners. Technologies are becoming available that enable L&D professionals to aggregate, evaluate, and measure many types of learning data in one place. Key Learning Evaluation Takeaways. So, what is learning evaluation? We'll explain how you can effectively evaluate your learning and development process from design through implementation.

Three Superheroes of Training Evaluation

Growth Engineering

Well, let’s look at the heroes of training evaluation and how they helped make sense of training evaluation! First came Donald Kirkpatrick. Followed by Roger Kaufman and then Jack Phillips. This trio laid the foundations that help you evaluate your training. Hero Profile 1: NAME: Donald Kirkpatrick. KNOWN FOR: The four levels of learning evaluation. He is, without doubt, the undisputed king of training evaluation.

Impact and ROI of Learning: Worth Pursuing or Not?

CLO Magazine

Instead, they call for showing the alignment of learning to business goals, focusing on easier-to-measure participant reaction, amount learned and application (levels 1, 2 and 3, respectively, of the Kirkpatrick Model/Phillips ROI Methodology) and finally focusing on employee engagement with learning (consumption of learning and completion rates). In conclusion, we must not shy away from higher levels of evaluation because they are harder or more subjective.

ROI 47

Evaluating Training – Capturing the Benefits Aspect of ROI

Obsidian Learning

New post Evaluating Training – Capturing the Benefits Aspect of ROI on Obsidian Learning. Training evaluation is necessary and, in many ways, critical to the success of a business. And even if training evaluation is undertaken, it is usually at the easiest and lowest level: the measurement of student reactions through simple surveys or happy sheets. Determining the benefits of the training is more difficult, and involves knowing how the training program should be evaluated.

What’s LEO’s Chain of Evidence Learning Evaluation Model?

Watershed

It roughly maps to Kirkpatrick’s learning evaluation levels one to four; however, LEO’s model has been developed through practical experience to be more pragmatic in a blended learning context. The chain of evidence model is similar to Kirkpatrick, but adjusted to incorporate the practical reality of corporate learning. Actually, we’ve called on a lot of the learning evaluation models you may be familiar with—Kirkpatrick, Kaufman, Phillips, Anderson, and the like.

What’s the ROI on Disengaged Employees?

Dashe & Thomson

Training evaluation appears to be the Holy Grail of the L&D world. It’s one of the hottest topics in the blogosphere and at conferences (see Barbara Camm’s highly trafficked posts on evaluation), and yet it remains extremely elusive. From Kirkpatrick’s Four Levels of Evaluation, to Michael Scriven to Jack Phillips [.] social learning evaluation millenials ROI social media social technology

ROI 154

Watershed's 7 Steps of Learning Evaluation

Watershed

Out of the dozens of different learning evaluation models currently in practice, which one is right for you? Meet our "super method" for learning evaluation. We've been using Watershed Insights to dig into several models of learning evaluation in light of what's possible with modern technologies such as xAPI. Here's an overview of our practical seven-step evaluation process that you can implement in your organization: Step 1: Align learning program evaluation with priorities.

Evaluation and ROI in Compliance Training

Interactive Services

But what about evaluation and return on investment (ROI)? . Evaluation and ROI are important indicators of how well a program is working and how much risk is mitigated. When it comes to evaluating and determining the ROI of your compliance initiative, heed the advice of the popular business book written by John Doerr and measure what matters. Evaluating Compliance Training Programs.

Commonly Used Training Evaluations Models: A Discussion with Dr. Will Thalheimer

Convergence Training

And that’s especially true when it comes to issues regarding learning evaluation. We were excited to be able to talk with Dr. Thalheimer about four common learning evaluation models, and we’ve got the recorded video for you below. If you prefer your learning evaluation information in written form, just scroll down for the transcript of our discussion. Many thank to Will for participating in this discussion on learning evaluation and for everything he does.

The Training Evaluation Framework of the Future: The GEvaluation Framework

Growth Engineering

When Donald Kirkpatrick first formed his training evaluation framework, it was 1964 and the world was a very different place. Back in these simple times, The Kirkpatrick Model worked well – because learning took place in a classroom it was much easier to measure its success. Even though a lot has changed in half a century, the models we use to evaluate our learning haven’t. So, it’s time we brought our evaluative frameworks up to date.

Is this thing on? Tips for measuring course effectiveness and return on investment

Obsidian Learning

The Kirkpatrick four levels of training evaluation. While later stages of evaluation measure more obvious aspects of quality—such as the impact of the training on the learner—it’s important not to overlook the less obvious factors, such as instructional design or the use of technology. Usability Testing: During course design and prototype development, evaluate the course for usability. Evaluation: The Four Levels. Level 2: Learning Evaluation.

Don Kirkpatrick’s Contribution to Learning & Development

CLO Magazine

dissertation at the University of Wisconsin and needed some good measures to evaluate the impact of a training program for supervisors. All four were important to evaluate a program. Throughout the ’50s he gave talks at conferences and referred to his evaluation framework, which he continued to use and refine. Once published, the four levels really took off and became the standard for evaluation.

Isolating the Results of eLearning Impact

Integrated Learnings

A recent project renewed my interest in Level 3 and Level 4 evaluation methodologies. That led me to purchase the book, Isolation of Results , by Jack Phillips and Bruce Aaron. To make sure we’re on the same page, Level 3 evaluation refers to measuring transfer of training to the job in terms of observable behaviors. For more detail, skim a quick review of Kirkpatrick’s four levels of evaluation. What have you read about evaluation? By Shelley A. Gable.

The Training Evaluation Framework of the Future: The GEvaluation Framework

Growth Engineering

When Donald Kirkpatrick first formed his training evaluation framework, it was 1964 and the world was a very different place. Back in these simple times, The Kirkpatrick Model worked well – because learning took place in a classroom it was much easier to measure its success. Even though a lot has changed in half a century, the models we use to evaluate our learning haven’t. So, it’s time we brought our evaluative frameworks up to date.

What's the problem with Kirkpatrick?

Clive on Learning

The aim of the session aim was to 'present a fresh perspective on evaluation', based on work carried out for the CIPD by the University of Portsmouth. While acknowledging that Kirkpatrick's four levels of evaluation had proved valuable over more than thirty years in helping to measure instructor-led, content-based 'training' interventions, Sloman felt that a new approach was necessary to support a more self-directed, work-based 'learning' process.

How to Improve Learning Retention with Contextual Feedback

CourseArc

The 4 Levels of Evaluation are: Reaction, Learning, Behavior, and Results. If you want to learn more about Donald Kirkpatrick and his training evaluation model, visit [link]. Kirkpatrick’s 4 Levels of Evaluation are: Reaction, Learning, Behavior, and Results. While the answer you selected includes some of Kirkpatrick’s Levels of Evaluation, the aspect of Return-on-Investment was developed by Jack Phillips.

Putting Thought to Work: Evaluation in Practice

CLO Magazine

Organizations that have reached higher measurement levels use a blended approach for the various frameworks and find ways to customize evaluation. The company organizes its learning evaluation data using a tool, Metrics That Matter, from KnowledgeAdvisors that connects directly to its learning management system. Then she said the tool automates levels 1 through 5 — including the Phillips ROI model — to help learning leaders report and analyze the data.

How do you measure how training creates value? – The 7 learning principles

Learning Wire

Let’s begin with the evaluation of learning. Why evaluating? Are there credible measurement methods to evaluate value creation? The Kirkpatrick/Phillips model shows us how and why to assess training outcomes. What should we evaluate? Evaluating value creation involves comparing the cost of an investment with what you get in return. According to Phillips (2012), intangibles are benefits that cannot be given a monetary value.

[Free Webinar] How to measure and maximize the ROI of eLearning

Adobe Captivate

The approach uses Kirkpatrick’s model of training evaluation and Kirkpatrick-Phillips evaluation model of training to measure training effectiveness and arrive at the ROI. What is Kirkpatrick’s model of training evaluation? What is Kirkpatrick-Phillips evaluation model of training? • How can you practically use these models of training evaluation and determine the ROI?

How do we measure value creation from training?

Learning Wire

In a previous article we looked at the five levels of impact in any exhaustive training evaluation process that reflect the key challenges involved in high-quality evaluation. We also looked at the limitations of evaluations due to non-measurable (intangible) factors. Training evaluation at a glance. Evaluating training is no easy task. If you restrict yourself to sending out a satisfaction form, that’s not going to be enough for a complete evaluation.

Are learning objectives really that important?

The eLearning Nomad

3) And they are useful when we evaluate courses: have the learners achieved the objectives we have set for the course? The predetermined objectives approach was first introduced in a school-based setting in 1932 by Ralph Tyler, an influential American educator, specialised in assessment and evaluation. To this extent it is unsuitable as a general purpose model of evaluation for adult learning.”.

Workplace Learning 2025 – What is the ROI of learning and development?

Learning Pool

The ROI Institute, which helps organisations evaluate the success of programmes using the ROI methodology developed by its chairman Jack Phillips, found that Global Systems Corporation achieved an almost eight-fold ROI of training tailored to help new recruits enter the corporate world, increase productivity and reduce attrition. For every organisation and any type of training, the same general formula for evaluating the success of L&D programmes applies,” she explains.

ROI 59