Expert insights. Personalized for you.
In a series of articles published in 1959, he prescribed a four-stage model for evaluating training programs, but it was not until 1994, that he published “ Evaluating Training Programs: The Four Levels “ According to Kirkpatrick, evaluating training programs is necessary for the following reasons: 1. Since Kirkpatrick stated his original model, other theorists like Jack Phillips, have referred to a fifth level, namely, Return On Investment (ROI). MORE
People talk about the Philips model (aka ROI) , the Kaufman model , Roger Kaufman’s work, and Rob Brinkerhoff as well through the success case methodology. And okay, if we can just walk through each of the four models you talked about–Kirkpatrick, Phillips, Kaufman, and Brinckerhoff–and maybe you can explain to people, especially people who may not have heard of any of these, what they are and what are some pros and cons of each. Jack & Patti Phillips: [link]. MORE
Phillips is the chairman, and Patti P. Phillips is president and CEO of the ROI Institute. In his 2017 Chief Learning Officer article, “The Business Case for Leadership Development,” William C. Byham, founder and CEO of Development Dimensions International, notes the following: “An article in McKinsey Quarterly accused U.S. The Wall Street Journal ran an article a few years ago titled “So Much Training, So Little to Show for It.” Jack J. MORE
Select the instructional methods that will best help employees satisfy the learning objectives (see our article on evidence-based training methods here). The Phillips “ROI” Model. We’ll try to cover those in some later articles. MORE
In this article, I outline how you can use the Kirkpatrick’s model of training evaluation to measure training effectiveness, its impact, and the ROI of eLearning. How To Determine The ROI Of eLearning. The measurement of ROI of eLearning needs an integrated approach that should begin during the Training Needs Analysis or TNA phase and should successively build up right up to the determination of its impact on business. MORE
Following a keynote by Ed Cohen, former CLO at Booz and now head of Satyam Computer Services’ School of Leadership, we launched into a ThinkTank session chaired by Cedric Coco of Lowe’s with observations from Kent Barnett (Knowledge Advisors), Michael Echols (Bellevue University’s Human Capital Lab), and Patti Phillips (of The ROI Institute). ROI is the answer. The conversation in the room implied there was one best way to measure ROI. MORE
The conversation that began with Kirkpatrick and continues with Phillips takes place at virtually every training conference, is represented as an article in almost all training publications and is a topic of debate whenever training professionals meet. The dialog focuses on metrics, or the lack thereof; ROI and the challenge of measuring it; and the meaning of “business impact.” MORE
In a series of articles published in 1959, he prescribed a four-stage model for evaluating training programs, but it was not until 1994, that he published “ Evaluating Training Programs: The Four Levels “. Since Kirkpatrick stated his original model, other theorists like Jack Phillips, have referred to a fifth level, namely, Return On Investment (ROI). It was while writing his thesis in 1952 that Donald Kirkpatrick became interested in evaluating training programs. MORE
Those benefits are exactly why our experts drafted this article for your team. In 2003, Dr. Jack Phillips introduced the fifth level as the return of investment for the training. ROI = (Total Program Benefits – Total Program Costs ROI/ Total Program Costs) x 100%. Dr. Phillip’s proposed addition created a training assessment model that shaped how online training can benefit teams by providing an accurate scope of insight to a business leader. MORE
Over the past decade or so, learning professionals have been publishing articles like: . “ In an Insider Higher Ed article, “ Learning Engineers Inch Toward the Spotlight ,” employers were looking to hire “for a new role, and they didn’t want to call it an instructional designer.”. MORE
While many acknowledge the value of human resource development, few companies are prepared to analyze ROI on an asset as “fuzzy” as a well-trained employee. For more information on evaluating ROI, check out my colleague Barbara’s informative post: More on Re-evaluating Evaluation – Jack Phillips and ROI. David Kirkpatrick, in an article for Forbes claims that Now Every Company Is a Software Company. MORE
This article is designed to be your A-to-Z guide to learning analytics. As a refresher , here is an article on What is Learning Analytics ?). Evaluating Learning Analytics and Measuring ROI. Data Points to Prove the ROI of Learning Analytics. Adding ROI to Kirkpatrick. MORE
For many years, calculating ROI (return on investment) of continuous training has been difficult, especially with the pressure of Direction Committees and stakeholders, with expenses sometimes hard to justify with actual and tangible results. ‘The main issue for Human Resources is the calculation of its return on investment.’ How do we calculate the ROI of continuous training? But there is a summary at the end of the page, for people in a rush Which ROI? MORE
Reading up on a topic in a handbook article written by an expert can only better prepare you to hunt down (and properly digest) whatever granular information you ultimately need to find. For example when contemplating the topic of evaluation several influential and respected names immediately come to mind: Kirkpatrick, Phillips, Brinkerhoff, and others. When comparing the two lists, you can see there are some immediate matches: ROI and Phillips; the Four Levels and Kirkpatrick. MORE
Determination of the ROI of online training is a hot topic today. In this article, I outline a popular ROI methodology (using Kirkpatrick’s model of evaluation), and 5 tips that you can use to maximize ROI in corporate training. Maximize The ROI Of Online Training In 5 Ways. Let me begin with a few basic definitions related to ROI of online training first: What Is Return On Investment (ROI) In Corporate Training? Which ROI Methodology Can Be Used? MORE
In a previous article we looked at the five levels of impact in any exhaustive training evaluation process that reflect the key challenges involved in high-quality evaluation. In this second article, we’ll be looking at the different aspects of training evaluation represented in diagram form, the three key principles that will help you to evaluate effectively, and a few tips to avoid the issue of intangibles. Phillips J, Pulliam Phillips P. MORE
In this article, I outline how you can use the Kirkpatrick’s model of training evaluation to measure training effectiveness, its impact, and the ROI of eLearning. How To Determine The ROI Of eLearning. The measurement of ROI of eLearning needs an integrated approach that should begin during the Training Needs Analysis or TNA phase and should successively build up right up to the determination of its impact on business. MORE
However, the focus is now shifting to determining its impact and the Return On Investment or ROI of online training. Measuring The ROI Of Online Training. In this article, I will begin with a quick summary of the benefits that eLearning offers, what ROI is, and how you can measure it. I will also outline the ROI methodology we use. I am quoting extensively from my earlier article Return Of Investment (ROI): Are You In? Do refer to this article. MORE
The return on investment concept for learning has been around since Jack Phillips introduced it about 40 years ago. Here is an example from an email I received: “Anyone who is involved in the business world is familiar with the concept of ROI. But when it comes to learning initiatives, gathering the necessary data to calculate ROI is difficult to put it mildly.” First, ROI used in the business world, should not be confused with ROI used in the learning world. MORE
Phillips is chairman and Patti P. Phillips is president and CEO of the ROI Institute. According to a September 2018 article in Forbes, Amazon’s Jim Bezos is the most innovative CEO by far. employees are given four hours each week of focused “bullet time,” according to an August 2018 article published in The Wall Street Journal. At Level 5, ROI can be calculated. Measurement great ideas innovation Multiple paths to yes ROIJack J. MORE
I hate ROI. Every learning professional reading this article can send me irate emails. I have been chasing ROI for the past 25 years. I’m not saying training does not contribute to a learner’s on-the-job performance, but in the sense of measuring a Kirkpatrick or Phillips level 4/5 impact, we all know “contributing” is not a strong word. For us to achieve a true ROI measure, it’s about proximity to performance. MORE
Following our previous article about the 7 learning principles of an impactful digital learning , we’re exploring each month one of them. The Kirkpatrick/Phillips model shows us how and why to assess training outcomes. Instead of a set of data, it involves a chain of measurable proofs of benefit, as shown in the Kirkpatrick/Phillips pyramid. The Kirkpatrick/Phillips pyramid identifies five dimensions that form a sequence: What is the value created for my company? MORE
If you’ve missed the previous article … Technology can make professional learning affordable, easy, and convenient. We address these issues, as we discuss return on investment (ROI), and the importance of establishing clear business needs and program objectives. Realizing that ROI Isn’t Difficult! Working out ROI for distance‐learning efforts needn’t be a mystery. Calculating ROI. Measuring ROI: Evaluation framework. MORE
By 1959, editors for the predecessor of ASTD’s T&D magazine had heard of his approach and asked him to write an article. He said there was too much content for one article; that he would need to spread it over four articles. Jack Phillips began applying the four levels in the late ’60s and later extended the work by adding ROI as the fifth level. I just saw an announcement that Don will retire from the speaking circuit this year. MORE
However, the focus is now shifting to determining its impact and the Return On Investment or ROI of online training. Measuring The ROI Of Online Training. In this article, I will begin with a quick summary of the benefits that eLearning offers, what ROI is, and how you can measure it. I will also outline the ROI methodology we use. I am quoting extensively from my earlier article Return Of Investment (ROI): Are You In? What Is ROI? MORE
Calculating Return on Investment (ROI) is no easy feat for most any project. And, measuring the ROI for your training programs is no exception. What Is ROI? In simplest terms, ROI is the gain that results from an investment. There are a number of online resources that explain ROI in more detail. Useful Resources for Making Your ROI Calculations. We’ve got a few articles of our own on the topic of ROI. MORE
CLO: Let’s talk about your new book, “ Forward-Focused Learning: Inside Award-Winning Organizations ” — how did you come up with the concept, and what would you like readers of this article to know? MORE
The longtime University of Wisconsin at Madison professor wrote a series of articles in 1959 for the American Society for Training and Development that outlined what became known as the four levels of evaluation — reaction, learning, behavior and results. Jack Phillips, chairman of ROI Institute Inc., Phillips, who started the ROI Institute in 1993 and has written a number of books on the subject, adds a fifth level, return on investment, to the taxonomy. MORE
The April 19 th issue of eLearning Guild’s Insider Newsletter has an article titled, The True Value of Online Training, which has a link to a whitepaper titled, How to Promote the Value of Online Training Within Your Organization. A simple ROI analysis of the savings can underscore the point within your own organization. chairman of the ROI Institute and a renowned expert on accountability and evaluation. “In MORE
You can read about these levels in-depth in my previous article, Getting to Know ADDIE: Evaluation. In this article, I would like to focus on the fifth level, which was suggested for addition by Jack Phillips. Return on Investment (ROI) When evaluating the effectiveness of training, it is customary to consider an additional level of the Kirkpatrick’s model, namely, the ROI methodology, developed by Jack Phillips in 1991. roi evaluation elearning MORE
Several articles in the past month have suggested the profession should step back from trying to isolate the impact of learning and the resulting return on investment. However, for certain types of learning there is still a need to measure impact and ROI (levels 4 and 5 of the ROI Methodology). Once the level 4 isolated impact is determined, ROI is very straightforward. MORE
Determination of Return On Investment , or ROI. We’ll cover QC in depth in another blog article, but here’s a summary of what to do: Storyboard/Content Validation: Test all course content for instructional effectiveness, grammatical accuracy, and stylistic clarity. ROI Results Evaluation. Return on Investment (ROI) measures the financial benefit to the organization of training. Phillips, J. Training Evaluation Kirkpatrick Levels of Evaluation ROI MORE
CLO Magazine
JULY 12, 2018
Several articles in the past month have suggested the profession should step back from trying to isolate the impact of learning and the resulting return on investment. However, for certain types of learning there is still a need to measure impact and ROI (levels 4 and 5 of the ROI Methodology). Once the level 4 isolated impact is determined, ROI is very straightforward.
Geenio
NOVEMBER 13, 2015
You can read about these levels in-depth in my previous article, Getting to Know ADDIE: Evaluation. In this article, I would like to focus on the fifth level, which was suggested for addition by Jack Phillips. Return on Investment (ROI) When evaluating the effectiveness of training, it is customary to consider an additional level of the Kirkpatrick’s model, namely, the ROI methodology, developed by Jack Phillips in 1991. roi evaluation elearning
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
CLO Magazine
SEPTEMBER 28, 2018
The return on investment concept for learning has been around since Jack Phillips introduced it about 40 years ago. Here is an example from an email I received: “Anyone who is involved in the business world is familiar with the concept of ROI. But when it comes to learning initiatives, gathering the necessary data to calculate ROI is difficult to put it mildly.” First, ROI used in the business world, should not be confused with ROI used in the learning world.
Adobe Captivate
MARCH 7, 2018
Determination of the ROI of online training is a hot topic today. In this article, I outline a popular ROI methodology (using Kirkpatrick’s model of evaluation), and 5 tips that you can use to maximize ROI in corporate training. Maximize The ROI Of Online Training In 5 Ways. Let me begin with a few basic definitions related to ROI of online training first: What Is Return On Investment (ROI) In Corporate Training? Which ROI Methodology Can Be Used?
CLO Magazine
OCTOBER 5, 2018
Phillips is the chairman, and Patti P. Phillips is president and CEO of the ROI Institute. In his 2017 Chief Learning Officer article, “The Business Case for Leadership Development,” William C. Byham, founder and CEO of Development Dimensions International, notes the following: “An article in McKinsey Quarterly accused U.S. The Wall Street Journal ran an article a few years ago titled “So Much Training, So Little to Show for It.” Jack J.
Coorpacademy
AUGUST 1, 2019
For many years, calculating ROI (return on investment) of continuous training has been difficult, especially with the pressure of Direction Committees and stakeholders, with expenses sometimes hard to justify with actual and tangible results. ‘The main issue for Human Resources is the calculation of its return on investment.’ How do we calculate the ROI of continuous training? But there is a summary at the end of the page, for people in a rush Which ROI?
EI Design
FEBRUARY 29, 2016
However, the focus is now shifting to determining its impact and the Return On Investment or ROI of online training. Measuring The ROI Of Online Training. In this article, I will begin with a quick summary of the benefits that eLearning offers, what ROI is, and how you can measure it. I will also outline the ROI methodology we use. I am quoting extensively from my earlier article Return Of Investment (ROI): Are You In? Do refer to this article.
EI Design
JULY 11, 2018
In this article, I outline how you can use the Kirkpatrick’s model of training evaluation to measure training effectiveness, its impact, and the ROI of eLearning. How To Determine The ROI Of eLearning. The measurement of ROI of eLearning needs an integrated approach that should begin during the Training Needs Analysis or TNA phase and should successively build up right up to the determination of its impact on business.
Adobe Captivate
OCTOBER 17, 2017
However, the focus is now shifting to determining its impact and the Return On Investment or ROI of online training. Measuring The ROI Of Online Training. In this article, I will begin with a quick summary of the benefits that eLearning offers, what ROI is, and how you can measure it. I will also outline the ROI methodology we use. I am quoting extensively from my earlier article Return Of Investment (ROI): Are You In? What Is ROI?
Mindflash
MAY 15, 2017
Calculating Return on Investment (ROI) is no easy feat for most any project. And, measuring the ROI for your training programs is no exception. What Is ROI? In simplest terms, ROI is the gain that results from an investment. There are a number of online resources that explain ROI in more detail. Useful Resources for Making Your ROI Calculations. We’ve got a few articles of our own on the topic of ROI.
CLO Magazine
JUNE 29, 2013
I hate ROI. Every learning professional reading this article can send me irate emails. I have been chasing ROI for the past 25 years. I’m not saying training does not contribute to a learner’s on-the-job performance, but in the sense of measuring a Kirkpatrick or Phillips level 4/5 impact, we all know “contributing” is not a strong word. For us to achieve a true ROI measure, it’s about proximity to performance.
|
Lambda Solutions
JUNE 14, 2020
This article is designed to be your A-to-Z guide to learning analytics. As a refresher , here is an article on What is Learning Analytics ?). Evaluating Learning Analytics and Measuring ROI. Data Points to Prove the ROI of Learning Analytics. Adding ROI to Kirkpatrick.
Jay Cross
SEPTEMBER 27, 2008
Following a keynote by Ed Cohen, former CLO at Booz and now head of Satyam Computer Services’ School of Leadership, we launched into a ThinkTank session chaired by Cedric Coco of Lowe’s with observations from Kent Barnett (Knowledge Advisors), Michael Echols (Bellevue University’s Human Capital Lab), and Patti Phillips (of The ROI Institute). ROI is the answer. The conversation in the room implied there was one best way to measure ROI.
The Performance Improvement Blog
APRIL 21, 2010
The April 19 th issue of eLearning Guild’s Insider Newsletter has an article titled, The True Value of Online Training, which has a link to a whitepaper titled, How to Promote the Value of Online Training Within Your Organization. A simple ROI analysis of the savings can underscore the point within your own organization. chairman of the ROI Institute and a renowned expert on accountability and evaluation. “In
CLO Magazine
JANUARY 23, 2019
Phillips is chairman and Patti P. Phillips is president and CEO of the ROI Institute. According to a September 2018 article in Forbes, Amazon’s Jim Bezos is the most innovative CEO by far. employees are given four hours each week of focused “bullet time,” according to an August 2018 article published in The Wall Street Journal. At Level 5, ROI can be calculated. Measurement great ideas innovation Multiple paths to yes ROIJack J.
Obsidian Learning
OCTOBER 30, 2015
Determination of Return On Investment , or ROI. We’ll cover QC in depth in another blog article, but here’s a summary of what to do: Storyboard/Content Validation: Test all course content for instructional effectiveness, grammatical accuracy, and stylistic clarity. ROI Results Evaluation. Return on Investment (ROI) measures the financial benefit to the organization of training. Phillips, J. Training Evaluation Kirkpatrick Levels of Evaluation ROI
Learning Wire
JULY 22, 2016
Following our previous article about the 7 learning principles of an impactful digital learning , we’re exploring each month one of them. The Kirkpatrick/Phillips model shows us how and why to assess training outcomes. Instead of a set of data, it involves a chain of measurable proofs of benefit, as shown in the Kirkpatrick/Phillips pyramid. The Kirkpatrick/Phillips pyramid identifies five dimensions that form a sequence: What is the value created for my company?
Dashe & Thomson
JUNE 22, 2012
While many acknowledge the value of human resource development, few companies are prepared to analyze ROI on an asset as “fuzzy” as a well-trained employee. For more information on evaluating ROI, check out my colleague Barbara’s informative post: More on Re-evaluating Evaluation – Jack Phillips and ROI. David Kirkpatrick, in an article for Forbes claims that Now Every Company Is a Software Company.
Learning Wire
SEPTEMBER 26, 2016
In a previous article we looked at the five levels of impact in any exhaustive training evaluation process that reflect the key challenges involved in high-quality evaluation. In this second article, we’ll be looking at the different aspects of training evaluation represented in diagram form, the three key principles that will help you to evaluate effectively, and a few tips to avoid the issue of intangibles. Phillips J, Pulliam Phillips P.
CLO Magazine
AUGUST 24, 2013
The longtime University of Wisconsin at Madison professor wrote a series of articles in 1959 for the American Society for Training and Development that outlined what became known as the four levels of evaluation — reaction, learning, behavior and results. Jack Phillips, chairman of ROI Institute Inc., Phillips, who started the ROI Institute in 1993 and has written a number of books on the subject, adds a fifth level, return on investment, to the taxonomy.
Training Industry
MARCH 21, 2017
The conversation that began with Kirkpatrick and continues with Phillips takes place at virtually every training conference, is represented as an article in almost all training publications and is a topic of debate whenever training professionals meet. The dialog focuses on metrics, or the lack thereof; ROI and the challenge of measuring it; and the meaning of “business impact.”
Coassemble
JULY 28, 2020
Those benefits are exactly why our experts drafted this article for your team. In 2003, Dr. Jack Phillips introduced the fifth level as the return of investment for the training. ROI = (Total Program Benefits – Total Program Costs ROI/ Total Program Costs) x 100%. Dr. Phillip’s proposed addition created a training assessment model that shaped how online training can benefit teams by providing an accurate scope of insight to a business leader.
Learning Wire
SEPTEMBER 14, 2015
If you’ve missed the previous article … Technology can make professional learning affordable, easy, and convenient. We address these issues, as we discuss return on investment (ROI), and the importance of establishing clear business needs and program objectives. Realizing that ROI Isn’t Difficult! Working out ROI for distance‐learning efforts needn’t be a mystery. Calculating ROI. Measuring ROI: Evaluation framework.
Learnnovators
JUNE 26, 2013
In a series of articles published in 1959, he prescribed a four-stage model for evaluating training programs, but it was not until 1994, that he published “ Evaluating Training Programs: The Four Levels “ According to Kirkpatrick, evaluating training programs is necessary for the following reasons: 1. Since Kirkpatrick stated his original model, other theorists like Jack Phillips, have referred to a fifth level, namely, Return On Investment (ROI).
Learnnovators
JUNE 27, 2013
In a series of articles published in 1959, he prescribed a four-stage model for evaluating training programs, but it was not until 1994, that he published “ Evaluating Training Programs: The Four Levels “. Since Kirkpatrick stated his original model, other theorists like Jack Phillips, have referred to a fifth level, namely, Return On Investment (ROI). It was while writing his thesis in 1952 that Donald Kirkpatrick became interested in evaluating training programs.
CLO Magazine
MARCH 1, 2011
By 1959, editors for the predecessor of ASTD’s T&D magazine had heard of his approach and asked him to write an article. He said there was too much content for one article; that he would need to spread it over four articles. Jack Phillips began applying the four levels in the late ’60s and later extended the work by adding ROI as the fifth level. I just saw an announcement that Don will retire from the speaking circuit this year.
CLO Magazine
MAY 21, 2021
Over the past decade or so, learning professionals have been publishing articles like: . “ In an Insider Higher Ed article, “ Learning Engineers Inch Toward the Spotlight ,” employers were looking to hire “for a new role, and they didn’t want to call it an instructional designer.”.
CLO Magazine
JANUARY 21, 2021
CLO: Let’s talk about your new book, “ Forward-Focused Learning: Inside Award-Winning Organizations ” — how did you come up with the concept, and what would you like readers of this article to know?
Convergence Training
NOVEMBER 19, 2019
People talk about the Philips model (aka ROI) , the Kaufman model , Roger Kaufman’s work, and Rob Brinkerhoff as well through the success case methodology. And okay, if we can just walk through each of the four models you talked about–Kirkpatrick, Phillips, Kaufman, and Brinckerhoff–and maybe you can explain to people, especially people who may not have heard of any of these, what they are and what are some pros and cons of each. Jack & Patti Phillips: [link].
Web Courseworks
AUGUST 4, 2014
Reading up on a topic in a handbook article written by an expert can only better prepare you to hunt down (and properly digest) whatever granular information you ultimately need to find. For example when contemplating the topic of evaluation several influential and respected names immediately come to mind: Kirkpatrick, Phillips, Brinkerhoff, and others. When comparing the two lists, you can see there are some immediate matches: ROI and Phillips; the Four Levels and Kirkpatrick.
Convergence Training
SEPTEMBER 13, 2020
Select the instructional methods that will best help employees satisfy the learning objectives (see our article on evidence-based training methods here). The Phillips “ROI” Model. We’ll try to cover those in some later articles.
122 
Let's personalize your content