More on Re-evaluating Evaluation – Jack Phillips and ROI

Dashe & Thomson

I just realized that I haven’t included a single word about Jack Phillips , who introduced Return on Investment (ROI) as Level 5 to Kirkpatrick’s Four Levels of Evaluation. My first exposure to PhillipsROI—although I didn’t realize it at the time—was through a colleague who introduced me to Kirkpatrick’s Four Levels. Her four levels were Level 1: Reaction , Level 2: Learning , Level 3: Behavior , and Level 4: ROI. Trend line analysis.

Interview with Jack Phillips on ROI for eLearning


Jack Phillips is founder and CEO of the ROI Institute, Inc., Phillips developed the ROI Methodology™, a critical tool he has used for measuring and evaluating programs such as training, human resources, technology and quality programs and initiatives. We also provide certifications so that people can do the ROI analysis on their own. The challenge is that people who own programs may not want to measure them at the impact and ROI levels.


Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

The Phillips ROI MethodologyTM – Measuring Data at All Levels – Part 5

CommLab India

In the fourth part of the series, we saw how the Phillips ROI Methodology TM suggests that measurements be taken at every level. An analysis tool must be used to measure outcomes. Level 5: ROI. Methods to measure: Several ROI calculating tools are available. The formula to measure ROI is: As you can see, the most important parts of the Phillips ROI Methodology TM are collecting and analyzing data, and we have seen how this can be done successfully.

Showing the ROI of key programs on a budget

CLO Magazine

Here are 7 ways to minimize resources and still show the value of a major program at the impact and ROI levels. It’s important for every member of the team to share the responsibilities for designing, developing and delivering impact and ROI. 2: Build in data collection and analysis.

ROI 67

The Elusive ROI for Learning Through Technology by Jack Phillips and Patti Phillips

Learning Solutions Magazine

ROI studies show the results, in terms of transfer to the job and impact on business results, are. Here is an analysis of the. Few would dispute the convenience, low cost, and high efficiency of eLearning. However, dozens of. usually less than facilitator-led versions. It does not have to be this way. reasons, and some recommendations that will make a difference in your real-world results. Emerging Topics Management Measurement Professional Development Training Strategies

Does coaching deliver impact?

CLO Magazine

And finally, it is possible to show the value of coaching at the impact and ROI levels. This usually means evaluating coaching programs to the impact and ROI levels. Taking a few straightforward steps can ensure coaching delivers value and eliminates the probability of a negative ROI.

Impact and ROI of Learning: Worth Pursuing or Not?

CLO Magazine

Instead, they call for showing the alignment of learning to business goals, focusing on easier-to-measure participant reaction, amount learned and application (levels 1, 2 and 3, respectively, of the Kirkpatrick Model/Phillips ROI Methodology) and finally focusing on employee engagement with learning (consumption of learning and completion rates). However, for certain types of learning there is still a need to measure impact and ROI (levels 4 and 5 of the ROI Methodology).

ROI 39

5 Tips To Maximize The ROI Of Online Training

Adobe Captivate

Determination of the ROI of online training is a hot topic today. In this article, I outline a popular ROI methodology (using Kirkpatrick’s model of evaluation), and 5 tips that you can use to maximize ROI in corporate training. Maximize The ROI Of Online Training In 5 Ways. Let me begin with a few basic definitions related to ROI of online training first: What Is Return On Investment (ROI) In Corporate Training? Which ROI Methodology Can Be Used?

ROI 55

[Free Webinar] How to measure and maximize the ROI of eLearning

Adobe Captivate

As new and innovative approaches are gaining momentum, the focus is also shifting on determining the impact and Return on Investment or ROI of online training. So how can you effectively measure the ROI of your online training? Join us for a free webinar hosted by Asha Pandey, the Chief Learning Strategist at EI Design, where she will showcases EI Design’s integrated approach with a focus on what is the right action at each stage that will impact the ROI of eLearning positively.

Determining The ROI Of eLearning – Using Kirkpatrick’s Model Of Training Evaluation

Adobe Captivate

In this article, I outline how you can use the Kirkpatrick’s model of training evaluation to measure training effectiveness, its impact, and the ROI of eLearning. How To Determine The ROI Of eLearning. The measurement of ROI of eLearning needs an integrated approach that should begin during the Training Needs Analysis or TNA phase and should successively build up right up to the determination of its impact on business. Training Needs Analysis (TNA).

ROI 56

Workplace Learning 2025 – What is the ROI of learning and development?

Learning Pool

Part 4: While the benefits of workplace learning are wide-ranging, measuring the return on investment (ROI) remains a challenge for many organisations. ROI of L&D is critical to ensure money is invested wisely and the programme is open to scrutiny.

ROI: Determine the Impact of Your Training Programs


Calculating Return on Investment (ROI) is no easy feat for most any project. And, measuring the ROI for your training programs is no exception. According to Research and Markets report, “Global E-Learning Market Analysis & Trends – Industry Forecast to 2025,” the e-Learning market is expected to grow at a compound annual growth rate 7.2% What Is ROI? In simplest terms, ROI is the gain that results from an investment.

ROI 40

Determining The ROI Of eLearning – Using Kirkpatrick’s Model Of Training Evaluation

EI Design

In this article, I outline how you can use the Kirkpatrick’s model of training evaluation to measure training effectiveness, its impact, and the ROI of eLearning. How To Determine The ROI Of eLearning. The measurement of ROI of eLearning needs an integrated approach that should begin during the Training Needs Analysis or TNA phase and should successively build up right up to the determination of its impact on business. Training Needs Analysis (TNA).

eLearning ROI Case Study: How To Maximize Your Returns

EI Design

I share an eLearning ROI case study that demonstrates how L&D teams can improve and maximize the ROI of corporate training. This demonstrates how you can maximize the ROI by adopting our approach, based on Kirkpatrick’s model of training evaluation. How To Maximize ROI In Corporate Training: A Case Study. What Is ROI On Corporate Training, And Why Is This Measurement Critical Gor L&D Teams? eLearning ROI Case Study.

Free eBook: Practical Approaches To Determine The ROI Of eLearning – Using Kirkpatrick’s Model Of Training Evaluation

EI Design

This free eBook, Free eBook: Practical Approaches To Determine The ROI Of eLearning – Using Kirkpatrick’s Model Of Training Evaluation, provides practical tips that you can use to measure the effectiveness of online training, leading to the ROI on your training spend. Practical Approaches To Determine The ROI Of eLearning – Using Kirkpatrick’s Model Of Training Evaluation. The Case Study demonstrates how these can help you improve and maximize your ROI.

The Essential Guide to Learning Analytics in the Age of Big Data

Lambda Solutions

Evaluating Learning Analytics and Measuring ROI. Data Points to Prove the ROI of Learning Analytics. Evaluating Learning Analytics and Measuring ROI. It’s the most recognized method of evaluating a training program’s ROI. Adding ROI to Kirkpatrick.

E-Learning vs. In-Person: A False Dichotomy

The Performance Improvement Blog

A simple ROI analysis of the savings can underscore the point within your own organization. They quote Jack Phillips: “Our studies have shown that participants in e-learning programs are less likely to follow through than in an instructor-led program,” notes Jack Phillips, Ph.D., chairman of the ROI Institute and a renowned expert on accountability and evaluation. “In Online training and conferencing advocates continue to ask the wrong question.

Introduction to Performance Consulting

Infopro Learning

You can measure the results from performance consulting initiatives using the following levels of evaluation (ref: Jack and Patti PhillipsROI methodology): Level 1: Have employees responded favorably to the new initiatives? If you are interested in learning more, Infopro Learning breaks learning performance consulting into three main categories; Business and Performance Gap Analysis , Learning Strategy , and Learning Effectiveness Evaluation.

Avoiding the Curse of the Consultant

CLO Magazine

accounting for nearly half (44 percent) of the global market, according to analysis from Greentarget, a public relations firm. Clients who fund consulting want to see the business value and sometimes even the ROI. In today’s climate, showing ROI for major projects can be a market differentiator. The fourth challenge is to explore the prospect of ROI forecasting and guaranteeing results. Make It Credible: Measuring Results and Calculating ROI.

Is this thing on? Tips for measuring course effectiveness and return on investment

Obsidian Learning

Determination of Return On Investment , or ROI. Note that while this kind of data analysis cannot prove a causal relationship between training and changes in organizational effectiveness, it can provide compelling evidence of improvement as a result of training (Kirkpatrick & Kirkpatrick, 2006). ROI Results Evaluation. Return on Investment (ROI) measures the financial benefit to the organization of training. Phillips, J. New post Is this thing on?

How to Evaluate Employee Training Programs (A Step-by-Step Guide)


The Phillips ROI Model. The Phillips ROI Model builds heavily off of the Kirkpatrick Model. Where the Philips Model takes things a step further is relative to the monetary side of things, specifically the fifth step: Return on Investment (ROI).

Looking Back and Ahead on the Measurement and Management of L&D

CLO Magazine

On the measurement side, I am thinking of Don Kirkpatrick, who gave us the four levels, and Jack Phillips, who gave us isolated impact for level 4 and ROI for level 5. On the management side, I am grateful again for the contributions from Don and Jack, and now Jim and Wendy Kirkpatrick and Patti Phillips as well, for their guidance in how to manage, particularly with respect to partnering closely with goal owners and focusing on what it takes to achieve level 3 application.

Here’s the latest thinking on measurement, analytics and reporting

CLO Magazine

Third, programs and computing power are facilitating the use of microdata, which simply means that data are available at the individual level for further analysis or for follow-up with the employee. Application, Impact and ROI. It is an exciting time to be a learning professional.

How do we measure value creation from training?

Learning Wire

Evaluating training ROI involves more than simply calculating a financial ratio. Even though certain things are hard to measure reliably, Phillips (2015) suggests a way of getting round this when evaluating the impacts of a training course. But this wouldn’t be a viable way of evaluating the course because there are so many unrelated factors that can affect your analysis. Phillips J, Pulliam Phillips P.

Evidence or Proof of L&D Impact? Why Not Both?


This is where you assemble significant amounts of data, run statistical models like regression analysis to show a link between the learning program and the business outcome. This may also be part of a periodic project where your team or a 3rd party consultant conducts an in-depth analysis on the program and its related impact and writes-up a report summarizing the findings. In the world of Learning & Development (L&D), impact is super important.

A Legend’s Look at Learning

CLO Magazine

Phillips is the chairman, and Patti P. Phillips is president and CEO of the ROI Institute. An ROI analysis is needed for a few projects, using the same calculation that the CFO would use when investing in a capital expenditure. When programs are designed to drive the business measures, they deliver results, removing the fear of a negative ROI. Jack J.

Free learning & development webinars for December 2021

Limestone Learning

In this webinar, Xyleme Chief Strategy Officer, Leslie Farinella, will break down how re-evaluating your content strategy and investing in the right technology can help organizations turn learning content into a competitive advantage to drive content management ROI.

Free 89

What is stopping companies from measuring learning: Skillsets, datasets, toolsets or mindsets?


How long can L&D organizations expect this to continue if they can''t prove to their stakeholders their training tools and programs are effectively influencing employee performance and driving ROI? L&D is no different—except to the extent that professionals must work even harder to demonstrate impactful training outcomes and ROI. "In Yes, organizations continue to view L&D as a line item expense part of loss/profit analysis and not a long term investment, which it is.

Expert Panel: Learning Transfer, Chatbots, and Measurement

Mobile Coach

Her second book, Making Change Work: How to Create Behavioral Change in Organizations to Drive Impact and ROI, co-authored with Jack and Patti Phillips of the ROI Institute, was published in 2016 by Kogan Page. Expert Panel: Learning Transfer, Chatbots, and Measurement.

Instructional Design Basics: What Is ADDIE?

Convergence Training

ADDIE is an acronym that stands for each of the steps of the model–analyze, design, develop, implement, and evaluation (or you might see it listed out as analysis, design, development, implementation, and evaluation). The five steps of ADDIE are: Analysis. Analysis or Analyze.


CFO, Friend or Foe?

CLO Magazine

For more than 100 years, CFOs have been using ROI for capital expenditures. CFOs may have no idea how to connect leadership development to a financial ROI, but chances are, you do. Have them review program costs that would be included in an ROI analysis. Secure their help developing an executive-friendly learning scorecard that moves the accountability reporting beyond inputs, reaction and learning to include data on application, impact and ROI.

Putting Thought to Work: Evaluation in Practice

CLO Magazine

Most rely on quantitative as well as qualitative measures, ensuring some human intuition and analysis is included. Then she said the tool automates levels 1 through 5 — including the Phillips ROI model — to help learning leaders report and analyze the data. She said they also calculate ROI, but the measure is analyzed in aggregate along with qualitative measures.

Free L&D webinars for December 2019

Limestone Learning

Training needs analysis. Wednesday, December 4, 2019, 9AM – 10AM PT: What Caused the ROI? In this session, Dr. Patti Phillips, CEO, ROI Institute, Inc., Here’s where ROI comes in. During this webinar, you’ll learn what ROI is, what it’s not, and how to make it work.

Discussing the ASTD Handbook, eLearning, and Published Expertise with Elaine Biech

Web Courseworks

I’m happy to say that the ASTD Handbook meets my demands in every regard, and although I haven’t read through all 980 pages of this massive reference guide, I’ve searched the handbook multiple times and always come away with excellent analysis and hints on what to explore next. For example when contemplating the topic of evaluation several influential and respected names immediately come to mind: Kirkpatrick, Phillips, Brinkerhoff, and others.

ASTD 135

Free learning & development webinars for June 2021

Limestone Learning

If you’re in self-analysis mode, you might benefit from checking out a free webinar or two. PT: Why Performance-based, Results-focused Instructional Design is Critical to Delivering Training ROI and More This presentation, hosted by Darryl Sink, President of Darryl L.

Free 69

Don Kirkpatrick’s Contribution to Learning & Development

CLO Magazine

The name evolved from the fact that they were published as four installments, each one adding another level to the analysis. Jack Phillips began applying the four levels in the late ’60s and later extended the work by adding ROI as the fifth level. I just saw an announcement that Don will retire from the speaking circuit this year. Few people have single-handedly made such an impact on our profession. I have known Don for years and his story is very interesting.

Brewing CALDO: How communications and learning professionals can thrive using DevOps

CLO Magazine

When was the last time you were able to do a formal needs analysis at the start of a project? Because of a focus on identifying and mitigating potential defects early, many believe these models yield significant ROI, but it has proven difficult to show ROI with quantifiable “hard numbers.”

Free L&D webinars for October 2019

Limestone Learning

Tuesday, October 1, 2019, 9AM – 10AM PT: Demonstrating the ROI in Leadership Development Organizations worldwide spend billions on leadership development; yet, for many organizations, little evidence exists that those investments are paying off. Join Patti Phillips, CEO of ROI Institute, and Katharine Aldana, Learning and Development Design Manager for Verizon, as they demonstrate how you can show the impact and ROI of your leadership development programs.

Commonly Used Training Evaluations Models: A Discussion with Dr. Will Thalheimer

Convergence Training

But anyway, when we think about data and analysis, you know, we need to collect data that’s accurate, valid, relevant, highly predictive of what we care about, what’s important, that’s also cost-effective. So we can also look at our designs, our analysis, our assumptions, and look at and get information about our learning by looking at the inputs as well as the outputs. The Philips/ROI LearningEvaluation Model. So this is what the ROI methodology looks like.

Free learning & development webinars for September 2020

Limestone Learning

Join Monica Cornetti, Founder and CEO of Sententia Gamification, and Sharon Goza, Integrated Graphics Operations and Analysis Lab Manager at NASA, for this webinar and you’ll learn how to: Start with a character your audience can relate to.