More on Re-evaluating Evaluation – Jack Phillips and ROI

Dashe & Thomson

I just realized that I haven’t included a single word about Jack Phillips , who introduced Return on Investment (ROI) as Level 5 to Kirkpatrick’s Four Levels of Evaluation. My first exposure to Phillips’ ROI—although I didn’t realize it at the time—was through a colleague who introduced me to Kirkpatrick’s Four Levels. Years later, I realized that she had skipped Kirkpatrick’s Level 4: Results and substituted Phillips’ ROI.

An Overview: Phillips' Model for Learning Evaluation

Watershed

While there are many other learning models we’ve yet to cover, the one we’re asked about most often that’s not mentioned in our original series is Phillips' Learning Evaluation Model. In a nutshell, Phillips' model focuses on how to: collect data, isolate the impact of training versus the impact of other factors, and. Tip: Find out even more about Phillips' Model for Learning Evaluation in Jack Phillips' book.).

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Why We Should Stop Talking About ROI in Training

Mindflash

Another model and methodology from Jack Phillips includes a fifth level – ROI – which adds an added financial metric to the mix. In many cases they are not being tracked, and when they are, we’re too focused on the metric itself. Metrics are important, but they require a level of context; they need to be part of a story in some way. We Don’t Need New Metrics. However, the ROI metrics themselves provide horrible language for telling the story of our efforts.

ROI 80

To Deliver Results Start with Why?

CLO Magazine

Phillips is the chairman, and Patti P. Phillips is president and CEO of the ROI Institute. Measurement business value measurement metricsTags: business value , measurement , metrics The post To Deliver Results Start with Why? Demonstrating the business value for learning is not a matter of providing more resources for measurement, evaluation and analytics. It’s a process that permeates the learning cycle, starting with why?

Employee performance goals; Choosing the right training evaluation model

Learning Pool

Phillips proposed adding a fifth level to the model, return on investment (ROI). Phillips is probably your guy. . Phillips adds a fifth level to Kirkpatrick, ROI, and gives a methodology for calculating the financial impact of programs.

How to Calculate the ROI of Training Programs: The Formula

LearnUpon

We want to help you tackle the unknown of ROI, and overcome the issues to track this holy grail metric. Each one will need to be divided into “primary metrics” and “secondary metrics”. Primary metrics are revenue-based metrics.

20 Key Takeaways from ATD 2019

Docebo

Panel discussion with learning leaders from PPD, Explorance and Verizon and also Patti Phillips from the ROI Institute covering how to measure and demonstrate the value of L&D. Don’t try and find a holistic all-LMS-encompassing metric, but instead, take a micro approach to individual programs and scale from there. Patti Phillips, PH.d,

Docebo 133

Workplace Learning 2025 – What is the ROI of learning and development?

Learning Pool

Time, costs, and measurement metrics are among drivers of this mismatch. While ROI can be elusive, organisations that do it well are starting with the business metric and examining performance outcomes,” says CIPD head of L&D Andy Lancaster. Six Part Series: Workplace Learning 2025.

ROI 77

Unlocked Learning—Training analytics made easy

Coassemble

Instead of hunting down metrics, imagine being able to receive every training metric in one dashboard? But for your admins, it allows them to create training content and view various metrics including: When and where a user has accessed and self-enrolled in a training session.

Why Don’t Businesses Measure Training ROI?

LearnUpon

This makes it difficult to pinpoint key metrics, leaving you and your team with a lack of understanding of what’s important across the organization. . Why not ask the sales and success departments for success metrics they have showing a decrease in time to launch for new customers.

ROI 52

Here’s the latest thinking on measurement, analytics and reporting

CLO Magazine

There are two very significant developments here, each of which will change the way human capital metrics are reported. In the future, why would any employee go to work for an organization that refuses to disclose its human capital metrics?

Watershed's 7 Steps of Learning Evaluation

Watershed

We've combined our favorite elements from Kirkpatrick , Kaufman , Anderson's Value of Learning , Brinkerhoff's Success Case Method , and Phillips' Model to create our version of a "super method" of how to evaluate learning. Step 2: Define metrics for effective evaluation. Identify success metrics most appropriate to the organization. Ensure that these metrics cover the complete story from learning through to achievement of the program goals.

The Essential Guide to Learning Analytics in the Age of Big Data

Lambda Solutions

What metrics and sources to use in implementing learning analytics. This data can be sorted, filtered, and correlated to specific metrics, such as activity and course completions. That means selecting the right metrics on actual job performance data—not just training performance.

Training Evaluation Beyond Kirkpatrick

Training Industry

Because showing the return on investment from training involves many components and metrics that are not readily measurable, such as the behavioral changes in the learner’s day-to-day work as a result of the training they attended. For those of us in the training and development industry, the quest to measure the results of training is almost as complex and even unattainable as the quest for the holy grail.

Unlocked Learning—Training analytics made easy

Coassemble

Instead of hunting down metrics, imagine being able to receive every training metric in one dashboard? But for your admins, it allows them to create training content and view various metrics including: When and where a user has accessed and self-enrolled in a training session. In 2003, Dr. Jack Phillips introduced the fifth level as the return of investment for the training. Being able to measure your team’s training is crucial to its effectiveness.

Training Evaluation – 5 Best Ways to Evaluate Training Effectiveness and Impact

Kitaboo

The Phillips ROI Model. Very similar to the Kirkpatrick model in approach, the Phillips ROI model has an extra step, which is to evaluate the program’s return on investment (ROI) by measuring the difference between training cost and training results.

What Is Learning Evaluation?

Watershed

An Overview: Phillips Model for Learning Evaluation. Step 2: Define metrics for effective learning evaluation. Knowing how to aggregate and evaluate all types of learning is a priority for many learning and development practitioners. In fact, many say there's a growing demand from their organizations to show L&D’s impact across the organization.

Evaluation and ROI in Compliance Training

Interactive Services

While ROI is an important metric, it shouldn’t be the sole focus of a comprehensive compliance training initiative. Learning management systems (LMSs) can provide metrics and generate reports that give you data pointing to the effectiveness of training, like time spent on each module, assessment scores, number of times an assessment was attempted, and course completion rates.

Stepping Up to the ROI Challenge

CLO Magazine

Phillips is the chairman, and Patti P. Phillips is president and CEO of the ROI Institute. Even more important is how the change in leader behavior in 43 companies affected important business metrics. Jack J. In his 2017 Chief Learning Officer article, “The Business Case for Leadership Development,” William C. Byham, founder and CEO of Development Dimensions International, notes the following: “An article in McKinsey Quarterly accused U.S.

ROI 59

Should Loyalty Be the New Measure of Training ROI

Training Industry

Jack and Patti Phillips of the ROI Institute argue that training organizations that don't have an extensive measurement and evaluation system are likely to see a reduction or elimination of their budget. Spyridon Arvanitis of Geniki Bank-Societe Generale Group, Hellas, refers to loyalty as a business metric accepted by executives. Loyalty is also a metric that can be easily measured and reported using net promoter score (NPS).

ROI 50

The Elevator Pitch for Trainers

Training Industry

The conversation that began with Kirkpatrick and continues with Phillips takes place at virtually every training conference, is represented as an article in almost all training publications and is a topic of debate whenever training professionals meet. The dialog focuses on metrics, or the lack thereof; ROI and the challenge of measuring it; and the meaning of “business impact.”

Putting Thought to Work: Evaluation in Practice

CLO Magazine

The company organizes its learning evaluation data using a tool, Metrics That Matter, from KnowledgeAdvisors that connects directly to its learning management system. Managers input progress measures into the LMS, which streamlines the data to Metrics That Matter. Then she said the tool automates levels 1 through 5 — including the Phillips ROI model — to help learning leaders report and analyze the data.

ROI: Determine the Impact of Your Training Programs

Mindflash

Return on Investment is a popular financial metric for evaluating the financial consequences of investments. Check out this interview with Jack Phillips of the ROI Institute. Calculating Return on Investment (ROI) is no easy feat for most any project. And, measuring the ROI for your training programs is no exception. Industry Growth Means Managers Need to Understand the Business Impact of Their Training Programs.

ROI 40

Weighing the Options: Different Schools of Thought

CLO Magazine

Jack Phillips, chairman of ROI Institute Inc., a learning metrics and measurement consultancy, is a proponent of this approach. Phillips, who started the ROI Institute in 1993 and has written a number of books on the subject, adds a fifth level, return on investment, to the taxonomy. To measure level 4, or impact, Phillips said learning leaders take the predetermined outcome measure. Next, Phillips said the company would determine the lifetime value of a new account.

10 New Years Resolutions for Learning and Development Professionals

Degreed

“Signify (the new company name for Phillips Lighting) is in the midst of disruptive change, and we are changing the way the learning function operates to align with the speed of our business. We will identify a metric which is a leading indicator of Learning Effectiveness which also correlates Employee NPS. “If you always do what you’ve always done, you’ll always get what you’ve always got” -Henry Ford.

ROI: It’s All About Proximity

CLO Magazine

I’m not saying training does not contribute to a learner’s on-the-job performance, but in the sense of measuring a Kirkpatrick or Phillips level 4/5 impact, we all know “contributing” is not a strong word. The next fair metric related to training is knowledge gain. Both of these measures fall comfortably within level one and two, and because of the proximity and intent of training, are fair metrics to gauge training effectiveness.

ROI 62

CFO, Friend or Foe?

CLO Magazine

To keep the relationship friendly: Consider them to be metrics experts. If you are having difficulty securing additional funds for metrics, analytics and measurement, you may have a sympathetic ear because the CFO understands the power of analytics. Phillips is the chairman and Patti P. Phillips is president and CEO of the ROI Institute.

Is this thing on? Tips for measuring course effectiveness and return on investment

Obsidian Learning

There are two sources of data gathered in the Level 4 evaluation process for the course: Metrics collected by the Support team (pulled from the Help Ticket System). The historical support metrics will be analyzed to determine support call trends before and after training. To determine the financial benefit of your course, you need to identify both hard data elements and soft data elements (Phillips, 1996): Hard data elements are benefits to which monetary amounts may be assigned.

Building a Better Working World

CLO Magazine

From left: Simon Berridge, Jan Cannon-Bowers, Christiana Zidwick, Brenda Sugrue, Eric Ellefsen, Shawn Phillips. New and redesigned programs doubled business metrics, including revenue and client satisfaction. Building a better working world — this is EY’s ultimate purpose, one it works to espouse through its Vision 2020 strategy launched in 2013. At the heart of this strategy is having the highest-performing teams provide exceptional client service worldwide.

Instructional Design Basics: What Is ADDIE?

Convergence Training

Use the information you discovered while learning about the task performance and organizational goals to begin creating evaluation strategies for your training and KPIs/metrics to assist with that evaluation. The Phillips “ROI” Model.

ADDIE 65

Free L&D webinars for December 2019

Limestone Learning

In this session, Dr. Patti Phillips, CEO, ROI Institute, Inc., While ROI has a long history of use, it remains one of the most misunderstood and misrepresented metrics in the learning and development data dictionary.

Free 70

What is stopping companies from measuring learning: Skillsets, datasets, toolsets or mindsets?

Xyleme

By adapting our language and perspective to integrate terms like traffic, bounce rate, conversion, time on site and social sharing into an L&D context, we create a dynamic system that uses data to drive smarter decision-making and draw on accurate metrics for more actionable insights. Yet, L&D professionals continue to herald formal evaluation - Kirkpatrick and Phillips - as industry standards, essentially disregarding new tools at their disposal.

High time for ROI

Jay Cross

Scouts live by metrics. Following a keynote by Ed Cohen, former CLO at Booz and now head of Satyam Computer Services’ School of Leadership, we launched into a ThinkTank session chaired by Cedric Coco of Lowe’s with observations from Kent Barnett (Knowledge Advisors), Michael Echols (Bellevue University’s Human Capital Lab), and Patti Phillips (of The ROI Institute). Issue: What are the biggest mistakes in installing HR metrics?

ROI 36

Free L&D webinars for April 2020

Limestone Learning

Join Patti Phillips, CEO of ROI Institute, and Katharine Aldana, Learning Design Manager at Verizon, as they describe four simple ways to forecast ROI for your programs. “ Flower, Yellow, Daffodil ” by Pixelia is licensed by Pixabay.

Free 78

A Review of the Elearnity 9-Grid

Unicorn Training

In the first of a series looking at published research findings and statistics in the e-learning space, Peter Phillips takes a look at the Elearnity 9-Grid™ True Grid? 3) Direction of travel – Finally, in a valiant attempt to include five metrics in a two dimensional Grid, Elearnity provide a measure of “our sense of their likely future direction of travel within the model”, by where they position the vendor’s blob within whatever square of the Grid they find themselves.

CLO Symposium

Jay Cross

It replaces the inward-looking mumbo-jumbo of Kirkpatrick and the Phillips with business measures. “Nobody cares about internal metrics.” Along with my Internet Time Alliance colleagues Jane Hart & Clark Quinn and several hundred chief learning officers, I attended the Fall CLO Symposium this week. Our theme was “Game-Changing Learning: Development for the New Normal.” ” Sports metaphors filled the air.

CLO 40

Free L&D webinars for May 2018

Limestone Learning

Discuss how to correlate learner engagement with other critical organization metrics to tell an important story about the value of L&D investments. What metrics for training ROI do you use? What metrics to set up, how to install, and how to follow them up to get results. Phillips, along with her husband and business partner Dr. Jack J. " Queen Victoria's Oak Tree, Sandringham " by Andrew Hugget is LICENSED under CC BY-SA 2.0.