More on Re-evaluating Evaluation – Jack Phillips and ROI

Dashe & Thomson

I just realized that I haven’t included a single word about Jack Phillips , who introduced Return on Investment (ROI) as Level 5 to Kirkpatrick’s Four Levels of Evaluation. My first exposure to Phillips’ ROI—although I didn’t realize it at the time—was through a colleague who introduced me to Kirkpatrick’s Four Levels. Years later, I realized that she had skipped Kirkpatrick’s Level 4: Results and substituted Phillips’ ROI.

An Overview: Phillips' Model for Learning Evaluation


While there are many other learning models we’ve yet to cover, the one we’re asked about most often that’s not mentioned in our original series is Phillips' Learning Evaluation Model. In a nutshell, Phillips' model focuses on how to: collect data, isolate the impact of training versus the impact of other factors, and. Tip: Find out even more about Phillips' Model for Learning Evaluation in Jack Phillips' book.).


Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Why We Should Stop Talking About ROI in Training


Another model and methodology from Jack Phillips includes a fifth level – ROI – which adds an added financial metric to the mix. In many cases they are not being tracked, and when they are, we’re too focused on the metric itself. Metrics are important, but they require a level of context; they need to be part of a story in some way. We Don’t Need New Metrics. However, the ROI metrics themselves provide horrible language for telling the story of our efforts.

ROI 80

To Deliver Results Start with Why?

CLO Magazine

Phillips is the chairman, and Patti P. Phillips is president and CEO of the ROI Institute. Measurement business value measurement metricsTags: business value , measurement , metrics The post To Deliver Results Start with Why? Demonstrating the business value for learning is not a matter of providing more resources for measurement, evaluation and analytics. It’s a process that permeates the learning cycle, starting with why?

CLO 38

Employee performance goals; Choosing the right training evaluation model

Learning Pool

Phillips proposed adding a fifth level to the model, return on investment (ROI). Phillips is probably your guy. . Phillips adds a fifth level to Kirkpatrick, ROI, and gives a methodology for calculating the financial impact of programs.

Three Challenges Faced by Today’s L&D Leaders

Your Training Edge

Rob Brinkerhoff, a professor at Western Michigan University and thought leader in the field, puts that number 80 to 85% (Phillips, 2016)! In a 2008-2009 study, Jack and Patti Phillips surveyed the CEOs of Fortune 500 and large, private-sector employers. Jack & Patti Phillips, Measuring What Matters: How CEOs View Learning Success 2009). Phillips, Jack & Phillips, Patti. Phillips, Ken.

20 Key Takeaways from ATD 2019


Panel discussion with learning leaders from PPD, Explorance and Verizon and also Patti Phillips from the ROI Institute covering how to measure and demonstrate the value of L&D. Don’t try and find a holistic all-LMS-encompassing metric, but instead, take a micro approach to individual programs and scale from there. Patti Phillips, PH.d,

Docebo 133

Why Don’t Businesses Measure Training ROI?


This makes it difficult to pinpoint key metrics, leaving you and your team with a lack of understanding of what’s important across the organization. . Why not ask the sales and success departments for success metrics they have showing a decrease in time to launch for new customers.

ROI 52

Watershed's 7 Steps of Learning Evaluation


We've combined our favorite elements from Kirkpatrick , Kaufman , Anderson's Value of Learning , Brinkerhoff's Success Case Method , and Phillips' Model to create our version of a "super method" of how to evaluate learning. Step 2: Define metrics for effective evaluation. Identify success metrics most appropriate to the organization. Ensure that these metrics cover the complete story from learning through to achievement of the program goals.

Unlocked Learning—Training analytics made easy


Instead of hunting down metrics, imagine being able to receive every training metric in one dashboard? But for your admins, it allows them to create training content and view various metrics including: When and where a user has accessed and self-enrolled in a training session. In 2003, Dr. Jack Phillips introduced the fifth level as the return of investment for the training. Being able to measure your team’s training is crucial to its effectiveness.

Continuous learning with a purpose

CLO Magazine

Do we have metrics that we could look at to see if they shift? We will do our studies along the way to make sure that those metrics are moving in the right direction.”.

Workplace Learning 2025 – What is the ROI of learning and development?

Learning Pool

Time, costs, and measurement metrics are among drivers of this mismatch. While ROI can be elusive, organisations that do it well are starting with the business metric and examining performance outcomes,” says CIPD head of L&D Andy Lancaster. Six Part Series: Workplace Learning 2025.

The Essential Guide to Learning Analytics in the Age of Big Data

Lambda Solutions

What metrics and sources to use in implementing learning analytics. This data can be sorted, filtered, and correlated to specific metrics, such as activity and course completions. That means selecting the right metrics on actual job performance data—not just training performance.

Here’s the latest thinking on measurement, analytics and reporting

CLO Magazine

There are two very significant developments here, each of which will change the way human capital metrics are reported. In the future, why would any employee go to work for an organization that refuses to disclose its human capital metrics?

Training Evaluation Beyond Kirkpatrick

Training Industry

Because showing the return on investment from training involves many components and metrics that are not readily measurable, such as the behavioral changes in the learner’s day-to-day work as a result of the training they attended. For those of us in the training and development industry, the quest to measure the results of training is almost as complex and even unattainable as the quest for the holy grail.

What Is Learning Evaluation?


An Overview: Phillips Model for Learning Evaluation. Step 2: Define metrics for effective learning evaluation. Knowing how to aggregate and evaluate all types of learning is a priority for many learning and development practitioners. In fact, many say there's a growing demand from their organizations to show L&D’s impact across the organization.

Training Evaluation – 5 Best Ways to Evaluate Training Effectiveness and Impact


The Phillips ROI Model. Very similar to the Kirkpatrick model in approach, the Phillips ROI model has an extra step, which is to evaluate the program’s return on investment (ROI) by measuring the difference between training cost and training results.

Putting Thought to Work: Evaluation in Practice

CLO Magazine

The company organizes its learning evaluation data using a tool, Metrics That Matter, from KnowledgeAdvisors that connects directly to its learning management system. Managers input progress measures into the LMS, which streamlines the data to Metrics That Matter. Then she said the tool automates levels 1 through 5 — including the Phillips ROI model — to help learning leaders report and analyze the data.

Creating Collaborative Spaces


Ten years ago, Phillip Greenspun, a founding engineer at ArsDigita, wrote a treatise about hiring and supporting great software engineers. Ping-pong stats have even made it onto our official metrics dashboards. . OpenSesame was born in a kitchen.

ROI: Determine the Impact of Your Training Programs


Return on Investment is a popular financial metric for evaluating the financial consequences of investments. Check out this interview with Jack Phillips of the ROI Institute. Calculating Return on Investment (ROI) is no easy feat for most any project. And, measuring the ROI for your training programs is no exception. Industry Growth Means Managers Need to Understand the Business Impact of Their Training Programs.

ROI 40

The Elevator Pitch for Trainers

Training Industry

The conversation that began with Kirkpatrick and continues with Phillips takes place at virtually every training conference, is represented as an article in almost all training publications and is a topic of debate whenever training professionals meet. The dialog focuses on metrics, or the lack thereof; ROI and the challenge of measuring it; and the meaning of “business impact.”

Should Loyalty Be the New Measure of Training ROI

Training Industry

Jack and Patti Phillips of the ROI Institute argue that training organizations that don't have an extensive measurement and evaluation system are likely to see a reduction or elimination of their budget. Spyridon Arvanitis of Geniki Bank-Societe Generale Group, Hellas, refers to loyalty as a business metric accepted by executives. Loyalty is also a metric that can be easily measured and reported using net promoter score (NPS).

Stepping Up to the ROI Challenge

CLO Magazine

Phillips is the chairman, and Patti P. Phillips is president and CEO of the ROI Institute. Even more important is how the change in leader behavior in 43 companies affected important business metrics. Jack J. In his 2017 Chief Learning Officer article, “The Business Case for Leadership Development,” William C. Byham, founder and CEO of Development Dimensions International, notes the following: “An article in McKinsey Quarterly accused U.S.

ROI 49

Free learning & development webinars for September 2021

Limestone Learning

Dr. Jack Phillips, Chairman at ROI Institute, will showcase the current status of measurement and evaluation by providing examples of measures at each level of the evaluation process.

Free 90

Weighing the Options: Different Schools of Thought

CLO Magazine

Jack Phillips, chairman of ROI Institute Inc., a learning metrics and measurement consultancy, is a proponent of this approach. Phillips, who started the ROI Institute in 1993 and has written a number of books on the subject, adds a fifth level, return on investment, to the taxonomy. To measure level 4, or impact, Phillips said learning leaders take the predetermined outcome measure. Next, Phillips said the company would determine the lifetime value of a new account.

Evaluation and ROI in Compliance Training

Interactive Services

While ROI is an important metric, it shouldn’t be the sole focus of a comprehensive compliance training initiative. Learning management systems (LMSs) can provide metrics and generate reports that give you data pointing to the effectiveness of training, like time spent on each module, assessment scores, number of times an assessment was attempted, and course completion rates.

10 New Years Resolutions for Learning and Development Professionals


“Signify (the new company name for Phillips Lighting) is in the midst of disruptive change, and we are changing the way the learning function operates to align with the speed of our business. We will identify a metric which is a leading indicator of Learning Effectiveness which also correlates Employee NPS. “If you always do what you’ve always done, you’ll always get what you’ve always got” -Henry Ford.

ROI: It’s All About Proximity

CLO Magazine

I’m not saying training does not contribute to a learner’s on-the-job performance, but in the sense of measuring a Kirkpatrick or Phillips level 4/5 impact, we all know “contributing” is not a strong word. The next fair metric related to training is knowledge gain. Both of these measures fall comfortably within level one and two, and because of the proximity and intent of training, are fair metrics to gauge training effectiveness.

ROI 53

CFO, Friend or Foe?

CLO Magazine

To keep the relationship friendly: Consider them to be metrics experts. If you are having difficulty securing additional funds for metrics, analytics and measurement, you may have a sympathetic ear because the CFO understands the power of analytics. Phillips is the chairman and Patti P. Phillips is president and CEO of the ROI Institute.

CLO 32

Free learning & development webinars for March 2022

Limestone Learning

PT: ROI on a Shoestring When it comes to measuring the success of talent development, no metrics are more important to top executives than business impact and ROI. Dr. Jack Phillips, Chairman of ROI Institute, will show how to meet these challenges with limited resources.

Free 62

Free learning & development webinars for October 2021

Limestone Learning

PT: Best Practices for Engaging Course Content Anyone in the L&D Industry knows you have to be a jack of all trades, which means you need to know the basics in everything from marketing to metrics. “ Flying dog, Batman, bats ” by Andrea Bohl is licensed by Pixabay.

Free 90

What is stopping companies from measuring learning: Skillsets, datasets, toolsets or mindsets?


By adapting our language and perspective to integrate terms like traffic, bounce rate, conversion, time on site and social sharing into an L&D context, we create a dynamic system that uses data to drive smarter decision-making and draw on accurate metrics for more actionable insights. Yet, L&D professionals continue to herald formal evaluation - Kirkpatrick and Phillips - as industry standards, essentially disregarding new tools at their disposal.

Building a Better Working World

CLO Magazine

From left: Simon Berridge, Jan Cannon-Bowers, Christiana Zidwick, Brenda Sugrue, Eric Ellefsen, Shawn Phillips. New and redesigned programs doubled business metrics, including revenue and client satisfaction. Building a better working world — this is EY’s ultimate purpose, one it works to espouse through its Vision 2020 strategy launched in 2013. At the heart of this strategy is having the highest-performing teams provide exceptional client service worldwide.

High time for ROI

Jay Cross

Scouts live by metrics. Following a keynote by Ed Cohen, former CLO at Booz and now head of Satyam Computer Services’ School of Leadership, we launched into a ThinkTank session chaired by Cedric Coco of Lowe’s with observations from Kent Barnett (Knowledge Advisors), Michael Echols (Bellevue University’s Human Capital Lab), and Patti Phillips (of The ROI Institute). Issue: What are the biggest mistakes in installing HR metrics?

ROI 36

Is this thing on? Tips for measuring course effectiveness and return on investment

Obsidian Learning

There are two sources of data gathered in the Level 4 evaluation process for the course: Metrics collected by the Support team (pulled from the Help Ticket System). The historical support metrics will be analyzed to determine support call trends before and after training. To determine the financial benefit of your course, you need to identify both hard data elements and soft data elements (Phillips, 1996): Hard data elements are benefits to which monetary amounts may be assigned.

Instructional Design Basics: What Is ADDIE?

Convergence Training

Use the information you discovered while learning about the task performance and organizational goals to begin creating evaluation strategies for your training and KPIs/metrics to assist with that evaluation. The Phillips “ROI” Model.


Free learning & development webinars for June 2021

Limestone Learning

Engagement and motivation are important drivers that impact other meaningful metrics, like productivity, profitability, customer satisfaction, retention, safety and quality. “ Cicada, close-up, insect ” by Pexels is licensed by Pixabay. Spring is flirting with us!

Free 70

Free L&D webinars for December 2019

Limestone Learning

In this session, Dr. Patti Phillips, CEO, ROI Institute, Inc., While ROI has a long history of use, it remains one of the most misunderstood and misrepresented metrics in the learning and development data dictionary.

A Review of the Elearnity 9-Grid

Unicorn Training

In the first of a series looking at published research findings and statistics in the e-learning space, Peter Phillips takes a look at the Elearnity 9-Grid™ True Grid? 3) Direction of travel – Finally, in a valiant attempt to include five metrics in a two dimensional Grid, Elearnity provide a measure of “our sense of their likely future direction of travel within the model”, by where they position the vendor’s blob within whatever square of the Grid they find themselves.

CLO Symposium

Jay Cross

It replaces the inward-looking mumbo-jumbo of Kirkpatrick and the Phillips with business measures. “Nobody cares about internal metrics.” Along with my Internet Time Alliance colleagues Jane Hart & Clark Quinn and several hundred chief learning officers, I attended the Fall CLO Symposium this week. Our theme was “Game-Changing Learning: Development for the New Normal.” ” Sports metaphors filled the air.

CLO 40