More on Re-evaluating Evaluation – Jack Phillips and ROI

Dashe & Thomson

I have been blogging a lot about Training Evaluation this year—mostly Kirkpatrick , but also Brinkerhoff and Scriven. I just realized that I haven’t included a single word about Jack Phillips , who introduced Return on Investment (ROI) as Level 5 to Kirkpatrick’s Four Levels of Evaluation.

An Overview: Phillips' Model for Learning Evaluation

Watershed

When we originally published our Learning Evaluation eBook and blog series, we covered four learning evaluation models representing a range of approaches and perspectives: Kirkpatrick, Kaufman, Brinkerhoff, and Anderson. While there are many other learning models we’ve yet to cover, the one we’re asked about most often that’s not mentioned in our original series is Phillips' Learning Evaluation Model. Meet the fifth level of learning evaluation.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Training Evaluation – 5 Best Ways to Evaluate Training Effectiveness and Impact

Kitaboo

Training evaluation refers to an attempt to obtain relevant information on the effects of a training program. The primary objective of evaluating any training program is to develop an understanding of whether it has achieved its stated objectives. The Phillips ROI Model.

What Is Learning Evaluation?

Watershed

Knowing how to aggregate and evaluate all types of learning is a priority for many learning and development practitioners. Technologies are becoming available that enable L&D professionals to aggregate, evaluate, and measure many types of learning data in one place.

Training Evaluation Beyond Kirkpatrick

Training Industry

Because showing the return on investment from training involves many components and metrics that are not readily measurable, such as the behavioral changes in the learner’s day-to-day work as a result of the training they attended. Today, when we discuss training evaluation, we likely reference Kirkpatrick’s model with the familiar four levels: reaction, learning, behavior and results.

Watershed's 7 Steps of Learning Evaluation

Watershed

Out of the dozens of different learning evaluation models currently in practice, which one is right for you? Meet our "super method" for learning evaluation. evaluate alignment with strategic priorities. Step 2: Define metrics for effective evaluation.

Why We Should Stop Talking About ROI in Training

Mindflash

Another model and methodology from Jack Phillips includes a fifth level – ROI – which adds an added financial metric to the mix. Reaction : There are reasons post-program evaluations are commonly called smile sheets. Most evaluations are not asking business-related questions.

ROI 106

To Deliver Results Start with Why?

CLO Magazine

Demonstrating the business value for learning is not a matter of providing more resources for measurement, evaluation and analytics. Finally, for programs that are very expensive, important and strategic, the evaluation should be pushed to ROI (Level 5), usually in 5 percent of programs.

Putting Thought to Work: Evaluation in Practice

CLO Magazine

Organizations that have reached higher measurement levels use a blended approach for the various frameworks and find ways to customize evaluation. The company organizes its learning evaluation data using a tool, Metrics That Matter, from KnowledgeAdvisors that connects directly to its learning management system. Managers input progress measures into the LMS, which streamlines the data to Metrics That Matter.

Three Challenges Faced by Today’s L&D Leaders

Your Training Edge

Rob Brinkerhoff, a professor at Western Michigan University and thought leader in the field, puts that number 80 to 85% (Phillips, 2016)! In a 2008-2009 study, Jack and Patti Phillips surveyed the CEOs of Fortune 500 and large, private-sector employers. Jack & Patti Phillips, Measuring What Matters: How CEOs View Learning Success 2009). Phillips, Jack & Phillips, Patti. Phillips, Ken.

Is this thing on? Tips for measuring course effectiveness and return on investment

Obsidian Learning

The Kirkpatrick four levels of training evaluation. While later stages of evaluation measure more obvious aspects of quality—such as the impact of the training on the learner—it’s important not to overlook the less obvious factors, such as instructional design or the use of technology.

Unlocked Learning—Training analytics made easy

Coassemble

Instead of hunting down metrics, imagine being able to receive every training metric in one dashboard? But for your admins, it allows them to create training content and view various metrics including: When and where a user has accessed and self-enrolled in a training session.

Workplace Learning 2025 – What is the ROI of learning and development?

Learning Pool

Time, costs, and measurement metrics are among drivers of this mismatch. While ROI can be elusive, organisations that do it well are starting with the business metric and examining performance outcomes,” says CIPD head of L&D Andy Lancaster. Six Part Series: Workplace Learning 2025.

The Essential Guide to Learning Analytics in the Age of Big Data

Lambda Solutions

How to use learning analytics for evaluation. What metrics and sources to use in implementing learning analytics. Evaluating Learning Analytics and Measuring ROI. This data can be sorted, filtered, and correlated to specific metrics, such as activity and course completions.

Here’s the latest thinking on measurement, analytics and reporting

CLO Magazine

There are two very significant developments here, each of which will change the way human capital metrics are reported. In the future, why would any employee go to work for an organization that refuses to disclose its human capital metrics?

Should Loyalty Be the New Measure of Training ROI

Training Industry

Only 35 percent of the 199 talent development professionals the Association for Talent Development surveyed for its 2016 “Evaluating Learning: Getting to Measurements That Matter” report said their organizations evaluated the business results of learning programs to any extent. Jack and Patti Phillips of the ROI Institute argue that training organizations that don't have an extensive measurement and evaluation system are likely to see a reduction or elimination of their budget.

ROI 55

ROI: Determine the Impact of Your Training Programs

Mindflash

Return on Investment is a popular financial metric for evaluating the financial consequences of investments. Executive teams evaluate training programs based on the costs and outcomes of your investments. Check out this interview with Jack Phillips of the ROI Institute.

ROI 52

CFO, Friend or Foe?

CLO Magazine

To keep the relationship friendly: Consider them to be metrics experts. If you are having difficulty securing additional funds for metrics, analytics and measurement, you may have a sympathetic ear because the CFO understands the power of analytics. Phillips is the chairman and Patti P.

Weighing the Options: Different Schools of Thought

CLO Magazine

The framework for learning evaluation and measurement embraced by most in the industry starts with Kirkpatrick. The framework for learning evaluation and measurement embraced by most in the industry starts with Kirkpatrick. The longtime University of Wisconsin at Madison professor wrote a series of articles in 1959 for the American Society for Training and Development that outlined what became known as the four levels of evaluation — reaction, learning, behavior and results.

10 New Years Resolutions for Learning and Development Professionals

Degreed

“Signify (the new company name for Phillips Lighting) is in the midst of disruptive change, and we are changing the way the learning function operates to align with the speed of our business.

Free L&D webinars for December 2019

Limestone Learning

While business measures may improve after a program’s implemented, unless evaluators take steps to isolate the effects of the project on that improvement, they’ve no credible claim to the improvement – leaving stakeholders to wonder, “What caused it?”

Free 70

Free L&D webinars for April 2020

Limestone Learning

It doesn’t replace the need to demonstrate the actual results, but it’s a useful tool and one worth adding to the training evaluation toolbox. “ Flower, Yellow, Daffodil ” by Pixelia is licensed by Pixabay. Crocuses and daffodils are peeking out, the days are sunny, and we’re … inside!

Free 83

Free L&D webinars for May 2018

Limestone Learning

Discuss how to correlate learner engagement with other critical organization metrics to tell an important story about the value of L&D investments. How managers might use these competencies to write job descriptions, and as the basis of performance plans and evaluations.

What is stopping companies from measuring learning: Skillsets, datasets, toolsets or mindsets?

Xyleme

By adapting our language and perspective to integrate terms like traffic, bounce rate, conversion, time on site and social sharing into an L&D context, we create a dynamic system that uses data to drive smarter decision-making and draw on accurate metrics for more actionable insights.

Free L&D webinars for October 2019

Limestone Learning

Join Patti Phillips, CEO of ROI Institute, and Katharine Aldana, Learning and Development Design Manager for Verizon, as they demonstrate how you can show the impact and ROI of your leadership development programs. “ Carrot harvest ” by woodleywonderworks is licensed under CC BY 2.0.

Free 77

Free L&D webinars for March 2018

Limestone Learning

However, one consideration that people often overlook is the type of technology that an LMS uses, which is important to consider when evaluating whether an LMS will be able to provide the type of functionality you’ll need to meet your goals. Evaluate the results of a blended learning campaign.

Free L&D webinars for October 2018

Limestone Learning

Join Neha Gupta of True Office Learning, as she evaluates common e-learning capabilities for cognitive soundness, meaningful engagement, impactful efficacy, and measurable ROI. “ Spider, Web, Tree, Branches ” by skeeze is Public Domain. Are you ready for October?

Free 68