Kirkpatrick Revisited | Social Learning Blog
APRIL 17, 2011
I have included Kirkpatrick’s Four Levels of Evaluation in every proposal I have ever written, and I wanted to hear from Kirkpatrick himself regarding his take on the current state of evaluation and whether his four levels are still viable. I wanted to get a more in-depth knowledge of the four levels and where the next generation, Kirkpatrick’s son James, was taking them.
Vale Don Kirkpatrick
MAY 28, 2014
Last week, Don Kirkpatrick passed away. The numbers have been documented (from a presentation with results from the ASTD Benchmarking Forum) as dropping in implementation from 94% doing level 1, 34% doing level 2, 13% doing level 3, and 3% doing level 4. Kirkpatrick rode this tool for the rest of his career, created a family business in it, and he wasn’t shy about suggesting that you buy a book to learn about it. Still, Kirkpatrick could’ve renumbered the steps, or been more outspoken about the problems with just step 1. That’s not the idea!
A tribute to Dr Donald Kirkpatrick, pioneer and veteran of Instructional Design
MAY 20, 2014
Dr. Donald Kirkpatrick is one of those few people that have achieved eternity by virtue of their contribution. A professor emeritus at University of Wisconsin and the creator of the Kirkpatrick Four-level Evaluation Model , Dr. Kirkpatrick passed away on May 9, 2014 at the age of 90. However, Dr. Donald Kirkpatrick shall live with us in essence and spirit.
Discussing the ASTD Handbook, eLearning, and Published Expertise with Elaine Biech
AUGUST 4, 2014
When my copy of the second edition ASTD Handbook arrived at the office this summer, I immediately felt the urge to explore my new resource and see what content made the final cut for an important industry asset. The second edition ASTD Handbook is filled with great insights from industry experts. The ASTD Handbook does just that. When the time was right ASTD made the move.
Don Kirkpatrick’s Contribution to Learning & Development
MARCH 1, 2011
By 1959, editors for the predecessor of ASTD’s T&D magazine had heard of his approach and asked him to write an article. I just saw an announcement that Don will retire from the speaking circuit this year. Few people have single-handedly made such an impact on our profession. I have known Don for years and his story is very interesting. Don got his start in learning when he was working on his Ph.D. dissertation at the University of Wisconsin and needed some good measures to evaluate the impact of a training program for supervisors. All four were important to evaluate a program.
Looking Forward to Busy Time at ASTD 2014
MAY 2, 2014
Looking forward to this year’s ASTD ICE conference in Washington, DC. Hoping to show them the highlights of the conference and all the wonderful events at ASTD this year. 1:15 pm – 2:00 pm: Join me for a great panel discussion with Ruth Clark and James Kirkpatrick. ASTD Handbook The book store will be in Salon GHI. ASTD Conference How awesome is that?
Re-evaluating Evaluation | Social Learning Blog
MARCH 15, 2011
Social Learning Blog Training and Performance Improvement in the Real World Home About Bios Subscribe to RSS Re-evaluating Evaluation by Barbara on March 16, 2011 in Project Management/Project Delivery For years, I have dutifully included a description of Kirkpatrick’s Four Levels of Learning Evaluation in every proposal for every company I have worked with. But it wasn’t. She enhanced it.
What's the problem with Kirkpatrick?
Clive on Learning
APRIL 21, 2008
While acknowledging that Kirkpatrick's four levels of evaluation had proved valuable over more than thirty years in helping to measure instructor-led, content-based 'training' interventions, Sloman felt that a new approach was necessary to support a more self-directed, work-based 'learning' process. It's worth checking first of all whether Kirkpatrick's model is still useful for evaluating the top-down stuff. ROI was mentioned frequently in the session in the context of Kirkpatrick's model, an association which I don't understand. Let's start with level 1. I think it does.
Become a Strategic Partner Through Effective Training Evaluation #ASTD2014 @Jim_Kirkpatrick
MAY 7, 2014
These are my live blogged notes from a concurrent session (W104) at the ASTD International Conference & Expo, this week in Washington, D.C. James Kirkpatrick, Senior Consultant Kirkpatrick Partners Sometimes we have to do the politically incorrect thing. Forgive any typos or incoherencies. In the new world, we start with Level 4 - we start with the results. It''s our GPS.
50 Years of the Kirkpatrick Model
AUGUST 31, 2009
In the fifty years since, his thoughts (Reaction, Learning, Behavior, and Results) have gone on to evolve into the legendary Kirkpatrick’s Four Level Evaluation Model and become the basis on which learning & development departments can show the value of training to the business. How has the model evolved over fifty years, is it still relevant? You can download the paper here.
The Power of Simplicity
JUNE 6, 2011
Don Kirkpatrick gave his farewell presentation on the four levels at ASTD’s annual International Conference & Exposition in Orlando last week. His work, popularized by his four-part series in the monthly ASTD magazine in 1959-1960, provided that measurement framework for the profession. The Business Of Learning Kirkpatrick scorecardsHe led two sessions, which were overflowing, and there was a long line at the bookstore to get his autograph. At 86, he is retiring from the speaking circuit. He answered the questions: 1) What should I measure? Do I need help?)
Stop Evaluating Training!
MAY 2, 2013
Kirkpatrick’s evaluation model has long been the holy grail of training effectiveness measurement to businesses. My response to clients has typically been: If you are referring to Kirkpatrick Levels 1 & 2 it’s not very difficult; ‘smiley sheets’ will tell us about Reaction, and ‘assessments’ can help determine Learning. Why is this happening?
Measuring And Evaluating Learning – The Top Skill For L&D Teams
DECEMBER 13, 2011
Recent research by ASTD and REED Learning reveals that measuring and evaluating learning is the top-most skill L&D teams are looking for. Even though it is a challenge to prove training/learning’s effect beyond Kirkpatrick’s level 1 & 2 -senior management continues to ask for numbers so here we are – the L&D folks want that skill the most. From the report, here is the list of top 10 skills in demand: How relevant do you think the following topics would be, as part of a qualification aimed at trainers and L&D practitioners? responses are in order). Coaching Skills.
Why We Should Stop Talking About ROI in Training
APRIL 4, 2012
Many people look at Don Kirkpatrick ’s work from as early as 1959 as the beginning of ROI in learning and development. It was in his early work that Kirkpatrick developed his four-level model: Level 1: Reaction. In concept, Kirkpatrick’s levels seem valuable. But no longer a welcome one. A Brief History of ROI in Training. Level 2: Learning. Level 3: Behavior.
Recent Research Review -- Reviewed (and Lamented)
Will at Work Learning
AUGUST 7, 2014
The Kirkpatrick framework has a number of theoretical and practical shortcomings…” (p. I''ll be speaking on this in an upcoming ASTD webinar. . Basing Our Evaluations on the Kirkpatrick 4 Levels is Insufficient!!! . The worst part about the Kirkpatrick model is that it controls the way we think about learning measurement. They did a really nice job! 86). . 87). .
How Can Infographics Produce Better E-Learning Courses?
MARCH 27, 2012
Understanding Kirkpatrick’s 4 Levels of Evaluation. April 10 : Jacksonville, FL ( NEFL ASTD ). April 17 : Virginia ( SEVA ASTD ). July: Knoxville, TN ( Smoky Mountain ASTD ). August: Houston, TX ( ASTD Houston ). October 17 : Bloomington, IL ( CIC-ASTD ). October 23 & 24 : Milwaukee, WI ( SEWI-ASTD ). Infographics are the hot! Keeping it Simple.
ROI and Metrics in eLearning
OCTOBER 16, 2007
Kirkpatrick's Level 3: Improving the Evaluation of E-Learning Level 3 evaluations measure whether the there was an actual transfer of learning to the actual work setting. The way to demonstrate that is by doing Kirkpatrick Level 3 (measuring the extent ROI of E-Learning: Closing In When it comes to e-learning, computing ROI suddenly becomes a complicated procedure requiring thoughtful chinstroking, serious seminar time, and earnest input from consultants and vendors. I'm working on an article around the use of ROI and metrics in eLearning. Use EVA instead. ROI is toast.
The Only Person Who Behaves Sensibly Is My Tailor
Performance Learning Productivity
NOVEMBER 24, 2014
ATD has been producing this annual report for years (originally as the ASTD). Brinkerhoff’s Success Case Method addresses systems impact rather than trying to isolate the impact of learning individually as the more simplistic Kirkpatrick approach attempts. “The only person who behaves sensibly is my tailor. He takes new measurements every time he sees me. It’s the sensible approach.
Tony Bingham #ASTDL20 keynote
APRIL 29, 2011
Today I’m at ASTD’s New England Regional Conference: Learning 2.0 These are my live blogged notes from the opening keynote with Tony Bingham, CEO and President of ASTD. CEOs don’t care about Kirkpatrick level 1, 2, 3. Don’t get left behind in lovely Westford, MA. Conference Board research from 2009: CEO challenges “how do we stay afloat?” IBM Global Student study 2010 IBM Study: inheriting a complex word [link] “Tomorrows leaders will be markedly different than previous generations.” Because learning is the center of strategy. We won’t go back to the way it was.
Allison Rosset Guest Post: Evaluation—Words Into Action?
The Learning Circuits
MAY 7, 2012
We speak fluent Kirkpatrick. An ASTD benchmarking study looked at course evaluations by Kirkpatrick level. ASTD’s own studies of practice, and others, show steady increases in the use of technology for learning and performance. This is one of those topics that never goes away. It reigns supreme in just about every needs study for workplace learning professionals. We say we want to do more and know more. We are eager to check out more tools, and get a better handle on the situation. The topic—evaluation. But knowing is not doing, not even close. Is it different today?
Why Corporate Training is Broken And How to Fix It
JULY 17, 2012
Don Kirkpatrick proposed a model for measuring the outcomes of training. The American Society for Training and Development (ASTD) reports that companies spend $1,067 per employee (2.7% However, go to any training conference, including ASTD’s own International Conference and Exhibition, and you don’t hear much about instructor-led training or self-paced learning. References 21.
How Big Data Will Reshape eLearning — and the Future of Work
FEBRUARY 21, 2012
This isn’t the traditional, Kirkpatrick -style learning data most people think about, like post-workshop evaluations and test scores. David Kelly is the director of training at Carver Federal Savings Bank and ember of the ASTD National Advisors for Chapters. Big Data is a term that you may not of heard of before but will hear a great deal of in the near future.
Cammy Beans Learning Visions: Beginning Instructional Designers Toolkit
MAY 1, 2007
My list begins as such: Important Theories and Terms: Gagnes 9 Events Kirkpatrick Levels ADDIE Cognitive Load Theory ARCS (John Keller) Books: Ruth Clark: e-Learning and the Science of Instruction Malcom S. About Kilpatrick Levels : do you mean Kirkpatrick levels ? Reflecting on #ASTD10 Recommended: "In Defense of the LMS" Chicago Bound: Kineo at ASTD ICE #astd10 One Year on at Kineo #yam Now that’s an Enterprise Moodle! -- Kineo Moodle f. ASTD Keynote: “People Lie Richard Hilleman Electr. Im not a big theory person. Blooms Taxonmy, of course.
Accelerated Learning: Where Does It Fit In? | Social Learning Blog
MARCH 27, 2011
Organizations such as PACT, ASTD, MNISPI, and the Digital Learning Forum get my creative juices going to generate new ideas and synergies. Social Learning Blog Training and Performance Improvement in the Real World Home About Bios Subscribe to RSS Accelerated Learning: Where Does It Fit In? I always thought that program was extremely innovative. The students directed their own learning.
Top 70 eLearning Posts for June and Hot Topics including iPad and Mobile Learning
eLearning Learning Posts
JULY 7, 2010
Evaluating Non-Formal Learning in the Context of Kirkpatrick’s Four-Level Model - The E-Learning Curve , June 7, 2010 The Four-Level Model is incompatible with the assessment of non-formal learning. Evaluating Non-Formal Learning in the Context of Kirkpatrick’s Four-Level Model is a post from: E-Learning Curve Blog. You might get ideas from this recording of a webinar that I gave today for the Baton Rouge ASTD. Best of eLearning Learning. June 2010. Another great month for eLearning content. don’t want anyone to get me wrong. like Kate a lot. Looking for THE SCORM Resource?
Brain and Memory with Arthur Kohn #astdtk13
JANUARY 31, 2013
I'm at the ASTD Tech Knowledge 2013 Conference kicking off today January 29, 2013 in San Jose, California. so here's the return on investments, here's your Kirkpatrick.). These are my liveblogged notes from a concurrent session with Arthur Kohn. Apologies for typos and incoherence. Brain and Memory: Seven Tips that Improve E-Learning. Arthur Kohn, PhD. Fulbright Professor of Cognitive Science. He starts us off with a memory test. He gives us 15 words in turn and then after he's said them all, we write them down in any order: (I remembered 12 out of 15 of them!). e.g., Coaching.
Hot List - Second Half of January
FEBRUARY 4, 2009
As I was at ASTD last week, I missed doing my Hot List for the week. Alternatives to Kirkpatrick Wiki as repository for a virtual community Wikipedia - tightening editing Remote Collaboration the Horizon report 2009 from Educause Items HP Learning Center - free online software, business, marketing and IT classes for small and medium business Keywords Social Media Work Skills Twitter Photoshop Captivate Adobe Google Articulate eBook Tools Demo ASTD TechKnowledge Notes on the list. You can find out more about this in Hot List and Hot Last Week.
Better Discussion / Debate in Learning
SEPTEMBER 6, 2006
In particular, he laments that as he was reading a discussion in a different domain and "working on the team picking out sessions for the 2007 ASTD conference " he found himself wanting to have better discussion and debate in our world of learning. Actually, I think that maybe FOO Camp and Bar Camp aren't quite the right format, but having the kind of dialog that occurs at unconferences is definitely needed at corporate learning conferences, e.g., ASTD, Training, etc. Kirkpatrick? Mark Oehlert post: I long for this kind of debate in our learning circles. Mark - I do too!
Cammy Beans Learning Visions: Youre So Immature: e-Learning in Some Organizations
OCTOBER 5, 2007
He was venting about Kirkpatrick evaluations. Reflecting on #ASTD10 Recommended: "In Defense of the LMS" Chicago Bound: Kineo at ASTD ICE #astd10 One Year on at Kineo #yam Now that’s an Enterprise Moodle! -- Kineo Moodle f. ASTD Keynote: “People Lie Richard Hilleman Electr. Cammy Beans Learning Visions Musings on eLearning, instructional design and other training stuff. Friday, October 05, 2007 Youre So Immature: e-Learning in Some Organizations Earlier this week I was musing about messes in Messy Learning OK. Messy Training Not OK. Games are scary.
NOVEMBER 18, 2009
Plug for ASTD - CPLP Certification includes all of these areas of expertise. Asif: Kirkpatrick talks about 'ROE -- return on expectations'. Clark Quinn and I led a discussion on Reinventing Organizational Learning at LearnTrends this morning. The recording will be up before the day is over, but I thought you might enjoy the discussion that went with it. tmast: yeah. Mars Chen: ok.