Kirkpatrick Revisited | Social Learning Blog
APRIL 17, 2011
It was an opportunity I couldn’t miss. I have included Kirkpatrick’s Four Levels of Evaluation in every proposal I have ever written, and I wanted to hear from Kirkpatrick himself regarding his take on the current state of evaluation and whether his four levels are still viable. Well, based on where Kirkpatrick and his son James are today, I was completely wrong. training?to
Vale Don Kirkpatrick
MAY 28, 2014
Last week, Don Kirkpatrick passed away. The numbers have been documented (from a presentation with results from the ASTD Benchmarking Forum) as dropping in implementation from 94% doing level 1, 34% doing level 2, 13% doing level 3, and 3% doing level 4. Kirkpatrick rode this tool for the rest of his career, created a family business in it, and he wasn’t shy about suggesting that you buy a book to learn about it. I certainly can’t fault him for it either, as he did have a sensible model and it could be put into effective use. This actually made a lot of sense.
A tribute to Dr Donald Kirkpatrick, pioneer and veteran of Instructional Design
MAY 20, 2014
Dr. Donald Kirkpatrick is one of those few people that have achieved eternity by virtue of their contribution. professor emeritus at University of Wisconsin and the creator of the Kirkpatrick Four-level Evaluation Model , Dr. Kirkpatrick passed away on May 9, 2014 at the age of 90. He recently reflected on his legacy in a thought-provoking post on the Kirkpatrick Partners blog.
Discussing the ASTD Handbook, eLearning, and Published Expertise with Elaine Biech
AUGUST 4, 2014
When my copy of the second edition ASTD Handbook arrived at the office this summer, I immediately felt the urge to explore my new resource and see what content made the final cut for an important industry asset. The second edition ASTD Handbook is filled with great insights from industry experts. The ASTD Handbook does just that. When the time was right ASTD made the move.
Evaluating Informal Learning
JULY 12, 2013
My colleague, Andrea May came back from ASTD International Conference & Exposition (ICE), which was held in Dallas in May of this year, raving about a presentation on “Evaluating Informal Learning.” She knows that I have been blogging about learning evaluation for the past couple of years—mostly Kirkpatrick but also [.] Evaluation Informal Learning
Don Kirkpatrick’s Contribution to Learning & Development
MARCH 1, 2011
By 1959, editors for the predecessor of ASTD’s T&D magazine had heard of his approach and asked him to write an article. I just saw an announcement that Don will retire from the speaking circuit this year. Few people have single-handedly made such an impact on our profession. have known Don for years and his story is very interesting. Don got his start in learning when he was working on his Ph.D. dissertation at the University of Wisconsin and needed some good measures to evaluate the impact of a training program for supervisors. All four were important to evaluate a program.
Looking Forward to Busy Time at ASTD 2014
MAY 2, 2014
Looking forward to this year’s ASTD ICE conference in Washington, DC. Hoping to show them the highlights of the conference and all the wonderful events at ASTD this year. ” Sunday, May 4, 2014. 1:15 pm – 2:00 pm: Join me for a great panel discussion with Ruth Clark and James Kirkpatrick. ASTD Handbook The book store will be in Salon GHI.
Re-evaluating Evaluation | Social Learning Blog
MARCH 15, 2011
Social Learning Blog Training and Performance Improvement in the Real World Home About Bios Subscribe to RSS Re-evaluating Evaluation by Barbara on March 16, 2011 in Project Management/Project Delivery For years, I have dutifully included a description of Kirkpatrick’s Four Levels of Learning Evaluation in every proposal for every company I have worked with. But it wasn’t. She enhanced it.
50 Years of the Kirkpatrick Model
AUGUST 31, 2009
In the fifty years since, his thoughts (Reaction, Learning, Behavior, and Results) have gone on to evolve into the legendary Kirkpatrick’s Four Level Evaluation Model and become the basis on which learning & development departments can show the value of training to the business. How has the model evolved over fifty years, is it still relevant? You can download the paper here.
What's the problem with Kirkpatrick?
Clive on Learning
APRIL 21, 2008
While acknowledging that Kirkpatrick's four levels of evaluation had proved valuable over more than thirty years in helping to measure instructor-led, content-based 'training' interventions, Sloman felt that a new approach was necessary to support a more self-directed, work-based 'learning' process. It's worth checking first of all whether Kirkpatrick's model is still useful for evaluating the top-down stuff. ROI was mentioned frequently in the session in the context of Kirkpatrick's model, an association which I don't understand. Let's start with level 1. think it does.
Become a Strategic Partner Through Effective Training Evaluation #ASTD2014 @Jim_Kirkpatrick
MAY 7, 2014
These are my live blogged notes from a concurrent session (W104) at the ASTD International Conference & Expo, this week in Washington, D.C. James Kirkpatrick, Senior Consultant Kirkpatrick Partners Sometimes we have to do the politically incorrect thing. Forgive any typos or incoherencies. In the new world, we start with Level 4 - we start with the results. It''s our GPS.
The Power of Simplicity
JUNE 6, 2011
Don Kirkpatrick gave his farewell presentation on the four levels at ASTD’s annual International Conference & Exposition in Orlando last week. His work, popularized by his four-part series in the monthly ASTD magazine in 1959-1960, provided that measurement framework for the profession. The Business Of Learning Kirkpatrick scorecardsHe led two sessions, which were overflowing, and there was a long line at the bookstore to get his autograph. At 86, he is retiring from the speaking circuit. He changed the profession and leaves us a tremendous legacy.
Stop Evaluating Training!
MAY 2, 2013
Kirkpatrick’s evaluation model has long been the holy grail of training effectiveness measurement to businesses. My response to clients has typically been: If you are referring to Kirkpatrick Levels 1 & 2 it’s not very difficult; ‘smiley sheets’ will tell us about Reaction, and ‘assessments’ can help determine Learning. Why is this happening?
Measuring And Evaluating Learning – The Top Skill For L&D Teams
DECEMBER 13, 2011
Recent research by ASTD and REED Learning reveals that measuring and evaluating learning is the top-most skill L&D teams are looking for. Even though it is a challenge to prove training/learning’s effect beyond Kirkpatrick’s level 1 & 2 -senior management continues to ask for numbers so here we are – the L&D folks want that skill the most. From the report, here is the list of top 10 skills in demand: How relevant do you think the following topics would be, as part of a qualification aimed at trainers and L&D practitioners? responses are in order). Coaching Skills.
Why We Should Stop Talking About ROI in Training
APRIL 4, 2012
Many people look at Don Kirkpatrick ’s work from as early as 1959 as the beginning of ROI in learning and development. It was in his early work that Kirkpatrick developed his four-level model: Level 1: Reaction. In concept, Kirkpatrick’s levels seem valuable. But no longer a welcome one. Brief History of ROI in Training. Level 2: Learning. Level 3: Behavior.
How Can Infographics Produce Better E-Learning Courses?
MARCH 27, 2012
Understanding Kirkpatrick’s 4 Levels of Evaluation. April 10 : Jacksonville, FL ( NEFL ASTD ). April 17 : Virginia ( SEVA ASTD ). July: Knoxville, TN ( Smoky Mountain ASTD ). August: Houston, TX ( ASTD Houston ). October 17 : Bloomington, IL ( CIC-ASTD ). October 23 & 24 : Milwaukee, WI ( SEWI-ASTD ). Infographics are the hot! Keeping it Simple.
Recent Research Review -- Reviewed (and Lamented)
Will at Work Learning
AUGUST 7, 2014
The Kirkpatrick framework has a number of theoretical and practical shortcomings…” (p. Will''s Note: I''m a big believer in the power of supervisors to enable learning. I''ll be speaking on this in an upcoming ASTD webinar. . Basing Our Evaluations on the Kirkpatrick 4 Levels is Insufficient!!! . They did a really nice job! THIS IS DEEPLY TROUBLING! 86). . 87). . 90). .
ROI and Metrics in eLearning
OCTOBER 16, 2007
Kirkpatrick's Level 3: Improving the Evaluation of E-Learning Level 3 evaluations measure whether the there was an actual transfer of learning to the actual work setting. The way to demonstrate that is by doing Kirkpatrick Level 3 (measuring the extent ROI of E-Learning: Closing In When it comes to e-learning, computing ROI suddenly becomes a complicated procedure requiring thoughtful chinstroking, serious seminar time, and earnest input from consultants and vendors. thought it would be worth posting this list here. Use EVA instead. ROI is toast. Use EVA instead.
The Only Person Who Behaves Sensibly Is My Tailor
Performance Learning Productivity
NOVEMBER 24, 2014
ATD has been producing this annual report for years (originally as the ASTD). Brinkerhoff’s Success Case Method addresses systems impact rather than trying to isolate the impact of learning individually as the more simplistic Kirkpatrick approach attempts. “The only person who behaves sensibly is my tailor. He takes new measurements every time he sees me. It’s the sensible approach.
Tony Bingham #ASTDL20 keynote
APRIL 29, 2011
Today I’m at ASTD’s New England Regional Conference: Learning 2.0 These are my live blogged notes from the opening keynote with Tony Bingham, CEO and President of ASTD. CEOs don’t care about Kirkpatrick level 1, 2, 3. Don’t get left behind in lovely Westford, MA. Conference Board research from 2009: CEO challenges “how do we stay afloat?” IBM Global Student study 2010 IBM Study: inheriting a complex word [link] “Tomorrows leaders will be markedly different than previous generations.” Because learning is the center of strategy. We won’t go back to the way it was.
Allison Rosset Guest Post: Evaluation—Words Into Action?
The Learning Circuits
MAY 7, 2012
We speak fluent Kirkpatrick. An ASTD benchmarking study looked at course evaluations by Kirkpatrick level. ASTD’s own studies of practice, and others, show steady increases in the use of technology for learning and performance. This is one of those topics that never goes away. It reigns supreme in just about every needs study for workplace learning professionals. We say we want to do more and know more. We are eager to check out more tools, and get a better handle on the situation. The topic—evaluation. But knowing is not doing, not even close. Is it different today?
Starting a conversation about learning analytics
JUNE 21, 2014
Evaluation has always been an important part of the process in training and development; the ubiquitous Kirkpatrick model was proposed in the 1950s. ASTD & The Maise Center, 2014) – this is a short book that does a terrific job giving an overview for training and development professionals, and it contains a number of case studies. By Ellen Wagner. (T+D, August 2012) ASTD members can view online here. This is the fourth in my series of posts sharing the reading list from my summer emerging technologies course. Introduction. Enter big data. Required. Recommended. Trends
Why Corporate Training is Broken And How to Fix It
JULY 17, 2012
Don Kirkpatrick proposed a model for measuring the outcomes of training. The American Society for Training and Development (ASTD) reports that companies spend $1,067 per employee (2.7% of payroll) to deliver an average of 32 hours of formal training annually. ASTD measures only formal training, the workshops, classes, and assignments meted out by training departments. Hart).
How Big Data Will Reshape eLearning — and the Future of Work
FEBRUARY 21, 2012
This isn’t the traditional, Kirkpatrick -style learning data most people think about, like post-workshop evaluations and test scores. Big David Kelly is the director of training at Carver Federal Savings Bank and ember of the ASTD National Advisors for Chapters. He Big Data is a term that you may not of heard of before but will hear a great deal of in the near future.
Cammy Beans Learning Visions: Beginning Instructional Designers Toolkit
MAY 1, 2007
My list begins as such: Important Theories and Terms: Gagnes 9 Events Kirkpatrick Levels ADDIE Cognitive Load Theory ARCS (John Keller) Books: Ruth Clark: e-Learning and the Science of Instruction Malcom S. About Kilpatrick Levels : do you mean Kirkpatrick levels ? Reflecting on #ASTD10 Recommended: "In Defense of the LMS" Chicago Bound: Kineo at ASTD ICE #astd10 One Year on at Kineo #yam Now that’s an Enterprise Moodle! -- Kineo Moodle f. ASTD Keynote: “People Lie Richard Hilleman Electr. Im much more into practical application. Amen, Cammy.
Using Evaluation Results Effectively
JANUARY 4, 2012
Kirkpatrick Quick Tip Vol. 2 #1. To start your year right we have a gift for you: a free copy of the Kirkpatrick chapter, Chapter 20, of the ASTD Handbook of Measuring and Evaluating Training. Unless evaluation results are effectively used, you have likely expended a lot of energy but stopped just short of realizing the ultimate purpose of training and learning - to improve organizational effectiveness
Getting to the Core
MARCH 16, 2011
This great question was posted by Ling-Huey on ASTD’s LinkedIn forum, and her question generated dozens and dozens of comments. “Core Competencies are usually skills and knowledge identified as being necessary for performing a function to an acceptable level of competency (i.e., “ability to properly structure instruction with respect to Kirkpatrick… or “ability to manage the learning environment ). “What Are the Core Competencies of a Trainer? involve the learners. exude enthusiasm and passion. understand how adults learn.
Accelerated Learning: Where Does It Fit In? | Social Learning Blog
MARCH 27, 2011
Organizations such as PACT, ASTD, MNISPI, and the Digital Learning Forum get my creative juices going to generate new ideas and synergies. I am an avid film goer, music lover, bridge walker, and supporter of the Dashe & Thomson running team. At first I couldn’t think whether I actually applied how training was delivered to these different types of learners. link] Blog this! Properly d.
Brain and Memory with Arthur Kohn #astdtk13
JANUARY 31, 2013
I'm at the ASTD Tech Knowledge 2013 Conference kicking off today January 29, 2013 in San Jose, California. month touch -- ask that same person -- "can you give us examples of how you've used this in your organization?" (so here's the return on investments, here's your Kirkpatrick.). These are my liveblogged notes from a concurrent session with Arthur Kohn. Apologies for typos and incoherence. Brain and Memory: Seven Tips that Improve E-Learning. Arthur Kohn, PhD. Fulbright Professor of Cognitive Science. He starts us off with a memory test. We are organic machines. Coaching.
Top 70 eLearning Posts for June and Hot Topics including iPad and Mobile Learning
eLearning Learning Posts
JULY 7, 2010
Evaluating Non-Formal Learning in the Context of Kirkpatrick’s Four-Level Model - The E-Learning Curve , June 7, 2010 The Four-Level Model is incompatible with the assessment of non-formal learning. Evaluating Non-Formal Learning in the Context of Kirkpatrick’s Four-Level Model is a post from: E-Learning Curve Blog. You might get ideas from this recording of a webinar that I gave today for the Baton Rouge ASTD. Best of eLearning Learning. June 2010. Another great month for eLearning content. don’t want anyone to get me wrong. like Kate a lot. Looking for THE SCORM Resource?
Hot List - Second Half of January
FEBRUARY 4, 2009
As I was at ASTD last week, I missed doing my Hot List for the week. Alternatives to Kirkpatrick Wiki as repository for a virtual community Wikipedia - tightening editing Remote Collaboration the Horizon report 2009 from Educause Items HP Learning Center - free online software, business, marketing and IT classes for small and medium business Keywords Social Media Work Skills Twitter Photoshop Captivate Adobe Google Articulate eBook Tools Demo ASTD TechKnowledge Notes on the list. You can find out more about this in Hot List and Hot Last Week.
Better Discussion / Debate in Learning
SEPTEMBER 6, 2006
In particular, he laments that as he was reading a discussion in a different domain and "working on the team picking out sessions for the 2007 ASTD conference " he found himself wanting to have better discussion and debate in our world of learning. Actually, I think that maybe FOO Camp and Bar Camp aren't quite the right format, but having the kind of dialog that occurs at unconferences is definitely needed at corporate learning conferences, e.g., ASTD, Training, etc. Kirkpatrick? Mark Oehlert post: I long for this kind of debate in our learning circles. Mark - I do too!
Cammy Beans Learning Visions: Youre So Immature: e-Learning in Some Organizations
OCTOBER 5, 2007
He was venting about Kirkpatrick evaluations. Reflecting on #ASTD10 Recommended: "In Defense of the LMS" Chicago Bound: Kineo at ASTD ICE #astd10 One Year on at Kineo #yam Now that’s an Enterprise Moodle! -- Kineo Moodle f. ASTD Keynote: “People Lie Richard Hilleman Electr. Cammy Beans Learning Visions Musings on eLearning, instructional design and other training stuff. Friday, October 05, 2007 Youre So Immature: e-Learning in Some Organizations Earlier this week I was musing about messes in Messy Learning OK. Messy Training Not OK. Games are scary.
NOVEMBER 18, 2009
Plug for ASTD - CPLP Certification includes all of these areas of expertise. Asif: Kirkpatrick talks about 'ROE -- return on expectations'. Clark Quinn and I led a discussion on Reinventing Organizational Learning at LearnTrends this morning. The recording will be up before the day is over, but I thought you might enjoy the discussion that went with it. Mars Chen: ok. it's 9AM.