Closing the gap between what assessments and eLearning evaluations are measuring and what they should be measuring is no simple task. A recent Guild research report by Will Thalheimer, Evaluating Learning: Insights from Learning Professionals, finds that most organizations measure attendance, completions, and learner perceptions of eLearning and other training. Instead, they should create and track evaluation objectives that target relevant results.
Capturing attendance and completion data allows organizations to demonstrate that they’ve sent factory workers to safety training, that managers know they are not supposed to harass their colleagues, that advisors are up-to-date on the latest contract regulations, etc. But learner surveys don’t measure knowledge, stickiness of training, or ability to transfer material learned to on-the-job application.
What is an evaluation objective?
Thalheimer, the report’s author and principal at Work-Learning Research, coined the term to describe a measurable outcome that is desired as the result of training. Examples he cites on his blog include “In the ‘Change-Management Simulation’ the learner will score 65 points out of a total possible of 90,” and “The learner will initiate a change effort within one month after the training ends and be successful in getting 75% of his/her colleagues to sign a public statement of support for the effort.”
These offer relevant activities that a learner can undertake. The activities apply the training. The desired result is clear—and can be measured. The data gathered can then inform instructional designers (and the learners’ managers) whether the training is effective. Can the person perform the stipulated task? Has performance improved since training was completed?
Thalheimer wrote that objectives focused on the big picture—job performance and business goals—are more relevant and therefore learners see them as valuable. In addition, they provide data that can be used to improve eLearning and drive impactful goals.
Evaluation objectives complement learning objectives
Evaluation objectives and learning objectives serve different purposes, but they might capture some of the same elements of learning.
“Before we even create a list of learning objectives, we should create a list of evaluation objectives,” Thalheimer wrote in the report. This allows evaluation to be “baked into our design and development process.”
Learning objectives are often written in broad terms; they stipulate that learners be able to describe or explain or determine something. While a learner might be able to meet these objectives, the results are difficult to measure. They are highly subjective and abstract. They might serve to focus learners on specific aspects of the material or tell learners how they will be expected to apply the material. Other learning objectives guide the design of the eLearning.
An evaluation objective can focus those goals and wrap them in measurable outcomes; a particular score on a test or a quantifiable boost in performance or reduction in errors or safety incidents.
Not all learning objectives are aimed at learners; some provide information for the instructional designers and the organizations—the managers and executives requesting and evaluating the eLearning. Evaluation objectives fall into this category. Learners might never see them!
<H2>Do a better job of evaluating learning
Evaluation objectives are just one way to get a more complete picture of the effectiveness and value of the eLearning you create. Learn more about what organizations are measuring, what they should be measuring, and how to evaluate eLearning more rigorously. Download your copy of Evaluating Learning: Insights from Learning Professionals today.