Training Evaluations 101: Benefits, How To, Plus Examples You’ll Want To Use

Organizations globally are investing $400 billion annually in training. Naturally, if you invest so much in training, you should show a measurable positive impact. That’s where training evaluations come in. But are all training evaluations the same? And are the benefits of training evaluations worth the effort? We answer these questions and more by providing examples of training evaluations you can apply in your specific situation.

What Are Training Evaluations?

Training evaluation was defined by A.C Hamblin as “any attempt to obtain information on the effects of training performance and to assess the value of training in the light of that information.” That’s a pretty broad definition, albeit true. And such a broad definition doesn’t provide a lot of guidance to create an evaluation of a training course.

So, the idea of training evaluations is simple, conceptually. Your organization has developed a training program and wants to see its success. But the devil is in the details.

Measuring The Impact Of Training

While the definition of a training evaluation isn’t complicated, measuring the impact of training isn’t so easy. There are examples of training evaluations that focus on the financial costs of changing behavior. And others that incorporate holistic measures of skill, ability, and knowledge. And in all fairness, isolating the impact of training is no straightforward task. For instance, an employee’s improved performance could result from several factors. Including office culture, financial perks, supervisory feedback, and growth possibilities. As well as the training they have recently taken.

For instance, perhaps you have designed examples of training evaluations for courses on improving employees’ productivity. Or the efficiency of procedures at the workplace. But significantly different training evaluations are necessary for other situations. For example, consider a course on selling instructional products to generate revenue or educating a customer about a new product they have purchased from you. These are all need different examples of training evaluations. So, finding a suitable evaluation method is essential. To this end, we focus on examples of training evaluations for these and other situations. But first, what are the benefits of training evaluations, and are they worth the effort?

Benefits Of Training Evaluations

While there are many examples of training evaluations, designing the right one isn’t a trivial task. But before we get into specific examples, let’s highlight the benefits. As you’ll see below, the benefits of training evaluations are multidimensional.

Areas Of Improvement

One of the main benefits of training evaluations is pinpointing shortcomings and areas for improvement. What was missing? What was irrelevant? Should we restructure the format?

From the content to the planning and tools used to deliver the courses, slight tweaks can improve results. Moreover, with the fast-paced world of technology, ideas can quickly become out-of-date. And so, it needs updating. All examples of training evaluations must provide recommendations on improving or updating your course.

Improving Participant Engagement

Another of the benefits of training evaluations is that the feedback can be used to increase the engagement of future participants. By including a few simple questions in your assessment, you can gain valuable insights.

For example:

• How did you find the course?
• Which parts of the training were most relevant to you?
• Can you describe the most important, or best, part of the course from your perspective?
• How could we improve the course?

Asking open-ended questions allows you to receive more feedback than asking closed-ended questions. Genuine feedback from open-ended questions can be precious. With it, you can find how to improve the training further. And are likely to identify improvements you wouldn’t have thought of, which will lead to enhancing participant engagement.

You can also use some of the positive responses from participants as testimonials to encourage others to take the training. And these outcomes from the training evaluations will build credibility in the course.

Measure Of Knowledge And Learning

One of the main benefits of training evaluations is to find out if the instructional goals of your training have been met or not. Typically, an assessment of knowledge transfer is assessed at different points throughout the course. And the specific approach will vary depending on the goals of the training. For example, learning specific knowledge and tasks will be easier to gauge. But, the success of less tangible outcomes, such as from a leadership course, may need to be observed in a role-playing situation. Or following completion of the training by assessment later. Such as a supervisor observing how a person interacts with their team members.

No matter the type of training course, you should always include one question in any examples of training evaluations you use. “What are you – the learner – going to do with this new training?”

Employee Retention

As firms seek growth opportunities, so do employees. They search for professional development and better pay. Investing in training signals that the company is also focused on their growth. And it has demonstrated that organizations providing training and development opportunities increase retention. So, quality training evaluations also improve employee retention by enabling future improvements in the training.

Efficient Use Of Resources

A focus of most organizations is the efficient use of their resources. And if used correctly, one of the benefits of training evaluations can be to enhance that. No one wants to continue offering a course that isn’t meeting expectations. So, find a way to improve it or stop running it. And divert those resources to areas that have a more beneficial outcome.

You can also use training evaluations to analyze the effectiveness between different training programs or methods of delivery. For example, how does an on-demand e-learning course compare to an in-person instructor-led course? How does the achievement of the instructional goals compare? How does the ROI compare? Do participants prefer one delivery method over another?

Additional Funding

Another one of the less common benefits of training evaluations is that they can help you receive extra funding. Use the results of your training evaluations to support requests for an increase in your budget to expand the program. Or, if you are a non-profit organization, apply for a grant to scale or expand the training you are providing.

Instructors Evaluation

If you use an instructor in part of your training, a benefit of training evaluations is that you can also provide feedback for them. Are there any areas for future improvement for the instructor or facilitator?

Reasons Training Evaluations Fail

Despite the benefits of training evaluations, there are several common reasons that they fail. These include the following.

Willingness To Complete Training Evaluations

For training evaluations to be successful requires at least some feedback from the participants. If the participants aren’t willing to spend the time to provide feedback, you are missing out on a lot of valuable information.

Often people are more willing to provide feedback if the time required to complete training evaluations isn’t extensive. Also, if participants understand there is a genuine interest in receiving their feedback and know why giving it is essential. It’s much more likely they will take the time to complete it.

Convoluted Or Complex Training Evaluations

If evaluations are complex, people will focus on getting through them rather than on the quality of the feedback they provide. And you’ll lose many of the benefits of training evaluations.

Poor Questions That Fail To Capture The Important Information

If you ask the wrong type of questions, you could miss out on many valuable insights. It’s easy to ask participants to complete a few multiple-choice or rate on a scale type questions. And it’s quick for them to respond. But is it getting at the information you need to improve the course and measure its effectiveness? Often the most valuable information is from an open-ended question. So in creating training evaluations, you have to be precise in the questions you ask and balance what you ask with the ease of completing it.

A Failure To Act

You could have the best training evaluations ever designed. But unless you act on the information they provide, it won’t help. You must use this valuable information to make decisions about the training.

Examples Of Training Evaluations

Below we discuss five very different examples of training evaluations. Each of these provides insights on how to develop different training evaluations. Our first example is training employees in a retail setting on new products.

1) Examples Of Training Evaluations – Selling Clothing In A Retail Store

You have probably heard of the Kirkpatrick method of training evaluations. While well known, this is only one method to evaluate training effectiveness. This method is suitable to use where there are very transactional aspects to the training. Such as in a training session on new products in a retail store selling garments in a clothing store.

It consists of 4 levels of training evaluation described below.

Level 1: Reaction

The degree to which participants find the training favorable, engaging, and relevant to their jobs

Level 2: Learning

The degree to which participants acquire the intended knowledge, skills, attitude, confidence, and commitment based on their participation in the training

Level 3: Behaviour

The degree to which participants apply what they learned during training when they are back on the job

Level 4: Results

The degree to which targeted outcomes occur as a result of the training

Let’s assume that the training consists of a 1 to 2-hour session to explain a line of new clothing products to store sales reps. In particular, you want the sales reps to understand the new clothing line. And the advantages of this line over other products to increase sales.

The first part of your training evaluations could be a few questions the participant rates from 1-10 to assess their reaction to the training. Was the session informative? Are they excited by the new product line etc.?

The next part of the training evaluation could be a short quiz testing their knowledge of the products. This could be using multiple-choice questions to test knowledge about the features of the products. Or advantages of them over similar products.

The third part determines if they can apply the knowledge they have learned when they are back working. One way to do this is to have their supervisor observe their interactions with customers. The added benefit of this is that the supervisor can also provide coaching and feedback.

The final part of the evaluation of training process assesses how well the training helps increase sales of the new products. To be effective, you need to have baseline data about product sales before the training. Or, if this is an entirely new product, you could have a few control groups. These are groups of sales reps who aren’t given the training, and you compare their sales with those of reps who have taken the training. Based upon these results, you can assess how well the training contributed to the sales goals.

2) Using The Learner Transfer Evaluation Model (LTEM) For Employee Training 

A better model for training evaluations in less transactional situations is Will Thalheimer’s LTEM. This model is described below.

Let’s apply this model to an employee diversity training session. These examples of training evaluations could be used in similar situations. In this instance, the main goal of this program is to create a positive work environment by helping employees recognize and be tolerant of differences among co-workers. Training evaluations of this type could be constructed as follows.

Tier 1 – Attendance. This is simply a measure of how many people (or percentage) attended the training session.

Tier 2 – Activity. For example, how many people started the course, and how much of it did they complete?

Tier 3 – Learner Perceptions. The part of the training evaluations could be in the form of a survey about the course. This could include feedback on a ranking scale about their perceptions. And it could consist of anecdotal feedback as well. Questions can include how effective they think the training will be for them. Or what they would do with this new insight after the training.

Tier 4 – Knowledge. Testing for knowledge, understanding, and retention would take the form of a quiz. They could take a test at the end of the course to test for learning. Completing the test a few days after the course would assess for retention. This tests factual knowledge and could take the form of multiple-choice questions.

Tier 5 – Decision Making Competence. This part of the training evaluation could be done in many ways. For example, if part of the training is an on-demand course, the learner could complete scenarios about diversity situations as part of the online course. This could include selecting different options within the scenario and receiving appropriate feedback.

Tier 6 – Task Competence. A great way in this type, of course, to test for decision-making competence during the session is to use role-playing scenarios. For example, attendees could split into groups of 3. Two members of the group have a role-playing scenario to act out. While the third member observes the interactions and, using a job aide or reference sheet, provides feedback. After the course, team meetings can include role-playing to evaluate training effectiveness further.

Tier 7 – Transfer. An employee’s supervisor can do continued coaching and assessment and it can be included as a formal part of the evaluation of training. As well, there is also the option of completing an on-the-job employee 360 evaluation. This can provide a range of feedback on how the individual contributes to a positive work environment.

Tier 8 – Effects of Transfer. Employee engagement and satisfaction surveys can assess the longer-term effects of knowledge transfer. If there are control groups that did not take the training, a comparison can show the effectiveness of the training.

Thalheimer-based examples of training evaluations are more involved and take longer to plan and undertake. But they do provide a lot more in-depth assessment of the impacts of training and transfer of knowledge.

3) Examples Of Training Evaluations in Regulatory Environments – Using the Behaviour Engineering Model

Thomas Gilbert (1927 – 1995) was a psychologist who used behavioral psychology to improve performance at work and school. Gilbert identified six variables that he believed were necessary to improve human performance: information, resources, incentives, knowledge, capacity, and motives. He called this his Behavioural Engineering Model (BEM). Gilbert believed that the absence of performance support at work was the most significant barrier to performance. And not an individual’s lack of knowledge or skill. So, he believed it necessary to focus on variables in the work environment before addressing them individually.

At the essence of the model, training is only one component needed to change behavior. And, to assess the success of the change, training evaluations have to be expanded to include these extra components.

At Spark + Co., we have adapted Gilbert’s original model to use in specific situations.

For this example of training evaluation, consider a situation in the Financial Services sector. A bank with a regulatory compliance problem often wants a simple solution and requests a training session. But we know that often this would very likely only make modest gains on its own. For instance, let’s use the example of a bank that needs to increase its compliance with Anti Money Laundering procedures.

To use this method effectively, you need to exert influence over more than just the training. Since you need to address other factors that impact the effectiveness of the process. To use this method, you will

• Set clear performance expectations
• Streamline the process used, in this case, for the Anti-Money Laundering procedures
• Provide quality and accessible resources to all employees
• Create feedback channels for supervisors to provide performance feedback
• Train supervisors to provide high-quality feedback and coaching

All these factors contribute to an increase in performance. And while it’s more involved and needs more time to design and measure performance, it leads to enhanced results and improvement. And each of the five areas we use contributes to the progress. In comparison, training alone can only improve about 20% of performance on its own.

So, in examples of training evaluations of this kind, you need to expand the assessment outside of solely the training component. And test all areas to measure effectiveness and make future improvements.

4) Examples Of Training Evaluations For Selling An Instructional Product

Training evaluations for a course you are selling directly to a consumer are often quite different from the previous examples of training evaluations we’ve covered so far. And in many ways, they are much more straightforward. Let’s take the example of a yoga instructor selling a course on yoga techniques and practice. As well as developing evaluations for the skill you want your learner to perform, the following are less obvious aspects to include.

Examples of Training Evaluations – Using Net Promoter Score

As with any course, you will want to assess the participants’ reactions and perceptions about the course. But the main question that you need to answer is how likely would the participant be to recommend your yoga course to a friend or colleague? This can determine your net promoter score (NPS). NPS measures a consumer’s experience and helps predict your business growth.

These training evaluations take the form of a question on a scale of 0-10. Those who score 9-10 are promoters. They are loyal enthusiasts who will buy your products and refer others. Thus increasing your growth. Those who score 7-8 are Passives. Although satisfied, they are unenthusiastic consumers. And are likely to be tempted by your competitors’ courses. People who score 0-6 are Detractors. And often unhappy customers who might damage your brand and slow growth through negative word of mouth.

By subtracting the percentage of Detractors from the percentage of Promoters, you get the net promoter score (NPS). It ranges from 0-100.

Additional Components For Training Evaluations For Instructional Products

So in this example, your training evaluations could be one question. But we wouldn’t recommend that since you have an opportunity to collect a lot of valuable information besides your NPS.

For example, what about the price of your course? During the course development, you’ll have done market research to identify a suitable selling price. But are you leaving money on the table, or do your consumers feel that your course is overpriced? Asking a simple scaled survey question in your training evaluations can shed light on this.

You also have the opportunity in your survey to ask for a testimonial. This is often an excellent way to get testimonials to advertise and market your course. And if you have more classes, you can ask them their interest in taking one of those. Or, if you are considering creating new courses, you can provide them with a series of topics asking which they’d be most interested in—helping you gauge the market for your next course.

5) Training Evaluations For Customer Education

Customer Education is often much more challenging to complete training evaluations. That’s because you are evaluating three different sets of training expectations. Let’s take the situation where you provide training for a software product sold to enterprise customers. Imagine you sell project management software to other businesses. You want to sell the software, but you also want people to use and love your product. And your customers want their employees to be able to use your product, and they want to justify spending money on the software and its training. As a Customer Education manager, you are often juggling three stakeholders.

1. The learner – most people in instructional roles want their “learner” to receive a great training experience.
2. Your boss – you want to prove that investing in education is a good thing for the business.
3. The enterprise customers for your product buy software on behalf of their employees and want a good training experience ensuring they get value for money.

So in customer education, examples of training evaluations of each of these perspectives need to be included.

Learner Focused Training Evaluation

When creating the learner-focused part of training evaluations, you will want to assess their

• Reaction, or learner perceptions about the training
• Knowledge
• Decision-making competence

But that’s not all. You also want to determine in the training evaluations the impact of the training. In this software training example, you want people to use the software. And many people feel that measuring the completion of the course is the critical metric. But that alone only has limited usefulness. It’s not much more than a Level 1 evaluation mentioned in the first example of training evaluations earlier.

Christy Hollingsworth (Senior Director Customer Education) at Heap agrees. Christy had this to say about designing training evaluations in a recent podcast.

“Over the years, I’ve started to care less and less about completions. You know, if they’re dropping off right away, I think that’s a very different problem. But if they’re getting 80%, through the content, and they’re not finishing the last few things, often, what I find is … they want to start applying it.”

So, measure activity and completion, but don’t focus only on that. Like with any other evaluation of training, you want to connect to application or transfer of learning. What would you see if your training was making an impact? How would you know? How would you measure that in your training evaluations?

Training Provider Evaluation

As the “training provider,” you are also under pressure to evaluate the impact of training on YOUR business. As a Customer Education professional, when your boss asks, “is the training working?” you want to be able to determine:

1. Of the customers who use your software, how many take the training?
   This will give you insight into the overall usage of the training. You can then compare that to the training design.
2. Of those who take training, what does their product usage look like?

As Christy H puts it,

“My favorite report is simply taking a course engagement, and then looking at the learning objectives for that, and then tying that to product behavior. So very specifically,…, what behaviors am I trying to drive with a particular course? And did I influence that in the product? When you can see that somebody hasn’t engaged with a feature or certain part of your product. They take the course … and then are now engaging … with that. And that is, I think, at the end of the day really going to tell you is your content successful at driving value.”

So in types of examples of training evaluations such as this you need to determine what your boss deems essential to the business:

• Reducing support costs?
• Growth?
• Reduced churn?
• Customer Satisfaction?

Then based on what’s important at your company, you can establish the story that your training is trying to tell. For instance,

Our training offering will _________________________

So that ________________

Leading to ___________________________________

We will know we are successful when _______________________________

Then design training evaluation metrics to measure the level of success.

Enterprise Customer Training Evaluations

What does your customer care about when investing in your training? Usually, they need to know the time for their employees to reach competency with the software has decreased. And that they are receiving greater value because of the training. Your training evaluations need to measure these. This helps them know that they made a good purchase and that your training has a positive bottom-line impact.

Measuring this in your training evaluations can often be the most challenging. But if you can, measure account level “penetration” and active user data. At the “account level,” are enough of the customer’s employees taking advantage of the training? And, as we discussed earlier, you might use a survey as part of your evaluation of training to measure net promoter score.

For your enterprise customer (and for your boss) the training, improves adoption, product usage, and value-added. As well it improves customer retention and benefits the bottom line. So extra benefits of training evaluations are that they can show success both to your customers and your boss!

Additional Training Evaluation Tips And Resources

Although we’ve covered a lot so far in this article on examples of training evaluations, we’re not entirely done yet. Below are five extra tips and resources to help you create training evaluations.

1. Pre-Training Assessment
   Before a learner starts a course, have them complete a pre-training assessment. This is invaluable when it comes to your training evaluations. A pre-training assessment provides a baseline of what the learner knows and doesn’t know. Using this, you can compare what the participant knows before and after the training.
2. Identify Performance Indicators
   One of the first things to do when creating training evaluations is to identify your performance indicators. Select specific data points that are most important to your organization. For example, product sales increases or improved employee engagement. These are the primary indicators you’ll use to determine how effective your training is.
3. Use Proxies
   Often in building training, we include a lot of content and then assess performance after the training. But it’s important that you include evaluative elements during the training. Or formative training evaluations. Having opportunities to practice and get feedback are important evaluative tactics. For example, a scenario during training is a proxy that you can use to determine “can they do it during the training?” If they cannot do the task during the training, they are not likely to be able to do it after the training. And your training evaluations will show a poor transfer of learning. Patti Shanks’s book “Write Better Multiple-Choice Questions To Assess Learning” is a deceptively simple title for a very nuanced book about the power of questions to shape and measure our training. A must-read to help you enhance your training evaluations.
4. Data-Driven Customer Education Training Evaluation
   Do you find data-driven training evaluations, particularly in customer education, intimidating? If so, you’re not alone, but you don’t need to be a data scientist to do this well. Here’s a great resource outlining how you might do it well, but simply, by Bill Cushard.
5. Do Some Research on Learning Transfer Learning Transfer
   A large part of successful training evaluations is figuring out if people can “do something different” after the training. If you’ve designed your training for optimal transfer, then the assessment should give you the confirmation you are on the right track. Two resources to start your research are Why is learning transfer so hard? by Connie Malamed, and When and Where Do We Apply What We Learn? by Susan Barnett and Stephen Ceci from Cornel University. The graphic opposite is a sample from the latter resource.

FINAL THOUGHTS ON TRAINING EVALUATIONS

Training evaluations are a simple concept, but to make them useful requires thoughtful planning. And as we’ve shown, there’s no one size fits all approach. The type of evaluation you use will depend on the circumstances. And by providing examples of training evaluations and the concepts behind them, you’re now better able to measure the impact of your training. And able to tap into of the benefits of training evaluations.

 

Spark + Co has worked with over 70 clients since 2007, providing custom online courses. We help businesses, non-profits, and government agencies to achieve their training goals.  Need to create custom online training?  Perfect, that’s what we do, and we’re ready to help. Book an initial consultation now, send us an email, or find out more from our website.