The Art of Incorporating Training Metrics

As a kid, do you remember an adult in your life making shapes with their hand, placing them behind a light, and casting a surprising shadow on the wall? I remember a barking dog and a simple bird with gentle flapping wings. There are artists that specialize in this kind of shadow art, such as Kumi Yamashita who carefully places everyday objects in front of a single light source to create striking silhouettes. (Please explore some of her amazing work at this link.)

For artists like Kumi Yamashita, the final product is a clear silhouette of some kind, like a profile of a face, person, or animal. Her building blocks are small pieces of common household objects like wood blocks, magnets, and other seemingly ordinary objects. In learning and development, the end result should be a clearly defined and measurable performance result with clear training metrics, such as reducing turnover, increasing customer satisfaction, increasing sales, or something similar. Our building blocks are online and classroom training, rich media, immersive simulations, on-the-job performance support, and whatever other modality the learners need.

And it all starts with knowing exactly what you’re trying to achieve—what that final picture will depict. To do this here at AllenComm, we use our Performance Mapping framework, which guides us to meticulously establish and define the training metrics, learning measurement, and training evaluation strategies we are going to use before we begin designing any program.

Establishing the goals and training metrics are like the light source. The design of the curriculum are the things that cast the final shadows and achieve the effect we want on the end audience. To begin mapping all of the variables and how they fit together, we establish the following, in this order:

• Ultimate Business Impact: what specific KPI is going to change and how
• Behavioral Proficiency: what specific behaviors must we see learners do differently
• Learning Objectives: based on the behaviors we are changing, which learning objectives must we target
• Modalities and Sequence: finally, what specific learning modalities and in what sequence are best for learners

This process incorporates Kirkpatrick’s Four Levels of Learning Measurement, which has been the gold standard in the industry for many years. However, in our experience most L&D professionals only measure to the lower levels. In fact, in a recent webinar hosted by AllenComm, the vast majority of L&D practitioners report only measuring level 1, and maybe level 2. There are many reasons for this: limited technology and budget are two of the biggest. However, we believe designing your solution by establishing training metrics first regardless of the size and scale of the program makes achieving level 3 and 4 more obtainable.

A Case Study with Training Metrics

For example, we recently supported Delta Air Lines as they sought a re-design of their customer service center onboarding program. Together, we established the measurable goals that were desired of the business and completed a data landscape discovery process to determine how the solution we designed would ultimately be tracked and measured. We made sure to document both what performance results we wanted, and what data source would be used to measure the training effectiveness. Spoiler alert: it wasn’t easy and the data came from many places. Here are just a few examples of what this Measurement Strategy Plan looked like:

Measurement Strategy Plan…
Desired Measured Change	Metric of Measurement and Method of Measurement
Increase proficiency and productivity	Call Handling Time; measured through phone system
Increase in customer satisfaction	Specialist Promoter Score; measured through post-call customer performance survey
Increase speed to resolution	First-contact resolution; measured through phone system

With this plan in hand, we then carefully curated existing content, and built new content to fill the gaps when needed. With our clearly defined goals we had a light shined in the specific direction we wanted to go, put all of the pieces into place, and launched the program with resounded success, including (but not limited to):

• 93% increase in Employee Net Promoter Score
• 207% increase in training capacity
• 15% increase in productivity
• 3% faster proficiency 

We’ve seen similar results with many of our other clients, including those with curriculums in manufacturing, sales, and compliance.

We understand that training evaluation is difficult and that it depends greatly on technology. When there isn’t an easy, all-in-one tool to help manage the many variables of training, it’s easy to become disheartened. However, the end result—that big picture—is worth it. Using the steps we’ve outlined, you are able to carefully map out the curriculum and determine what and how can be measured. This detailed and cohesive approach enables more insights into training metrics and how to achieve (and measure) the ultimate impact your program has on the business.