Addressing The Elephant In The Room With Effective Learning Evaluation: Aliana Szlachta

There’s never been a better moment to address L&D’s persistent elephant in the room: evaluation. Advances in technology mean measurement and evaluation is now more accessible than ever. That’s why Alaina Szlachta, doctor in education and human behavior, thinks it’s time we woke up to all the data around us. Hear why she says this starts with asking the right questions. Explore how you can make data-informed decisions, and dive in and discover some ways Artificial Intelligence (AI) can help.

Top tips for effective learning measurement and evaluation

Don’t have time to listen now? Here are some top tips from Alaina:

• Address the elephant in the room: L&D needs to be accountable for delivering business impact. And this can’t happen without effective methods for measurement and evaluation.
• Wake up to the data around you: We’re surrounded by data. Get comfortable finding and working with these numbers. Develop data literacy through practice.
• Ask the right questions at the right times: Don’t jump straight into the data. Start by asking questions to identify the best metrics to measure your impact.
• Get experimenting with AI data analysis: Use tech to save time. AI tools can sift through the data, so you can focus on the analysis.

Interested in even more tips? Here are 20 learning measurement ideas to show impact and improve performance. 

1. Address the elephant in the room

For some departments – like Product, Sales and Marketing – measurement and evaluation is part of the fiber of their work. But historically, L&D professionals haven’t had this accountability. They haven’t consistently been asked: What is the outcome? Have you met the mark? What are the numbers behind what you’ve done?

For Alaina, this is the biggest challenge. Not only are L&D unused to the practicalities of measuring impact or calculating Return on Investment (ROI). They’re not designing learning experiences with an intended outcome in mind.

”The elephant in the room is that often learning programs aren’t designed in ways that actually facilitate behavior change…There’s a lot that goes into making sure that we’re designing and supporting people for making the changes that we wish to see. And if we don’t design for the kind of change that we wish, or even know what’s the change that we want to see, then we have nothing to measure.”

2. Wake up to the data around you

With technology making measurement more accessible than ever before, it’s time for L&D to catch up. But without the fundamentals of data literacy, this can be intimidating.

Alaina explains that you don’t need a doctorate to get data literate. It’s about being comfortable working with numbers. And you can only do this through practice. Whether at work or in your personal life, look for the available data. Practice using it to make more informed decisions.

”Wake yourself up to the data that’s around you. We’re swimming in it…And it’s hard for us to [see it and] name it, to call it out and to talk about it. Today and going into the future, we’re swimming in data. Think about your Apple Watch and how much data they’re getting just from one small device. Now multiply that by our cell phones, Zoom, our cars… data literally lives all around us. We just have to open our eyes and then – more importantly – do something with it.”

3. Ask the right questions at the right times

Official standards have been released for L&D metrics and measurement. These are great for getting everyone speaking a common language. But before you can use these standards, you need to get to grips with the fundamentals.

Don’t overcomplicate things by jumping straight into the data. For Alaina, effective measurement always starts with asking the right questions. Before you measure anything, you need to know what you’re doing, and why. If an employee is engaging in a learning experience, what do you want to see when it’s complete? Is it changing performance, creating a feeling or providing knowledge? If you’re aiming for a simple outcome, you won’t need a complex measurement strategy.

”If we don’t ask the question: ‘What are we doing and why?’ and get really clear on what’s the point of this program in the first place, then how do we know that we’ve been successful? That’s when data can come into play. But it can be as simple as: Do people know that they can go to the intranet or our LMS? Or that they can go to these three people to get their questions answered? Then our people are using those resources: the people, the technology, the systems and services. That’s a really easy thing to track and monitor.”

4. Get experimenting with AI data analysis

With so much data everywhere, getting started with evaluation can be daunting. So, it’s no wonder that people are looking to technology to help them find that magic metric.

AI presents lots of opportunities. Although it can’t tell you what you’re aiming to achieve, it can help you speed up your data analysis. For example, Alaina used a program to record a series of discussions. She knew it had an AI tool built in and wanted to find out what it could do, so she started experimenting. For each transcript, she asked the tool questions about the data. What are the main themes in each conversation? And, could the tool pull out quotes representing these main themes? The results were great and it saved Alaina lots of time.

”My first call to action, before I say anything else, is start experimenting with these tools… If you’re not already using technology to save time, and to help you sift through qualitative and quantitative data, there are tools that you can use to make that process much easier. That’s where we are today. And then getting comfortable with that will make it so much easier for you to add more complexity in terms of what generative AI can do for you.”

A quick recap

Want to make sure you’re designing learning experiences with data in mind? Alaina has four strategies for embracing learning measurement and evaluation.

• Start making evaluation a priority at the beginning of every learning project.
• Recognize the data that surrounds you and practice analyzing it.
• Don’t jump into the data without asking what you’re aiming to achieve.
• Explore how AI tools can help you speed up the process.

Want to find out more? Check out the full podcast.

About Alaina

While working in academic and corporate roles, Alaina has developed a passion for magnifying the results of L&D through good data. She’s the founder and host of Measurement Made Easy – A community of practice elevating data literacy.

You can find out more and get connected with Alaina on LinkedIn.

Join the conversation!

We’d love to hear your thoughts on our podcast, so feel free to get in touch at kirstie.greany@elucidat.com. As always, don’t forget to subscribe to Learning at Large in your favorite podcast app and leave us a 5-star rating if you enjoyed it. Thank you for joining us, and see you next time.