Few things in the L&D profession attract as much disdain as the Level 1 evaluation survey.
Known colloquially as the “happy sheet”, the humble questionnaire is typically considered too onerous, too long, too shallow, and it gives whiners a platform to complain about the sandwiches or the air conditioning.
And yet a long time ago, someone told me something that has stuck with me ever since: you don’t want to not do it.
Why? Well first of all, it’s easy. Given the availability of online forms these days, rustling one up is like falling of a log, and the back-end compilation of the results is rather impressive.
And the survey shouldn’t be too long; that’s the fault of its design, not of the concept. In fact, I advocate only two questions…
- The net promoter: How likely are you to recommend this learning experience to a colleague?
- The open-ended: How might we improve this learning experience?
The net promoter score (NPS) has itself been criticised, but I like it because it’s a simple number that’s easy to track and report. By no means an in-depth analysis, it’s a summary indicator to keep an eye on. The standard to which it adheres is quite high – a promoter is a 9 or a 10 – and a negative score is a sure-fire sign you’re not hitting the mark.
The open-ended question shores up the NPS by enabling the respondent to explain their rating. If there’s a problem, this is where it will appear.
Indeed, the sandwiches and the air conditioning are favourite punching bags, but we L&D pro’s do bang on a lot about the learning environment. You don’t get much more environmental than shelter and sustenance, so why not turn the dial and mix up the menu?
I also advocate the following…
- Resist asking the participants to hand their feedback to you in person; or worse, share it aloud. This will probably result in platitudes which – unless that’s what you’re really after – are effectively useless.
- Allow the participants to submit their feedback anonymously. Again, you want them to be honest. If they still lie, your organisational culture has way bigger issues to address!
- Allow the provision of feedback to be voluntary. You need actionable insights, so invite only the feedback that someone feels strongly enough is worthwhile actioning.
- Invite feedback less frequently over time. We all suffer from survey fatigue, so after you’ve got a good gauge of what’s going on, keep your finger on the pulse with the occasional spot check.
If we look at Level 1 evaluation through this lens, we consider the feedback form less a “happy sheet” and more an “unhappy sheet”. It exposes hidden ailments that you can subsequently remedy.
It’s not the be-all-and-end-all of evaluation, nor is it meant to be. Rather, it’s the canary in the coal mine that alerts you to a risk before it gets any worse.
E-Learning Provocateur 
Hi Ryan,
I’m not really a fan of the NPS for learning solutions, for a couple of reasons. Firstly, it’s based on whether someone would recommend but that doesn’t mean that they actually do recommend it. Secondly, as a scorer, what’s the difference between an 8 and a 9? For NPS, the difference between the two is significant.
Having said that I don’t have a better solution if we don’t use NPS!
I definitely like the second, open-ended question as a way to gather specific areas for improvement.
Fair call, Matt. I guess for me it really boils down to the weight of the numbers towards the negative: a consistent 6 or less indicates something needs to be addressed.
Great article, Ryan. You point out a number of deficiencies within the traditional happy sheet process. I think an NPS adoption might work to at least give you a sense of acceptance/dissatisfaction with the learning. Everyone needs a starting point. Making things as simple as possible but not simpler might apply here. :)
Nicely put, Bill!
Hi Ryan
Nice article. Thoughts on NPS for learning are that it needs to be used for very specific purposes. It’s not an indication of effectiveness (though not much else on a happy sheet is either) but in some businesses where the learning is mandatory or there’s an incentive to do anything to get away from “work” you can get warped results. I think it’s more useful for optional and professional development type learning activities, where there could be actual benefit from peer recommendations.
For the record, I’ve used it in my surveys in the past for optional programs and it seems to have been beneficial.
Cheers
Sam
I think you make important points, Sam.
I would add that as learning in the workplace continues to informalise, we increasingly rely on the motivation of employees to drive their own development.
If they don’t like a learning experience, or they heed the negative recommendations of their peers, they’ll avoid it and hence its effectiveness in improving performance will be limited.
Two questions, I like the concept. Effective use of the concept of NPS. Loved the conpet of the sandwich and the airconditioning
Thanks
Kam
Cheers Kam :)