Learning Assessment

For March the LCBQ is:

How do you assess whether your informal learning, social learning, continuous learning, performance support initiatives have the desired impact or achieve the desired results?

Jay Cross responds:

You need to wait a while before taking the assessment. Smile-sheets and test scores prove nothing because they are administered before the forgetting curve sets in. The reason only 10%-15% of what is learned shows up on the job is that most of what you learn disappears rapidly unless it’s reinforced by reflection and practice. That’s why it’s a good idea to wait three to six months — to see what sticks.

In the Canadian military, it is called Validation, as opposed to Evaluation. The latter is to make sure your training program is using appropriate methods and resources. The former is to ensure that you’re meeting operational needs.

We used rolling validations in the Air Force. First, they were based on training that had clear Performance Objectives (PO’s). We would follow up with a quick survey to course graduates, their immediate supervisors and perhaps other senior operational staff. Six months after each course we sent out a survey. It listed all the PO’s on the course. Graduates were asked if 1) they had learned the skill on the course; 2) if they had to use that skill in the past six months and 3) if they felt competent with that skill [yes, it’s subjective]. Supervisors were asked if that graduate had performed the skill in the past six months and if so, whether the graduate was competent.

This was a very quick pulse-taking that could then be followed up with more in-depth interviews.

Could you do this with informal or social learning? Of course you could. You just need to describe the capability you wish to measure.

For example, after 6 months on the job, with access to micro-sharing, company blogs and the development of a personal knowledge management framework, you could ask similar questions.

Do you have to regularly access information and knowledge to do your job?

Have you learned to access information and knowledge faster than when you started 6 months ago?

Supervisors can be asked if people are effective at getting access to information in a timely fashion.

The Big Question is, what do you expect to achieve? In a complex environment, there is no linear relationship between cause and effect, so we know these measurements are only indicators. Used correctly, and administered as lightly as possible, they can help make informed decisions on priorities, support or resource allocations.