6 Questions That Do The Trick
Between that picture you uploaded yesterday, and the new comment you posted in your office discussion board, 2.5 quintillion bytes of data is created every day; tracked, reported and analyzed in one form or another. This data powers Amazon to recommend products to you and banks to check if you are eligible to have your loan approved. The same data mining principle can be applied to your workplace training programs—in the form of learning analytics.
However, there is a 3-fold problem that needs to be addressed when it comes to implementing analytics in L&D.
- First, most businesses have not established a proper strategy, resources, or instruments for proper training evaluation.
- Second, many businesses do have a system or tools to mine data but do not know how to go about utilizing them.
- The third and most common obstacle is that most stakeholders have not moved their training evaluation beyond course completion and qualifying (passing) percentages.
To define, design and enable learning analytics, it is essential to have a clear evaluation strategy in place. Here are answers to 6 questions you need to ask, to create an effective approach.
1. Who Are My Primary Stakeholders?
More often than not, you will be met with a lot of friction while attempting to convince your stakeholders to implement proper training evaluation. This makes it very important to know and recognize who your stakeholders are, their business goals, values, and priorities.
Only then, you can have the conversation to make them aware of the benefits of learning analytics. Mutual collaboration among all stakeholders is the key to establishing a strategy that will positively impact your business ROI.
2. What Information Would Benefit My Stakeholders?
After identifying all your stakeholders and recognizing their individual priorities, the next step is bringing them on to the same page about the:
- Evaluation strategy
- Training goals
- Performance metrics
Asking the right questions is the only way to get the information that would benefit your stakeholders in reaching their set performance goals.
The first question you need to ask your stakeholders before any conversation about strategy is "what is the goal of the program?" There is a chance you'll get the answer that they want employees or teams to do better. While 'doing better' is very appropriate, it needs to be quantified.
To measure the performance impact, you need to know the right performance metrics. So, the next question you need to ask is, "what will happen if they perform better?"
3. Where Will I Get My Information From?
Once you have a training program in place, you will need to align your evaluation instruments with the performance goal to get the information you need. Learners and training supervisors who are directly involved in the process will likely serve as your primary source of information.
Apart from the people involved, you can use the LMS for finding information such as the time taken by learners to finish courses, and which courses had high completion and qualifying rates. You can also mine the data from survey software apart from L&D to see what real impact training has made in meeting the business goals.
| Performance Goal | Sources/Methods |
| Evaluating learners' response to training | Feedback documents, reaction sheets, ratings |
| Evaluating knowledge difference pre- and post-training | Assessments (completion, qualifying rate) |
| Evaluating skill improvement | Focused groups, mentoring, reports from survey software, feedback from supervisors and peers |
| Evaluating the overall impact on the business goal | Reviewing projects and finances of the organization through survey software, before and after training |
4. How Will I Collect The Data?
When it comes to training evaluation, Learning Management Systems are the ultimate tool for data mining and collection and reporting. LMS come with evaluation features such as assessments and feedback forms. You can generate different reports based on learners' roles and experience, departments, and geographical locations. You can also embed a learning analytic engine in case your LMS doesn't have these analytical features.
With the evolution of xAPI, the Learning Record Store (LRS) can track and record learning experiences beyond completion and qualifying percentages. xAPI-enabled LMSs can track learning experiences even when the learner is offline. It can track details such as web searches and activities on social platforms/discussion boards.
You can also employ data visualization tools such as the GapMinder engine to integrate reports from other survey software to generate reports on how exactly learners' improvement in performance contributes to the overall increase in business.
5. How Will I Go About Analyzing The Report?
Unless you have an AI-enabled fully adaptive engine in place to analyze the results of your LMS reports and create learning paths based on it, you need experts to do the job. Analyzing reports will involve:
- Looking at the details of the learners' performance.
- Evaluating if there are any gaps between the desired and current performance.
- Deciding how to go about bridging the gap.
Report analysis will also involve finding the impact of training on business ROI.
6. How Will I Report My Evaluation Result?
In the end, all these findings, analyses, and reports need to circle back to the stakeholders. You will need to provide them with the evaluation report(s) in the form of charts, graphics, and executive summaries, depending on whom you're reporting to.
You will also need to keep in mind that reports can be quantitative and qualitative to address your stakeholder's questions and inform their decisions. You will also need to set a timeline for regular interim meetings before the final report.
So, if you are unsure of the evaluation metrics for learning analytics, these guiding questions will help you determine how you can decide on the questions, sources of information, instruments to use, and more. Download the eBook Leveraging Learning Analytics To Maximize Training Effectiveness - Practical Insights And Ideas for an incredible in-depth view of the topic. And for more insights on this, watch the webinar and discover how to quickly plan for learning analytics to ensure utilization.
