Designing Next Level Learning – Reviews And Learner Evaluation
We have two Instructional Designers for a large system, so we don’t spend as much time in analysis and evaluation as we should. We do have a modified analysis process that works for us, however. When we receive a request for a new learning course, we meet with our SMEs to determine need, purpose, objectives, audience, timelines, and priority. We try to work out who our primary SME reviewer will be, because it is a lot easier to deal with one reviewer than 10. This way, we can send everything to a single person who can then share it with whomever they see fit. We request that our primary SME collate all comments so we aren’t hit with multiple, often conflicting, reviews.
During the analysis meeting, we collect draft materials and work with requesters to determine the best solution for their needs. We then formulate the request into a basic blueprint and send it back for their review, thoughts, and additions before sending to our education governance committee for approval.
Design Review
For our design reviews, we usually write storyboards or audio or video narration scripts based on our analysis and the draft materials. Microsoft Word is our design application of choice for these documents. We begin with a peer review, because peers are colleagues who can provide insight into the content and design. In fact, a good peer reviewer is your best friend, so be sure to find and cultivate these special people. Your best choice is a skilled Instructional Designer. If you don’t have that luxury, seek out fellow educators with a sharp, critical eye, and a flair for language.
You want feedback, so give your reviewer the following suggestions before handing over your documents:
- If you don’t like how a sentence is worded, rewrite it to what makes sense to you.
- Feel free to move whole sentences, paragraphs, topics, or sections.
- Use general comments sparingly for suggestions such as overall observations, questions, or concerns.
- Do not worry about hurting my feelings; we both want what is best for the learner.
The ideal reviewer will go through your design documents both as an adult learner and as an educator while thinking critically about the following questions:
- Are the objectives clear?
- Does the content cover the objectives?
- Is everything clear and concise?
- Is the content too wordy?
- Is it in active voice?
- Are there any spelling or grammatical oversights?
- Can you answer the quiz or review questions based on the content?
- What changes might bring more value to the learner?
If using Microsoft Word, ask your reviewer to turn on track changes to make edits so you can easily find their modifications. When reviewing the document, use the simple markup viewing option to keep the document from becoming too confusing. (You can toggle between that and “all markup” to see big changes.)
Once the peer reviewer is done, incorporate the feedback. Don’t feel you have to accept everything, however. No one knows the project at this point more than you do.
The next step is to send your design document for SME review. Use the same process for SMEs, but ask that they focus solely on the content. Not all SMEs are created equal. You will have differing layers of involvement and expertise with your SMEs. Use timelines as needed. Three days is standard turnaround time for a reviewer. Sometimes, though, you won’t hear back in a timely manner. If deadlines are a factor, be clear that you cannot guarantee any previously agreed upon schedules if you don’t receive needed feedback on time.
Development And Implementation Reviews
Once you have your SME’s edits, you can develop your eLearning course using whatever software and techniques work for you. When the course development is complete, send it out for peer review and try to use the same person who reviewed the design documents. Prior knowledge is invaluable. This time, you want your peer reviewer to focus on the following questions:
- Does the sequencing make sense?
- Is the navigation intuitive?
- Is it aesthetically pleasing?
- Is it professionally laid out?
- Do images enhance the objectives?
- Do the audio and video play correctly?
- What changes would bring more value to the learner?
No matter how detailed your written design document is, most people can’t envision a course until they can see it in action. We upload our published course files to an internal web server, then send a link to reviewers so they can click and view the course. If you have an editable version of the course to send with your review request, use it. Adobe Captivate can generate Word versions of the content, and we’ve found the Handout option offers the most user-friendly format. Trivantis Lectora will also generate a Word version, but it is a little harder for reviewers to navigate. Again, ask your reviewer to turn on track changes and make whatever edits they want. Different authoring tools also have nifty reviewing features. If you have one available, try it out and see if it works well for you.
Once the peer reviewer is done, incorporate the feedback and send the course on to the SME review. Certain techniques and processes will work with some SMEs more than others. We’ve skipped the storyboard processes for some courses. Feel free to speed processes up as needed depending on timeline and priorities. We’ve used more iterative processes at times, with more back-and-forth conversations and to give the SMEs a push when needed. With one high-priority course, we weren’t getting answers to our questions, so we developed a very draft version of the course with highlighted questions right inside of it. This helped the SME visually understand what we needed to move forward.
When the SME has provided final feedback and you’ve weaved in any final tweaks, release the course to your Learning Management System. At this point, use a tester to review the eLearning course, check all links and interactions, and check for any technical issues.
Learner Evaluations
We add a link to a SurveyMonkey evaluation on the final slide of our eLearning courses and look at comments frequently. Our learners let us know about anything that managed to make it through all previous reviews. No matter how many people are involved in the process, slips happen—a word out of place, a confusing question, or a link that doesn’t work. If you follow a consistent review process using the ADDIE model, you will find that these slips are small and rare.
Checklist For Designing Learning
- If you don’t like how a sentence is worded, rewrite it to what makes sense to you.
- Feel free to move whole sentences, paragraphs, topics, or sections.
- Use general comments sparingly for suggestions such as overall observations, questions, or concerns.
- Do not worry about hurting my feelings; we both want what is best for the learner.
- Are the objectives clear?
- Does the content cover the objectives?
- Is everything clear and concise?
- Is the content too wordy?
- Is it in active voice?
- Are there any spelling or grammatical oversights?
- Can you answer the quiz or review questions based on the content?
- What changes might bring more value to the learner?
- Does the sequencing make sense?
- Is the navigation intuitive?
- Is it aesthetically pleasing?
- Is it professionally laid out?
- Do images enhance the objectives?
- Do the audio and video play correctly?
- What changes would bring more value to the learner?
If you want to learn more about taking your learning to the next level, download the eBook Trainer's Guide To Designing Next Level Learning.
Originally published at www.td.org.
