Streamlining Your E-Learning Production Process for Efficiency—Q&A
Diane and I had the pleasure of sharing some ways we have streamlined our production process in last week’s “Best of DevLearn” Webinar presented by The eLearning Guild, which you can view here. (Log in as at least a free Guild Associate to view the recording.)
During the session we talked about:
- The data types to capture and analyze about your production process
- How to map out your production process at both a macro and micro level
- How to create a knowledge base to help train team members and reduce effort
- How to create systems for effectively managing production
At the bottom of this post, you can find the main topics and some of the tools we shared.
There were some great questions posed that we were able to address in the webinar, but there were several we weren’t able to get to. Here are the questions…and our answers.
Q: How do you estimate a course that has several interactions in it?
A: We use some base assumptions about interactivity in a course. We assume that any course we will create will be focused on performance with application-based opportunities for students to try what they’ve learned. We will assume that many will be scenario-based, but a few may need to be more knowledge checks. We assume many of the interactions will use a standard question type, and that a handful will be fairly more involved (like a drag-and-drop activity). Our base assumption does not include games or branching simulations. So we know how long it takes to build a course with these assumptions, and if a particular course is going to need something special (such as a game, branching, etc.) then we adjust our estimates up accordingly.
Q: What is the average project load for a Project Manager?
A: That is really difficult to answer because it depends on how big the projects are, where the projects are in the process (start-up? storyboard? online draft? editing? waiting on client review?), how aggressive the timeline is, and even how much experience we have with the client. I have regular check-ins with the Project Managers to try to manage everyone’s load, because being “full” one week could change drastically the next. For example, a project could go on hold, a problem could arise, or we might get go-ahead on a large task that had been on hold for weeks. Unfortunately, managing load is a lot like nailing jelly to the wall, and we don’t always get the workload right. If I had to state a number, I’d say between 4 and 8 projects at any given time.
Q: What is the skillset for your team’s “Developer” and what are the skill sets for the “Traffic Manager?”
A: The Developer is the person who takes the storyboard and develops the course in the authoring tool. This person needs to be able to use the authoring tool, have a strong attention to detail, and ideally have strong graphic design skills.
The Traffic Manager is a flexible, skilled communicator who is good at moving the balls through the hoops to get the project complete. The perfect Traffic Manager has an extremely strong attention to detail and very strong editing/proofreading/quality assurance skills.
Q: What advice do you have for one person or smaller teams?
A: We mentioned in the webinar that we typically have six people on a project. Smaller teams don’t have that luxury, so you may have to wear all the hats yourself. Going through the micro-role process might be helpful anyway, because you can identify skills you may not have so you can (A) get them or (B) find someone to help with that. If I could only have one other person help me out, I would pick a course tester/editor. We get too close to our own work and may not see mistakes that would leap off the screen to someone else. There was a time when the company was just me and Diane. We’d pass the course back and forth for editing. The first contractor we added to the team was someone who could be another set of eyes for us.
Q: In the microprocesses spreadsheet, I don’t see any feedback from the end-user incorporated to pilot test for human factors issues. Is that something that you are considering for the future?
A: Any end-user testing usually falls into one of the client review cycles, which we did not document to the same degree. (If you are an internal practitioner, our process would have more steps than ours does.) Sometimes getting end-user feedback is done at the prototype phase, and sometimes again at online draft phase. I don’t think we’ve ever done a project where the end-user is involved at the storyboard phase. It is often challenging to get an end-user to slug through a storyboard document.
Q: Do you start early with a prototype?
A: We do start with a prototype with every new client. And, depending on the project, we may need to rework the prototype for each new project for a client. We’ve found that this allows them to envision what the course will look like, helps us validate design decisions, and lets us do early technical testing. This way we can get any feedback and make adjustments to make sure the final course is what the client expects.
Q: How important is Design Specification document to be approved before building Storyboard? Or is a Design Specifications document even needed in all projects?
A: We create a Design Specification Document for every project. We use it for two things: It helps make sure we’ve asked/answered all of the needed questions, and it is a great reference later when we need to come back to a decision and see why we are doing something the way we are. In our sign-off approval process, we usually have the prototype and the Design Document approved as a unit.
Q: How did you come up with that 15% figure [in the Review Language sample]?
A: I’m afraid it isn’t very scientific! We always try to strike a balance of letting our customers make changes throughout the whole process without letting the changes take over. If we had no guidelines, we could literally spend three times as long on edits as we did on initial development. There has to be a limit if we hope to stay within a schedule and a budget. At the same time, things get missed in a review. New stakeholders surface in the middle of a project. Some things seemed fine in the storyboards, but then don’t quite work well once it’s built. So we need to include some flexibility. 15% seems like that number. We have some clients who take the review process very seriously and can do everything they need to within that limit. In other cases, any limit would be difficult. At the end of the day, we’ll make whatever changes they need—but it may affect the schedule and the cost. So we have a discussion about the relative importance of the changes versus the cost versus the schedule and come to a joint conclusion.
Q: We frequently have one slide with an amount of content which should equal three slides. When and how do you distinguish this so that 15% max of the total number of slides changing is an accurate as possible?
A: We rarely enforce the 15% mathematically. Not all slides are created equal. Not all changes are created equal. We look at the overall list of changes, and if it is in the spirit of the 15%, then we generally don’t raise a flag. For example, we recently had a course where the client requested at least double the permitted number of changes. But all of them were simple text changes that didn’t involve programming, graphics, or audio—so we decided to make all of them. We want to remain flexible, because the course should be right. The 15% just protects us if things get out of control. We are actually thinking about changing it to a time and materials limit instead of a slide limit.
Q: In the sign off review process, is this done for each module/unit or all at once?
A: It really depends on how the information is divided. If we’ve created a storyboard for each module and each module has its own timeline in the project schedule, we’ll get the sign-off for each module. However, when we’ve created a several-module course where the storyboard is a single unit, we’ll generally request that sign-off for all modules at once.
Q: Is the review language shared with the client at the beginning or parsed out by phase?
A: The review language matches what was in our initial proposal, so none of it should be a surprise to the client. We send out only the language which applies to the current deliverable. However, that language includes a preview of what the next review will look like. For example, it is helpful to know that on a given draft you can make unlimited changes. But it is just as important to know that your next review is limited to 15% of the slides.
Q: Do you have clients impose content from the first time they come to you?
A: Clients generally come to us with what content they believe should be in the course; they are the subject-matter experts. However, we know it is our job to give students exactly the information they need to meet the desired performance and business results. From there, we may need to go back and forth about whether something really does need to be included, or we may find that something needs to be added. Diane gives some suggestions on how to negotiate out “nice-to-know” information in this post.
Q: Can you shoot emails from Google Docs [QA Spreadsheet] to people to let them know a task has been added?
A: We don’t do a lot of formal assignments of specific line items. In general, the Traffic Manager keeps track of what is happening through the online draft phase and lets whoever is next in the process know their time is needed to push the course to the next step. We do have an assignment column, but generally the whole list is assigned to one person, which is communicated by the Traffic Manager to the person doing the work. If there is a specific assignment, then the Traffic Manager would notify that person. For example, the Traffic Manager may have someone working on client edits, but there are three requests that need a decision to be made first. My initials would go in the Assignment column, and the Traffic Manager would let me know.
Q: What is behind the color-coding in the cells of the QA Spreadsheet?
A: During the Webinar, several people were in the document at the same time taking a look at it, and each cell someone was in was coded to a color that Google Docs gave them. During the Webinar, it had no significance. However, it does show you who is in the spreadsheet if you are in it at the same time and the person is logged in to Google. This is great when multiple members of your team are working at the same time on the spreadsheet because you can see who is looking at the spreadsheet, what they are changing, and you can even chat in the Google Document if you need to.
I do have to admit, though, that we do use color-coding sometimes. Especially at the end of a project when we have only a few outstanding items, we’ve been known to color-code the cells that need someone’s attention so they don’t get lost in the mix!
Q: What column in the QA spreadsheet does the client fill out?
A: The client fills out columns A through E during a first review. Then, during a second review where they are verifying that the work was done properly and approving the change, the client makes any comments in column H (Updates) and then changes the Status to either Client Approved (hopefully) or Reopened.
Q: Is your Knowledge Base one long running document?
A: Our Knowledge Base is a library of items (Word documents, .pdf documents, Screenrs, etc.) that currently lives in Google Sites in a topic outline form. Each topic is a separate documents so we can control the content and who has the ability to access that content. We are still looking at the best way to store and distribute this information, but currently Google Sites is what we are using.
Q: For the Knowledge Base, should each person have his/her own version of the document that makes sense to her and special things she likes?
A: The goal of the Knowledge Base is to have a single place someone can go to get needed information. I suppose that depending on what the document was, it might make sense to have a document with personal notes; however, we don’t currently have it laid out that way. If there is a different process or something someone would like to have more information on, Diane and I would like to know what that need is so we can improve the documentation (or even the process). We are open to new ideas, so a discussion about what is needed to make the document benefit everyone is our preference. However, in the future, we could add forms, etc. that might make sense to make available for individual versioning.
We do have two versions of a few documents that we are working on, such as our storyboard style guide. One version has been signed off on by the team and made available to all of our contractors. But then we have another version that we can use to suggest/debate possible changes. Then periodically, we go through those suggestions, decide how to handle them, and then re-issue a new version.
Q: Where would you store the Startup or the Summary of anything noteworthy?
A: Most of the key details on a project go into the Design Document. (Because of that, I don’t know that the title Design Document is completely accurate anymore.) If there is anything noteworthy about the project, that’s where we put it, making that document the single source of knowledge. For example, there might be a little workaround we need to do to get the course to work in the LMS—that workaround would be added to the Design Document. It’s a living, breathing document that changes throughout the life of the project.
Q: Can you provide an example of one of your storyboards?
A: Absolutely! You can get our storyboard template in one of my posts, The Artisan E-Learning Storyboard. It gives our template, as well as our goals in creating it.
Q: Do you have any recommendation of PM software?
A: The best thing we can do is NOT recommend a specific software platform, as each team’s needs are different. Instead, you can view a webinar on the process we used to select ours. Perhaps you can use a similar decision-making process to select the right tool for you. When you click that link, scroll down to the bottom to find it in the archived section.
Q: What is the cost of project management as a percentage of the project cost?
A: Depending on the type of project, we plan for about 10%.
Main Points and Resource Links
Data Analysis
Length Analysis
Micro-Roles
Micro-Processes
Review Language
QA Tracking
Online QA spreadsheet (Google doc)
File Structures
Knowledge Base
PM Software
Previous Post
Next Post
Great tips! With the growing demand for impactful and innovate e-learning solutions, L&D managers are often not equipped to drive an e-learning initiative completely in-house. So choosing an e-learning partner is then the logical solution. But it is important make sure that you can utilize the strengths of the development team to the fullest – here are a few ways to ensure you extract the best out of them – http://www.gc-solutions.net/blog/deriving-the-best-out-of-e-learning-outsourcing-creating-impactful-courseware/
Interesting article! Successful e-learning courses have to make sure that they are not content centric but learner centric. This is to say that e-learning should be created by keeping the learner in mind and aligning the content design accordingly to suit their learning preferences. Our article on ‘Taking the boring out of e-learning’ has some interesting pointers that maybe useful in the context. http://www.gc-solutions.net/blog/effective-e-learning-how-to-take-the-boring-out-of-e-learning/