Posted by Let’s face it – RFI/RFP documents sent over to learning system vendors are massive tomes that confuse, challenge and even baffle some vendors. They can be 25 pages. 50 pages, even 100 pages or more. They go way beyond what is needed OR not enough needed. Some companies have every department give their two cents and then you get bloated materials – because no one checks to see if there are redundancies, and oh, yeah, none of those departments oversee L&D or Training or whatever department – let’s say Marketing or HR. Thus, when folks start to gripe about how much they hate or dislike or lack of usage, enjoy – because they are coming to you; and not those departments who gave their two cents.
Then you have RFP/RFIs that lack substance. The Use Case is ambiguous or a convoluted mess that only the person creating it can figure out. Some have terminology that isn’t clear, jargon that means nothing to the vendor, and the assumption that whoever is completing the RFP on the vendor side even knows the industry (L&D or Training, for example). Maybe it is your salesperson. Maybe it is the product. Maybe it is a combo (which is what you want), or maybe it is some intern who just got shoved the 500-page questions for them to respond to. From a vendor side, a good way to go about it is the infamous cut-and-paste angle – because they see the questions so often – they keep a list, and bam – cut-and-paste. You’re welcome!
The fact of the matter is nobody loves creating RFP/RFIs. They are time-consuming. They open up a lot of unknowns – because if you do not know, it isn’t in the RFP. Too many hands can get involved, which slogs the process, and since they don’t know what is “in” or “out” or what to seek out, they regurgitate the usual inquiries. Up next, IT is getting involved (Boo! Hiss!). They are all over the map; some of them think this is an ERP or HCM, and thus can’t understand why they can’t get access to certain things OR why their requests for specific information aren’t clear cut. Oh, here comes Compliance. Yeah, they are focused on essentials, again, forgetting this is a learning system and is for learning. Hold on! In the end, you need some Tylenol and a stiff drink. Then you send it. Wait for the responses, which usually include the vendor following up with questions that could have been resolved if you sent the correct document with the right approach (although I swear some call anyway even though it is clear what you are seeking, and yet they still can’t figure it out).
Fear not, though. My newest learning system template is now available, and it packs a wallop. Okay, one version does. The other is more streamlined. Thus, two options – Go full throttle (the one I always use – for my award considerations, or assess a vendor – analysis-wise) or streamline! It’s like selecting between high caffeine or decaf. I recommend you use this version – whichever you choose – because even in the Robust I removed some areas, cleaned up areas, and added new stuff, sections, and features.
Both contain key essentials, but streamlining most functionality out – well, it is the minimums. I added a couple of new areas, plus additional information to provide when you send the RFP. I always recommend adding anything else you want OR removing items you don’t want – for example – remove e-commerce, but keep the rest. I never recommend removing something you never heard of. There is a reason to use the template – it includes the stuff you may never have heard of.
I added a “Craig’s Comments” column, too – which you should remove before you send the template – so make a copy! These comments are for you, not the vendor.
I’d keep some parts (below in the breakdown of the tabs – I state which ones to remove) of Craig’s Resources because there will be vendors with zero clue about a few areas. Thus, the premise they will surf the net to find this information is about as strong as me going out into the desert land surrounding my home, and playing putt-putt with the rocks. And lizards as my co-players.
The LINKS
The Learning System Template – Robust Edition
The Learning System Template – Streamline Edition
The Details and Features – That are NEW, plus a couple of areas to just say “yes”
I’m going to start at the beginning of the template – remember this is RFP/RFI here. So this is the second tab. The first is just the logo of the template – so anybody receiving it knows it is a RFP template. You can delete this tab. You wont’ hurt my feelings.
Tab Two – Your Information
Break it Down
Most are pretty self-explanatory – but the header says “Customer Completes” – that means you, not the vendor.
NEW Stuff
- What department(s) will oversee the learning system? Maybe it is just L&D or Training or HR. But it could be marketing for external (not a good idea, but hey, it happens), L&D internal. OR maybe it is sales, the call centers, or IT (hopefully just for themselves – I kid, I kid). By providing this information, the vendor knows – no ambiguity here.
- Will the Learning System be available to everyone OR rolled out to certain departments/divisions etc.? – I like this question because it is pretty common to do say a rollout for the call centers, and then it goes to another department – maybe external. OR you are going customer training – but what to limit the scope to specific product lines. I say this is common because you will see vendors list these big names on their marketing “who are our clients.” Failing to note it is in one or a few areas and not the entire company. Which is why I always ask.
- How many learners do you anticipate accessing the learning system in the first year? Again, if you have 5,000 learners but only believe 1,000 will access (based on your knowledge or some other data point), then note the 1,000 but add some cushion, say 250 or 500. If you are rolling this out, to specific departments, use those numbers of employees. For customer training – it can be all over the map, but always go conservative to start (if you are truly starting). If you know – mention it. Vendors base their pricing on the learner numbers. Plus, they always want the numbers. BTW – never give them your budget. NEVER, EVER, EVER.
Tab three – Use Case
Write this out in a document and include it. There are two ways to do the use case – the most common way (oh, and be very clear, no ambiguity) Or the newest way – which I wrote about – HERE.
I think the new way is the way to do it – from now on. But, hey – you choose!
Tab four – Terminology
This is for you and the vendor. Trust me; a lot are not familiar with certain terms.
Tab five – Craig’s Resources
As mentioned in the resources area:
“If you plan to send this template to the vendor as the RFP, I strongly recommend removing the items around the salesperson question, demo, and pricing. This is the inside info for you. Many people who end up regretting the system they bought, tend to make the mistake (without realizing it) when they view a demo, support questions (which are listed in this template), and the salesperson questions.”
Tab Six – The Functionality
The first behemoth area. I have had folks ask me whether a vendor will complete this entire section. The answer is yes – if they want to try to land your business. When you send the template out – regardless of the edition; always tell them to complete the column “vendor details” for each feature they say yes to. You want to know more here. When vendors leave it blank – I think lazy.
- Features – Here is where the features for each “area” is listed.
- Your Response – This is where you get to select of of the following – Yes, No, Unsure – This is achieved by clicking the cell next to each feature
- Your Details – If you as the “client,” which means prospects – which is YOU – here you can add comments – most you won’t, but for, say, integrations, you likely will if it is something unusual or multiple items. You can also leave it all blank.
- Vendor Responses – This is where the vendor selects one of the following choices (by clicking on the cell). Yes – it means they have it TODAY. No – means no. Roadmap23 – this means they plan to roll it out by the end of 2023. Roadmap24- means they are projecting to rollout it out by the end of 2024. Always be aware that some vendors will say yes to Roadmap23, then it goes into 2024 or later before it happens. This is why I always ask – what percentage of their roadmap did they deliver in 2022?
- Vendor Comments/Details – Seems pretty obvious, and yet, over and over again, I see vendors ignore it. Seriously. If you are a vendor, complete as much as you can. Remember, somebody, is deciding on scheduling a demo (if they went RFP first) or deciding if you make it to the next round OR are the vendor to go with – based on those comments.
- Craig’s Comments – I left a variety of comments or insights if you will, on various features in the template. These are some key takeaways to be aware of.
Break it Down
This is where I focus on the new areas added to the template – or a few new items in a section OR in the case of the Learning Environment – say yes to 98% of it because you want it. All. – Uh, my recommendation. The 2%, though – well, some vendors have it, some don’t – so those are the wildcards. And thus, you can say nay if you do not want it. Remember, when you are buying a learning system, do not think now for what you need – think three years down the road. Thus, the vendor may not have those new items today, but that is what that roadmap is for.
The Wild Cards
These are the ones where you can go yes or no, and recognize that not every vendor offers it. If they should – I will note it below.
- Catalog Search by skill or interest – If they offer, it is by skill. Not a dealbreaker IMO, just nice to have. This is the catalog, BTW, not the search in general.
- Catalog Search by category/topic – A vendor should have this for their catalog – especially if they offer 3rd party content. That said, expect plenty who don’t.
- Search Bar – Search by content, topics, skills, whatever – by typing in the entire word. If a vendor has a search function in their system – it is usually very visible – this is the standard way to go. i.e. they can do this. If they lack a search option, that should be a minus.
- Search Bar by metadata – where you start with a letter, then another, and bam – a bunch of textual items pop up. I love this feature approach, but it’s not universal.
Content Creator – NEW TERM in the INDUSTRY – WARNING! WARNING!
The days of a vendor calling their content whatever tool, “built-in authoring tool,” or “authoring tool” are being phased out. The new term is “content creator.” The angle – Learner can create content, which is somewhat misleading – because from that perspective, I slide that under content curation (for now), because the Learner can only upload their video, materials, and recordings from mobile, add links and that approach. What they are not doing is getting access to the content creator tool that the admin or whoever is in charge of creating content. Now some vendors allow SMEs to gain access to create content, but again, that is on the back end OR limited access on the front end. I would never let an SME create content. Bad idea! Anyway, I recently saw a vendor, where the learner actually does see the content creator tool on the learner side and thus can create courses/other types of content and publish it to all learners or learners in a specific community, group, and, yes, cohort. That, to me, is the big win. Therefore, today, the Learner can see the content creator tool – the same the admin sees is very rare – I think that will change by the end of 2024.
Generative AI is gaining interest among vendors as it relates to creating content (i.e., courses, etc.). You type in what you want, “leadership,” or “Create a course for blue-collar workers on XAT.” The system then outputs the course itself. Some include a TOC; others break out by chapters. The time to do so can be short or several minutes. Text-based is really how it works here. One I saw, enabled you to add video and had this “Virtual Avatar” and a background, that could be added to the course. I was underwhelmed. Now, do I think fully text-based courses/content is effective learning? NO. You need engagement and interaction here. That said, I see a lot of static (the term for text-based) courses – which add an image or a question or drag and drop and call it, engaging. Sorry, it’s boring. Anyway, for those who do that, or just push out text content and call it a course, this Gen-AI rapid build of courses/content, based on what is noted above, will be amazing.
The content creator tools/or creation tool uses a What You See is What You Get angle – so you want that. I didn’t include the ability to upload a PowerPoint and convert it to a course (see ya Studio), but some can do that. Ditto on the lesson’s angle, adding learning objects, etc. The standard fare here.
If you have an ID background, you will be underwhelmed here. I did see one content creator tool, where you can add a SCORM object – i.e., a SCORM course to it. Still, the content creator tools are not designed for folks with high-tier ID skill sets. Rather, it is for anyone, who doesn’t have those skill sets or needs to publish content quickly.
AI – The Double Angle
AI – Machine Learning, AI- Generative AI
This is where I think a lot of folks are going to get confused, because how vendors are pitching AI. You are really forced to ask them what type of AI. I dealt with this the other day with a vendor. They kept saying AI, and when I asked if it was machine or Generative, they said machine. Okay, fine. Then after that, they kept saying AI. If they say that to me, you can be their salespeople are likely just saying AI (and may not know the difference). Then, as the potential buyer, you will get confused, and many will automatically think Generative AI (although I think they will think ChatGPT, which is just one LLM model, and not even that good).
Hence the two options under AI. Don’t be surprised if a vendor is using both. Why? Token fees! Pricey to the vendor, so how do I as a vendor resolve that? Some parts are Gen-AI, and other items are machine learning.
The future, and what you hear all the time, when someone says AI (nowadays) is Generative AI. I’ve written about Gen-AI – recommend the READ. I placed in that the system only uses ChatGPT – because that is what many folks are unaware of.
The LLMs listed on the Template
- GPT Model (popular with learning system vendors) – They are going ChatGPT or GPT-4 here. The latter is better, but pricey (for the vendor). GPT is from OpenAI.
- ChatGPT – Very popular. This is pure text-based. The way I see it being used – is along the lines of Answer Engine. Whereas you ask a question, and it gives you an answer. You can do this answer engine with GPT models, Azure, and other LLMs.
- Azure OpenAI – From Microsoft. I’ve seen only a couple of vendors use this.
- Uses a different LLM – This is where it goes if they use an LLM for their Gen-AI. I haven’t seen any vendor yet, going this route. PaLM 2 is out there. LLaMA (from Meta) is out there; however, from a vendor perspective, there is no usage. I think Gemini from Google has potential- it hasn’t been released yet. Anyway, from your perspective, I would say yes to the GPT Model, ChatGPT (maybe), Azure (maybe), and Other LLMs (maybe). Once more vendors go beyond OpenAI, then that will change. If a vendor is on Azure, then it should be a no-brainer that they use Azure OpenAI.
I didn’t list Answer Engine, because this can be confusing enough. Thus, if a vendor does a Q/A angle – hence the answer engine, it will be via one of the LLMs – and then they provide that in their details/comments area.
System Scrubs the System (the Learning System)
For scrubbing the system it works this way (and I believe it will become trendy by the end of 2024) – right now extremely limited – i.e., who offers it.
- Client uploads their materials – PDFs, Word docs, whatever into the learning system – it is only the Client’s materials
- End user types in a word or phrase, and the Gen-AI goes right to that word or phrase in the text. It could also be a video too. – This might appear via the Answer Engine too – i.e. it goes to the area.
You can even have your materials sitting on external such as SharePoint for example and it connects. Right now, though it is uploaded.
System Scrubs the Internet and pulls down free content – materials, etc. based on what you asked or sought.
This is where it can get dangerous, even with guardrails in place. It is searching the net and pulling down content from wherever OR in, say an answer engine; the responses come from various Internet sources.
Expect to see a higher probability of hallucinations. You know what I am talking about if you have used Bing or Google Bard. If you have used ChatGPT, you know what I am referring to. Basically, a hallucination is fake or false information. The Gen-AI makes up stuff.
The reality is that many people think that whatever is outputted by Gen-AI is 100% true – all the time. That it is infallible. It is not. There are already a lot of cases validating this. Hence the need for human validation. Or Human touch.
If a vendor tells you that they have no hallucinations – they are lying. My gut tells me that plenty of salespeople will never mention the whole hallucination or bias issue – both of which exist. The bias comes from the source materials – that the LLM is trained on- and is a growing concern in the AI community. I recommend this read or this one or one more around hiring bias.
System creates a learning journey or pathway
I’ve seen this with a few systems, Filtered for one does this. So you enter a word or specific statement, and the system outputs your learning path for you. The downside is at this time, the content you want added or assigned to each path (topic, area) you must add it yourself (Filtered, for example, is this approach).
AI Coach
Really rare. Has lots of potential. So far, the best one I have seen comes via a learning technology solution called Bongo Learn.
Regarding Generative AI, be aware that it is at a very infant stage in the industry. I would expect more Roadmap24 responses than say, “yes, we have it.” Remember, though, you are seeking a system for the future. Thus, seek this out.
What you want to say YES to
- Everything in the Administration section. The dashboard is nice to have – I am a huge fan – but it shouldn’t be a dealbreaker for you if a vendor doesn’t have it. The other stuff? You want it.
- Manager area – If you are buying this for your employees, say yes.
- Instructor area – Yes, for nearly all. The only exception is “Instructor can scan badge codes/or codes (QR) of individuals entering the class (for verification purposes) (Mobile).” It is a nice to have but not common in the industry. Odd, especially for on-site.
- Mobile – You want a native app and on/off synch. Trust me on this. And you want it in both stores – Google and iOS. Never assume they exist in each. Some vendors will white-label the mobile app for you (thus, instead of their name in the app store, it is your name. This is usually an additional fee. I’m a fan of doing this because who really wants to see the vendor name for the app. If I am doing customer training? I want mine. Internal? Why not?
- Assessment Tool – Could be called Qiz/Assessment – The timed option, again, is a nice to have if you want it, I am not a fan of it, but some folks are and want it. So nearly yes to all.
- Multi-Tenant/Portal – If you are doing customer training/distributor training/partner training/B2B – you want to say yes to all. If you are doing internal, but seek a multi-tenant – say yes to all. If you are an association who has different chapters and they want their own look and access, say yes to all.
- Certificates – Excluding the LinkedIn one, yes to all. Not every vendor will have them, but at least you get a general idea.
- Classroom management – If you plan to have on-site training
- Event Management – This is for webinars/seminars – Excluding cells 144 and 145, ideally you want the other items. I am a huge fan of waitlist- auto, which means if you drop out, and someone is next in line, they get in, and the system notifies them. Time saver! I also like the connection to the person’s email account – i.e. calendar, when they sign up for a seminar/webinar in the system.
The Other Tabs
If you are seeking cohort-based learning – then there is a tab for you. If not, remove the tab.
Skills tab exists. I made a few updates and changes.
Gen-AI Questions – You want the vendor to respond if they have Gen-AI. If they have Gen-AI and leave this tab blank, select someone else. Again, if they say their LLM will not generate hallucinations – they are either lying or have no idea what a hallucination is, and thus, don’t pick them. I started out with some very basic questions. Over time, I will expand.
The other tabs are pretty self-explanatory.
Bottom Line
I modified the UI/UX slightly, because everybody says they have a Modern UI, even if they don’t. Nobody is going to say – “Our UI is from 2000.” Albeit, I have seen some systems, where I seriously believe it is from 2000 or 1996.
If I am a vendor, I would download this template, and start completing in the fields and especially the responses ahead of time. Then when one comes there way, all set to go. I’d include the head of product, and someone technical to review the tech or Gen-AI questions (if the vendor has it). Then I would double-check.
If I am looking for a system or considering a switch, the template will make a major difference, because the legwork has been done for you. Along with some guides too to provide additional insight. And best of all, you can trust what you see, and recognize what is coming soon – okay 2024. Always remember – don’t look at just what you need now – look at what you might want three years from now.
Because when you sign a contract, you can’t decide a year later to bolt – well you can, but there are no refunds. And thus, you are stuck.
And miserable.
E-Learning 24/7