Posted by “Hey everyone! As dad’s favorite dog (according to me), I’m back to announce the nominations and winners for this year’s Turkey Awards. Did I mention I’m number one? Spirit always takes over these awards, and Voodoo is on a walk, plus Nava (our newest member is busy chewing on dad’s shoes), so it is me, Cali.” “I love food. And because Dad no longer eats meat, I’m stuck eating plant-based food, but it still tastes delicious to me. Plus, dog treats. I droll for these. Do you have any turkey dog treats? I won’t tell Dad.”
“Okay, on to the awards. Oh uh, Dad sees me, got to run, I hear my name being called. Must be time for a car ride. I love car rides. Maybe we will go and get a turkey dog treat. Or a treat. Did I mention I’m number one?”
Thanks Cali, I will take it from here. She’s quite a cutie, except when she barks. Clearly, she is saying, let’s get to the nominees and award winners.
Category – Time to End these specific items.
Nominee #1
End the SCORM/xAPI etc. nomenclature on the back end of the learning system.
It absolutely makes no sense. The number of people who have no idea on what you are talking about increases each year. As a vendor said to me, a few weeks ago, they see more and more people just uploading content, all types of content, not necessarily courses built by 3rd party authoring tools.
I always laugh when a vendor says, “legacy,” as though everyone in L&D, Training, HR and whatever other department is overseeing the learning system, knows what that term means. Unless you rumbled in a century or two ago, you have no idea. It’s a tech term honestly, used quite a bit during the ERP days. Vendors push it still for learning and training.
Now, think how many people are overseeing a learning system who are not in L&D or Training? Do you think marketing is going to know what SCORM is? Do you think after the L&D department is decimated and the system heads over to HRIS or HR is going to know what xAPI is?
Do you think they care? No. No. And no.
If even you are using a 3rd party authoring tool, or an expert in ID or e-learning development, if you see “upload content” I am very confident this means you can upload any type of content. Regardless of it is a course standard or not. I mean, if someone is buying a system, they usually will ask whether it takes XYZ course standards or not. And if not, you should tell them. Thus, if I know it takes SCORM 1.2, and I have built my courses in SCORM 1.2, and I see upload content/courses – you should trust that the person can figure this out.
It really makes no sense on why vendors still do this. How many people today can actually tell you what SCORM stands for, let alone a SCO, let alone the whole manual for that thing? I worked at a F50 one time, and they had one person dedicated to SCORM, and the whole manual. They were the experts. But they didn’t upload the courses.
I digress.
Let’s add another piece to this whole listing of the term SCORM, xAPI (I’ve also seen TinCan), and other course standards where it should just say upload content.
Your built-in content creator. Take a look at your data and see how many people are using this to create content within your system. Now compare that to the number of people who are clicking on the SCORM button. Now cross-reference that over a time period of one year, with data for each month. Next, cross-reference to size of client (end-user wise), department overseeing it, experience of the admin and so on.
How many people create courses in PPT and just upload it?
The point here is you can’t push “system X is so legacy or traditional,” when you have a button or something that clicks for folks to upload courses, and note it as a course standard.
Talk about traditional.
Nominee #2
Stop saying “authoring tool.”
Who says this anymore in the generations that are creating content? It’s not just millennials that are overseeing L&D or training. Besides kids on Tik Tok, you see adults – 30, 40, 50, 60 and so on years old. The term content creator is widespread these days.
I know authoring tools continue to hard press the authoring term, but just because they are still in the stone age, doesn’t mean you need to be. I mean there are authoring tools who are still desktop only.
Yowsa!
I say content creator tool when I talk to vendors these days, ditto on my recent template. End the authoring tool term, and while you are at it, stop saying slides. If you are creating a course, you are not creating a presentation, or playing as though you are PowerPoint or whatever crummy name Apple uses to call their presentation program.
Lessons? What am I – 10 years old? Who says Lessons, beyond you being in primary or secondary school? I never said it, when running corporate training or an association, but vendors slide that in (bring back the slide rulers in that case). Seriously, I wonder how many vendors spent like 30 minutes on the internet identifying the proper terms to list when someone creates a course. This should be a third term to stop using, but I have other nominees.
Nominee #3
Completed on the learner side.
Which is a button the learner clicks, in order for the metrics to show they completed the course or content or whatever they are viewing, even if they never did. I know of vendors who use the “complete” for their machine learning algorithm, so that if you don’t click it, it either leaves it out OR it penalizes the learner.
Okay, so you click it, then the recommended content shows what? All that wonderful similar assigned content. As for the auto-complete, I have seen this on a customer training focused-system. Why would I want that? If I am learning a product, or software, or a service, and I am only interested in one area of that, because I already know or that is just my interest, why do I have to select complete?
Finally, the whole reason asynchronous-based content (now referred to as self-paced) was created, is that you can go anywhere you want, at any time, without having to complete it. Focus on what is relevant to you.
Complete takes the entire premise and throws it into the garbage can with the left over turkey bones.
And the winner is:
Eliminate SCORM and similar course standard button the back-end.
Just say content. Trust me, people will figure it out.
Category 2 – Hello Learning System vendors (and learning tech for that matter) – Are you paying attention?
I totally understand, every vendor has their own take and approach to creating x capability or Y capability and not some other option. And yes, cost always plays a role and even resources – albeit there are some vendors whereas “resources,” doesn’t seem to apply when they head down to meet the animal in the rabbit hole with Alice in Wonderland.
What is the result of this magical tour? Potential missteps.
Nominee #1
Clients drive the roadmap way too much.
Recently I was talking to a vendor who told me their roadmap and other items is based on 70% or more from their clients. They acknowledged they needed to drop that and felt in the mid-40s should be right.
I’ve heard from vendors who say 25% to 30%. All of them always say that they – the vendor themselves does not focus solely on the clients and will come up with their own capabilities and features.
The numbers should be no more than 10%. If your clients don’t know about the latest features or capabilities, why do you expect them to tell you? The last time I looked, the vendor is supposed to be the expert not the client.
I’m equally not a fan of vendors that add features because their biggest clients tell them they need to have it. The worst of this is when the majority of their customers are in the 500 to 2,500 range.
I’ve seen the end results here, and it isn’t pretty. Think about it. The system is heavily driven by what is needed with the monster size clients, despite the target of marketing and user range is less than 2,500.
Funny, I never hear a vendor say, we listen to what are clients say at the 500 range and add those features.
Nominee #2
Missing Gen-AI space – LLMs and well, scenarios
Oh, this is about to get far worse. A lot of vendors just dived right into Generative-AI without having another LLM, to add some additional guardrails. Despite many AI experts recommending that any solution have at least two LLMs, the market (with the exception of two vendors), went with one. ChatGPT. If you haven’t heard the latest news, one of the founders was fired, and went over to create a new AI lab at Microsoft (as a separate company). While OpenAI is now trying to woo him back – due to 700 of their 770 employees threatening to quit, so far nothing. I won’t get into the whole game plan of OpenAI originally after firing him, because one of the biggest proponents of going back to their original premise, is now trying to get him back.
The point here, regardless of whether he returns or not, is that vendors did not think thru all potential scenarios, before going solo. Some have said, prior to him getting fired, that if OpenAI isn’t popular in say two years or something else comes along, they can switch out quickly and go with another Gen-AI offering.
None of them to date, those that went solo, went thru a model as a service, which I believe will play a key role with companies going into the Gen-AI space. Google Cloud has a model-as-a-service, Amazon has one too. There a few others.
I was quite surprised on how many vendors are not paying attention to what is coming down the pipe with LLMs being built by the behemoths (exlcluding Open AI and the promise of ChatGPT-5). Olympus by Amazon is expected to be a major game-changer, far exceeding two trillion parameters. It makes ChatGPT seem like puny in terms of parameters. Gemini by Google is expected to be a game-changer too. Meta, is working on their own new LLM, which they expect to compete heavily with ChatGPT.
Anthropic’s Stability.AI is no longer garbage. It’s promising, and one vendor, Cypher Learning uses it, along with OpenAI.
One would think that vendors who are adding Gen-AI would be paying close attention to the market – and what is out there now and what is coming. Were you aware that one company (small) is close to launching an LLM that has full empathy? (a major issue in today’s LLM market)
Nominee #3
Vendors forgetting to tell their salespeople about the Gen-AI challenges.
You want Gen-AI. You have heard about. You ask about ChatGPT (because that is the only one you hear, and you are using it in the workplace). Good news, the vendor offers it!
Let’s see it you say. And off we go.
Problem? The salesperson lacks basic knowledge around Gen-AI, and as a result, isn’t telling you the full story – because they don’t know, nor has anyone told them (I met two vendors who have Gen-AI, and their execs didn’t’ know about certain items around Gen-AI, which thus they failed to have in their system).
If you have a trainer or an instructor and they are going to provide a session on whatever topic, you would expect them to know that subject matter inside and out. You would never let them just go with whatever. I am talking to anyone in L&D or Training here. Yet, again and again, you ignore this when it comes to your salesperson, and worse, with Gen-AI is getting to be massive.
There are a lot of vendors who do NOT TRAIN their salespeople, despite them focusing on training and L&D. There are plenty who lack the general basic information/knowledge and thus, they aren’t sharing it with the rest of the class, you know the folks selling the product, and yes, in many cases, the solutions consultant showing it (when it comes time).
A couple of basics here, that a simple web search will provide, but I will save you some time, and provide them – there are quite a few, but these are the essentials – that you should tell everyone you are talking to, that asks about Gen-AI (and those numbers are increasing each day), regardless of if they say ChatGPT or not.
Oh, and don’t wait for them to ask you, because they may not know, okay, less than 2% will know.
- All Gen-AI produces hallucinations – which means fake or false information. Thus it is extremely important that you always have someone review the output before using it. Today, the vendors who have Gen-AI are focused on content/course creation, assessment tools and some skill building. Only three have a “window or information before you begin, that identifies the potential for fake or false information with Gen-AI and that you should always review before publishing.
- Gen-AI may produce AI bias. This is a reality. Think this way – you are adding your own content/materials, and unbeknownst to you, it has bias in it. AI can’t figure that out. And depending on the data sets the vendor used initially, it may have bias in it.
- Gen-AI is not 100% accurate. It doesn’t matter what LLM you are using. Plus don’t buy into “well, we have x guardrails and this and that, and it is private – i.e. self-contained and thus, it will produce higher accuracy numbers.” – There is no consensus on how higher that is, and yet I’ve heard vendors say 98%. It might be 92%, or 95%. Even if, hypotheticallly it is 98%, that means 2% inaccurate. If you heard that your car has a 2% chance of blowing up when you start it, would you want to know that ahead of time with the salesperson telling you? As long as you, the customer is aware of the understanding that no Gen-AI is 100% accurate, you should be fine. But, not knowing is not an excuse – i.e. the vendor must tell you – and don’t rely on the higher accuracy numbers, with private Gen-AI. Sounds great, but Gen-AI is so early stages, it is just not a valid guarantee.
A couple of other points that you should ignore – as the prospect:
- Vendor says they are the leader in Gen-AI or the first with Gen-AI. The latter is extremely unlikely, because they are not looking at the entire market. The former, please. Marketing spin, wash and dry. I’ve seen the leader in AI for Skills or something like that. Then you see it, and you go, “Really?”
- Vendors who push they are the only ones or leaders for course/content creation with Gen-AI. The only vendor today, whose content creation tool I have seen with Gen-AI that blows me away is Cypher Learning. That’s it. And I have seen a lot of systems that have a content creator component.
- Any marketing around Gen-AI as thou that vendor is the dominator in the space. No, no you aren’t.
Be aware that if a vendor is using ChatGPT, it is likely they are using ChatGPT 3.5 turbo, which does not have the full power or functionality of ChatGPT-4 (now known as GPT Plus) nor the Enterprise ChatGPT. If a vendor is using ChatGPT (and again, nearly everyone is – see further up), ask them if it is only 3.5 turbo or 4 or a combo. I can easily tell which one they are using, but my expertise in the space, isn’t the same as everyone else, which is why I can’t stress enough, that you find out. And any vendor who tells you, they can’t tell you their LLM, should be removed from consideration. You are not asking for the secret to getting to Mars thru a portal in time. If they have the LLM, they need to provide it to you. I would think a vendor who has two, would be thrilled to tell it. I have to pull teeth with some vendors to get that information – but trust I stay on them, like ticks on a hound dog (remember when folks would say that?).
And the winner is
Vendors who fail to tell their salespeople basic knowledge around Gen-AI, who thus need to tell their customers.
Turkey of the Year Award
We have come to the conclusion of the Turkey Awards. Except the biggest one, the one folks always expect to see and find out. Past winners include SAP SuccessFactors and Workday Learning. This analysis is just based on the vendors I have seen this year in 2023, which are a lot – the most ever! And we are not even done with the year. Plenty of pickings to choose from, just like that sweet potato pie, Grandma ma makes, or the green bean casserole that nobody wants to touch, because your aunt, always uses something that is unknown to the human population.
With everything going on this year, we all need some joy. Some happiness, to make our day fulfilling, which is why for 2023, there is no Turkey of the Year Award.
A pardon has been pronounced. This is based on my own in-depth research of asking a group of experts in the turkey treat industry.
Voodoo voted to abstain – he was eating a cookie. Cali said pardon, because she was promised gravy as a bribe. Nava is still eating something that I own, despite having lots of toys. However, if she did vote, it would be pardon. Which she probably thinks means treat. Spirit is the docile one. So, pardon is clear. But he is open to turkey treats, or uh, any treat or belly rub.
There you go. Two award winners and a pardon for TOY Award.
Speaking of toys – “Who’s a good doggie?”
E-Learning 24/7