LMS Decision Making Criteria B4 U Buy

If you are like me, you have probably seen dozens of posts or articles out there telling you here is “how to pick an LMS”, “criteria for selecting an LMS”, “best practices for buying an LMS” and so forth.

The problem I have with all these items, is wondering how many of these people have bought an LMS, i.e. been on your side – the buyer’s side?  And if yes, what verticals? What challenges did they face? How long have they been on the buyer side? Where they gave budget control or not?

As you can see, providing insight when you haven’t been standing in your “shoes” in a manner of speaking, really doesn’t tell you the reality, because they haven’t had to be in reality.

I have.

I’ve had budget control and yes, at a few places not so much.  I’ve had to go into places who wanted e-learning didn’t have it and wanted to move fast on it, just as I went to places who either had it or didn’t have it poor about getting it done – even though they wanted it done – management, company culture, etc.

I’ve had high budgets and really low budgets. 

Timetables everywhere and yes, at one place nowhere – I wanted to scream.  It was so bad, a slug could out run that place.  Each place wanted an LMS, and if they didn’t know what it was (and no one did), I had to explain it, sell it to the executives, tell them the benefits and blah, blah.

The fact of it all was that it wasn’t easy.  Culture change was everywhere. 

Some people just won’t buy-in no matter what I did.  Others went “let’s go.”  And oh, don’t let me talk about the association world and those professional education/development/training committees.  Working for a small company, mid-size, large, F100?  Been there.  Involved in associations, non-profits, B2B, B2C, global? I know your challenges, been there, done that.

I mention all of this, because my selection criteria is based on years and years of being on the buyer side, and having experienced it all first hand with no guidebook or assistance along the way, enables me to provide you with the realities that exist in the e-learning world and in our world (training, L&D and yes, for those folks in HR).

Here are the factors that will be involved in your decision-making process.  Please note, that there are always unknown variables or factors which are specific to your organization or which are required from your CEO.  Plus as with anything, there are always unknown external variables which cannot be identified at the present time – hence unknown.

Not all of these factors will be relevant to you, but this is what I have found to be the essentials that go into the decision-making process.

Factors that go into your buying decision process

Factor Explanations

Again, as noted, the above list is in no particular order.  One item that has to be in your buying process, but not cited above – it liking the system.  It can meet every one of these factors above, but if you do not like the system, then time to move on.

When I hear people say they hate their system, my retort is always the same.  “You picked the wrong system.”

You should like, heck, love your system.  If you don’t or you didn’t (which resulted in a move or soon to be move), then somewhere along the way, something went wrong – factors, criteria, etc. – and as a result zing.

For some, the “brand” factor is where it went all down hill.  

I’ll explain brand in a few moments, but there are some “brands” whose track records IMO do not live up to the hype or promises.

Tying into the above graphic, I will go around the “LMS” circle.  You can easily replace, “LMS” to reflect your learning solution – i.e. any one of the subsets, such as employee engagement platform, learning platform, social learning platform, gamification only platform, knowledge management platform, learning system, micro-learning system or platform, courseware LMS and so forth.

You can easily replace, “LMS” to reflect your learning solution – i.e. any one of the subsets, such as employee engagement platform, learning platform, social learning platform, gamification only platform, knowledge management platform, learning system, micro-learning system or platform, courseware LMS and so forth.

Goals and Objectives

Simply speaking, what are your goals and objectives in regards to your

a. E-Learning program – which the LMS will play a part in

b. The LMS itself.  What do you want out of the system?  What objectives are you trying to solve or resolve?  How will the system attain those goals for you (in an ideal world)?  If you have impact of learning objectives include them.  If you are using ROI objectives, again post them here (i.e. when you are considering the objectives).

Poor Way:  Our goals are to find an LMS that will help us with compliance.    OR Our objective is to find/select an LMS that will enable us to move forward on our L&D program.

Why are these poor?  They are too vague. Too ambiguous.   It sets you up for picking the wrong system from the get go.

Get specific on your goals and objectives.  You should have no more than five goals and objectives. Three is ideal.

When you start going above five, you are pushing beyond realism.  I mean, if you cannot identify the top three goals/objectives in all honesty, again no more than five, then how may I ask will you be truly happy with what you get?

Better:  1. Track and capture compliance data based upon specific courses that we provide to our learners.  2.  Enable our learners to take courses, content and share that content with others in our company – i.e. we want a learning community.   3. Expand our offerings and generate additional revenue by offering courses and content for sale online.

See the difference?

Specific goals and objectives will get you farther ahead right out of the gate, rather than the vanilla ambiguity too many people start with and in the end, counter the objectives they had in the first place (and not in a good way).

Meets Use Case

Regardless if it is employees, fellow businesses/clients (B2B), customers or some/all of the above, you will have a use case – a learner case.

This isn’t the same as a user case study you see on vendors websites (which btw are nothing more than marketing materials wrapped up in a “research” like angle.  Same thing with white papers. Wonderful marketing materials.  Anyway that is my two cents or three pence.)

Use case aka learner case is in essence, is an explanation of what your learners are currently or if new to the world of e-learning/LMS you want them to do within the system.

  For example:  “We have 35,000 employees.  Of the 35,000, 15,000 of them are taking two ILT courses a month.  We want them to continue to take ILT courses, but register online and take at least one online course a month.  We are supporters of professional development, so we are going to have a catalog of courses that people can select from on an on-going basis.”

For example:  “We have 35,000 employees.  Of the 35,000, 15,000 of them are taking two ILT courses a month.  We want them to continue to take ILT courses, but register online and take at least one online course a month.  We are supporters of professional development, so we are going to have a catalog of courses that people can select from on an on-going basis.”

We want them to continue to take ILT courses, but register online and take at least one online course a month. 

We are supporters of professional development, so we are going to have a catalog of courses that people can select from on an on-going basis.”

The above example is a newbie and just one set example.  I’ve seen use cases that are just a couple of lines and use cases that are multiple pages. 

Again, shorten down.  Your use case should be no more than three or four paragraphs.  Anything more – is too much and guess what? It will impact your decision-making process.

Functionality

What are the features you need (must-haves)? What are the features that you would like, but may not need now?  What are the capabilities that you definitely need? What is the roadmap of the vendor and track record of meeting the new functionality/capability?

Functionality to me is one of the biggest parts of the decision-making process.  The system has to have what you need in functionality and if it doesn’t  – can you wait until it does?  I’m a big believer in the “roadmap” track record. 

As someone who once worked at a tech company, the old phrase, “yeah it  is on our roadmap” was often used in order to entice someone to ideally buy.   That roadmap could be 10 years from now,.

I’m a big believer in the “roadmap” track record. 

As someone who once worked at a tech company, the old phrase, “yeah it  is on our roadmap” was often used in order to entice someone to ideally buy.   That roadmap could be 10 years from now.  This is why I ask vendors what was their roadmap for the previous two years (if they have been around that long) and how many of those items were completed and met the timelines.   I also want to know their current and upcoming roadmaps.

Additionally, I will ask whether the updates are updates in terms of new functionality, or are they fixes to the system or both.   Some “updates” are really fixes.

Business Requirements, Technical/Security Requirements

I would list what are my requirements today and ideally if I have an idea, the business requirements in a year or two.  The same with my technical and security requirements.

Yes, you may not know the future, but a good training, L&D, HR leader will be someone who constantly is reading (thus learning) about what is going on now and what might be possible down the road.

For your requirements focus first and foremost in the present time and the following year. Let’s say one technical requirement is AES 256 with the LMS. 

But you are reading an article which says that in two years, you will want AES 960 (I’m making this up, so don’t go looking for that article).

Thus, your security requirement is AES256, but in your mind, is the ideal AES 960.  How can you find out if they are exploring future security?  Ask the person running  technical/security if they keep up with the latest security/privacy issues and/or solutions. Ask if they have heard of AES 960 (don’t ask, again it is fake).

This is an opportunity to understand who is running the security show there and how often they are consuming or exploring future security information.  If you don’t care, then clearly ignore.

Too many people, just ask the basic security questions. Some folks go on for dozens of pages.  Here is a clue to save you massive time and not waste theirs either.

1. Ask where they host their system.  Many vendors use sites such as Rackspace, Amazon or Microsoft Azure.
2. Go to the site and explore the technical side of the house – the information is fully visible.
3. From there, you can now inquire with the vendor if they are doing something above and beyond what the server site offers and includes.

The above by the way would include infrastructure from a technical standpoint.

But I want to know the infrastructure for the company itself.

Internal Infrastructure

• How many people are full-time support?  Do they have a support/customer service manager or someone running the team?  Do they use multiple team members to handle support/customer excellence (example: one from project team, one from customer support, one from technical, etc.)
• What was/is your turnover rate? Voluntary? Total?  A vendor may disclose or not disclose this, so just by asking employee count from last year to now would work.
• How many new clients do you have? What is your total number of clients? What is the avg. user base?
• What is the total number of users that you have on the system at any given time – an average is fine.
• How many people are on my project team?  Who is my POC during the project and after the system goes live?

These are just a few questions, others could include financial/sales info such as

• What is their growth rate from the last year to this year?
• What is your debt load?

Be aware that the majority of vendors are private, so getting some of this info is going to be tough.  Again, if it doesn’t matter than ignore.

On the internal infrastructure, I want to make sure that this vendor isn’t going to have difficulty with me, because they have the right number of employees, and right processes in place.

At the end of the day, you want to know the internal workings at your soon to be buddy.

Brand Y or N

Does it matter whether or not the vendor is well -known in the industry?  Let’s be frank here, the LMS space (inc. subsets) as a whole, really stink at marketing, especially getting their name and message out there, so it is on everyone’s tongues or more importantly, eyeballs.

Ask someone to name a few LMSs or at least one, and I suspect it will be Cornerstone OnDemand or SumTotal.   To me, these are brand names.  If they are in education/higher education it is likely to be Blackboard or Instructure, which some folks refer to as Canvas.

Say open source? Moodle rolls right out.  Ever heard someone go, oh, yeah Sakai!

I’d say highly unlikely.  Each of the above are brand names.

Here is what I have learned in my years of doing this.

• People either love or hate Cornerstone OnDemand, SumTotal Learn, Saba, SuccessFactors
• Support track records for the above, are generally average – I’ve spoken with clients who say they want to leave one of the above (shall not be named) because they failed or were unable to do the integration they promised; they ignored multiple calls/emails for support; they never responded to an issue;  they didn’t deliver what they were supposed to do; and so on.

These vendors can argue that this happens with plenty of other systems out there, and they are right – but the downside of being a well-known brand name is that your issues are seen or widely heard/repeated more so, than someone who has a great system but not as well-known.

Everyone who is a reader of the blog likely knows my thoughts and feelings about each one of the brand names listed above.  Some I like, some I don’t – but I have reasons for doing so, and not just because they once spit in my coffee (uh, no one did that, so don’t ask me who).

If you are seeking an HCM or a system and plan to buy two or more modules, then Cornerstone, SumTotal, Saba are definite routes to consider.

When I hear that nasty “traditional” term, I already know the person is referring to one of the bigger brands – i.e. Cornerstone, SumTotal.  

Let’s get it straight here – “traditional” is a marketing term, to spin and scare you to go elsewhere.  Less than 10 vendors in the entire industry have multiple modules.

Presentation/Demo

This is one factor that way too many people fail on and as a result get the shock of their life, when they buy the system, it goes live and they go “oh cr–” that isn’t what I wanted or asked for or it isn’t working this way or is not responding that way…

Online Demo and some are also F2F (I have bolded those that are F2F too)

• Did they ask you want you wanted to see ahead of time?
• Did they skin it with your colors and logo ahead of time? 
• Did they show you a couple of other client sites so you could get an idea on what it could look like out of the box?
• Who was on the call?  Just the sales person or a sales person and technical person or some combination of other folks?
• Did they actually listen to what you were asking?
• Did they respond accordingly?
• Did they sound interested and passionate about their product?
• Did you feel a connection?
• Did you get the feeling they were reading off a script?
• When you asked questions did they stop right there and answer or ignore and continue their demo view?
• After the demo, did they send you a thank you note?  If yes, how long did it take? 
• Did they send you a snail mail thank you note (very rare these days, but I’ve seen a few)?
• Did they include the “if you have any additional questions, etc.” type of line? Or was it simply a thank you note?
• Who sent the thank you note? The salesperson?  CEO?  Some other executive?  Multiple people?
• Did they constantly bombard you with next steps or when you would respond?  Even if you told them at the end of the demo, your next steps, and follow-up communication, did they listen or ignore?

Face to Face Presentation

• Who was at the presentation?  I wanted more than just my sales person.  Having a senior exec was always a plus.  Even if it is just you and one other person, the F2F is the opportunity for the vendor to shine.   A tech person for them to have there is equally good, or at least on the phone to answer additional questions
• Did they show up ahead of schedule to set up?  You want a YES here. I mean if your trainer showed up late or failed to setup correctly would you be fine with that?
• Did they dress professionally?  I’m very laid back, but when I had to do presentations, I wore a suit, even if the customer said you can dress casual.
• Did they make eye contact with you during the presentation?
• Did they appear to be genuinely excited to be there?  Did you feel like they cared about you and wanted you to be a customer?
• Did they answer all of your questions?  If they didn’t know did they say so and then write down the question(s), so they could follow up with you?  Did they give you a timeframe for doing so?  (This can also be with an online demo option too)
• Did they schedule the session accordingly?  Did they provide breaks?  Did they provide an agenda?  In essence, you want to see how prepared they were and professional.

These are the core ones to me.  Sure you can add/remove, but at least you get an idea.

UI/UX

Extremely important. What does the Learner UI look like? Easy to use and navigate?  What about on admin side?  Mobile?  What is the learner experience (UX)?  Same with admin.

These are just the basics, you can go as in-depth as you want.  At the end of the day if your learners can’t figure out how to take a course or courses and navigate, then your system will fail to attain mass.

Service/Support

I’ve covered quite a bit earlier, but I also want to know where their support staff is located, what are the hours, is everything included i.e. email/phone support, do they charge extra for certain levels of support,  do they offer end user support (for me, I’d never want it, but there is a slight increase in consumers seeking it),  what languages does their support agents speak and support themselves, what is their SLA – and you want a copy of it.

What is their turnaround time on average to solve a simple issue?  How does their support process work?  What do they define as critical?  What has been their average time frame to solve a critical issue, as defined by the vendor?

If you want them to provide you analytical data on some level of support – ask them.  Many vendors have solid amounts of support/service numbers/stats, nothing wrong with asking to see them – before making your decision.

Remember, the number one reason people leave their LMS (and subsets) is because of poor support and service.

Implementation

If they can’t meet your implementation time-frame, then I’d call that a deal breaker.  The basics to ponder and ask (in no particular order)

• Who is on my project team?  Who is my POC during implementation?
• What is your implementation process?
• Once I sign the project plan, how long will it take to build, QA and go live (pre-live before humans enter it)?
• Will I be able to see the project plan in real-time?  (They might have an online site they use – if you don’t care, ignore the question)
• If I have to make changes to the project plan, what impact will that have on the implementation process?  Are there any additional fees for making changes?
• We want to be able to do Q/A prior to the system going live.  How can we access to do so?
• As part of your implementation process, do you do Q/A too?
• When do you provide training to our administrator(s)/Training/L&D/HR executive? (whoever is running the show)
• On the day we go live, will there be a person at your company, we can reach quickly if an issue arises?  Trust me, you want one.

Price

The big elephant in the room.  I’d say the dealbreaker.  Vendors usually want to know your budget.  I say never provide it.  Provide them all your information (that you want to), include the number of learners (an important piece), seek a three-year deal and a discount, have them itemize each item and provide you with a proposal.

If you are at an early stage, tell them the information – goals/objectives, use case, learner projections and ask for an estimated range.  Let them know you are unaware of other variables or items that could come up, but you just need to get a ballpark here.

I would always be provided with a ballpark range.  Then they would ask whether that is within my budget or not.  If it was, let’s schedule.  If not, I would either say it isn’t  (but it is close) or would say that even if they could deal down, it is highly unlikely to fit into my budget

I always ask what is the setup fee and what is included with that out of the box.  If a vendor can’t tell me or pulls the “Well it depends” angle, then I’d say thank you and end the conversation.

If you start to argue with the vendor or feel frustrated, then this is not the right vendor for you, regardless if the price point is there or not.

Bottom Line

There isn’t any way to make this post shorter and it could go on much longer.

The reason is that the information is so relevant and the factors are based on the real world and real life experiences, not some theory or idea a vendor or person has who hasn’t been there or done that.

Whether you are new to the search or re-exploring, know this.

Based on my experience, these are the factors in the LMS decision-making process.

Ignore a few, tweak a couple and move on.

That is your choice

Good or bad.

E-Learning 24/7