Of late, I find a number of my posts are about taking down silly articles. And, I find that the claims are increasingly irritating. Now this probably isn’t new, I’m just more ‘sensitized’, but I see some recurrent themes, and I think it’s worth thinking about what they’re doing and what you should watch out for. These efforts may not be deliberately misleading, but they’re problematic regardless. The point here is to equip you to be cautious in your evaluation of claims.

Why is it a problem? Several reasons: for one, some of the problematic claims are myths, and they can cause you to be discriminatory. While looking for the non-existent magic bullet, you might use a categorization (e.g. personality or generation) that isn’t real.  To the extent you do, you can be unfairly stereotyping people. Also, it might cause you to invest money in things that aren’t helpful; you’re spending money on things that won’t have an impact. This is money you could be using to do something more useful, like build in the basics of better learning science. The worst outcome, of course, is doing something that’s actually harmful, interfering with the very goals you’re trying to achieve.

The formats differ; one was a white paper, one was a blog post, one was a magazine article. What’s shared are some recurrent patterns, some particular techniques and flaws that are worth examining. So let’s examine them.

Myths

One of the most prevalent problems, and warning signs, is the presence of myths. Whether it’s learning styles, attention span of a goldfish, millennials, or more, it’s wrong. Now, you may not know all the potential contenders, but you should have your antenna tuned to the potential. And the fact that they’re myths means science has shown them to be wrong.

To be fair, learning styles is subtle. I’ve seen it tossed in casually in claims about a tool or platform, something like “can meet different learner styles”.  And that’s appealing, because we know learners differ. But not in systematic ways that we can leverage for design, as research has shown us. It’s misleading and reemphasizes bat patterns.

Learning styles and generations are simplifications that make us think we can accommodate differences, and that we’re therefore being considerate. However, research tells us that the best way to design our learning is to the learning outcome, not the learner. There are no shortcuts; no brain-training that’s not specific to what learners need to be able to do is going to help. You need to practice what you need to do.

Other claims, like gender differences, or dropping attention spans and how technology is changing the way we think, are similarly appealing. People seem to be behaving differently, and we want explanations. But we have to be wary of ‘easy’ ones. Can you create a simpler explanation? Many of these purported changes are supposedly about how technology is affecting our thinking. And this is legitimate if it’s not proposing a fundamental change in our brains. Yet…here’s a clue: our cognitive architecture doesn’t shift that fast. While we can learn new things, and new technologies can open up new affordances, we still need to perceive through our sensory systems, practice, reflect, and other aspects of our cognitive makeup.

Hype

If it’s not myths, and they fit here too, it’s could be the latest fads.  For instance, you’ll see buzzwords that aren’t well differentiated. Microlearning and gamification are both terms that have a couple of different interpretations. Maybe it’s just that uncertainty makes it hard for people to call you out? Then there are others that are trying to capitalize on associations (neuro-<anything>). I understand wanting to ride the latest meme, but use it to mean something!

One of the prevalent tricks is to phrases that mean something but are already old-hat. Mobile went mainstream in 2010, when Google said they were going ‘mobile first’, yet recently an article claimed that mobile was ‘on the rise’. Yes, there are more people (population worldwide is still increasing), so more devices; however, in places like the US there are already more devices than people! This isn’t new. You have to pay attention to the details.

A similar trick is to tap into people’s fear or excitement. Something like “Learning Is Changing Forever” makes a bold, but ultimately vacuous claim. If it sounds unbelievable, it probably is!  No, there’s nothing that’s going to make everything different. It may, ultimately, make a change, but it takes time. Even the internet took time to take off, and it’s arguably the best case for something that’s been world changing. Not as many things will have a fundamental effects as are claimed.

There’s also the ‘bait and switch’. We’re going to tell you about the ‘new learning’! And then, when you pull it apart, it’s the same old-school view. Similarly, you’ll hear some recommendations for behavior based upon something ‘new’! “Use simulations because millennials will demand it!” However, when you unpack the recommendations, they’re true for everyone. And these recommendations aren’t new. They’re just good principles. I suppose we should be grateful for good recommendations, but I think if they come for the wrong reasons, they could cover some bad practices too. Indeed, in one I saw one of the recommendations actually recommended something contrary to best principles.

Data

One of the other recurrent elements, besides hype, is the wrong use of data. For instance, they’ll use people’s preferences as a basis for decisions. One I saw said people prefer small chunks, so that’s what you should do. Okay, but preferences aren’t always right. For instance, whether people prefer a learning experience has essentially no correlation with its actual effectiveness.

Another misuse is using anecdotes to imply authenticity. Folks who’ve invested in a solution have a greater likelihood of finding it valuable because they’ve invested in it. It doesn’t mean you’ll get the same results. Would it work in your context? Are they talking about similar people, goals, and environment?

And of course, there’s misinterpretation and over-generalization of data. The goldfish myth actually came from studies on web page behavior several years apart. The average time on a page dropped from something like 13 seconds to around 8. However, can you think of any other reasons we might spend less time on pages several years later? Pages loading faster? Getting better at evaluating pages quickly? Human attention is a complex phenomena, and again our ‘wetware’ doesn’t change that fast.

Caveat Emptor

Again, I’m not saying all these folks were deliberately trying to be deceptive. There’s no clear evidence that any of these were consciously choosing to misrepresent things. Some are just marketers more focused on sizzle than substance. However, you will see articles on sites affiliated with products, and not surprisingly the recommendations do align with the products. That’s just marketing. But, just as you should at home, you need to be a skeptical consumer.

Our industry doesn’t have the equivalent of Consumer Reports. There are independent evaluations of LMSs, with greater or lesser credibility. Similarly with authoring tools. And you have to expect that they’ll do their best to get your attention. The onus is on us as buyers, however, to know this and to ask for credible evidence.

We need to be professional. We need to know the underlying foundations for our work (e.g. learning science), and also how to evaluate claims. We should accept that there will be attempts to sway our judgment, and we also should be prepared to counter those attempts. Our learners, and our organizations, depend on us to use tested and sound approaches. It’s our responsibility.