Clark Quinn

DECEMBER 22, 2020

Someone on LinkedIn asked about a way to evaluate orgs on their learning infrastructure. We also consider the use of metrics in the organization, both to evaluate and to improve. We move our focus from internal to a broader picture, and start evaluating our processes themselves, and move to looking at these as a synergistic whole .

Performance Support 295

Performance Support Performance Metrics Organizational Learning 295

Clark Quinn

APRIL 26, 2022

The correlation between people’s evaluation of the learning and the actual impact is essentially zero. Yet the variability of people, the uncertainty of the approach, and more, suggest that courses should be trialed, evaluated, and revised until actually achieving the necessary change. At least for novices.

Metrics 262

Metrics Performance Support Pedagogy Behavior 262

Clark Quinn

FEBRUARY 12, 2019

The original brought people into a room, had them generate ideas, and evaluate them. Then, you share, and still refrain from evaluation, looking for ideas sparked from the combinations of two individual ideas, extending them (even illogically). Finally, you get critical and evaluate the ideas. And that’s a problem.

Evalution 165

Evalution Research Instructional Instruction 165

Clark Quinn

APRIL 18, 2023

Instructional differences become more apparent when evaluated from the perspective of how well the learning transfers to new problems and settings. That’s where this quote comes from: Many approaches to instruction look equivalent when the only measure of learning is memory.

Pedagogy 232

Pedagogy PDF Instructional Instruction 232

Clark Quinn

AUGUST 12, 2020

A step-by-step process for evaluating your instructional content, providing you with an actionable blueprint on transforming your content. A step-by-step process for evaluating your instructional content, providing you with an actionable blueprint on transforming your content to be effective and engaging.

Market 339

Market Tips Cognitive Brain 339

Clark Quinn

OCTOBER 17, 2023

What we want is an impartial evaluation. Worse, the folks conducting the study had. Sorry, that’s not an unbiased observer. They have a vested interest in the outcome. This simply couldn’t be it. Not least, the author was the CEO of the platform. It got worse.

Research 219

Research Study Statistics Issue 219

Clark Quinn

NOVEMBER 9, 2023

We’ll work from analysis, through specification, and on to evaluation (we won’t talk much about implementation, because of my quip that getting the design right leaves lots of ways to create the solution, and not doing so renders everything else extraneous.

Trust 206

Trust Analysis Suite Evalution 206

Clark Quinn

NOVEMBER 14, 2023

Guy’s book is a way to avoid doing that, and systematically evaluating what the pivotal point should be for determining whether we can successfully intervene or not, and how. Quite simply, L&D has a problem of going from go-to-whoa without considering whether a course is the right solution.

Performance Support 204

Performance Support Problem Open Analysis 204

Clark Quinn

JUNE 28, 2022

Even your evaluation is different. Implementation is also more iterative. You’ll be investing slowly, to allow pivots and to keep the overall costs contained. Postponing programming and preferring paper are components of this. You are testing, now, not only effectiveness, but also the experience.

Design 300

Design Cognitive Analysis Integrate 300

Clark Quinn

NOVEMBER 23, 2016

You evaluate impact by learner feedback. You work from PDFs and PPTs to develop your course. You have a knowledge quiz at the end. You use libraries to add graphics & interactions. Your learners avoid the courses. You track completion.

eLearning Strategy 0

eLearning Strategy Strategy Design Behavior 0

Clark Quinn

SEPTEMBER 25, 2018

In it, a guy who’d done his PhD thesis on decision making talked about how when you evaluate the outputs of decisions, to determine whether the outcome was good, you needed values. And, of course, you analyze the decision afterward to evaluate the outcomes. Now this to me ties very closely back to the Sternberg model of wisdom.

Technology 151

Technology Evalution Guidelines Analysis 151

Clark Quinn

MAY 2, 2023

Which may be more accurate, in that if you need to attend to what’s in working memory, and manage the process of attending to information while evaluating what decision to make. They have it influencing most of the above. You must maintain conscious focus on what you want to learn. Which folks tout as less than that of a goldfish.

Attention 195

Attention Movie Picture Streaming 195

Clark Quinn

JUNE 14, 2022

It doesn’t have to be ‘adrenaline in the blood’ or even galvanic skin response, subjective evaluations by your learners is just fine. Test usability first, then effectiveness, and then engagement. Yes, you want to quantify engagement.

Cognitive 286

Cognitive Metrics Exercise Instructional Design 286

Clark Quinn

FEBRUARY 6, 2018

A vigilant evaluation of what’s presented is healthy for dialog and moving the industry forward! Worst case, you listen to those who carry the standard of rigor in evaluation. Then, I reckon we need to be rigorous in evaluating new approaches. The alternative, a cynic still looking for good, is less optimal.

Cognitive 116

Cognitive Ratio Alternatives Evalution 116

Clark Quinn

JUNE 13, 2023

Here I’m talking about things like the need for iteration and formative and summative evaluation, (proper) brainstorming, etc. Particularly the things that are our adaptions to the flaws in our cognitive architecture, but as learners and as designers.

Cognitive 152

Cognitive Skills Instructional Design Calendar 152

Clark Quinn

NOVEMBER 2, 2021

This should be coupled with providing development of writing good search strings and evaluating search outcomes. At the individual level we can be doing things like implementing federated search, to support individual learning. This also means curating a suite of resources aligned with learning directions and future opportunities.

Policies 266

Policies Search Digital Alternatives 266

Clark Quinn

MAY 24, 2022

Let’s evaluate the late adopter strategy. At the same time, there’s a discussion happening about learning affordances of the metaverse. I realize my strategy is the same, which I learned many years ago (wish I could remember from whom!). The short version is, wait until the dust settles.

Adoption 274

Adoption Adopt Strategy Cognitive 274

Clark Quinn

AUGUST 8, 2023

Further, reliable research to evaluate whether they have a measurable impact comes up saying ‘no’ So, despite fervent (and misguided) support, folks shouldn’t promote learning styles as a basis to adapt to. Here’s my take on why I say don’t use AI unsupervised.

Trust 149

Trust Social Networks LinkedIn Learning Styles 149

Clark Quinn

NOVEMBER 22, 2017

These still typically require human oversight to evaluate. An approach we used in the past was to have learners create a complex answer, but have the learner evaluate it! Of course, you’d want to pair this with an evaluation guide as well. Then we ask the learner to self evaluate against the rubric.

Assessment 120

Assessment Solution Cognitive Evalution 120

Clark Quinn

JUNE 22, 2020

There are also debates on games, evaluation, to e- or not to e-, at least. And, a reprise of the great debate on evaluating learning or impact (4PM PT 25 June). There are panels on, for instance, on diversity & inclusion, state of learning, learning technology, women in learning, as just a few. Lots of interesting topics.

Diversity Training 158

Diversity Training Asynchronous Cognitive Synchronous 158

Clark Quinn

AUGUST 19, 2020

Which means we have to evaluate the case for paying attention to research. In a perfect world, we’d spend all the time we want on learning. However, we don’t live in that world, we live in the real world. Which means our decisions are about tradeoffs. So here’s a stab at the case for learning science.

Learning 309

Learning Brain Research Cost 309

Clark Quinn

APRIL 12, 2022

Initially, we need to evaluate the reason. There are two issues, to me: what we should say, and how we should say it. For instance, I think when someone says something wrong, we need to educate. It could be that they don’t know any better. Or it could be that they’re deliberately trying to mislead. .

Behavior 259

Behavior Social Issue Suite 259

Clark Quinn

OCTOBER 19, 2016

Now, the model of self-regulated learning I was reading about talks about personal goals, learning actions, and evaluation. It seems to me that learning goals sit outside of SSS, the SSS serves as the actions, and then evaluation comes after the action. And of course, the evaluation feeds new goals. Two additional things.

PKM 0

PKM Evalution Search Personal 0

Clark Quinn

JUNE 2, 2015

And while these are desirable things, they’re hard to evaluate. The problem is that our technology to evaluate freeform text is difficult, let alone anything more complex. An alternative is to have the learner evaluate themselves. That is, they use a rubric to evaluate their response and the model response.

RFP 100

RFP Evalution Alternatives Asynchronous 100

Clark Quinn

NOVEMBER 13, 2018

And how to evaluate what you get. If we evaluate our learnings and apply the ones that are improvements, we’re innovating. Obviously, there’s more to it than just that, so it’s about seeking actively but also setting up systems to continually feed you new, potentially tangential thoughts. Nor do our organizations.

Skills 163

Skills Develop PKM Learning 163

Clark Quinn

OCTOBER 5, 2021

When I evaluate learning technologies, for instance, I look to see how well they do things like provide meaningful practice: active and contextualized. So these days, when I look at things, I try to look with a cognitive (ok, post-cognitive) perspective. I look to see how things align, or not, with how our brains work.

Cognitive 286

Cognitive Virtual World Brain Second Life 286

Clark Quinn

FEBRUARY 4, 2021

We want to diverge and converge, exploring ideas broadly before evaluating them. And this is where what’s known about creativity matters. I’ve written before about brainstorming, and in brief, there are things that work and things that don’t. And we need individuals to think on their own before sharing those ideas. .

Cognitive 244

Cognitive Analysis Agile Diversity Training 244

Clark Quinn

JULY 19, 2016

So, first of all, my focus was on formative evaluation. They are fans of Make It Stick (mentioned in my Deeper eLearning reading list ), so it was easy to help them recognize that good activities require learners retrieve the information in context , so each formative evaluation should be a situation requiring a decision.

Evalution 100

Evalution Format Learning Assessment 100

Clark Quinn

MAY 9, 2013

I don’t want to preclude essays or other interpretive work like presentations or media content, and they are likely to require human evaluation, even with peer marking. Writing evaluation rubrics is also a challenge for untrained designers or experts. accreditation) is going to continue to be a role for some time.

Assessment 184

Assessment Online Cognitive Simulations 184

Clark Quinn

MAY 30, 2012

Inspired by Dave Cormier’s learning contract , and previous work at learner-defined syllabi and assessment, I had a thought about learner-created project evaluation rubrics. I’d like to start having the learners propose evaluation criteria for rubrics.

Assessment 189

Assessment Evalution Analysis Project 189

Clark Quinn

DECEMBER 1, 2020

I’d like to go through them and evaluate each. However, the trigger for the post was from the results of Chief Learning Officer 2020 State of Learning report. And, when I saw them, I saw some problems. The question is whether we’re measuring impact, or not. Back to my usual prose, as I need visual support for this. ;).

Metrics 260

Metrics ROI ASTD Cost 260

Clark Quinn

JULY 20, 2021

Let’s include the ability to balance a bankbook, to project manage, to critically evaluate claims, to communicate visually, and the like. Our curriculum is a holdover from what was stipulated at the founding of this country. Let’s get a curriculum that’s looking forward, not back.

Project 209

Project Pedagogy Open Source Lecture 209

Clark Quinn

SEPTEMBER 11, 2018

Looking at your processes and evaluating them. An early exercise where you evaluate your own network systematically, for instance, struck me as really insightful. I note that part of the meta-learning here is the ongoing monitoring of your own processes (the starred grey clouds).

Personal Learning 127

Personal Learning Personal Personalization Cognitive 127

Clark Quinn

JUNE 8, 2021

While focused on parsing the broader picture of societal claims (and social media hype), it is relevant to evaluating learning science as well. . It is a really well done read about some simple but useful rules for how to be a more careful reviewer of statistical claims. I hope you find my new recommended readings of interest and value.

Cognitive 210

Cognitive Statistics Social Media Predictions 210

Clark Quinn

JANUARY 28, 2020

We need to evaluate any claims by several methods. And more doesn’t equal ‘faster’ Yes, we’re processing in parallel, but nerve firing rates change based upon activation, not modality. And this means that we have to have our ‘hype’ shields up. Who else is saying this?

Images 174

Images Research Learning Styles Trust 174

Clark Quinn

OCTOBER 29, 2019

For instance, we can ask that the objective include elements of measurable evaluation, and carry that forward through the final practice design. Or we could structure practice as decision making. But we can have templates at higher levels, too. We can go beyond that, and have structures to guide doing good curricula design.

Templates 115

Templates Cognitive Pattern Quiz 115

Clark Quinn

APRIL 30, 2015

Depending on the ability of the learners, you may start with challenges that are like but not necessarily real challenges, such as evaluating a case study or working on a simulation. Similarly, the freedom to share work for evaluation is a valuable move on the path to a learning culture.

Activities 100

Activities Integrate Case Study Social Media 100