Do you think it’s valuable to know your learners’ level of confidence in their knowledge? I say a hearty, “yes.” In fact, confidence that you genuinely know something is requisite for true behaviour change. Assessment 2.0 is the next generation of assessments. It includes confidence as a weight in calculating scores. Read on to learn how confidence-based assessment works and why it’s so powerful.
Assessment 2.0 works on two principles: certainty-based scoring and confidence checking (also known as confidence-based assessment). The concept has been around for some time and is championed by Nic Price, Lead Learning Designer in our Sheffield office.
How does certainty-based scoring work?
Certainty-based scoring is a mechanism by which you ask learners knowledge-based questions and how sure they are that their answers are correct. You get more points if you’re correct and feel more confident in your answers. This separates out learners who genuinely know something and those who genuinely don’t. Adding in certainty-based scoring makes your assessments more valid and it’s quite powerful.
Here’s an example:
Let’s say I ask you what the capital of France is. You might say, “well, it’s Paris and I’m willing to bet quite a few points on that answer because I’m quite certain I’m right.” Now if I ask you what the capital of Australia is, you might say, “Ah, I think it’s Canberra. But maybe it’s Sydney. I’m not willing to bet as many points on that answer.” In this scenario, you’ve been asked to reflect on your certainty in how much you genuinely know the material.
Now imagine this in an elearning or online assessment. You can’t score 100% by simply answering the questions correctly; you also have to be certain. You can get all the answers correct, but if you weren’t sure on some of them, you get a lower score. The inverse is true. If you are incorrect but were convinced you were right, your score will be lower.
In this way, you really do separate out people who genuinely know from those who genuinely don’t.
What is confidence-based assessment?
Confidence checking is the other principle behind Assessment 2.0. As L&D professionals, we are always striving to change behaviours. That’s the goal, right?
We have seen that behaviour change happens when there is confidence. We see this especially in sales performance or any sort of triage performance in the medical sector.
Checking confidence provides new insight into capability and metacognition (knowing what you know and knowing what you don’t).
Consider this graph that looks at the relationship between confidence and competence.
A new look at assessing learners
Learners who are highly confident and highly competent are your star performers. These are the people who are selling the right things and selling more than the rest of the team. These are the people who save lives and perform well in high-stakes situations.
Learners who are highly confident but not competent are dangerous. These are the people who sell the wrong things, things that potentially your company cannot deliver on. These are the people who might mistake the person with superficial bleeding as the one who needs more immediate care than the triage patient who is completely unconscious having a stroke.
Now let’s look at the learners in the bottom two quadrants. These learners have low confidence and varying levels of competence. These are the people who you can generally reach through some performance support or learning intervention. You can increase their confidence and competence, and you can measure it over time. These learners should be moving towards the right and towards the top as they learn more and practice more.
Confidence-based assessments provide great learning data
Even a simple application of a certainty-based scoring system should be able to give you a much better idea of how much people know.
The data can be reported into any appropriate LMS, and it’s most useful if you can take two separate data streams out of a module and collate them.
xAPI tracking – the most powerful way of monitoring and analysing data – can also be applied to confidence-based assessments.
When you use it, you should expect learner confidence to rise over time if you use the correct questions. However, there have been some exceptions to this in cultures where people are prone to adopting a lack of confidence (for example, a culture that values modesty),even in instances of high competence.
Applying confidence-based assessments
Misplaced confidence can have damaging effects in any sector but more so for high-consequence industries. L&D teams want to reward learners who are genuinely competent, as well as people who are realistic in their evaluation of their knowledge.
Certainty-based scoring is an effective solution to this challenge, and can be tailored to suit your development targets.