Confidence-based learning

Confidence-Based Learning (CBL) is a methodology used in learning and training that accurately measures a learner's knowledge quality by determining both the correctness of the learner's knowledge and his or her confidence in that knowledge. Additionally, the patented CBL process is designed to increase retention and eliminate the contaminating effects of guessing which often skew the results of traditional, single-score assessments. This combination yields a precise profile of the individual's knowledge base and identifies the difference between what the individual thinks he or she knows and what that individual actually does know.

Once knowledge and confidence gaps have been identified, the CBL system creates a customized learning program for each learner to fix these gaps. Then CBL re-evaluates the learner's knowledge quality. The process, similar to quality improvement processes such as Six Sigma, is continued until the learner achieves total mastery of the knowledge they need for a particular skill. The CBL system defines mastery as the validated achievement of confidence and correctness for 100% of the content. This means that a learner must answer a question with confidence and correctness two consecutive times. Mastery then becomes confidently-held, correct knowledge put into practice.

In CBL, learning is not about a score; it is about determining what a person needs to be taught and then providing the learner with the information necessary to achieve mastery of the content. CBL builds the learner's knowledge quality through multiple sessions instead of simply presenting the content and then assessing a learner's knowledge through one, single-score assessment.

CBL can stand on its own as a discrete and self-contained educational activity. However, it also is a flexible and versatile learning platform that can be implemented and blended with other training modalities. When learners achieve mastery prior to live training (i.e. instructor-led program, mentoring/coaching session), they are able to focus on the areas where learning gaps were identified.

History

CBL is a culmination of more than 70 years of academic, commercial, and governmental research into the connection between confidence, correctness, retention, and learning. The first academic paper on the subject was written in 1932 and asserted that measuring confidence and knowledge was a better predictor of performance than measuring knowledge alone, which can be prone to guesswork.

Extensive research and technological advances ultimately led to the commercial development of CBL.

Research and conceptual development

The framework for Knowledge Factor's CBL system is based primarily around the research of Drs. Darwin Hunt, Dieudonne LeClerq, Emir Shuford, and James E. Bruno. Significant advances in the knowledge and confidence connection were made by all of four researchers, but Bruno brought their collective work together in a methodology that made it possible for knowledge and confidence to be effectively measured.

Fundamental research that helped form the framework for CBL includes:

• Dr. Darwin Hunt – New Mexico State University

Hunt conducted significant research with the US Navy and focused on the dimensions of knowledge, linking confidence and correctness with retention. His process involved a two-step approach – (1) answer the question (objective measurement of correctness), and then state your confidence in your answer (subjective confidence statement). According to Hunt, research shows that the retention of newly learned material is systematically related to "how sure" people are about the correctness of their answers when they learn it.

Hunt, in an article titled, "The Concept of Knowledge and How to Measure It," stated that "Knowledge has many dimensions and the importance of confidence in one's knowledge state cannot be overlooked."

• Dr. Dieudonne LeClerq – University of Belgium

LeClerq focused on item bank testing - the process of using a pool of questions, from which questions are drawn and randomly delivered to learners to see how well they answer questions without pattern recognition or order influencing the process. The result is higher quality knowledge and information. The outcome was a report on the quality of information/knowledge that shows where misinformation exists.

• Dr. Emir Shuford – University of California at Los Angeles (UCLA)

Shuford developed a measurement algorithm that focused on the determinations of the reliability of someone's knowledge and how a learners' knowledge reliability was improved or diminished based on their level of doubt or confidence in it.

• Dr. James E. Bruno – University of California at Los Angeles (UCLA)

Bruno, a Professor of Education at UCLA, began his work on understanding the connection between knowledge, confidence, retention, and the quality of knowledge while consulting for the North Atlantic Treaty Organization (NATO) and the RAND Corporation in the 1960s. Bruno eventually turned his focus to the K–12 market and packaged his developments into practical application for the K–12 educational market in the early 1990s.

Bruno's research on the linkage between knowledge, confidence, and behavior led to the intuitive conclusion that knowledge alone is necessary but not sufficient to create behavior. Rather, it is the fusion of knowledge and confidence that leads to behavioral outcomes and empowers people to act. People who are confidently correct will take actions that are productive. The reverse is also true in that individuals who are confident about misinformation will take action, with consistently negative (and potentially dangerous) results.

This measurement of knowledge quality was initially called Information Reference Testing (IRT). The IRT process uses a unique two-dimensional assessment process in which a single answer for each question generates two metrics simultaneously – correctness and confidence.

Numerous research studies in the 1980s and 1990s established a link between correctness and confidence, but most of these testing approaches first asked learners to state what they believed to be the correct answer to a question, then asked them to state their confidence in the answer just selected. Even though these approaches help us understand the level of confidence a learner has in his or her answers, research shows that when learning and confidence measurements are two discreet actions, learners have a tendency to overstate their confidence level. Confidence, in this setting, is a logical response to a question, not a spontaneous measurement of true emotion.

While developing the IRT methodology, Bruno pointed out something that other researchers failed to address. When evaluating the accuracy of testing and assessment outcomes, one needs to take into account the level of honesty in each learner's answers. If a person is provided with several answer choices and does not know the answer, that person's only option is to guess the correct answer. If he or she happens to guess the answer correctly, that person is given credit for knowledge he or she does not have.

Bruno tested his IRT process extensively among students, including successes with juvenile offenders in the LA County Corrections System who had dropped out of mainstream society because they were unable to adapt to, and succeed in, a traditional learning environment. He found that with IRT, these juveniles were able to achieve academic success while enjoying the process. Bruno's focus on this audience led to the game-like simplicity of the IRT process, which encouraged honesty in the evaluation and learning process in a safe environment where the emphasis was on knowledge quality rather than a score. Bruno ultimately was able to develop a simple methodology for measuring knowledge quality that became the basis for CBL.

Commercial development

Bruno's work met with considerable success and was highlighted in a national symposium where its potential as a commercial training methodology was first identified. Knowledge Factor, Inc. acquired the intellectual rights to the IRT process in 2002 and, working in conjunction with Bruno, further developed the process. This ultimately led to contextually smart learning, for which a patent was awarded in 2005. At that time, the name of the IRT process was changed to Confidence-Based Learning (CBL).

How Confidence-Based Learning works

CBL is deployed to learners through an online, Internet-based application that uses iterative learning sessions to measure a learner's knowledge and confidence and then provides targeted feedback to close learning gaps.

CBL learning process

CBL begins the learning process by asking the learner a set of questions and then filling knowledge gaps with critical content, whereas most traditional online learning approaches deliver content first and then test to validate each learner's understanding of the content. The CBL approach is similar to the centuries-old Socratic Method, which despite an abundance of advanced technologies and approaches to delivering training, has proven to be one of the most effective methodologies ever developed when it comes to ensuring that learning takes place.

By combining a knowledge diagnostic and prescriptive learning into one process, the CBL System offers a contextually-smart learning environment that is akin to a rigorous quality improvement process with a focus on learning.

1. Diagnose – The CBL System starts by diagnosing the true knowledge of learners (i.e., what they actually know vs. what they think they know.) This diagnosis is achieved through CBL's patented answer selection process which determines the knowledge quality for each learner and his or her confidence in that knowledge. This knowledge quality is then categorized by objective into one of four knowledge quadrants in the Learning Behavior Model. The two top quadrants are likely to result in action because the learner's confidence in the knowledge is high. The two bottom quadrants are likely to result in action because the learner’s confidence in the information is low.
2. Prescribe – After the learner's knowledge gaps are identified through the diagnostic session, the CBL system immediately presents an individualized learning plan for the learner in the CBL Learning Center. The Learning Center provides a visual representation of the learner's results in the following order: areas in which the learner is misinformed, uninformed, contains doubt, and has achieved mastery. The prescriptive plan includes feedback on the learner's performance and the learning content needed to fill knowledge gaps.
3. Learn – Once the individualized learning plan is provided to the learner, he or she can begin filling knowledge gaps. The personalized learning program enables the learner to click on the highest priority items first (e.g., areas where he or she held misinformation). As the learner reviews each categorized question, he or she discovers the correct answer, his or her answer, and the explanation as to why the correct answer is correct and why the incorrect answers are wrong. The explanation contains the critical information necessary for the learner to obtain mastery of the content. It also contains key information explaining the risks associated with the incorrect knowledge and the consequences of taking the wrong action.

Continue... The CBL system uses iterative learning sessions to present information to the learner; therefore learners will continue the diagnose-prescribe-learn process until all knowledge gaps are closed. CBL's individualized approach narrows down the content as the learner demonstrates mastery. This eliminates the need for users to spend time learning what they have already mastered and makes the learning process more efficient. Because the CBL learning process is based on an individual's knowledge and confidence, the number of learning sessions needed to achieve mastery for a module will vary per learner. However, CBL's innovative process enables the learner to quickly learn and retain the material with confidence.

Learning behavior model knowledge quadrants

The knowledge quadrants in the Learning Behavior Model and their associated learner behaviors are as follows:

• Misinformation—knowledge a learner confidently believes to be correct, but which is actually incorrect. Those who have confidence in wrong information (misinformation) will very likely make mistakes on the job, which puts companies at the most risk.
• Mastery—knowledge a learner knows confidently that is correct, and which will likely be applied correctly in practice. Learners who have correct knowledge and a high degree of confidence in their knowledge (mastery) are masters of that knowledge domain. These learners are likely to act and act correctly, resulting in higher performing and more productive learners who make fewer mistakes.
• Doubt—knowledge a learner believes to be correct, but an element of doubt exists that may cause the learner not to act on that knowledge. Someone who harbors doubt may be correct on a certification, but is likely to act with hesitation or not act at all.
• Uninformed—knowledge that a learner has not acquired yet. Someone who is uninformed is unlikely to act, which can result in a state of paralysis.