Certainty levels allow students to indicate how sure they are about their multiple choice answer during an assignment. This indication can be used to score the students' answers, encouraging them to indicate a lower certainty instead of guessing if they don't know the answer. Especially in the medical domain, where not being wrong can be more important than being right, we see this being applied to encourage students to gain insight into their understanding of the subject matter.
The potential benefits of using certainty levels are:
- Encourages students to reflect upon how sure they are of their answer
- Rewards students for answering honestly instead of guessing an answer
- Penalties students for confident wrong answers
- Provides students with more detailed insight into their grasp of the subject
Setting certainty levels
Setting certainty levels
lightbulb_outline If guess correction and certainty levels are both selected, the scoring of certainty levels will replace the effect of the standard guess correction.
Certainty levels can be set on an assignment as a guess correction option using a template file. To set certainty levels on an assignment, follow the steps below.
- Navigate to the Digital test tab of your assignment.
- Click settings Digital test settings in the menu at the top.
- Scroll down to Certainty degrees
- Click plus to add a new level and fill-in the name to be shown to participants
- Indicate the certainty used to calculate the average certainty
- Indicate the score students should get in case the answer is correct or incorrect. You can set this as a negative
- Click update to save the levels and refresh the page to see these in the preview
error_outline If a participant does not indicate a certainty level with a question, the question will be regarded as 'unanswered' and the student will receive 0 points.
Insights
Insights
Using certainty levels provides additional insights into the result of a participant for instructors. These insights can be found in the Certainty analysis tab of a result with certainty levels.
-
Capability levels
The distribution of six capability levels of a participant are shown in a pie chart. The chart shows the percentage of questions a student has answered correctly or incorrectly, broken down by whether they expected this outcome or not. For example, if a student answers a question correctly and with a high certainty, this is categorised as a 'Known capability'. If a student answers a question incorrectly with a high certainty, this is categorised as a 'Unknown incapability'. Instructors can choose to publish this chart to students in the publication settings. -
Centration
Centration is calculated by subtracting the percentage of correct answers of a participant from their average certainty. This results in a score from -100 to 100, where a score near zero indicates the participant made a good estimation of their own knowledge. A negative score generally indicates the student has underestimated their knowledge (the percentage of correct answers is higher than the average certainty of the participant). A positive score indicates an overestimation of knowledge. Centration cannot be published to students.
Examples score calculation
Examples score calculation
As explained before, Certainty levels can be used to score the student's answers. In this part, two examples are provided on the score calculations with certainty levels.
For both examples, an assignment is used that contains 5 multiple choice questions, with each 4 answer alternatives. The following certainty levels are used:
| name | certainty degree (certain - partially certain - uncertain) | certainty (%) | score when correct (%) | score when incorrect (%) |
| 0%-25% | uncertain | 0% | 50% | 10% |
| 25%-50% | uncertain | 25% | 80% | 0% |
| 50%-75% | partially certain | 50% | 90% | 0% |
| 75%-100% | certain | 75% | 100% | 0% |
Example 1
Student Alba Smitham has the following result:
- Question 1: Correct, with a chosen certainty of 75%-100%
- Question 2: Correct, with a chosen certainty of 75%-100%
- Question 3: Correct, with a chosen certainty of 75%-100%
- Question 4: Correct, with a chosen certainty of 75%-100%
- Question 5: Correct, with a chosen certainty of 75%-100%
For each question, there is a certainty of 75% and a score of 100% because all answers given are correct.
- Average certainty = sum of all selected certainties / number of answered questions = (75% + 75% + 75% + 75% + 75%) / 5 questions = 75%
- Centration = average certainty - percentage correct answers = 75% - 100% = -25%
Example 2
Student Claud Boyle has the following result:
- Question 1: Incorrect, with a chosen certainty of 0%-25%
- Certainty = 0%
- Question 2: Correct, with a chosen certainty of 50%-75%
- Certainty = 50%
- Question 3: Correct, with a chosen certainty of 50%-75%
- Certainty = 50%
- Question 4: Incorrect, with a chosen certainty of 75%-100%
- Certainty: 75%
- Question 5: Correct, with a chosen certainty of 25%-50%
- Certainty: 25%
3 out of 5 questions are correct, so the percentage correct answers is 60%.
- Average certainty = sum of all selected certainties / number of answered questions = (0% + 50% + 50% + 75% + 25%) / 5 questions = 40%
- Centration = average certainty - percentage correct answers = 40% - 60% = -20%
Comments
0 comments
Please sign in to leave a comment.