Certainty levels allow students to indicate how sure they are about their multiple choice answer during an assignment. This indication can be used to score the students' answers, encouraging them to indicate a lower certainty instead of guessing if they don't know the answer. Especially in the medical domain, where not being wrong can be more important than being right, we see this being applied to encourage students to gain insight into their understanding of the subject matter.
The potential benefits of using certainty levels are:
- Encourages students to reflect upon how sure they are of their answer
- Rewards students for answering honestly instead of guessing an answer
- Penalties students for confident wrong answers
- Provides students with more detailed insight into their grasp of the subject
Setting certainty levels
Certainty levels can be set on an assignment as a guess correction option using a template file. To set certainty levels on an assignment, follow the steps below.
- Navigate to your assignment.
- Click settings Settings in the menu at the top.
- Click Mark calculation in the menu on the left.
- In the Correction for guessing-section, click Upload. The upload will then upload as a background job.
- After a few moments, refresh your browser. The option Calculate by certainty level will be checked by default.
- If you navigate back to the assignment, you will see the certainty levels has been applied to your multiple choice questions.
lightbulb_outline If guess correction and certainty levels are both selected, the scoring of certainty levels will replace the effect of the standard guess correction.
Import file
Within the import file, you can determine several options per certainty level:
| Header | Value | Description |
| name | any | Displayed to participants during the assignment |
| certainty degree (certain - partially certain - uncertain | certain, partially certain, uncertain | determines the certainty level used for analysis |
| certainty (%) | 0% - 100% | used to calculate the average certainty |
| score when correct (%) | ≤100% | percentage of total points the participant scores for a correct answer with this certainty level |
| score when incorrect (%) | ≤100% | percentage of total points the participant scores for an incorrect answer with this certainty level |
Insights
Using certainty levels provides additional insights into the result of a participant for instructors. These insights can be found in the Certainty analysis tab of a result with certainty levels.
-
Capability levels
The distribution of six capability levels of a participant are shown in a pie chart. The chart shows the percentage of questions a student has answered correctly or incorrectly, broken down by whether they expected this outcome or not. For example, if a student answers a question correctly and with a high certainty, this is categorised as a 'Known capability'. If a student answers a question incorrectly with a high certainty, this is categorised as a 'Unknown incapability'. Instructors can choose to publish this chart to students in the publication settings. -
Centration
Centration is calculated by subtracting the percentage of correct answers of a participant from their average certainty. This results in a score from -100 to 100, where a score near zero indicates the participant made a good estimation of their own knowledge. A negative score generally indicates the student has underestimated their knowledge (the percentage of correct answers is higher than the average certainty of the participant). A positive score indicates an overestimation of knowledge. Centration cannot be published to students.
Examples score calculation
As explained before, Certainty levels can be used to score the student's answers. In this part, two examples are provided on the score calculations with certainty levels.
For both examples, an assignment is used that contains 5 multiple choice questions, with each 4 answer alternatives. The following certainty levels are used:
| name | certainty degree (certain - partially certain - uncertain) | certainty (%) | score when correct (%) | score when incorrect (%) |
| 0%-25% | uncertain | 0% | 50% | 10% |
| 25%-50% | uncertain | 25% | 80% | 0% |
| 50%-75% | partially certain | 50% | 90% | 0% |
| 75%-100% | certain | 75% | 100% | 0% |
Example 1
Student Alba Smitham has the following result:
- Question 1: Correct, with a chosen certainty of 75%-100%
- Question 2: Correct, with a chosen certainty of 75%-100%
- Question 3: Correct, with a chosen certainty of 75%-100%
- Question 4: Correct, with a chosen certainty of 75%-100%
- Question 5: Correct, with a chosen certainty of 75%-100%
For each question, there is a certainty of 75% and a score of 100% because all answers given are correct.
- Average certainty = sum of all selected certainties / number of answered questions = (75% + 75% + 75% + 75% + 75%) / 5 questions = 75%
- Centration = average certainty - percentage correct answers = 75% - 100% = -25%
Example 2
Student Claud Boyle has the following result:
- Question 1: Incorrect, with a chosen certainty of 0%-25%
- Certainty = 0%
- Question 2: Correct, with a chosen certainty of 50%-75%
- Certainty = 50%
- Question 3: Correct, with a chosen certainty of 50%-75%
- Certainty = 50%
- Question 4: Incorrect, with a chosen certainty of 75%-100%
- Certainty: 75%
- Question 5: Correct, with a chosen certainty of 25%-50%
- Certainty: 25%
3 out of 5 questions are correct, so the percentage correct answers is 60%.
- Average certainty = sum of all selected certainties / number of answered questions = (0% + 50% + 50% + 75% + 25%) / 5 questions = 40%
- Centration = average certainty - percentage correct answers = 40% - 60% = -20%
Comments
0 comments
Please sign in to leave a comment.