The automatic scoring option is intended to be used for questions with binary (correct/incorrect) answer options where multiple answers can be selected. Among these question types are the multiple response, match and order questions. Instead of defining the points for each option you only need to declare if it is correct or incorrect. The score for the student is calculated based on the fraction of correct options selected and the fraction of incorrect options selected. The rule offers a standardised way for correcting the possibility that students receive a good score simply by selecting all options. If a student is certain of an answer, it is more beneficial with this rule to select that single answer rather than selecting all options. We apply the product rule to multiple response question, the product rule is defined as:
*Score = proportion correct * (1 - proportion incorrect * deduction factor)
proportion correct = number of correct options selected / number of correct options
proportion incorrect = number of incorrect options selected / number of incorrect options
deduction factor = (1/a + (((N-C)/N)*(1-1/a)))
proportion incorrect = number of incorrect options selected / number of incorrect options
deduction factor = (1/a + (((N-C)/N)*(1-1/a)))
N = total number of options
C = number of correct options
a = weight factor (set standard to 2.0)
C = number of correct options
a = weight factor (set standard to 2.0)
The product rule is an alternative to using negative points for answer options. Consider for comparison a question using negative points with two correct and two incorrect options. You could give each correct option 1 point and each incorrect option minus 1 point. If a student selects one correct option he/she will score half of the maximum points. If he/she guesses or selects all options the student will score zero points on average.
The product rule will, in this scenario, result in half the points if one correct option is selected, but will still give a quarter of the points if all options are selected. This way the product rule will make the scoring for this question equivalent to guessing in the case of a multiple choice question with one correct option and three incorrect options scoring zero points.
For multiple response and match questions the numbers of correct and incorrect options are clear. For an order question we decided that each ordered pair counts both as a correct option as well as an incorrect option. With 5 items to order, you will have 10 order pairs. This means N is 20 and C is 10 for the product rule formula.
The automatic scoring option is also available for the hotspot match and fill-in question types, however the scoring for these question types does not use the above formula. This is because the formula is not appropriate because you can not decide to do the equivalent of 'selecting all options' with these question types. Therefore, there is no logic in using the automatic scoring in this case. Instead the points will be awarded evenly for every possible correct answer. For example, if 2 of the 5 matches are correct, you will score 40% on that question if the the automatic scoring option is enabled.
*E. Terwisscha van Scheltinga (2009). Van quotiëntregel naar productregel. EXAMENS, 1, 26-27.
Comments
0 comments
Please sign in to leave a comment.