By viewing the insights of questions, you can see the Pvalue, Ritvalue, Rirvalue, p', Rk and Guess score values of each question. You can also see the quality of the question and if it is Good, Easy, Difficult or Needs attention. If needed, you have the option to change the contributions of a question to Bonus, Full points or Disabled.
To view insights into the questions of your assignment, follow the steps below.
 Click the domain School name in the menu on the left.
 Click label_importantCourses in the menu at the top.
 Select your course or use the search bar.
 Select your assignment or use the search bar.
 Click analytics Insights in the menu at the top.
 Click Questions in the menu on the left.
From top to bottom, the following elements are included in the analysis:
 Changing the question contribution
 Question analytics
 Group comparison
 Multiple choice or grading scheme breakdown
Changing the question contribution
The option to change the question contribution is on the top of each question insight. In the dropdown menu, four options are shown:
 Contributes: This is the standard option; the question contributes to the grades of the students.
 Bonus: The question will not be taken into account in the maximum score that can be gained, but it can be a bonus for the students who answer correctly.
 Full points: Every student is awarded the maximum amount of points possible for this question.
 Disabled: The question and the points awarded for the question will be taken out of the assignment and will not be taken into account for the grading.
If you select another contribution type in the dropdown menu, the consequences of this edit will be displayed. You can check the consequences and click Confirm to edit the contribution.
Question analytics
In the questions insights, each question is analysed individually. As mentioned, the following seven elements are calculated by Ans:
Pvalue, Ritvalue, Rirvalue, p', Rk , Guess score and Advice on question quality
The visibility is applicable depending on the question type:
Guess correction possible for question  No guess correction possible for question  
Advice on question quality  Yes  Yes 
Pvalue  Yes  Yes 
Ritvalue  Yes (if not randomly generated)  Yes (if not randomly generated) 
Rirvalue  Yes (if not randomly generated)  Yes (if not randomly generated) 
p'  Yes  No 
Rk  Yes  No 
Guess score  Yes  No 
The meaning of these values are as follows:

Pvalue: Rate of difficulty
The pvalue represents the difficulty of the question. It shows how many students correctly answered the question on a scale from 0 to 1. The ideal value depends on the goal of the assignment. For example, if the assignment is the only assignment in the course, the aim could be to have a pvalue of 0.8. A lower pvalue is acceptable as well, for example in case it's a formative assignment which is part of a series of assignments. The pvalue is calculated as follows:
p = [number of students that answered this question correctly] / [total number of students]

Ritvalue: Relation of the question to all questions in the test
The Ritvalue indicates how well the question fits the test; the higher, the better.
A value greater than 0,40 is very good.
A value between 0,300,39 is good, but the question still has room for some improvement.
A value between 0,200,29 is a doubt.
Any value lower than 0,19 indicates that reconstruction of the question should be considered.
The equation for how Rit calculation is as follows:
Yg = average total score of students that answered correctly
Yf = average total score of students that answered incorrectly
Sy = standard deviation of the total score
Sy = standard deviation
q = (1p)

Rirvalue: Relation of the question to all other questions (so all questions except this one)
This value measures the same as the Ritvalue, but the Rirvalue is more exact, while it is not compared to itself.
A value higher than 0,40 is very good.
A value between 0,300,39 is good, but the question still has room for some improvement.
A value between 0,200,29 is a doubt.
Any value lower than 0,19 indicates that reconstruction of the question should be considered.
The equation is given below.

p': Rate of difficulty corrected for the guess correction
The corrected pvalue (p'value) represents the difficulty of the question as well, however, it corrects the value for the statistical possibility of guessing the right answer. The value is scaled from 1 to 1. If the p'value is close to 1, the question was too easy and it didn't separate students based on their performance. For extreme low p'values, the question was likely to be difficult. The p'value is calculated as follows:
p' = p  [ (1p) / (number of options for this multiple choice question  1) ]

Rk: Corrected guess correction
Rk is the corrected guess effect, where the choices from others are taken into consideration in order to judge such information more accurately.
In a question with, for example, 4 choices and 1 correct answer, there's a 1 in 4–or 25%–chance the answer can be guessed correctly. This is translated into a guess score (see below). The Rk value also takes the actual results of the students into account. In the example of 4 possible answer options, there are 3 incorrect answers (distractors). Ideally, the distribution of incorrect answers of all students is evenly divided over the distractors (roughly). The Rk value takes this distribution into account. The Rk value is calculated as follows:
 In the example to calculate the Rk value, we use a multiple choice question with 1 correct answer out of 4 possible answers. The following distribution is applicable:
 Answer A (correct answer): 40 answers
 Answer B (distractor): 22 answers
 Answer C (distractor): 3 answers
 Answer D (distractor): 14 answers
 The total amount of answers: 79
 Calculate the number of incorrect answers: 22+3+14 = 39
 Determine the ideal (evenly) distribution of the incorrect answers per distractor: 39 divided over 3 distractors is 13 answers per distractor
 Determine per distractor the amount of students that need to be 'moved' to get the ideal distribution: (2213) + (1413) = 9 + 1 = 10 answers need to be 'moved'.
 The ration 10 / 39 on three distractors is (10/39 * 3) = 0,77 alternatives.
 The amount of correct alternatives is 4  0,77 = 3,23 alternatives
 The Rk value is 1 / 3,23 = 0,31 (or 31%). The original guess correction was 25%. The closer the Rk value and the guess score are, the better the ideal distribution of the distractors.
 In the example to calculate the Rk value, we use a multiple choice question with 1 correct answer out of 4 possible answers. The following distribution is applicable:

Guess score: The score that can be statistically scored by guessing the answer
For all closedended questions where the answer can be guessed, a guess score is calculated. The overview of applicable question types and the calculation of the guess score for different situations can be found in the guess correction article.
Conclusion of Pvalue, Ritvalue, Rirvalue and p'
For every question, based on the p and Ritvalues, one of the following conclusions is made:
 Good: All values are within the appropriate ranges.
 Easy: The pvalues is above 0.9.
 Difficult: The pvalue is below 0.1.
 Needs attention: The Ritvalues is below 0.2.
Grading scheme breakdown
For each question, the breakdown of the answers of the students or from the review of the teacher is given. The representation depends on the question type and chosen grading method. All possible combinations are shown below.
Question type  Grading scheme breakdown 
Fillin question 
Ans shows for each unique answer given the number of students that gave that answer. Additionally, it is possible to approve additional answer alternatives via the grading scheme option. 
Hotspot Hotspot match Order Numerical 
No breakdown is shown. 
Match  For each possible combination of rows and columns, Ans shows the number of students that gave that answer combination. 
Mathematical equation 
Automatic grading: No breakdown is shown. Manual grading: Per criterion, the number of students that received the points is shown 
Multiple choice Statement 
For each answer alternative and for the 'no answer chosen', Ans shows the number of students that selected the answer. Also, for the distractors (the incorrect answers), the avalue is shown. The avalue represents the amount of students that selected the distractor, on a scale of 0 to 1. 
Code Editor Drawing File upload Open question Worksheet 
Points per criterium: Per criterion, the number of students that received the points is shown. Slider: No breakdown is shown. Rubric: Per rubric level, the number of students that received the points of the level is shown. 
error_outline In a "randomly generated" assignment, not all participants will receive the same questions. The Rit and Rir values indicate a correlation between the questions and the final result of an assignment. Because not every participant is given the same questions, this correlation cannot be calculated.
Comments
0 comments
Article is closed for comments.