Step 2 - Questions insights

In the previous step, you learnt how to view the overview insights of your digital test. In this step, you will learn how to view the Questions insights. 

To gain insights into the questions of your digital test, click on analytics Insights from within your digital test. You will then be brought to the Insight overview page. Click Questions in the box on the left, in which you'll find the following:

Contributes

Below every question, you will see the dropdown menu Contributes. This allows you to change the contributes of the question to Bonus, Full points or Disabled. Before selecting a contribute, you can preview how it will effect the overall insights of that question/exercise.

Contributes
This is the default option; the question contributes to the grades of the students.

Bonus
The question will not be taken into account in the maximum score that can be gained, but it will count as a bonus for the students that answered correctly.

Full points
Every student is awarded the maximum amount of points possible for this question.

Disabled
The question and the points awarded for the question will be taken out of the exam and will not be taken into account for the grading.

Analysis of the Questions

Quality of the question
The first section of the box will display the quality of the question. The quality shown is either Good, Easy, Difficult or Needs attention. If you click the dropdown menu at the top of the page, you have the option to filter the questions by quality.

• Good: All values are within the appropriate ranges.
• Easy: The p- and p'-values are probably too high.
• Difficult: The p- and p'-values are probably too low.
• Needs attention: The Rit-value- and/or Rir-values can be too low (< 0.30). The question is not representative for the assignment.

Next to the quality of the question, there are six elements relating information on the questions:

P-value, Rit-value, Rir-value, p', Rk , and Guess score

P-value: Rate of difficulty
The p-value represents the difficulty of the question. It shows how many students correctly answered the question on a scale from 0 to 1. The ideal value depends on the goal of the assignment. For example, if the assignment is the only assignment in the course, the aim could be to have a p-value of 0.8. A lower p-value is acceptable as well, for example in case it's a formative assignment which is part of a series of assignments. The p-value is calculated as follows:

p = [number of students that answered this question correctly] / [total number of students]

Rit-value: Relation of the question to all questions in the test
The Rit-value indicates how well the question fits the test; the higher, the better.
A value greater than 0,40 is very good.
A value between 0,30-0,39 is good, but the question still has room for some improvement.
A value between 0,20-0,29 is a doubt.
Any value lower than 0,19 indicates that reconstruction of the question should be considered.
The equation for how Rit calculation is as follows:

Yg = average total score of students that answered correctly
Yf = average total score of students that answered incorrectly
Sy = standard deviation of the total score
Sy = standard deviation
q = (1-p)

Rir-value: Relation of the question to all other questions (so all questions except this one)
This value measures the same as the Rit-value, but the Rir-value is more exact, while it is not compared to itself.
A value higher than 0,40 is very good.
A value between 0,30-0,39 is good, but the question still has room for some improvement. 
A value between 0,20-0,29 is a doubt.
Any value lower than 0,19 indicates that reconstruction of the question should be considered.
The equation is given below.

p': Rate of difficulty corrected for the guess correction
The corrected p-value (p'-value) represents the difficulty of the question as well, however it corrects the value for statistical possibility of guessing the right answer. The value is scaled from -1 to 1. If the p'-value is close to 1, the question was too easy and it didn't separate students based on their performance. For extreme low p'-values, the question was likely to be difficult. The p'-value is calculated as follows: 

p' = p - [ (1-p) / (number of options for this multiple choice question - 1) ]

Rk: Corrected guess correction
Rk is the corrected guess effect, where the choices from others are taken into consideration in order to judge such information more accurately.
In a question with, for example, 4 choices and 1 correct answer, there's a 1 in 4–or 25%–chance the answer can be guessed correctly. This is translated into a guess score (see below). The Rk value, does also takes the actual results of the students into account. In the example of 4 possible answer options, there are 3 incorrect answers (distractors). Ideally, the distribution of incorrect answers of all students is evenly divided over the distractors (roughly). The Rk value takes this distribution into account. The Rk value is calculated as follows:

• • In the example to calculate the Rk value, we use a multiple choice question with 1 correct answer out of 4 possible answers. The following distribution is applicable:
    • Answer A (correct answer): 40 answers
    • Answer B (distractor): 22 answers
    • Answer C (distractor): 3 answers
    • Answer D (distractor): 14 answers
    • Total amount of answers: 79
  • Calculate the amount of incorrect answers: 22+3+14 = 39
  • Determine the ideal (evenly) distribution of the incorrect answers per distractor: 39 divided over 3 distractors is 13 answers per distractor
  • Determine per distractor the amount of students that need to be 'moved' to get the ideal distribution: (22-13) + (14-13) = 9 + 1 = 10 answers need to be 'moved'.
  • The ration 10 / 39 on three distractors is (10/39 * 3) = 0,77 alternatives.
  • The amount of correct alternatives is 4 - 0,77 = 3,23 alternatives
  • The Rk value is 1 / 3,23 = 0,31 (or 31%). The original guess correction was 25%. The closer the Rk value and the guess score are, the better the ideal distribution of the distractors.

Guess score: The score that can be statistically scored by guessing the answer
For all closed ended questions where the answer can be guessed, a guess score is calculated. The overview of applicable question types and the calculation of the guess score for different situations can be found in the guess correction article. 

Duration: The duration indicates the average time students spent on the question.

You can read in further detail about the question values in this article.

Next step: Step 3- Objectives Insights