Analyse questions

To learn how to analyse questions, you can follow the steps below.

Here, you will find Analysis on your Exercises.

On this page, each question is analysed individually. There are five elements pertaining information on the questions:

p, p', Rit, Rir, and Rk

All of these values are calculated for multiple choice questions, while the p-, Rit-, and Rir-values are calculated for open questions.

The meaning of these values are as follows:

• p: Rate of difficulty
  The p-value is the percentage of students that correctly answered the question. The higher the value, the easier the question.

p = [number of students that answered this question correctly] / [total number of students]

• p': Rate of difficulty corrected for the guess correction
  The p'-value is the percentage of students that know the question. This value ranges between a minimum of 0,25 and a maximum of 0,9. If the value equals 0,5, then it can be said that a maximum discrimination between strong and weak students is found.

p' = p - [ (1-p) / (number of options for this multiple choice question - 1) ]

• Rit: Relation of the question to all questions in the test
  The Rit-value indicates how well the question fits the test; the higher, the better.
  A value greater than 0,40 is very good.
  A value between 0,30-0,39 is good, but the question still has room for some improvement.
  A value between 0,20-0,29 is a doubt.
  Any value lower than 0,19 indicates that reconstruction of the question should be considered.
  The equation for how Rit calculation is as follows:

Yg = average total score of students that answered correctly
Yf = average total score of students that answered incorrectly
Sy = standard deviation of the total score
Sy = standard deviation
p = p - max points awarded for the question
q = (1-p)

• Rir: Relation of the question to all other questions (so all questions except this one)
  This value measures the same as the Rit-value, but the Rir-value is more exact, while it is not compared to itself.
  A value higher than 0,40 is very good.
  A value between 0,30-0,39 is good, but the question still has room for some improvement. 
  A value between 0,20-0,29 is a doubt.
  Any value lower than 0,19 indicates that reconstruction of the question should be considered.
  The equation is given below.

• Rk: Corrected guess effect
  In a question with, for example, 4 choices and 1 correct answer, there's a 1 in 4–or 25%–chance the answer can be guessed correctly. If the question is worth, for example, 1 full point, then the student is awarded a score of 0,25 points for guessing correctly. This does not inherently accurately reflect the student's grasp on the knowledge required.
  Rk is the corrected guess effect, where, within the score calculation, the choices from others are taken into consideration in order to judge such information more accurately.

For every question, based on the p-, p'-, Rit-, Rir- and Rk-values, one of the following conclusions is made:

• Good: All values are within the appropriate ranges.
• Easy: The p- and p'-values are probably too high.
• Difficult: The p- and p'-values are probably too low.
• Needs work: The Rit- and/or Rir-values can be too low (< 0.30). The question is not representative for the assignment.

Aside from these elements that provide insight to the questions, there is also an option for the contribution status for each question. This option is in the form of a drop-down menu.
In the menu, four options are shown:

1. Contributes: This is the standard option; the question contributes to the grades of the students.
2. Bonus: The question will not be taken into account in the maximum score that can be gained, but it can be a bonus for the students answer correctly.
3. Full points: Every student is awarded the maximum amount of points possible for this question.
4. Disabled: The question and the points awarded for the question will be taken out of the exam and will not be taken into account for the grading.

If you select another contribution-type in the drop-down menu, the consequences of this edit will be displayed. You can check the consequences and click Confirm to edit the contribution.

error_outline In a "randomly generated exam" not all students will get the same exercises. The Rit and Rir values indicate a correlation between the exercises and the final result of a test. Because not everyone is given the same exercises, this correlation cannot be calculated.