Skip to main content

Analyse the alignment of reviewers

Mike Rietveld
  • Updated

Analysing the alignment of reviewers can provide different insights in regards to an assignment after it has been graded. Using this feature achieves fair and transparent grading.

To analyse the alignment of reviewers, follow the steps below.

  1. Click on your course or use the search bar in the Courses-section. 
  2. Click on your assignment or use the search bar. 
  3. Click Insights in the menu at the top.
  4. Click Alignment in the menu on the left.


On this page you can gain insight into:

  • The alignment of the grading by different reviewers
  • The number of results
  • The number of minutes spent reviewing the question.
  • The average number of points given

Grading alignment

The grading alignment is calculated as follows:

((average_question_score - grader_average) / points).abs > DEVIATION_THRESHOLD

We use a DEVIATION_THRESHOLD of 0.2

So for example:
average_question_score = 1.0
grader_average for a reviewer is 1.5
points = 5
(1.0 - 1.5) / 5 = 0.1 which is smaller than 0.2 so there is no deviation detected.

Related articles

Comments

0 comments

Please sign in to leave a comment.