Analysing the alignment of reviewers can provide different insights in regards to an assignment after it has been graded. Using this feature achieves fair and transparent grading.
To analyse the alignment of reviewers, follow the steps below.
- Click on your course or use the search bar in the Courses-section.
- Click on your assignment or use the search bar.
- Click analytics Insights in the menu at the top.
- Click Alignment in the menu on the left.
On this page you can gain insight into:
- The alignment of the grading by different reviewers
- The number of results
- The number of minutes spent reviewing the question.
- The average number of points given
The grading alignment is calculated as follows:
((average_question_score - grader_average) / points).abs > DEVIATION_THRESHOLD
We use a DEVIATION_THRESHOLD of 0.2
So for example:
average_question_score = 1.0
grader_average for a reviewer is 1.5
points = 5
(1.0 - 1.5) / 5 = 0.1 which is smaller than 0.2 so there is no deviation detected.