In the review overview, there is the option to review the students for a second time. In the points and mark calculation you will see a range indicating the reviews given by the first and second reviewer. Upon publishing the results students will also see the scores given by both reviewers. However, when exporting, only the grade obtained during the first review will be shown.
Some benefits of a second review include:
- Review each student twice
- Discover grade differences
- Insights in agreement
To enable this setting, follow the steps below.
- Click the domain School name in the menu on the left.
- Click on label_importantCourses in the menu at the top.
- Click on your course or use the search bar.
- Click on your assignment or use the search bar.
- Click settings Settings in the menu at the top.
- Click Review options in the menu on the left.
- Enable Second review.
- Click content_paste_search Review in the menu at the top.
- Click the drop down menu First review and select Second review.
- You can then start a second review.
When a reviewer grades an exercise, they are not able to grade this exercise during the second review. Only (department) administrators can do this.
If you use the second review, Ans will show an additional option in the Insights menu: Reviewer reliability. In this menu, a matrix of all reviewers is shown. Per combination of reviewers, Ans shows the reliability in percentages. Ans compares the scores given by a reviewer and these are compared with the scores of each other reviewer. When the percentage is 100%, both reviewers gave the same scores in all cases. When the percentage is 0%, none of the given scores was the same.