The peer-reviewed digital test

To create a peer-reviewed digital test, follow the steps below.

1. Click the domain School name in the menu on the left.
2. Click label_important Courses in the menu at the top.
3. Select your course or use the search bar. 
4. Click on New assignment and fill in the name of the test. If you use assignment templates, click the dropdown menu No template and select your template.
5. Select Peer-reviewed digital test and click Create.

You can now start adding exercises to your peer-reviewed digital test in the same way as you would for a digital test. If you want to create an assignment from within a question bank, you can select the option 'Digital test' when copying the question bank assignment to your course. Once the digital test is copied to your course, you have the option to change the Grading method. This is explained below.

After creating a peer-reviewed digital test, you will need to set up the peer review options. This is to confirm the settings for how the digital test will need to be reviewed.

To set up a peer review, follow the steps below.

1. Navigate to your assignment or use the search bar.
2. Click settings Settings in the menu at the top.
3. Click Peer review options in the menu on the left. 
4. Under Options for reviewing, fill in the number of reviews each participant has and click Update. 
5. Enable Grade anonymously if you prefer for the participants to grade each other anonymously.
6. Click New timeslot. Fill in the Start time* & Deadline* and click Save. Participants can then review their peer's work within this timeslot. 

Grading method

• Peer review
  If you have created a peer-reviewed digital test from the course overview, Peer review is the default setting. If you change the grading method at the dropdown from Peer review, the assignment type will no longer be a 'Peer-reviewed digital test'.
  
  
• Self review
  If you select Self review from the dropdown menu, the grading method is changed to self reviewed and the assignment type will be updated to a 'Self-reviewed digital test'. Participants will then be required to review their own assignment instead of having it reviewed by their peers. There are no additional settings for this selection. 
  
  
• Review
  If you select Review from the dropdown menu, the 'Peer-reviewed digital test' is updated to a standard 'Digital test'.

The dropdown menu is only visible if the participants have not yet started the assignment. After the first participants starts, it is no longer possible to adjust the review options. 

Options for reviewing

• Number of reviews 
  With this setting, you can determine how many reviews the participant can do. After they are finished with their own assignment, they will see the option to start reviewing. Based on the number confirmed in this setting, the platform will indicate how many peer reviews are required. The participants will also see additional slots for optional peer reviews. 
  
• Grade anonymously
  If this option is enabled, participant's names are not visible during grading. The option is enabled by default.
  

Timeslots

A timeslot needs to be set for participants to start reviewing their peers. The availability of the assignment is set for all participants. Click the button New timeslot. Fill in the Start time* & Deadline* and click Save.

To view the results of an assignment that has been peer reviewed follow the steps below. 

1. Navigate to your assignment or use the search bar. 
2. Click beenhere Results in the menu at the top.
3. Click on a result number to view the result of an individual student.

Results overview

In the results overview you will see the marks received by each participant and the corresponding number of peer reviews in parenthesis. This can be seen in the image below. The result is obtained by taking the average of the points awarded by each participant. For example, in the screenshot below "Learner 2" has a grade of 7.8 as a result of 3 reviews.

Furthermore, you will see the following indications at the results overview:

• Reliability
  Indicates to what extent the different gradings are similar to this result. The peer reliability compares the average deviation in points for open questions in reviews received for the given result across various peer reviews. It represents the percentage of similarity in the points awarded by different graders.
  
  
• Progress
  Indicates the progress of peer reviews conducted by participants.
  
  
• Consensus
  Indicates to what extent the grading given by this participant corresponds with the grading of other participants. The consensus is calculated as follows:
  
  (1 - ((absolute(points given for this result in this review - average points for this result excluding this review)) / average points for this result including this review)) * 100 

Result per participant

To see how an individual participant was reviewed on each criteria or question, click on the result number of the participant at the result overview. By default, you will be directed to the the Exercises tab. Here, you can see how many points the participant received for each question from their peer. In the example below, the student received 10 points for Question 1 from one peer, 5 points from a second peer, and 7 points from a third peer.