Start reviewing

On the left side of the review overview, the amount of results (number of participants) and the amount of exercises included in the assignment is displayed. If you are an instructor of the course, you will see the button how_to_reg. If you only have a 'reviewer' role in the course, this button will not be visible. If you click on this button, you can assign reviewers to help distribute the workload. Alternatively assigning reviewers can be done from the settings Settings menu at the top. 

At the top of the page, you can make use of the three filters. The 'Grading progress' filter allows you to filter result on 'All questions', 'Graded questions' or 'Ungraded questions'. If you prefer to review based on group(s) or class(es), you can use of these two filters. If you begin reviewing when a filter is active, Ans will only show submissions of the participants included in the filtered grading progress and group(s) and/or class(es). 

On the right side of the review overview, you will see the Review button and a dropdown menu. The dropdown menu allows you to choose how you would like to review the submissions of the participants.

Review per result

To view a submission of a participant as a whole, you can choose to review per result by clicking the Review button or Continue reviewing button. 

To view the grading progress of all participants, you can hover over the purple grading progress bar.

Review per exercise

It is also possible to review a submission per exercise. By doing so, you will review one exercise for all participants before navigating to the next exercise. The same options three filters apply for Review per exercise as they are for Review per result. In addition, you have the possibility to view either all subquestions or a selection of subquestions. 

To view the grading progress of an exercise, you can hover over the purple grading progress bar.

It is important to note, that multiple reviewers cannot grade the same submission. Once a reviewer is grading a submission, it is automatically locked for other reviewers. If multiple reviewers are grading at the same time, Ans automatically distributes them over different submissions. However, there is an exception when another user accesses a student's answer through the student's result overview or a shared link. This exception is necessary for situations where teachers need to discuss a result together.

When you begin reviewing submissions, you will be directed to the following screen. Please note that the screen differs based on the question type, assignment type and the chosen grading scheme.

When reviewing a question, the scroll will focus on the given answer. This could mean, for longer questions, you may need to scroll up to view the entire question. The reasoning behind scrolling to the answer is that as a teacher/grader you probably know the question, and want to focus on reading the students reply. The question you are reviewing is shown on the rights, so if you don't know the question, you can check on the right side of the screen.

When reviewing a question, the view will automatically scroll to the given answer. For longer questions, this might require you to scroll up to see the entire question. The reasoning behind this approach is that, as a teacher or grader, you're likely already familiar with the question and want to focus on the student’s response. The exercise name and question are also displayed on the right-hand side (above the grading scheme) of the screen for easy reference.

Top menu

• The arrow_back-icon: Clicking on this will direct you back to the Review overview.
• The assignment name: The assignment name is displayed.
• beenhere # 942690: This indicates the result number. This is not the same as the student number.
• The keyboard_arrow_left x/y keyboard_arrow_right-icon: switch between questions or participants (depending if you chose to Review per result or Review per exercise). When viewing the answers of an individual result, you can easily move between all the attempts using the arrows. For peer-reviewed digital tests, you can navigate between all the peer reviews from a specific result.
• Review:
  • Grading scheme: Here, you can grade the participant's answer or make adjustments if needed.
  • Flags: Add, edit or delete flags. Flags are often used in cases where you need a second opinion from another reviewer on how to grade the participant's answer. A comment can be added to the flag.
  • Discussion: If this option is enabled, participants can start a discussion. The messages of the participant of this question will be displayed. 
• Annotations: You can add annotations to the answer of a participant.
• The search -icon: By using the search function, you can  find previously viewed results. Results can be filtered according to grading progress, such as fully graded, partially graded, or not graded. In instances where grading is not conducted anonymously, a search for a specific student is also possible. 
• The settings-icon: This option allows you to edit the grading settings. The grading settings are personal settings and do not apply for other graders. The following grading settings can be enabled:
  • Display exercise
  • Display question
  • Display model answer
  • Display objectives

lightbulb_outline The results are sorted first by the date they were created and then randomised within each day. Suppose there are 3 results: 1 and 2 started yesterday and 3 started today, 1 and 2 are randomly ordered and 3 will always be the third.

lightbulb_outline For open questions in digital tests, it is possible to navigate through the saved answers by using the navigation buttons of the timestamp just below the question.

The reviewing menu differs per grading scheme. For open-ended questions, there are three possible options:

• Points per criterion
• Rubrics with levels
• Slider

Adjustments can be made to to the grading types points per criterion and rubrics with levels. This is explained below.

Points per criterion

Points per criterion is the default grading type. All criteria are uncoloured (grey) when reviewing begins, indicating an unsaved state. If a criterion has been met, click the applicable criterion, and it will turn green. Your changes will be automatically saved. If no criteria is met, click None of the above.

When grading with points per criterion, you can make use of the following shortcuts:

• 1, 2, 3, ... : (un)toggle criteria
• Spacebar: (un)toggle all criteria
• 0, § (Apple), ~ (Windows): (un)toggle "None of the above"
  
  

Rubrics with levels

When using the grading type 'rubrics with levels,' you can define levels (e.g., insufficient, sufficient, good), include descriptions, and assign points to each level. You can select one level per criterion when grading. This is the key difference between points per criterion and rubrics with levels, as with points per criterion, you can select multiple (or all) criteria.

When grading with rubrics with levels, you can make use of the following shortcuts:

• 1, 2, 3, ... : to select a level
  
  

Slider

With the grading type 'slider,' you can either manually enter the number of points or use the slider to assign points. To include decimals, use a period to separate the whole number from the fractional part.

When grading with slider, you can make use of the following shortcuts:

• 1, 2, 3, ... : to fill in the points
  
  

Adjustments to the grading scheme

If you need to deviate from the grading scheme, you can use the option to adjust question points to either award or deduct points for a question. Adjustments to question points are possible for all question types, provided the 'Adjustments to question points' option is enabled in the Review Options tab within your assignment settings.

When making adjustments, you must first select the number of points before applying the adjustment. The adjustment is then applied afterward. This calculation method is designed to provide maximum flexibility, allowing students to be awarded extra points that exceed the question's original point value.

This flexibility enables you to give students a point for their effort, even if they scored only on negative criteria, or to reward exceptionally good answers with points above the maximum.

To make adjustments to question points, follow the steps below.

1. Navigate to the question you want to review/make adjustments.
2. You will first need to select the amount of points before you can add an adjustment.
3. Click the tab +Adjustment.
4. Fill in the adjustment (either positive or negative), add a comment and click Save.
5. To make another adjustment to the points for that question, click the tab +Adjustment.
6. Fill in the amount of points, add a comment and click Save.

A useful way to leave feedback for open questions is by using annotations. When adding annotations, you can highlight text in a student's answer, and a tooltip will appear. From there, you can add a comment, change the highlight colour, or delete the highlight.

To make use of annotations, follow the steps below.

Add a comment

A helpful way to provide feedback for participants while grading is to leave a comment under the awarded points. Adding comments is possible for all open-ended question types.

To add a comment, follow the steps below.

1. Navigate to the question you want to grade and leave a comment on.
2. Grade the question accordingly and then click +Comments. The exact position of this button depends on the chosen grading type.
3. A text editor will open where you can add your comment.
4. Fill in the comment or select a Quick comment.
5. Click Save.

Use comment suggestions

To save time while reviewing, you have the option to reuse the same feedback that you gave to a previous participant. When you have added a comment to a criterion and you are giving a comment to a different participant for the same criterion, Ans will show you the three most used comments you have given for that specific criterion. Your suggested comments will be displayed under the text editor and can be used by clicking on them.

Use quick comments

If you often use the same comments, you can save them as quick comments. This way you just need to click on quick comments and select the comment you want to use. In every place where you can add comments, it is possible to save comments for later use or reuse previously given comments.

In order to do so, click on the + Comment button. In the screen that pops up, you can click on Quick comments. You can show and choose an existing quick comment by clicking Show quick comments. It's also possible to store text as quick comment by clicking Save as quick comment. A comment always has a name and the content of the comment:

lightbulb_outline You can also type a hashtag (#) with the name of a saved quick comment in the comment field. Ans will directly show the comment after pressing enter. 

After participants have submitted an assignment and the results have been published, they have the option to start a discussion (if enabled with the publication of the assignment). More information about using flags can be found in this article.

Reviewers can flag answers if they are uncertain about the number of points to award the participant for a specific question. Instructors can then take a look at the flagged answer to give a decisive answer. More information about using flags can be found in this article.