Order question

To create a an order question, follow the steps below.

1. Navigate to your question bank or your course assignment. 
2. Click New exercise, fill in a name and click Create.
3. Click New question and select Order.
4. Formulate the question in the question content field.
5. Fill in the number of points the question is worth.
6. Add the alternatives by clicking on + Add alternative. The order in which you add your alternatives is the same order the participants will be required to rank the alternatives. Students will be shown the alternatives in a random order. If preferred, you can also click on the push_pin-icon. This allows you to keep the position of the alternative. To delete the alternative, click the delete-icon.
7. Optionally, by toggling on Edit feedback it is possible to provide feedback on alternative level. This feedback will only be visible during the publication.   
8. Click the Orientation dropdown menu and select Horizontal or Vertical. 
9. Optionally, click the dropdown menu QUESTION FEEDBACK to:
   • Add a model answer. 
   • Show feedback on answer level. Feedback on answer level can be made visible during the publication or during the taking of a digital test if the option 'Feedback' is enabled in the Accessibility settings. This is explained further here. You can choose to fill in feedback that is shown to students when:
     • The question is answered correctly.
     • The question is answered partially correctly (only when the 'Custom' tab is selected).
     • The question is answered incorrectly.
10. Optionally, add learning objectives in the 'Objectives' field.
11. Click Create.

For additional options, click the settings-icon. You will see the following:

• Bonus
  This marks the question as a bonus question, which means that its points are not included in the maximum amount of points achievable for that assignment. If a student answers the question incorrectly, this will not be detracted from the number of points and the mark. However, when awarding points, it is possible to achieve more points that the maximum points achievable for the assignment.
   
• Partial scoring
  Participants will receive points for partially correct answers. For example, if the total number of points for a question is 2 and their answer is half correct, they will receive 1 point. Without this option, they would only be able to receive either 0 or the maximum number of points.
  Enabling partial scoring can increase the applied guess correction. Since students can earn points for partially correct answers instead of an all-or-nothing approach, they may be more likely to guess additional answers. This article provides a more detailed explanation of how the guess correction is calculated. 
   

• Use automatic scoring
  When using automatic scoring to grade an order question, the product rule is applied to calculate the points achieved. Every element ordered in the learner's answer is compared to the subsequent elements to determine whether that pair is correctly placed compared to each other. This way, every pair in the answer is checked.

  The ratio of the number of correct pairs to the total possible pairs is calculated. For regular grading, this ratio is multiplied by the total available points for the question. 
   

• Limit minimum to zero points
  With this option enabled, it’s impossible to go below zero points when selecting the different criteria. It is still possible to manually adjust the number of points to go below zero with the adjustment option during the reviewing.
   
• Limit maximum to total points
  In the grading settings it’s possible to set the maximum amount of points achievable with the question, separate from the points per criteria When this number deviates from the maximum amount achievable by adding up the criteria points, this option will cap the maximum amount of points to this manually set number.

All options mentioned below for the grading scheme can also be added or adjusted from the exercise tab. However, if the assignment has been taken, it is only possible to adjust the grading scheme from the grading scheme tab.

1. Click Grading scheme tab at the top of the assignment or question bank exercise.
2. Navigate to the question for which you want to set the grading scheme.
3. Click the more_horiz-icon and click Edit. 
4. Fill in or adjust the amount of points the question is worth. 
5. Optionally, add or adjust the model answer. 
6. Adjust or delete an alternative, click the delete-icon. If you need to add an alternative, you will need to add it via the Exercise tab.
7. Optionally, by toggling on Edit feedback it is possible to provide feedback on alternative level. This feedback will only be visible during the publication. toggle the Edit feedback button, directly below the scoring methods to provide feedback on alternative level. This feedback will only be visible during the publication.  
8. Optionally, click the dropdown menu QUESTION FEEDBACK to:
   • Add a model answer. 
   • Show feedback on answer level. Feedback on answer level can be made visible during the publication or during the taking of a digital test if the option 'Feedback' is enabled in the Accessibility settings. This is explained further here. You can choose to fill in feedback that is shown to students when:
     • The question is answered correctly.
     • The question is answered partially correctly (only when the 'Custom' tab is selected).
     • The question is answered incorrectly.
9. Optionally, add or adjust the learning objectives in the 'Objectives' field.
10. Click Update.

If you have not used the product rule option ('Use automatic scoring'), an order question will be graded as follows:  

We determine the number of correct ‘pairs’. For example, 4 options for the following pairs are used:

1-2 ; 1-3 ; 1-4 ; 2-3 ; 2-4 ; 3-4

We compare this with the student's answer and if for example, the student identified 4 correct pairs, the student receives a score of 4/6 * the number of points.

So, in general: grade = (number of correct pairs set by the student / total number of correct pairs) * number of points set for this question.