Automatic scoring is an option which can be enabled for a question where students can select multiple answers. The option allows you to select which answers are correct, instead of providing scores per answer alternative. The score for the student is calculated based on the fraction of correct options selected and the fraction of incorrect options selected. The automatic scoring option can only be applied to the following question types:

- Match question
- Multiple choice question
- Order question
- Fill-in question
- Hotspot match question

For all question types mentioned above, an example of automatic scoring is provided at the bottom of the article.

To enable automatic scoring, follow the steps below.

- Click the
*domain***School name**in the menu on the left. - Click
*label_important***Courses**in the menu at the top. - Select your course or use the search bar.
- Select your assignment or use the search bar.
- Navigate to the question that you want to change the question type of.
- Click the
*more_horiz***-icon**and click**Edit**. - Click the
*more_vert***-icon**and click**Use automatic scoring**. - Click
**Save**.

*lightbulb_outline* It is possible that the automatic scoring option is enabled as default by your administrator for all new questions that are created. In that case, you don't have to enable the option yourself.

## Advantages of using automatic scoring

The automatic scoring option prevents a student from receiving points for a question when the student selects all possible answers. In case automatic scoring is enabled, Ans also takes the fraction of incorrect answers selected into account to calculate the score for a student. In case you don't want to provide full points when a student selects all possible answers, you can use the automatic scoring option to prevent this from happening. If automatic scoring is enabled, the score for a student that selects all answers for a question where multiple answers can be selected should be (approximately) equal to the guess score of a question.

## Differences between enabling and disabling automatic scoring

For questions without the automatic scoring option enabled, it is possible to determine the number of points per answer alternative. For questions with automatic scoring enabled, you can only select which answers are correct and set the total amount of points for the question. In the figures below, you can see the difference for a multiple choice question answer alternative.

Automatic scoring disabled: enter the number of points to be earned for the answer alternative.

Automatic scoring enabled: select whether an answer alternative is correct or not.

## Score calculation when the automatic scoring is enabled

To calculate the score Ans uses a formula. The formula uses the total amount of correct and incorrect options. For the multiplechoice, match and order question types, it's possible to determine the amount of correct and incorrect options. For these question types, the formula is used.

For the fill-in and the hotspot match question type, the amount of incorrect options is not known. Therefore, Ans applies the automatic scoring option in a different way for the fill-in and the hotspot match question type.

## Calculate the score with the formula

As mentioned above, the formula is used for the multiple choice, match and order question types. Ans uses a formula defined by E. Terwisscha van Scheltinga (2009)*.

The formula is as follows:

- Score = (proportion correct * (1 - proportion incorrect * deduction factor))*points for the question

Where:

- proportion correct = number of correct options selected / number of correct options
- proportion incorrect = number of incorrect options selected / number of incorrect options
- deduction factor = 1/a + ((N-C)/N)*(1-1/a)
- N = total number of options
- C = number of correct options
- a = weight factor (set standard to 2.0)

## Other applications of the automatic scoring option

For the fill-in and the hotspot match question types, automatic scoring is applied in a different way. The total amount of incorrect options are not known for these question types. Because this is not known, the result of the formula cannot be calculated.

#### Fill-in question

For the fill-in question type, Ans evenly distributes the number of points over the different gaps. In case the question is worth 2 points and there are 2 gaps in the question, both gaps are worth 1 point.

#### Hotspot match question

For the hotspot match question type, Ans awards the points evenly for every marker that is placed correctly. In case the question is worth 4 points and has 4 markers, a student will receive 1 point per marker which is placed correctly.

For both question types, an extended example can be found below.

*E. Terwisscha van Scheltinga (2009). Van quotiÃ«ntregel naar productregel. EXAMENS, 1, 26-27.

## Example: Multiple choice question

Automatic scoring can be applied to multiple choice questions with multiple correct answers. If you use automatic scoring when only one answer is correct, the student will automatically receive full points for the selected answer.

In this example, a multiple choice question is used with four answer alternatives in total. Two answer alternatives are correct, and two alternatives are incorrect. The total amount of points to be earned for the question is 2. In the example shown below, the student needs to select which cities are the capital cities of their respective countries. London and Amsterdam are correct, while Munich and Milan are incorrect.

## Calculation of the score if automatic scoring is disabled

In case automatic scoring is disabled, you enter the number of points a student earns when selecting an answer. In this example, London and Amsterdam are worth 1 point and Munich and Milan are worth 0 points. In case a student selects all four answer possibilities, Ans only takes the number of correct answers into consideration for the score calculation.

#### Example situation A:

When a student selects all 4 answers, the student did also select the 2 correct answers. Therefore, the student receives the maximum score of 2 points, despite having selected 2 incorrect answers as well.

#### Example situation B:

When a student selects 2 correct answers and 1 incorrect answer, the same applies. A student will receive 2 points, as both correct answers are selected and the incorrect answer is not taken into consideration for calculating the score.

## Calculation of the score if automatic scoring is enabled

#### Example situation A:

When all 4 answers are selected, the following applies:

- proportion correct = 1 (2 correct options selected / 2 correct options)
- proportion incorrect = 1 (2 incorrect options selected / 2 incorrect options)
- deduction factor = 0,75 because:
- 1/a+((N-C)/N)*(1-1/a) where:
- N = 4 (total number of options)
- C = 2 (number of correct options)
- a = 2 (standard weight factor)

- 1/a+((N-C)/N)*(1-1/a) where:
- points for question = 2
- This results in:
- Score = (proportion correct * (1 - proportion incorrect * deduction factor))*points for question
- Score = (1 * (1-1*0,75))*2
- Score = 0,5

#### Example situation B:

When all 2 correct answers and 1 incorrect answer is selected, the following applies:

- proportion correct = 1 (2 correct options selected / 2 correct options)
- proportion incorrect = 0,5 (1 incorrect options selected / 2 incorrect options)
- deduction factor = 0,75 because:
- 1/a+((N-C)/N)*(1-1/a) where:
- N = 4 (total number of options)
- C = 2 (number of correct options)
- a = 2 (standard weight factor)

- 1/a+((N-C)/N)*(1-1/a) where:
- Points for question = 2
- This results in:
- Score = (proportion correct * (1 - proportion incorrect * deduction factor))*points for question
- Score = (1 * (1-0,5*0,75))*2
- Score = 1,25

## Example: Order question

For an order question type, the automatic scoring option can be enabled as well. To calculate the score for an order question, Ans checks the amount of correct 'pairs' of options. A correct pair is considered as an answer alternative that have been placed correctly compared to the other alternative of the pair. We have provided an example with four answer alternatives. The student needs to put the Dutch cities in the correct order, from North to South. The correct order is: Groningen - Utrecht - Eindhoven - Maastricht.

In total, there are 12 pairs that can be made. 6 out of 12 are correct and 6 out of 12 are incorrect. The overview has been placed in the table below.

Correct pairs |
Incorrect pairs |

Groningen - Utrecht | Maastricht - Eindhoven |

Groningen - Eindhoven | Maastricht - Utrecht |

Groningen - Maastricht | Maastricht - Groningen |

Utrecht - Eindhoven | Eindhoven - Utrecht |

Utrecht - Maastricht | Eindhoven - Groningen |

Eindhoven - Maastricht | Utrecht - Groningen |

## Calculation of the score in case automatic scoring is disabled

If automatic scoring is disabled, Ans will only consider the correct pairs when calculating the score. For simplicity, the total points available have been set to 3. With 6 correct pairs, this means a student can earn 0.5 points for each correct pair.

#### Example

For this example, we assume that the answer of the student is:

- Eindhoven
- Maastricht
- Groningen
- Utrecht

In this scenario, the student has correctly matched 2 out of the 6 possible pairs. The pairs Eindhoven - Maastricht and Groningen - Utrecht have been placed correctly. As a result, the student receives 1 point in total (0.5 points per correct pair).

## Calculation of the score in case automatic scoring is enabled

For the calculation of the score with automatic scoring, Ans takes the amount of correct and incorrect alternatives into account. For the order question, an alternative equal a pair. For the formula, some details are different compared to others. If we use the same example as above, the answer of the student is:

- Eindhoven
- Maastricht
- Groningen
- Utrecht

Using the automatic scoring formula, this answer results in:

- proportion correct = 0,33 (2 correct option selected / 6 correct options)
- Eindhoven-Maastricht and Groningen-Utrecht are selected

- proportion incorrect = 0,66 (4 incorrect option selected / 6 incorrect options)
- deduction factor = 0,75 because:
- 1/a+((N-C)/N)*(1-1/a) where:
- N = 12 (total number of options)
- C = 6 (number of correct options)
- a = 2 (standard weight factor)

- 1/a+((N-C)/N)*(1-1/a) where:
- points for question = 3
- This results in:
- Score = (proportion correct * (1 - proportion incorrect * deduction factor))*points for question
- Score = (0,33 * (1-0,66*0,75))*3
- Score = 0,50

## Example: Fill-in & hotspot match question

For the fill-in and the hotspot match question, the automatic scoring rule can be applied. However, for these question types the number of incorrect answers cannot be calculated. Therefore, the formula as used in the other question types cannot be applied. Instead, the automatic scoring option will make sure that points will be awarded evenly for every possible correct answer.

## Fill-in question

For the fill-in question, the automatic scoring option will take a look at the amount of gaps that has been inserted in the question. In the example below, the student needs to fill-in three capital cities of Dutch provinces. The correct answers from top to bottom are: Groningen, Maastricht and Haarlem.

When the automatic scoring option is enabled, Ans will check how many gaps have been answered correctly. In case 2 out of 3 answers are answered correctly, the student receives 2/3 of the number of points (in this case 2 points). For the fill-in question type, it is possible to enter multiple answers that are correct. If all answers that are correct for a gap have an equal amount of points, automatic scoring can be enabled. However, if you for example would like to give half of the points for one of the answer alternatives, the automatic scoring option cannot be used. In this case, you need to disable the automatic scoring option and insert the number of points per answer alternative per gap.

## Hotspot match question

For the hotspot match question, the number of incorrect answers cannot be determined as well. Therefore, the calculation in case automatic scoring is enabled is similar to the fill-in question. Ans checks all correctly placed markers and distributes the scores evenly. As an example, we use a question where the student needs to place three markers on the cities of Groningen, Maastricht and Haarlem on a map of the Netherlands.

If a student places 2 out of 3 markers correctly, the student will receive 2/3 of the number of points for the questions (in this case 2).

## Comments

0 comments

Article is closed for comments.