Service Incident - 16th of July 2025 - Maximum points incorrect after changing question points in the grading scheme

All times in the document are recorded in UTC+2 (CEST).

Summary
Two users reported that they found assignments where the maximum points per result differed, when it should not have. This was forwarded to the Technical team on July 4th 9:17, after which it was escalated to a priority 1 incident by the Technical team at 12:05 of the same day.

We discovered that when a question’s points were changed from zero to a non-zero value, results for submissions that earned zero points on that question were not recalculated. As a result, both the total mark and maximum score remained based on the old value until the mark was recalculated. Any action performed that triggered the recalculation of a result, such as changing the mark formula, points, or question contribution would update the points and mark to the correct values.

The incorrect maximum points issue only applies when the question’s points were changed after the assignment had been taken. In such cases, some results may have an incorrect maximum score and total mark, specifically for submissions where the student received zero points on the affected question.

As a result, participants may have received an incorrect mark.

A hotfix was deployed on July 10th, 2025, ensuring that marks are recalculated whenever a question’s point value changes from zero to non-zero. This guarantees that the maximum points are always recalculated upon changing a question’s points, resulting in a newly calculated grade, even if a question was previously answered incorrectly.

Lead-up
On June 26th 13:37, a user reported that they had found strange behaviour with fill-in questions, where changing the questions’ maximum points would not update some marks of students. On June 27th 14:26, the Support team responded that this would be investigated by them. On June 30th 10:34, the Support team confirmed that there was an issue which was classified as a prio-3 and a task to fix the behavior was created.
On July 3rd 14:48, another user reported that they found an assignment where the maximum points per result differed, when it should not have. This was forwarded to the Technical team on July 4th 9:17, after which it was escalated to a priority 1 incident by the Technical team at 12:05.

Fault
It was discovered that this issue had always existed and would occur under the following conditions:
 

1. A question is worth zero points
2. A participant answers the question in a way that awards them no points. For example, a multiple choice question where an incorrect answer alternative is selected or an open question where none of the criteria that award points are selected.
3. The question is updated to have non-zero points

This would result in the mark not being recalculated with the new maximum points, when it should have.


Impact
An impact analysis was performed from July 14th 2025 to July 15th 2023 in order to determine the affected results. This was scoped to results created within the past 730 days (2 years).

The investigation resulted in 154 affected assignments, with 3785 impacted results.

Detection
The issue was first reported by users via two different support tickets, on June 26th 13:37 and July 3rd 14:48. Upon review, the support team escalated the problem to the technical team for further investigation.

Response
On July 4th 9:17, the Support team forwarded the ticket to the Technical team and informed both users that further investigation was required. The Support team assured the users that updates would be provided as more information became available.

Recovery
A hotfix was deployed on July 10th, 2025, ensuring that marks are recalculated whenever a question’s point value changes from zero to non-zero. This guarantees that the maximum points are always recalculated upon changing a question’s points, resulting in a newly calculated grade, even if a question was previously answered incorrectly.

Additionally, we sent a list of impacted results, including the difference in points and potentially marks. The mark would only be impacted when the mark calculation “formula” was chosen. This is because marking with table uses earned points only, which were not impacted.

Timeline
26th of June, 2025

• 13:37 - Support team receives a ticket indicating that there appears to be a discrepancy relating to the maximum points when switching a question’s points from zero to non-zero.

27th of June, 2025

• 14:26 - Support team responds to the customer explaining that their request has been received and it will be investigated.

30th of June, 2025

• 10:34 - Support team verifies that the report is an issue on our end and a priority 3 task is created to solve the issue.

3rd of July, 2025

• 14:48 - Support receives another ticket indicating that there appears to be a discrepancy relating to the maximum points that can be earned within an assignment.

4th of July, 2025

• 09:17 - Support forwards the information to the technical team and the technical team begins their investigation.
• 12:05 - Technical team categorises the problem as a priority 1 issue.

10th of July, 2025

• 11:28 - Technical team deploys a hotfix for the issue, to prevent it from occurring again.

15th of July, 2025

• 10:52 - Technical team informs support with an overview of the issue alongside the impact analysis of all assignments from the past two years.

Reflection
This issue was caused by overly complicated logic in the handling of changing points to zero and from zero. This has now been improved upon, including the writing of more automated tests. We also plan to improve the handling of zero points even further in a future release. With the January 5th release, we removed the option to limit the amount of points for an open question to zero or its amount of points. This is not handled well at the moment when assigning zero points to a question, as awarding a criterion with 1 point, does not change the score to 1, but keeps it at 0.

The issue was not picked up with the correct priority upon receiving the first ticket. This has been discussed internally and the Technical team will be approached more promptly, to ensure that the priority is more accurate in the future.

Developing the hotfix took a week, which was longer than we anticipated, however given the importance of calculating the grades accurately, we took the time to thoroughly test the changes.

The steps taken in the handling of this incident were followed in procedural order. The cause was found effectively and internal communication throughout the process was strong.