Service Incident - 9th of September 2025 - Incorrectly updated point adjustments

All times in the document are recorded in UTC+2 (CEST). 

Summary
During internal testing, an issue was discovered regarding point adjustments to submissions, where editing these adjustments to assign a different number of points would add the new value to the previous adjustment instead. Further details, including examples can be found in the "Fault" section below. 

This issue originated from the most recent production release (17th of August 2025), in which the grading scheme interface was updated.

The technical team was made aware of the issue on September 3rd and a fix to prevent the issue from occurring was deployed on September 5th. An impact analysis began on September 4th and completed by September 8th.

With this issue, applying changes to an existing adjustment would add those changes to the previous version of the adjustment, rather than updating the adjustment altogether. 

This would result in an incorrect score for the submission that the adjustment was made to, and with that, the result.

Lead-up
During our continuous internal testing, an issue was discovered with updating adjustments. This was communicated to the technical team on September 3rd.

In the release of 17th of August 2025, a change was made to the grading of results. With this change, the rubrics and criteria grading were updated. This included changes to the adjustments that were not intended to change the behaviour, but did. 

This unintended change in behaviour led to the issue described in more detail in the “Fault” section below.

Fault
With the issue present, editing an adjustment to assign a different number of points adds the new value on top of the previous adjustment instead of replacing it.

To better comprehend the issue, please see the examples below:

Example 1:

• Submission is initially scored at 0 points.
• An adjustment of +2 points is added.
• The adjustment is edited to +1 point instead of +2.
• Resulting score: 3 points (0 + 2 + 1).
   

Example 2:

• Submission is initially scored at 1 point.
• An adjustment of −1 point is applied.
• The adjustment is edited to −2 points instead of −1.
• Resulting score: −2 points (1 - 1 - 2)

Impact
An impact analysis was performed to find submissions affected by this issue. This analysis was scoped on adjustments that were updated after the release of August 17th, 2025. The results from this analysis are as follows:

• A total of 74 submissions were affected among 7 institutions

Detection
The issue was detected during internal quality assurance continuous testing of the platform. This was communicated to the technical team in the morning of September 3rd.

Response
This was escalated to a higher priority issue later that day, after which work on the fix was started immediately. The fix was completely finished (written, tested, reviewed and deployed) in the afternoon of September 5th.

An impact analysis was started in the morning of September 4th and this was concluded in the afternoon of September 8th.

Recovery
We fixed this issue by identifying the problematic code that produced it and rewriting the affected logic. Now, when an adjustment is created or updated, the submission score is fully recalculated to ensure accuracy.

Additionally, institutions affected by the issue are offered the option to recalculate both the submission and the final result to correct any discrepancies caused by the issue.

Timeline
3rd of September, 2025

• 08:26 - Technical team are informed of the issue by our Quality Assurance tester.
• 9:03 - Technical team begins investigating. 

4th of September, 2025

• 9:00 - Technical team works on a fix for the issue and writes code to determine the impact of the issue on affected submissions

5th of September, 2025

• 13.22 - The hotfix is approved after being reviewed and tested
• 16.15 - Technical team deploys a hotfix for the issue, to prevent it from occurring again.

8th of September, 2025

• 10:52 - Technical team informs support with an overview of the issue alongside the completed impact analysis of the impacted submissions. 

9th of September, 2025

• 10.00 - Technical team starts on Post-mortem
• 16.53 - Support team informs all impacted schools 

10th of September, 2025

• 16.42 - Post-mortem is published

Reflection
This issue was introduced by newly added code that changed the logic of how adjustments were applied. Tests existed for creating and removing adjustments, but this specific case was not covered.

The issue was found by continuous internal testing of the platform and action was quickly taken upon being notified. The steps taken in the handling of this incident were followed in procedural order. The cause was found effectively and internal communication throughout the process was strong.

Developing and deploying the hotfix and finding the impact of affected submissions happened relatively quickly.

To prevent this in the future, we expanded our test coverage to include edge cases and scenarios introduced by new logic changes.