Service Incident - 11th of July 2025 - Guess score was calculated after the mark was recalculated for changing contribution of a question

All times in the document are recorded in UTC+2 (CEST).

Summary
On June 30th 2025, a user reported that changing a guess scored question's contribution to disabled or full points, caused the dynamic variables in the mark calculation to function incorrectly. Since the question lost its guess score, it reverted to being treated as open‑ended rather than closed-ended. This change was not taken into account when the mark was recalculated, which meant the marks had to be recalculated again to ensure the variables applied correctly. Consequently, the question contributed to an incorrect dynamic mark calculation variable.

These variables (total points for open questions, total points for closed questions, obtained points for open questions and obtained points for closed questions) use the guess score of questions to determine the category they belong to. Since the calculation of the guess score had not yet been performed, the questions could count towards the opposite variable type.
The issue was first reported via a support ticket and escalated to the technical team. On July 1st, the technical team began developing a hotfix and conducting an impact analysis. By July 4th, the support team informed the impacted users about the impact on their institution and confirmed that a hotfix and postmortem were in progress.

A hotfix was deployed on July 4th, 2025, ensuring that marks are recalculated an additional time whenever the guess score of a question changes, for any reason. This prevents order of operations between the recalculating of the mark and updating of the guess score to have any impact, resulting in a consistently correct mark.

Lead-up
On June 30th at 16:29, a user reported an issue in which marks seemed to be calculated inconsistently when changing the contribution of a question. This resulted in the Support team sharing the potential issue with the Technical team the morning after.

Fault
It was discovered that this issue had been present ever since dynamic mark calculation variables were introduced into the platform and would occur under the following conditions:

1. An assignment with closed-ended questions was taken
2. The assignment uses dynamic variables for open-ended and closed-ended questions in its mark calculation
3. One of the questions had its contribution changed from contributing or bonus to full points or disabled, or the other way around.

This would result in the mark being calculated with the new contribution being taken into account correctly, but the question could count towards the wrong variable due to the guess score changing later.

Impact
An impact analysis was performed on July 3rd in order to find the results that were affected. All assignments using dynamic variables were included in this impact analysis, and it is therefore believed that all occurrences have been covered.

The investigation resulted in 4 affected assignments, with 97 impacted submissions.

Detection
The issue was first reported by a user via a support ticket. Upon review, the support team escalated the problem to the technical team for further investigation.

Response
On July 1st, 2025, the support team forwarded the ticket to the technical team and informed the user that further investigation was required. The support team assured the user that updates would be provided as more information became available.

Recovery
On July 4th, 2025, a hotfix was released to ensure that the marks are recalculated an additional time whenever the guess score of a question changes, for any reason.
Additionally, impacted schools were given the option to have the marks of the impacted assignments recalculated, which would result in a correct grade.

Timeline
30th of June, 2025

• 16:29 - Support receives a ticket indicating that there appears to be a discrepancy relating to the grading formulas and changes to the question grading from the insights tab.

1st of July, 2025

• 10.29 - Support forwards the information to the technical team and the technical team begins their investigation.
   

2nd of July

• 16:00 - Technical team finds issue and impact 
   

3rd of July

• 10:56 - The technical team informs support with an overview of the issue alongside the impact analysis.
   

4th of July

• 15:37 - Hotfix was deployed
• 16:04 - Support informed the user that reported the issue and the administrators about the impact on their school and confirmed that a postmortem report was being written.

Reflection
This issue was caused by insufficient testing of the interactions between the dynamic mark calculation variables and the question contribution features. That testing has now been added. It is also worth noting that this bug comes from a longer time ago, and processes have been improved upon since, with more rigorous testing as well as a dedicated Quality Assurance department being set up.

The steps taken in the handling of this incident were followed in procedural order. The cause was found effectively and internal communication throughout the process was strong.