Service Incident - 22nd of February 2024 - Points given to unanswered numerical questions – Ans

Incident Summary

All times in the document are recorded in UTC+1 (CEST).

On the 6th of February, 2024, at 16:04, the support team received a ticket that described the possibility to receive full points for numerical questions if the question remained unanswered and the correct answer was exactly zero. At 16:37, the technical team discovered that empty answers were changed to 0.0 instead of being seen as unanswered. On the 7th of February at 9:14, a hotfix was released to prevent the issue from occurring again. Moreover, an investigation was started to determine the impact.

Lead-up

The support team received a ticket on February 6th, at 16:04. In this ticket, the sender reported that unanswered numerical questions would give the participants full points if the correct answer was zero.

Fault

During the investigation, it was discovered that this issue was introduced on December 11th, 2023. On this date, a hotfix was released which fixed another issue where the answers of numeric questions were compared as strings instead of decimals.

Impact

An impact analysis was performed by the technical team in order to find the results that were affected. This analysis was scoped to December 11th as this was the date the change for the answer comparison was released. The results from this analysis, are as follows:

A total of 707 submitted results were found to be affected among 67 assignments.

Detection

The findings of the user who sent the ticket had been investigated and confirmed. Once the issue had been confirmed, a hotfix was created and deployed to prevent the issue from occurring.

Response

The support team forwarded the ticket to the technical team on the 6th of February, 2024, at 16:34. The technical team immediately investigated the issue. Once the issue was confirmed, a hotfix was developed and then deployed the next day. The technical team started with an impact analysis after the hotfix had been released.

Recovery

Ans will provide a list of impacted results to the institutions that were affected by the issue and offer the possibility to rectify the results.

Timeline

6th of February

16:04 - Support team receives ticket containing the issue
16:30 - Support team forwards the issue to the technical team
16:37 - Issue is confirmed by the technical team and a hotfix is created

7th of February

09:32 - Hotfix has been deployed to the production environment

12th of February

09:21 - Start of impact analysis

14th of February

16:35 - Impact analysis finalised and shared with the support team

19th of February

15:54 - Findings shared with the CSM team

Reflection

The steps taken in the handling of this incident were followed in procedural order. The impact analysis took longer to finalise as there was a significant amount of data to process. We have included an internal test alongside the hotfix which will be run in our continuous integration process to prevent the issue from occurring again.