19th of July 2023 - Marks without limits applied

Incident Summary

All times mentioned in this document are recorded in UTC+2 (CEST). The support team received several tickets about the limit of a mark not being applied to a student's result. The first ticket was sent on July 17th, 2023, at 10:06.

The issue was introduced in a previous release when support was added for the rounded mark in the API. The issue only applies for results where the mark is below the value set for “Limit the lowest mark” or above “Limit the highest mark” in the mark calculation. Marks that are within the threshold of the limits are not affected by this issue.

Lead-up

The support team received a ticket on July 17th, 2023, at 10:06, stating that the student received a mark lower than the value set for the “Limit the lowest mark” option in the mark calculation. The technical team was informed on July 17th at 10:20 and started an investigation.

Fault

This issue was introduced in the release of June 26th, 2023, where a change was introduced in order to support the rounded mark attribute in the API. This addition resulted in the backend logic to reverse the order where the rounding and mark limitation are applied.

Impact

Please note that the calculation of the mark was not incorrect, and that this issue only affected the visual representation of the mark based on the rounding that was set in the assignment settings. The issue only applies for results where the mark is below the value that is set in the “Limit the lowest mark” option or above the “Limit the highest mark” option within the mark calculation.

Marks that are retrieved via the API using the “grade” or sent to the LMS or Osiris, were not affected by this issue. If the “rounded_mark” attribute was to retrieve the mark in the API, then the incorrect grade would have been retrieved.

Marks that are within the threshold of the limits are not affected by this issue.
During the investigation, an impact analysis was performed by the technical team. The results of the impact analysis display results (some which have already been published) that were affected by this issue. The support team received several tickets regarding this issue.

Detection

The incident was detected when the support team received several tickets stating the limits set for the marks in the mark calculation were not being applied to new results.

Response

The support team informed the technical team about the issue on July 17th at 10:20, after which the technical team started an investigation. The technical team was able to reproduce this issue on 18th of July at 10:51 and started working on an impact analysis and a hotfix.

Recovery

On July 18th at 17:02, a hotfix was deployed to prevent this issue from occurring again.
Ans will provide a list of the impacted results and offer the possibility to rectify the mark.

Timeline

17th of July 2023
10:06 - Initial ticket with the issue about mark limitations not being applied is sent.
10:20 - Issue is shared with the technical team.

18th of July 2023
10:51 - Technical team has successfully reproduced the issue.
11:13 - Technical team has started the impact analysis.
13:47 - Technical team developed a hotfix.
17:02 - The hotfix was deployed.
17:15 - Technical team has finished the impact analysis.

19th of July 2023
09:50 - Support team has informed the customers that sent in the ticket that the issue has been resolved on production.

Reflection

The issue remained unnoticed as there were not many occurrences nor any errors which were directly visible to the user or would show up in Ans’ internal monitoring system. We have included an internal test alongside the hotfix which will be run in our continuous integration process to prevent the issue from occurring again.