Error while creating Subtask on a ticket
Incident Report for Jira Work Management
Postmortem

SUMMARY

On Tuesday, September 5, 2022, at 10:20 PM UTC, Atlassian customers using Jira Service Management, Jira Software, and Jira Work Management products started getting errors while creating subtasks for issues with required fields. The error was limited to users who had required fields for creating subtasks. This incident was caused by a change pushed to production that started deploying on September 5, 2022, at 10:15 PM UTC. The incident was detected within one hour and 20 minutes  and the problem was mitigated for 90 percent of affected customers in the next 6 hours, by 5:48 AM UTC on September 6.  The first mitigation to unblock the majority of customers was to revert back to the old issue create experience. The problem was completely resolved for the remaining ten percent in seven hours, by deploying the actual fix. The total time to resolution for all customers was ten hours and 45 minutes.

IMPACT

The overall impact was from 10:20 PM UTC on September 5, 2022, to 9:05 AM UTC on September 6, 2022, and affected customers of Jira Service Management, Jira Software, Jira Work Management products. The incident caused service disruption to users when they were creating subtasks with required fields in Jira.

ROOT CAUSE

The root cause was a recent change in create issue code which was part of an ongoing improvement task. This change was missed by our automated continuous deployment suites and manual tests in our testing environment and got deployed to the production environment. This change affected the creation of subtasks with required fields in Jira. We acknowledge that the incident could not be mitigated earlier because we missed following our own process, to Feature Flag every single change we make, for part of the change.

REMEDIAL ACTIONS PLAN & NEXT STEPS

We know that outages impact your productivity. While we have a number of testing and preventative processes in place, our automated continuous deployment suites and test scripts didn’t cover this use case.

We are prioritizing the following improvement actions to avoid repeating this type of incident:

  • Implementing additional automation testing for all the changes affecting issue creation flow.
  • Adding an additional check in our review process of going through rigorous QA checkpoints for all the trigger points and then pushing it to production.

We apologize to customers whose services were affected during this incident; we are taking immediate steps to improve the platform’s performance and availability.

Thanks,

Atlassian Customer Support

Posted Sep 15, 2022 - 06:39 UTC

Resolved
Between 12:20 AM UTC to 04:50 AM UTC, 6 September 2022, some customers experienced errors when creating subtasks in Jira Cloud. We have deployed a fix to mitigate the issue and have verified that the services have recovered. The conditions that cause the bug have been addressed and we're actively working on a permanent fix. The issue has been resolved and the service is operating normally.

Please contact us at https://support.atlassian.com/contact/ if you are still experiencing issues.
Posted Sep 06, 2022 - 05:26 UTC
Identified
We continue to work on resolving the creating subtasks error for Jira Cloud. We have identified the root cause and expect recovery shortly.
Posted Sep 06, 2022 - 04:53 UTC
Investigating
We are investigating an issue with creating subtasks which require mandatory fields that is impacting some Jira Cloud customers. We will provide more details within the next hour.
Posted Sep 06, 2022 - 03:30 UTC
This incident affected: Create and edit.