US data center issues

Incident Report for Cronofy

Resolved

AWS have closed their incident for the underlying issue with SQS and other services.

AWS's SQS service appeared to be unavailable from around 20:47 UTC through to 20:57 UTC in our US data center, hosted in us-east-1.

As SQS is our primary messaging queue between parts of the Cronofy platform, many operations will have been severely degraded during this period.

We are confident that service has returned to normal as our own metrics have returned to normal levels.

Posted Mar 09, 2022 - 21:37 UTC

Monitoring

AWS have created an incident relating to the SQS outage in us-east-1 available here: https://health.aws.amazon.com/health/status

Our US data center appears to have fully recovered and we are continuing to monitor.

Posted Mar 09, 2022 - 21:17 UTC

Update

AWS's SQS service appeared to be unavailable from around 20:47 UTC through to 20:57 UTC in our US data center (us-east-1).

As SQS is our primary messaging queue between parts of the Cronofy platform, many operations will have been severely degraded during this period.

Since SQS has become available again our systems appear to be recovering as expected.

We continue to monitor the situation.

Posted Mar 09, 2022 - 21:04 UTC

Investigating

We're seeing signs of underlying issues in the US relating to SQS, our core messaging queue. This may have side effects on all operations in the US data center.

We are investigating.

Posted Mar 09, 2022 - 20:55 UTC

This incident affected: Scheduler, API, Background Processing, and Developer Dashboard.