Delayed notifications from monitors

Incident Report for Datadog US1

Resolved

This incident has been resolved.

Posted Jun 20, 2022 - 10:04 EDT

Monitoring

We are back to a normal situation. We're still monitoring actively to be sure there are no other problems.

Posted Jun 20, 2022 - 09:33 EDT

Update

Events are back to normal. Parts of notifications are still delayed and are recovering.

Posted Jun 20, 2022 - 09:19 EDT

Update

Everything is still recovering, with delays of a few minutes for events, and a few notifications pending to be sent.

Posted Jun 20, 2022 - 08:54 EDT

Identified

We identified the root cause. It's recovering and notifications are being processed again. There is still delay for events that happened during a 35 minute window (until 12.18 UTC).

Posted Jun 20, 2022 - 08:26 EDT

Investigating

We are investigating delays in notifications, which began at 20 June at 11:43 UTC.

Posted Jun 20, 2022 - 08:14 EDT

This incident affected: Monitors.