Delays ingesting a subset of logs, APM, and profiling data

Incident Report for Datadog US1

Resolved

We've resolved the root cause, and our systems are now operating normally. We're working on backfilling the remaining delayed data and affected customers have an in-app banner explaining what data remains to be backfilled.

Posted Feb 09, 2021 - 21:59 EST

Update

We're continuing to work towards resolution. Query latencies have returned to normal. We are making progress on backfilling historical data and returning live traffic to the impacted customers and services.

Posted Feb 09, 2021 - 20:52 EST

Identified

We have identified the issue, and are continuing to work towards resolution.

Posted Feb 09, 2021 - 19:39 EST

Investigating

We're investigating an issue causing delays in a subset of logs, APM traces, and profiling data, as well as slower queries across these data types. To avoid spurious alerts, we've disabled monitors on the subset of data that is impacted.

Posted Feb 09, 2021 - 19:12 EST

This incident affected: APM and Log Management.