[Piano Analytics] Delays for some scheduled reports
Incident Report for Piano Software
Postmortem

Incident explanation:

  • The rights of a user were changed, which generated a large number of error exports and a large number of attempts, which slowed down the whole system.

Actions performed:

  • The exports of this user were disabled and the client isolated. This allowed the delay to be caught up.

Actions to come:

  • We will work on a better isolation of customers and users.

Posted May 23, 2022 - 18:37 UTC

Resolved
The issue has been resolved and all exports have been sent.
Posted May 23, 2022 - 18:36 UTC
Identified
We have fixed the issue. We are now working on sending exports.
There is a delay for several customers.
Posted May 23, 2022 - 13:51 UTC
Investigating
We are currently experiencing delays in the generation of some Dataquery 2 and Dataflow scheduled exports for 1 customer.
Our technical teams are working to resolve this issue so that these exports may be sent as soon as possible.
We apologise for any inconvenience caused.

Affected customers - Less than 1%
Posted May 23, 2022 - 07:09 UTC