The incident has been resolved and user mining information has been updated.
Our investigation concluded that this incident started due to an issue on AWS and we have received the following confirmation from them about it: "Between 12:15 PM UTC and 1:55 PM UTC we experienced impaired connectivity, increased API faults, latencies and launch times affecting some instances in a single Availability Zone (euc1-az3) in the eu-central-1 Region."
Posted Sep 08, 2023 - 08:37 UTC
Update
Our monitoring is still showing that service has been fully restored. Some applications may still see outdated user mining information for the 13 minute affected period. A full reindexing will take place within the next 12 hours, and we will resolve this incident at that time.
Posted Sep 07, 2023 - 19:54 UTC
Monitoring
A fix has been applied, the team are monitoring the system.
Posted Sep 07, 2023 - 13:17 UTC
Identified
VX read only replica in the EU region has been down since 12:27 UTC. The issue has been identified and Piano is working on a fix. Possible impact includes failure on some API read requests, email sending delays and report generation delays.
Posted Sep 07, 2023 - 13:06 UTC
This incident affected: Email Distribution (Piano ID and VX transactional Emails) and API Availability (API Endpoints - EU - api-eu.piano.io).