Performance issues on main cluster

Incident Report for Bubble

Resolved

We've confirmed that the failover resolved the issue. We're currently investigating the source of the bad query.

Posted Jan 14, 2021 - 11:38 EST

Monitoring

Failover is complete, monitoring to see if systems are back to normal

Posted Jan 14, 2021 - 11:19 EST

Identified

We've found the source of the issue: a long-running, frozen query on one of our main user databases. Unfortunately we are unable to manually terminate the query, so are going to have to failover the database. We expect 30 seconds to a couple minutes of downtime during the failover

Posted Jan 14, 2021 - 11:12 EST

Investigating

We're seeing some degraded performance on the main cluster and are investigating

Posted Jan 14, 2021 - 11:06 EST

This incident affected: Bubble Core (Main Bubble Environment).