The post-mortem from today's outage can be found below.
Incident Impact
From 9:43 AM PT/12:43 PM ET - 9:58 AM PT/12:58 PM ET on December 19th, 2019, Bolt had an incident that prevented shoppers from checking out. More specifically, shoppers who clicked the Bolt checkout button could not load checkout and received a message stating that an error occurred. The merchant dashboard was also inaccessible over the duration of the incident period, which lasted 15 minutes.
Root Cause Analysis
At 9:43 AM PT/12:43 PM ET, Bolt ran a database migration in production as part of a routine product update. This migration did not successfully complete, causing the Bolt database to lock and become temporarily unavailable.
Resolution
At 9:56 AM PT/12:56 PM ET, Bolt restarted the database to resolve the issue. The restart was successful, and service was fully restored by 9:58 AM PT/12:58 PM ET.
If you have further questions, please contact merchantsupport@bolt.com and cc your Success Manager.
Additionally, as a reminder, Bolt has a planned code freeze starting on Friday, December 19th, 2019 and ending on Thursday, January 2nd, 2020. No further code changes will be made to Bolt until the code freeze is lifted.
Posted Dec 19, 2019 - 23:34 UTC
Update
Services continue to function as normal following the fix implemented at 12:58 PM ET. A full post-mortem and root cause analysis will be available by the end of day today.
Service has been fully restored to checkout and the merchant dashboard as of 12:58 PM ET. We are monitoring for any further issues and will provide another update regarding root cause shortly.
Bolt is currently experiencing an issue causing checkout and the merchant dashboard to be inaccessible. Our engineering team is investigating the issue with the highest level of urgency, and we are aiming to resolve the issue as soon as possible.
We will keep you updated as we work through resolution; please reach out to merchantsupport@bolt.com if you have questions.
Posted Dec 19, 2019 - 17:54 UTC
This incident affected: Checkout and Merchant Dashboard.