We have identified the primary factor for this incident was a bug in the Ubuntu operating system that was introduced into our environment on March 9, 2023, and has since been removed. For more information regarding this bug please refer to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2009325
Posted Mar 23, 2023 - 16:47 MDT
Resolved
At 10:00 PM MT on March 9th, 2023 the PrinterCloud servers started experiencing high-load in the US region. The web application was unavailable or intermittently loading with slow response times.
A system update on our servers caused the high-load, and excessive connections to our databases.
Our current status is: We identified the initial issue, and rolled back the system updates. We will continue to monitor our environment and ensure we are stable.
We are working on identifying why the update caused the increased load on our servers.
Posted Mar 10, 2023 - 07:07 MST
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Mar 10, 2023 - 04:59 MST
Update
We are continuing to work on a fix for this issue.
Posted Mar 10, 2023 - 03:56 MST
Update
We continue to experience intermittent login failures and slow response times. We are continuing to investigate this issue.
Posted Mar 10, 2023 - 03:42 MST
Identified
The issue has been identified and a fix is being implemented.
Posted Mar 10, 2023 - 02:34 MST
Update
We are continuing to investigate this issue.
Posted Mar 10, 2023 - 01:53 MST
Update
We are continuing to investigate this issue.
Posted Mar 10, 2023 - 01:01 MST
Investigating
We are currently investigating this issue.
Posted Mar 10, 2023 - 00:14 MST
This incident affected: PrinterLogic | SaaS US (PrinterLogic | SaaS, MSP Portal - US).