A fix has been implemented and we are monitoring the results.
Posted Aug 10, 2021 - 18:01 CDT
Investigating
At approximately 10:50 am PDT / 12:50 pm CDT Tuesday Aug 10, 2021 for 12 minutes the following pages were unreachable: Jobs page Dashboard page Customers page We have installed logic that will speed-up database calls from the main database to read or view data. What is supposed to happen is when a request comes into the database the software will redistribute that load to multiple servers so that the main database is not overly burdened.
We needed to run this during high traffic hours to locate any issues. A conflict was found between our logic and our hosted AWS load balancers which is inherent with our servers.
We quickly figured out what was causing the high error rates and dialed the load back onto the main database. This brought the pages back.
We are going to continue to test while fully monitoring the situation and hopefully we will not have any further issues with this enhancement.
We apologize for the inconvenience this has caused your workflow today. Once fully implemented you will see an enhanced experience with your customer and job data appearing more quickly on the pages.
Posted Aug 10, 2021 - 18:00 CDT
This incident affected: Web Application (app.kickserv.com).