Data Center Connectivity Issues
Incident Report for Redtail Technology
Postmortem

On Thursday, October 1st, all Redtail services were interrupted due to a Power Distribution Unit (PDU) that became de-energized which resulted in equipment associated with it not failing over properly. This impacted our end users for a small window of time. 

Upon being alerted of the issue by our active monitoring solutions, we initiated our Incident Response protocols. Our technicians were dispatched to the Data Center experiencing the issue. Once the proper steps were taken to correct this issue, services promptly return to operational. As a result of the issue, we are proactively running more tests as well as looking into any additional monitoring and process that would ensure issues like these are mitigated and avoided in the future. 

We at Redtail would like to extend to you our humble and sincere apology for any negative impact these issues had on you and your business. We understand how critical it is that we deliver maximum uptime to support your daily operations, and will increase our efforts to meet and exceed your availability expectations. Please rest assured that we will do everything we can to learn from this event and use it to strive for improvement across all of our services.

Posted Oct 01, 2020 - 18:40 PDT

Resolved
The issue has been resolved. We at Redtail would like to extend to you our humble and sincere apology for any negative impact these issues had on you and your business. We understand how critical it is that we deliver maximum uptime to support your daily operations, and will increase our efforts to meet and exceed your availability expectations. Please rest assured that we will do everything we can to learn from this event and use it to strive for improvement across all of our services.
Posted Oct 01, 2020 - 16:39 PDT
Monitoring
Our technicians have corrected the issue impacting all Redtail services. You should start seeing access return. We are monitoring everything currently to ensure everything is ok.
Posted Oct 01, 2020 - 07:34 PDT
Identified
The issue has been identified and a fix is being implemented.
Posted Oct 01, 2020 - 07:12 PDT
Update
We are continuing to investigate this issue.
Posted Oct 01, 2020 - 06:46 PDT
Investigating
Some users may be experiencing latency or connectivity issues when attempting to access Redtail's various services. We are currently experiencing some connectivity issues with one of Data Centers which in turn is impacting our services. We are in communication with their engineers as they investigate the issue.
Posted Oct 01, 2020 - 06:41 PDT
This incident affected: Redtail API (REST), Redtail CRM, Redtail Imaging, and Redtail Email.