On June 21, 2022 from 6:27 UTC until 7:42 UTC almost all users received errors when trying to access any UiPath cloud product. While some users in some regions were still able to get through, we are treating it as a total outage.
UiPath relies heavily upon Cloudflare. In addition to DNS, we use it to proxy all of our traffic to cloud.uipath.com and related domains. This allows us to take advantage of Cloudflare’s DDOS protection, Web Application Firewall, caching, and more.
At the exact same times listed above, Cloudflare had a major outage in 19 of their data centers. These data centers handle the vast majority of all UiPath traffic, and as a result we were unavailable.
Cloudflare has posted an excellent deep dive into exactly what happened here: https://blog.cloudflare.com/cloudflare-outage-on-june-21-2022/
Our automated alerting system notified oncall engineers about the problem within minutes of the outage. They were able to quickly identify the root cause and post to status.uipath.com. Unfortunately, due to the nature of our dependency on Cloudflare, there was no action they could take to mitigate the problem. They saw that Cloudflare had already identified the problem and were working on a fix, so they waited until that fix was available. At that time, they confirmed that all UiPath services became available again with no further problems.
UiPath selected Cloudflare, in part, due to its very high reliability with an SLA of 100%. We are currently working with engineers at Cloudflare to see if there is anything more we can do to protect ourselves from these sorts of outages, but for the time being we believe that taking a dependency on an industry standard services such as Cloudflare is still better than any alternative.