Infinity Portal uses a Gateway service with an external cache mechanism. Some workflows caused an overload on our cache, making every cache request extremely slow thus the login process using the cache showing high latency and timeouts.
Summary
The user experience of the Portal’s login process is highly critical. We are committed to improve and will soon add healing procedures to keep the cache healthy. Thank you for your patience during the incident.
Posted Jun 14, 2022 - 15:21 UTC
Resolved
We have identified the specific place that caused the latency and applied an operation that have resolved the issue. The Infinity Portal performance is back to normal. We will make an RCA to learn and improve further.
Posted Jun 14, 2022 - 13:33 UTC
Update
We are continuing to investigate this issue.
Posted Jun 14, 2022 - 13:31 UTC
Investigating
We've noticed high latency and some timeouts to login to the portal for most of the users. We're investigating and will update accordingly. Thank you for your patience
Posted Jun 14, 2022 - 12:15 UTC
This incident affected: Infinity Portal (Infinity Portal EU Region, Infinity Portal US Region).