Cornerstone Status - Minor Issue - Latency

Minor Issue - Latency - UK-SL1 PROD

Incident Report for Cornerstone

Postmortem

On May 16th &17th, 2022 Cornerstone engineers were alerted by internal monitoring tools for portal latency & availability issues. Clients across the UK swimlane experienced slowness and portal availability issues intermittently.

Cornerstone engineers were engaged for investigation and troubleshooting and issues pointed to the storage issues. Further review revealed that the issue was due to storage not performing optimally due to I/o issues. This was resolved after adding more resources to storage.

To mitigate this situation from occurring in the future and to reduce the risk of a future impact Cornerstone is actively working internally to review the incident with the vendor. Internal teams are working with our third-party vendors to establish the issue and conducting a thorough investigation and creating remediation plans and capacity planning to avoid the recurrence of this incident.

Posted May 31, 2022 - 00:07 PDT

Resolved

The CSOD Technology Team observed a service disruption on this swimlane. The problem began at 1:07 pm Pacific Time and was resolved at 2:11 pm Pacific Time. During this time, clients with portals on this swimlane were not able to access the application.

Posted May 17, 2022 - 14:44 PDT

Update

Posted May 17, 2022 - 14:43 PDT

Investigating

The swimlane is experiencing some latency issues. Clients with portals in this swimlane may experience delays or issues accessing certain pages in the application. This is our top priority and we are working to resolve the problem as soon as possible. Please check back periodically for additional updates, which will be posted as they become available.

Posted May 17, 2022 - 13:12 PDT

This incident affected: UK SL1 (AWS) (Response Time).