Vault Login in EU Data Center

Incident Report for Keeper Status Page

Postmortem

The Engineering team found and resolved the cause of the API errors which occurred on Jan 28, 2021, described in this postmortem report.

It was determined that a particular “SCIM PATCH” for Group association requests from the Microsoft Azure portal was returning an invalid response from the Keeper servers based on the number of users within the group. The size of the response grew exponentially in size based on the number of users within the group. This issue caused memory exceptions to occur on all affected application servers, which subsequently generated API errors for login requests.

We have corrected the SCIM PATCH response in our Backend release 15.2.1 which went live on Saturday, Jan 30, 2021.

The SCIM specification https://tools.ietf.org/html/rfc7644#section-3.5.2 allows returning 204 as a result of GROUP's patch, which can greatly improve performance for large groups.

Jira Ticket: KA-3782

Posted Feb 01, 2021 - 09:18 PST

Resolved

This incident has been resolved.

Posted Jan 28, 2021 - 02:12 PST

Monitoring

Issue has been resolved, we are investigating the root cause and will update this statuspage shortly.

Posted Jan 28, 2021 - 02:06 PST

Investigating

We are investigating API errors in the EU region. Click "Work Offline" if experiencing login issues.

Posted Jan 28, 2021 - 01:42 PST

This incident affected: Keeper Infrastructure (EU Data Center) (Keeper Web Vault (EU)).