On May 25th at 14:11 UTC we were made aware of an issue preventing a portion of LKE users in our Frankfurt Datacenter from reaching their kube-apiservers via kubectl.
At 14:27 our administrators were able to identify the root cause as memory constraints leading to failed backend services beginning at 14:00.
Steps were taken to increase memory limits at 14:29. By 14:34 we were able to confirm a full system recovery and restoration of customer access to their kube-apiservers.
As a result, we are taking further steps to increase resilience, redundancy and monitoring for the components of our LKE service that were impacted.