Based on the monitoring, all packet routers have fully stabilized.
Posted May 05, 2023 - 05:42 UTC
Update
The core team has been monitoring the rollback for almost 1hr and packet routers have stabilized. We will monitor them for 1 more hour before resolving the incident.
Posted May 05, 2023 - 04:33 UTC
Update
We are continuing to monitor for any further issues.
Posted May 05, 2023 - 04:02 UTC
Update
Based on the monitoring, the core team has begun rollback on the packet routers due to high memory usage.
Posted May 05, 2023 - 04:01 UTC
Monitoring
The core team is rolling out packet routers across all regions to optimize GRPC process leaks. Data transfer may be temporarily unstable during the rollout. Should stabilize within 30 minutes.