At 09:16 UTC on 2021-03-18, we observed an increase in latency and errors for the History service in the South Asia PoP. We determined that there was an issue with our storage services vendor and immediately opened a ticket with them, the vendor restarted the Storage processes, and error rates dropped significantly at 09:45 UTC, and the incident was fully resolved at 10:21 UTC.
To prevent a similar issue from occurring, we have made optimizations to the Message Actions service, and we are actively working with the vendor to align on their mitigation steps.