On May 18, 2022, elevated latency and error rates with the conversational assistant and workflows were noticed. Scaling up related services allowed workflows to function as expected and the conversational assistant at a degraded capacity. Upon resolving the root cause, the traffic on all related services quickly dropped and was running at normal capacity.
An assisted conversation entered a bad state that elevated traffic in the conversational assistant and workflow at 1:35 PM EST. Related services were scaled to match the increase in traffic soon after, stabilizing the platform. Further investigation indicated that the error rates had not been fully resolved. The related assistant was modified to alleviate any issues and affected service latencies quickly dropped.