Incident Summary
From approximately 16:18 UTC to 17:00 UTC on Saturday, April 23 due to name resolution issues, some requests being served from the NYM2 datacenter experienced delays resulting in UI inaccessibility for users located near the East Coast of the U.S.
Scope of Impact
During the incident window, customers experienced intermittent delays with accessing our Invest, Monetize, and Curate UI.
Timeline (UTC)
2022-04-23 16:18: Incident Started: Impact of issue was felt by clients
2022-04-23 16:28: Incident Escalated: Engineers notified to existence of incident
2022-04-23 16:58: Issue Identified: Engineers identified issue causing incident
2022-04-23 17:00: Issue Resolved: Final fix released to address underlying issue
Cause Analysis
Due to intermittent name resolution issues in NYM2 datacenter, some requests going through this host were dropped causing the inaccessibility of our UIs, namely Monetize, Invest, and Curate.
Resolution Steps
Teams identified the problematic service and removed that from production, which immediately resolved this issue.
Next Steps
- Further investigation will be carried out by the teams on this issue.
The incident has been fully resolved. We apologize for the inconvenience this issue may have caused, and thank you for your continued support.
We have patched the issue and are monitoring our systems closely. We will provide an update as soon as the issue has been fully resolved.
We are currently investigating the following issue:
We will provide an update as soon as more information is available. Thank you for your patience.