Marking this issue as fully resolved. In working with our Vendor (AWS), the timeline to full recovery was reached at approximately 12:50AM Eastern on 27MAY22. All tests conducted on behalf of the Sharpen platform have been successful since that time-frame, and all health-checks/network monitoring tools/telemetry are showing that the system is performing as expected.
Posted May 27, 2022 - 12:25 UTC
Monitoring
Working with our Vendor AWS, we have confirmed a hot-fix has been implemented to address the DNS inconsistencies/routing errors seen across various infrastructure end-points. Preliminary validation testing has shown across-the-board recovery. We're actively monitoring the situation and still engaged with our Vendor - waiting for additional verification before designating an "All Clear".
Posted May 27, 2022 - 03:57 UTC
Investigating
We're aware of an active/ongoing issue with the API Gateway, where various API calls or queries to certain end-points are being met with a failure or timeout response. This has been tracked back to a DNS issue with one of our Vendors (AWS) and we're actively working on investigating the cause of misbehavior with them.
Posted May 27, 2022 - 03:31 UTC
This incident affected: Isolation Zone 0 (API) and Isolation Zone 1 (API).