AWS DNS Resolver Issue
Incident Report for Chargebee
Resolved
This incident has been resolved. Jobs that had failed due to this incident have been rescheduled. Based on our logs, very few api calls were affected during the incident.
Posted Jul 28, 2020 - 13:15 UTC
Update
At 8:30 a.m UTC, we started noticing few application errors due to DNS resolution problems at one of the Availability zones in AWS (US-EAST-1 region). Servers hosted in other availability zones were not affected.

Primary failures were for scheduled jobs. Few api requests that were served from that region were also affected.

Post identification, we immediately disabled serving requests from that availability zone.
Posted Jul 28, 2020 - 12:15 UTC
Monitoring
One of the AWS availability zone where our application is hosted, is affected by DNS resolver issue. Our technical team is working towards moving the request load to another stable zone.
Posted Jul 28, 2020 - 09:27 UTC
Identified
We are noticing intermittent errors in our service due to AWS DNS resolver issue. We have raised this with AWS and closely monitoring the situation.
Posted Jul 28, 2020 - 09:00 UTC