Degraded API Status
Incident Report for OpenAI
Resolved
Marking this incident as resolved on our end. All systems appear to be working properly and we do not appear to be affected by other Azure degraded services.
Posted Apr 01, 2021 - 16:40 PDT
Update
All of our systems appear to be operational; however, problems with our primary service provider, Azure, appear to have gotten worse https://status.azure.com/en-us/status. We will be leaving the status at Monitoring until these abate or until we have further confidence that none of our systems are subtly adversely affected.
Posted Apr 01, 2021 - 16:24 PDT
Monitoring
The Playground Site at beta.openai.com is resolving properly again. We will be monitoring the status of DNS propagation.
Posted Apr 01, 2021 - 16:13 PDT
Identified
The API appears to be fully functional. DNS for api.openai.com is working properly.

Unfortunately the Playground Site at beta.openai.com is not resolving right now. This is due to unintended side effects from restoring DNS to api.openai.com. We have identified the issue and are rolling out a fix now.
Posted Apr 01, 2021 - 16:05 PDT
Monitoring
DNS for api.openai.com should be properly resolving now. We have moved our DNS to Route 53 and are seeing good resolution. You should be able to use the domain name again instead of our hard-coded IP address.

We will continue to monitor the situation. While we believe our systems are now fully operational, Azure is still showing problems: https://status.azure.com/en-us/status
Posted Apr 01, 2021 - 15:47 PDT
Update
You can bypass the DNS issue in the short term by hard-coding our hostname 40.124.113.113

echo "40.124.113.113 api.openai.com" | sudo tee -a /etc/hosts

[edit] If you can't access your /etc/hosts file in your environment, you can also use a command similar to the following:

curl -k -H "Host: api.openai.com" "https://40.124.113.113"

Also note that due to the probabilistic nature of Azure's DNS issue it's possible that the hostname will intermittently resolve and then break again.

NOTE that we do not guarantee stable IP addresses. It is important that you undo this temporary patch once the issue has been resolved. We will update this Status Page as soon as we confirm the system is back to normal.
Posted Apr 01, 2021 - 15:04 PDT
Identified
We now believe that there is a widespread outage with upstream Azure DNS and associated services. We are actively monitoring the situation with Azure here: https://status2.azure.com/. We are working through alternatives on our end should our vendor not be able to resolve this in a timely manner.
Posted Apr 01, 2021 - 14:48 PDT
Update
We are seeing widespread connectivity problems. Current investigations lead us to believe that this may be due to DNS problems upstream. We are continuing to investigate.
Posted Apr 01, 2021 - 14:42 PDT
Investigating
We are actively investigating a degradation in API traffic as well as access to the Playground site.
Posted Apr 01, 2021 - 14:35 PDT
This incident affected: API and Playground.