Google wide-spread outage
Incident Report for CircleCI
Resolved
This incident has been resolved.
Posted Dec 14, 2020 - 12:39 UTC
Update
We are continuing to monitor for any further issues.
Posted Dec 14, 2020 - 12:38 UTC
Monitoring
Our internal monitoring suggests that Google seems to be recovering from the incident.

All builds are now running fine.

We are still actively monitoring the situation.
Posted Dec 14, 2020 - 12:37 UTC
Update
We believe that machine jobs should eventually run once the Google incident is resolved. However, remote docker jobs will have to be restarted manually, as they will fail with a timeout due to the aforementioned third-party incident.
Posted Dec 14, 2020 - 12:28 UTC
Investigating
Google appears to have a wide-spread outage, which is affecting our ability to boot Google Cloud VMs. This will prevent builds using the machine (Linux) and remote docker executors from starting.

Note that this doesn't affect MacOS jobs.

Please refer to the Google Status pages
> https://www.google.com/appsstatus#hl=en-GB&v=status
> https://status.cloud.google.com

We are monitoring the situation, and assessing the impact of this outage on our service.
Posted Dec 14, 2020 - 12:23 UTC
This incident affected: CircleCI Dependencies (Google Cloud Platform Google Cloud DNS, Google Cloud Platform Google Cloud Networking, Google Cloud Platform Google Cloud Storage, Google Cloud Platform Google Compute Engine) and Machine Jobs, Pipelines & Workflows.