The Emergency Maintenance has been finished. All systems are up and running again in normal state. We would like to excuse for the inconveniences caused today. As announced earlier, we will provide a full report.
Preliminary Incident Report:
Incident Report for Wednesday 3rd March 2021
12:39 UTC: We are currently seeing network issues at our Datacenter in our network switching backbone, affecting one member switch. Network connections should be mostly redundant (link aggregates), but you might experience package loss or network disruption. Investigation ongoing.
13:45 UTC: We believe that member switch 3 is causing the issues and will perform a reboot of switch at 14:00 UTC (15:00 CET) with the intention to resolve these.
14:15 UTC: Unfortunately the reboot led to a total outage. API and Web Interfaces are offline. Investigation ongoing.
15:00 UTC: Our systems are back online, API and Web Interfaces are available again.
17:30 UTC: Emergency Maintenance to ensure all systems will continue to work properly. We expect to be offline for ca. 15 minutes.
17:45 UTC: Emergency Maintenance finished. All systems back online again.
Posted Mar 03, 2021 - 17:57 UTC
Investigating
We will perform an Emergency Maintenance at 17:30 UTC and expect to be offline for ca. 15 minutes. This step is necessary to ensure all systems will continue to work properly. We kindly ask to excuse any inconvenience caused.
Posted Mar 03, 2021 - 17:11 UTC
This incident affected: XRRP API, EPP API, SOAP API, XMLRPC API, HTTPS API, Web Interface, OT&E Environment, WHOIS, and RDAP.