On October 24th we relaunched our company website, migrating the site to a new hosting provider as part of this launch. The migration revealed unanticipated DNS routing dependencies with Onfido’s dashboard in the EU region. As a result, after re-pointing onfido.com at 5:07 UTC, EU clients were prevented from logging into their dashboard accounts. The DNS change was rolled back at 07:30 UTC to restore the dashboard service.
When launching our new website, an interdependent infrastructure component was not migrated from one hosting provider to the other. This component provided request routing functionality to our dashboard that enabled login behaviours.
05:07 UTC: We released the new Onfido website, re-pointing onfido.com to a new hosting/DNS provider.
06:12 UTC: Having become aware of issues with the dashboard login, our investigations surfaced legacy dependencies with our routing between onfido.com and our Dashboard web application in the EU region.
07:10 UTC: A potential fix was identified. However, it was decided to rollback the website change to restore service, until that fix was available.
07:30 UTC: We completed the rollback to restore EU dashboard login for all clients.
Our immediate actions were to remove or redirect dependencies between the dashboard application and onfido.com. This enabled logins on the dashboard to work correctly with the new hosting configuration.
To avoid similar issues from recurring in the future, we are reviewing how we plan and test projects and changes of this nature. In particular, we are working on the following actions in the short term:
Additionally, we have a mid-term initiative to remove all remaining dependencies between onfido.com and our production applications.