Application Outage
Incident Report for Tray.io
Resolved
The incident has been resolved, all systems are back to normal. Executions which took place during the incident and upgrade have caught up.

If you have any questions or concerns, please feel free to reach out to support@tray.io. Thank you for your patience and understanding.
Posted Jul 06, 2022 - 13:04 BST
Update
The upgrade was successful. It started at 12:12pm and completed at 12:13pm BST, our engineering team are now monitoring the situation.

The application is back to normal. New executions will be successful, however executions that took place during the incident and upgrade are catching up, and will eventually succeed

We will be back with further updates within 30 minutes or as soon as we have new information.
We apologise for any inconvenience this incident may cause.
Posted Jul 06, 2022 - 12:20 BST
Update
Our engineering team will be upgrading a core database as part of the remedy to increase the rate of which affected executions can be resolved.

We are expecting several minutes of downtime, during the upgrade the system will be unavailable.

We will be back with further updates as soon as the upgrade is complete.
We apologise for any inconvenience this incident may cause.
Posted Jul 06, 2022 - 12:04 BST
Update
Our engineering team is still monitoring the situation.

The application is back to normal. New executions will be successful, however executions that took place during the incident are still catching up, and will eventually succeed.

We will be back with further updates within 30 minutes or as soon as we have new information.
We apologise for any inconvenience this incident may cause.
Posted Jul 06, 2022 - 11:17 BST
Monitoring
A fix has been implemented and our engineering team are currently monitoring the system.

The system is back up and operational however, workflow execution is currently still in degraded performance at this current moment, as we are trying to catch up on the executions that were affected during the incident.

We will be back with further updates within 30 minutes or as soon as we have new information.
We apologise for any inconvenience this incident may cause.
Posted Jul 06, 2022 - 10:29 BST
Update
The root cause of the failures is currently unknown but our engineering team is investigating the issue tirelessly.

Users will continue to see errors within the application or will be unable to access it. Some workflow executions will also be delayed, fail or won't initiate at all.

We will be back with further updates within 30 minutes or as soon as we have new information.
We apologise for any inconvenience this incident may cause.
Posted Jul 06, 2022 - 09:50 BST
Investigating
We are currently investigating an issue which is causing unexpected errors within the application.

Customers may have trouble loading and accessing Tray. A small number of workflow executions may be affected as well.
Posted Jul 06, 2022 - 09:25 BST
This incident affected: Workflow Execution, Workflow Builder (Workflow Builder API, Workflow Builder App, Workflow Builder Logs), and Embedded (Embedded API, Embedded App).