Synergy Cloud - Platform performance issues.
Incident Report for Total Synergy
Postmortem

Between the hours of 10:00 to 13:20 (AEST) on 07/11/2022 multiple users encountered slow performance when using Synergy Cloud 

The event was triggered by a change introduced as part of the Synergy November release which related to making global search more reliable.  

A complex retry issue caused degraded performance to customers using Synergy. 

The event was detected by multiple customers as well as our internal systems. A change has been tested and implemented to revert the previous change and was confirmed working around 13:20 (AEST) 07/11/2022.

Posted Nov 07, 2022 - 15:51 AEDT

Resolved
The fix we applied in our earlier update has resolved this incident.
Global search is now functioning correctly and the loading speeds across Synergy are back to normal.

Our deepest apologies once again for the disruption this caused you today, and a huge thank you for your patience while our teams worked to get this resolved.
Posted Nov 07, 2022 - 14:36 AEDT
Monitoring
Our Engineering team have applied a fix, this fix has resolved both the platform slowness and the Global Search and everything is running as expected again.

The support team will continue to keep an eye out for any further reports of slowness, and our Engineering team will continue to monitor performance in the background.
Posted Nov 07, 2022 - 14:11 AEDT
Update
Our Engineering team will be deploying a fix, there will be a 5-10 minute downtime from 1:20 PM AEST.
Posted Nov 07, 2022 - 13:10 AEDT
Update
Our Engineering team have made some changes to resolve the platform slowness, however while we have seen some improvement on speeds, we're still getting some reports of slowness, and Global Search is still not loading anything at all.

They are continuing to work on a permenant fix to deploy over the next hour and we will continue to keep you updated on the progress and inform you of any further downtimes required.

In the mean time please use the dedicated search for every area, rather than Global Search (ie. Go to Projects and search for a project, Contacts to search a contact etc.)

This will remain a top priority until it is completely resolved and we're incredibly grateful for your patience and understanding.
Posted Nov 07, 2022 - 12:36 AEDT
Identified
Our engineering team have idetified the problem causing the current platform slowness in Synergy and will be applying a fix within the next 10 minutes.

There will be downtime of 5-10 minutes from 11:25 AM AEST, we will continue to provide updates as they become available.
Posted Nov 07, 2022 - 11:17 AEDT
Investigating
We're investigating multiple reports of platform slowness this morning, 07/11/2022 (11:10 AM AEST)
Our engineering team are investigating this with high prioirty and we will provide updates as they become available.
We're incredibly sorry for any disruption this is causing you today and endeavour to have this resolved as quickly as possible.
Posted Nov 07, 2022 - 11:12 AEDT
This incident affected: Synergy App.