Outbound emails delayed due to email provider
Incident Report for Dixa
Postmortem

On Monday the 8th of March 2021, 11:50 pm CEST / 5:50 pm EST we started getting alerts and seeing that our outbound emails were being delayed by our carrier. It didn’t cause any disruption to work for agents in Dixa, as the outbound emails were just added to a queue, and successfully sent later.  

After investigation, the root cause was due to an email that was in an everlasting loop which caused our email provider to experience delays handling the load. 

The issue was resolved on Tuesday the 9th at 02:49 pm CEST / 08:49 am EST. 

We performed a Root Cause Analysis of this incident and are implementing solutions in order to prevent such instances from happening in the future. We sincerely apologise for any inconvenience that this may have caused.

Posted Mar 22, 2021 - 08:57 CET
Resolved
All known issues to this incident have been resolved. We thank you for your patience and cooperation.

Post mortem about this incident will be posted within 5 business days.
Posted Mar 09, 2021 - 17:45 CET
Update
We are continuing to monitor for any further issues

Next update at 18.00 CET
Posted Mar 09, 2021 - 16:58 CET
Update
We are continuing to monitor for any further issues

Next update at 17.00 CET
Posted Mar 09, 2021 - 15:56 CET
Monitoring
We are happy to inform, that our team have deployed a fix for the issue. We are seeing significant improvements to outbound emails being send and the backlog of queued emails from earlier is down to zero. We will continue to monitor the results.

Next update at 16:00 CET
Posted Mar 09, 2021 - 15:06 CET
Update
We are continuing to work on a fix for the issue

Next update at 15.30 CET
Posted Mar 09, 2021 - 14:31 CET
Update
We are continuing to work on a fix for the issue

Next update at 14.30 CET
Posted Mar 09, 2021 - 13:27 CET
Update
We are continuing to work on a fix for the issue

Next update at 13.30 CET
Posted Mar 09, 2021 - 12:53 CET
Update
We are experiencing a delay in outbound emails due to our email provider. We are actively working with our email provider to send queued emails. Agents are still able to receive their login email and all emails will be sent. We understand the inconvenience and are working as quickly as possible to resolve this

Next update at 12.30 CET
Posted Mar 09, 2021 - 11:47 CET
Update
We are continuing to work on a fix for the issue

Next update at 11.45 CET
Posted Mar 09, 2021 - 10:43 CET
Update
We are continuing to work on a fix for the issue

Next update at 10.45 CET
Posted Mar 09, 2021 - 10:17 CET
Update
We are continuing to work on a fix for the issue

Next update at 10.15 CET
Posted Mar 09, 2021 - 09:46 CET
Update
We are continuing to work on a fix for the issue

Next update at 09.45 CET
Posted Mar 09, 2021 - 09:12 CET
Update
We are continuing to work on a fix for the issue

Next update at 09.15 CET
Posted Mar 09, 2021 - 08:43 CET
Update
We are continuing to work on a fix for the issue

Next update at 08.45 CET
Posted Mar 09, 2021 - 08:13 CET
Update
Log in tokens/links are once again is being sent.

We are continuing to work on a fix for the issue related to outbound emails.

Next update at 08.15 CET
Posted Mar 09, 2021 - 07:43 CET
Update
We are continuing to work on a fix for the issue

Next update at 07.45 CET
Posted Mar 09, 2021 - 07:13 CET
Update
We are continuing to work on a fix for the issue

Next update at 07.15 CET
Posted Mar 09, 2021 - 06:13 CET
Update
We are continuing to work on a fix for the issue

Next update at 06.15 CET
Posted Mar 09, 2021 - 05:10 CET
Update
We are continuing to work on a fix for the issue

Next update at 05.15 CET
Posted Mar 09, 2021 - 04:08 CET
Update
We are continuing to work on a fix for the issue

Next update at 04.15 CET
Posted Mar 09, 2021 - 03:42 CET
Update
We are continuing to work on a fix for the issue

Next update at 03.45 CET
Posted Mar 09, 2021 - 03:12 CET
Identified
Our team have identified the issue and we are working on a fix, that will resolve the issue.

This is also causing new email login tokens to not being sent.

Next update at 03:15
Posted Mar 09, 2021 - 02:43 CET
Update
We are continuing to investigate the issue

Next update at 03.06 CET
Posted Mar 09, 2021 - 02:34 CET
Update
We are continuing to investigate the issue

Next update at 02.36 CET
Posted Mar 09, 2021 - 02:05 CET
Update
We are continuing to investigate the issue

Next update at 02.06 CET
Posted Mar 09, 2021 - 01:35 CET
Update
We are continuing to investigate the issue

Next update at 01.36 CET
Posted Mar 09, 2021 - 01:10 CET
Update
We are continuing to investigate the issue

Next update at 01.06 CET
Posted Mar 09, 2021 - 00:35 CET
Investigating
We are experiencing some difficulties with emails not being sent. We are investigating the matter.

Next update at 00:36 CET
Posted Mar 09, 2021 - 00:07 CET
This incident affected: Email (Outbound).
← Current Status Powered by Atlassian Statuspage