Summary

On August 10th, 2022, at 3:20 AM EDT, the Kustomer team started to receive transient alerts for a high error rate in the Instagram service that was caused by errors in our requests to the Instagram API. After discovering the root cause, the incident was resolved at approximately 2:20 PM EDT.

Impact

Users of the application across organizations would have been unable to send messages via Instagram that specifically included a link inside of the message body from 3:20 AM - 2:20 PM EDT. Outbound Instagram messages that did not contain a link would still have been successful during this time.

Next Steps

The engineering team will be updating the runbook for handling potential Instagram API issues to increase the speed with which we identify the root cause of any potential issues.

Posted Aug 12, 2022 - 17:37 EDT

Resolved

The issue with instagram outbound messages has been resolved,

we are will be re-driving any stalled messages now.

Thanks for your patience.

For any questions,
Please reach out to our Support team with any additional questions.
You can reach us by going to https://help.kustomer.com/ and clicking "Contact Support" at the top of the page.

Posted Aug 10, 2022 - 14:29 EDT

Monitoring

Outbound Messages should now send appropriately but there is still an issue with sending outbound messages that contain a Link(hyperlink), Causing outbound messages containing a Link to stay in "sending" status.

We will be able to re-drive these messages once Messages recover to full functionality.

We are continuing to Monitor this and will relay any updates.

For more information you can visit https://metastatus.com/

For any questions,
Please reach out to our Support team with any additional questions. You can reach us by going to https://help.kustomer.com/ and clicking "Contact Support" at the top of the page.

Posted Aug 10, 2022 - 12:23 EDT

Identified

Instagram is currently experiencing API issues, this is causing delays with Comments and DMs.
We will provide updates on any changes.

For more information you can visit https://metastatus.com/

For any questions,
Please reach out to our Support team with any additional questions. You can reach us by going to https://help.kustomer.com/ and clicking "Contact Support" at the top of the page.

Posted Aug 10, 2022 - 10:04 EDT

Investigating

Posted Aug 10, 2022 - 09:16 EDT

This incident affected: Prod1 (US) (Channel - Instagram) and Prod2 (EU) (Channel - Instagram).