Twilio - Errors for the Twilio Rest API Impacting Multiple Twilio Services
Incident Report for Ciptex
Resolved
The issue impacting multiple products has been resolved, and Twilio services are operating normally at this time.
Posted Feb 18, 2023 - 05:05 GMT
Update
The Twilio engineering team continues to observe indications of recovery on our TaskRouter and Flex services, but with continued intermittent issues, which are being identified and addressed.

In parallel, the Twilio teams are continuing to work on the remediation of the following services: Voice Insights, Verify, Sync, Debugger & Alerts, Event Streams, Rest API, Twilio Notify, IoT - Programmable Wireless, and Serverless Functions logs services.

The Twilio engineering teams will continue to work towards full restoration of all affected services.

Updates will be provided in 3 hours or sooner if we have further pertinent information.
Posted Feb 17, 2023 - 23:40 GMT
Update
The Twilio engineering team is observing further indications of recovery on their TaskRouter and Flex services, but with continued intermittent issues, which are being addressed.

In parallel, their teams are working on the remediation of the following services: Voice Insights, Verify, Sync, Debugger & Alerts, Event Streams, Rest API, Twilio Notify, IoT - Programmable Wireless, and Serverless Functions logs services.

The Twilio engineering teams will continue to work towards full restoration of all affected services and we will provide updates every 30 minutes or sooner if we have further pertinent information.
Posted Feb 17, 2023 - 23:09 GMT
Update
The Twilio engineering team is observing further indications of recovery on their TaskRouter and Flex services, but with continued intermittent issues, which are being addressed.

In parallel, their teams are working on the remediation of the following services: Voice Insights, Verify, Sync, Debugger & Alerts, Event Streams, Rest API, Twilio Notify, IoT - Programmable Wireless, and Serverless Functions logs services.

The Twilio engineering teams will continue to work towards full restoration of all affected services and we will provide updates every 30 minutes or sooner if we have further pertinent information.
Posted Feb 17, 2023 - 22:31 GMT
Update
Twilio continues to observe the intermittent recovery of TaskRouter, Flex console login and other underlying services within Flex. Separately, our teams continue to observe degradation in Twilio Notify, IoT - Programmable Wireless, and Serverless Functions logs services which are being triaged and assessed for remediation. The Twilio engineering teams are continuing to work towards full restoration of services and we will continue to provide updates every 30 minutes or sooner if we have further pertinent information.
Posted Feb 17, 2023 - 21:52 GMT
Update
Twilio continues to observe the intermittent recovery of TaskRouter, Flex console login and other underlying services within Flex. Separately, the Twilio teams have observed degradation in Twilio Notify services which are being triaged and assessed for remediation. The Twilio engineering teams are continuing to work towards full restoration of services and we will continue to provide updates every 30 minutes or sooner if we have further pertinent information.
Posted Feb 17, 2023 - 21:20 GMT
Update
Twilio continues to observe the intermittent recovery of TaskRouter, Flex console login and other underlying services within Flex. Separately, the Twilio teams continue to observe degradation in Twilio Notify and IoT - Programmable Wireless services which are being triaged and assessed for remediation. The Twilio engineering teams are continuing to work towards full restoration of services and we will continue to provide updates every 30 minutes or sooner if we have further pertinent information.
Posted Feb 17, 2023 - 20:49 GMT
Update
Twilio continues to observe the intermittent recovery of TaskRouter, Flex console login and other underlying services within Flex. Separately, their teams have observed degradation in Twilio Notify services which are being triaged and assessed for remediation. Their engineering teams are continuing to work towards full restoration of services and we will continue to provide updates every 30 minutes or sooner if we have further pertinent information.
Posted Feb 17, 2023 - 20:22 GMT
Update
Twilio is observing intermittent recovery of TaskRouter, Flex console login, and other underlying services within Flex. The Twilio engineering teams are continuing to work towards full restoration of services and we will continue to provide updates every 30 minutes or sooner if we have further pertinent information.
Posted Feb 17, 2023 - 19:44 GMT
Update
Twilio is observing intermittent recovery of Flex/RACE console login and other underlying services within Flex/RACE. The Twilio engineering teams are continuing to work towards full restoration of services and we will continue to provide updates every 30 minutes or sooner if we have further pertinent information.
Posted Feb 17, 2023 - 19:16 GMT
Update
Twilio is continuing to see the intermittent recovery of the Flex Console login, but the underlying services within Flex are still facing issues.

In reference to customers who are already logged into the Flex/RACE console, Twilio are seeing increased recovery. In reference to customers not logged into the Flex/RACE console, please refrain from logging into the console as the Twilio engineers are restricting access to aid the ongoing mitigation process.

The Twilio engineering teams are continuing to work to remediate these issues, and updates will be provided every 30 minutes or sooner if we have further pertinent information.
Posted Feb 17, 2023 - 18:38 GMT
Update
Twilio is continuing to see the intermittent recovery of the Flex Console login, but the underlying services within Flex are still facing issues.

In reference to customers who are already logged into the Flex/RACE console, Twilio are seeing increased recovery. In reference to customers not logged into the Flex/RACE console, please refrain from logging into the console as the Twilio engineers are restricting access to aid the ongoing mitigation process.

The Twilio engineering teams are continuing to work to remediate these issues, and updates will be provided every 30 minutes or sooner if we have further pertinent information.
Posted Feb 17, 2023 - 18:10 GMT
Update
Twilio is seeing intermittent recovery of Flex Console login, but the underlying services within Flex are still facing issues. The Twilio engineering teams are continuing to work to remediate these issues. Updates will be provided every 30 minutes or sooner if we have further pertinent information.

Please note that automated SMS routing and SMS conversion rates are not impacted at this time.
Posted Feb 17, 2023 - 17:43 GMT
Update
The Twilio engineering teams are continuing to implement the redeployment of the underlying service. No additional information is available at this time. Updates will continue to be provided every 30 minutes or sooner if we have further pertinent information.
Posted Feb 17, 2023 - 17:08 GMT
Update
The Twilio engineering teams are continuing to work on the redeployment of the underlying service. No additional information is available at this time. Updates will be provided every 30 minutes or sooner if we have further pertinent information.
Posted Feb 17, 2023 - 16:48 GMT
Identified
The Twilio engineering teams are continuing to work on the redeployment of the underlying service. No additional information is available at this time. Updates will be provided every 30 minutes or sooner if we have further pertinent information.
Posted Feb 17, 2023 - 16:27 GMT
Update
The Twilio engineering team have identified that a degraded underlying service is causing failures on upstream services affecting multiple products. They are redeploying the underlying service across all other services. Further updates will be provided every 30 minutes or sooner if we have further information.
Posted Feb 17, 2023 - 15:49 GMT
Investigating
Twilio have declared a Major Incident related to ongoing issues affecting multiple products, including Flex, Programmable Voice, ElasticSearch, Insights, Voice insights, Programmable SMS, Automated SMS routing, Task Router Statistics, Agent Login, and Fraud Guard. All necessary teams have been engaged to remediate the issue and restore services as soon as possible. Further updates will be provided in 30 minutes or as soon as we have further information.
Posted Feb 17, 2023 - 15:08 GMT
This incident affected: Twilio (Twilio Sync, Twilio TwiML Bins, Twilio SMS, Europe, Twilio Assets, Twilio SMS, Twilio Studio, Twilio Voice Insights, Twilio TwiML, Twilio TaskRouter, Twilio Flex Application Platform, Twilio Flex Insights, Twilio SMS, Europe).