Spreedly experienced an issue within an automated deployment process, which resulted in a degraded performance for the API with a partial outage for gateways.
What Happened
On January 4, 2021, Spreedly engineers released a new version of the application, the deployment of which failed due to an issue within an automated process. This made a negative impact to the production servers. Spreedly’s internal systems recognized communication issues with some payment gateways. Internal response teams immediately started to investigate the issue. While investigating, it became clear that the communication issues were not limited to communication with some gateways, but also degraded performance for the Spreedly API.
Spreedly engineers released an update to the deployment process which fixed the issue with the internal automated process. As a result, our internal systems showed signs of the API and gateway connections returning to normal.
Next Steps
- Investigating how to ensure changes to automated deployment processes will not affect production traffic.
- Proactively add notices to Status Page when the Spreedly team has discovered degradation.
We are still investigating to understand the specific causes of the incident and will publish a post-mortem.
We apologize for any inconvenience and disruption to service.
We are actively monitoring the results.
This is impacting a subset of transactions and requests to Spreedly's API.
Updates will be provided as they become available.