[Production] - V2 + V3 - APP + API - Slowness on our platform (V2) & signature processing delays (V2 & V3)
Incident Report for Yousign
Postmortem

Incident Impact and Resolution Report

Dear Yousign Users,

On 2024-02-29, an unexpected service disruption has occurred, affecting every customer using Yousign V2 both via the API or Web Application. As a side effect, this disruption also impacted Yousign V3 application with increase signature delays. The incident lasted for approximately 3 hours and 47 minutes.

On Yousign V2, the disruptions started at 15:47 CET, initially manifesting with increased response times and occasional timeouts. The situation gradually escalated to more frequents timeouts, HTTP 500 errors but also delay in notifications like SMS. All services were back to normal at 19:42 CET.

On Yousign V3, the signature slowdown started at 15:47 CET. The situation was back to normal at 19:27 CET.

The impacted components during this incident were:

We want to assure you that despite the disruption, no information was lost, and all received signatures were properly processed.

Root Cause Analysis

One of our customer had an unexpected and inappropriate use of our Yousign V2 API product. The application is not able to react and block this particular use-case properly.

This particular use-case led to a bottleneck in the Yousign V2 backend, but also in our signature engine as it is common to both Yousign V2 and Yousign V3.

Correction

To mitigate the issue on Yousign V2, we had to manually stop some non-critical processes to reduce the pressure on our backend and aleviate the bottleneck.

We also deployed an hotfix on the signature engine in order to mitigate the impact of Yousign V3.

Permanent Fix

To prevent this issue from occurring again in the future, the following actions will be performed:

  • Isolate Yousign V2 and Yousign V3 from a signature workflow perspective so that slowdowns in Yousign V2 does not impact Yousign V3 and vice versa
  • Setup a watchdog that will reduce the impact on requests related to an inappropriate use of Yousign V2 API
  • Perform a deeper investigation internally in order to be more reactive

Thank you for your continued trust in Yousign.

Posted Mar 04, 2024 - 10:55 CET

Resolved
This incident has been resolved.
Posted Feb 29, 2024 - 19:42 CET
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Feb 29, 2024 - 19:34 CET
Update
Signature processing delays concerns now only our V2. There is no more delay for V3.

Our teams are mobilized to resolve this incident as quickly as possible. We apologize for any inconvenience.

V2 - APP + API - Slowness on our platform (V2) & signature processing delays (V2)
Posted Feb 29, 2024 - 19:27 CET
Update
Our teams are mobilized to resolve this incident as quickly as possible. We apologize for any inconvenience.

V2 + V3 - APP + API - Slowness on our platform (V2) & signature processing delays (V2 & V3)
Posted Feb 29, 2024 - 18:51 CET
Investigating
Our V2 platform is impacted by a general slowness.

Signature processing is taking longer than expected (V2 & V3)

SMS are stuck in queue and therefore not sent in time (V2 only)

Start date : 29/02/2024 - 15:47
Posted Feb 29, 2024 - 16:27 CET
This incident affected: Yousign V2 (API V2 - https://api.yousign.com, APP V2 - https://webapp.yousign.com) and Yousign V3 (APP V3 - https://yousign.app, API V3 - https://api.yousign.app).