For more information about our Incident Response and Communications please read this support article.

We also maintain a list of Known Product Issues separate from this site here.

[Major] Service Degradation - Public API, Downloads, Collections, Notifications, Annotations & Conversion.
Incident Report for Box
Postmortem

We recently addressed issues affecting uploads, downloads and the public API. We would like to take the opportunity to further explain these issues and the steps we have taken to keep them from happening in the future.

On January 19th, 2021, between 5:21 PM PST and 7:03 PM PST some users may have experienced difficulties while working in Box. During this time, impact to the downloads, uploads, and the public API were observed. The issue occurred as a result of hardware failure after scheduled maintenance to the upgrade infrastructure of our internal DNS grid. We were able to resolve the issue by disabling the degraded member. In addition, this hardware upgrade process is being replanned and increased monitoring and alerting is being implemented to prevent similar issues from occurring in the future. 

Analysis 

  1. Hardware failures led to connection issues and ultimately degradation of DNS.

  2. Insufficient edge-case alerting/monitoring prolonged detection of the issue.

Corrective Actions

The following corrective actions have been completed or are planned:

  • Hardware upgrade process changes

  • Augmentation to current monitoring & alerting

  • Configuration tuning to mitigate similar issues faster

  • Working with vendor on next steps to address hardware issues

We are continuously working to improve Box and want to make sure we are delivering the best product and user experience we can. We hope we have provided some clarity here and we would be happy to answer any questions you may still have regarding this matter. 

 

Sincerely,

The Box Team

Posted Feb 04, 2021 - 07:01 PST

Resolved
We have completed our monitoring period and are considering the issue fully resolved. If you are seeing any new issues, please let us know at https://support.box.com.
Posted Jan 19, 2021 - 19:48 PST
Monitoring
We have taken steps to to resolve this issue and are monitoring for any further impact. If you continue to see issues, please contact Box Support at https://support.box.com.
Posted Jan 19, 2021 - 19:22 PST
Investigating
We are investigating reports of an issue impacting Public API, Downloads, Collections, Notifications, Annotations, Conversion. We will provide updates here as soon as possible.
Posted Jan 19, 2021 - 18:19 PST
This incident affected: Box Platform / API (Content API) and Box Web Application (Uploads/Downloads, Email Notifications).