Access Log Processing Delayed
Incident Report for Cloudsmith
Resolved
This incident has been resolved.
Posted Mar 02, 2021 - 17:19 UTC
Monitoring
Investigation showed that: the log processing issue got progressively worse and after investigating it seemed that the cause was a combination of a small number of inefficient database queries and the logic for how to split logs up for processing among workers was causing database contention.

We have applied a number of fixes and brought the log processing queue back to zero. Please note; we may have some inconsistent download counts, over the last 8 hours, but this is limited to the Docker format. Bandwidth usage metering is unaffected. If this is a significant issue for you; please contact us.

We are actively monitoring the situation.
Posted Mar 02, 2021 - 10:46 UTC
Investigating
Our data processing infrastructure is running behind which is causing inaccuracies in the reporting tools. This impacts accounting (e.g. usage, metrics and entitlement restrictions), as well as retrieval and display of recent download logs for repositories. Package downloading/uploading is not affected.

No data has been lost and the system should be caught up shortly.
Posted Mar 01, 2021 - 17:05 UTC
This incident affected: Backend Processing.