Incident Summary
From approximately 14:24-15:33 UTC on Friday, April 29, 2022, Ad Serving was reduced for the LAX1 and NYM2 Datacenters
Scope of impact
During the incident window, clients sending requests from regions near the East and West Coasts of the United States, may have seen increased bid request timeouts causing increases in blanks, defaults, and PSAs
Timeline (UTC)
2022-04-29 14:24: Incident started: an invalid object was created
2022-04-29 14:31: Batches were frozen
2022-04-29 14:33: Invalid object was disabled
2022-04-29 15:08: Hot fix release begun
2022-04-29 15:33: Incident resolved
Cause Analysis
The issue occurred when an invalid object was created using cross-device data without the proper enablement settings
Resolution Steps
Engineering disabled the invalid object and a hot fix was applied to the bidder
Next Steps
• Ensure unit tests cover all cases to catch unusual situations that trigger code issues
The incident has been fully resolved. We apologize for the inconvenience this issue may have caused, and thank you for your continued support.
We have patched the issue and are monitoring our systems closely. We will provide an update as soon as the issue has been fully resolved.
We have identified the following issue:
Our engineers are actively working to release a fix to the issue, and we will provide an update as soon as possible. Thank you for your patience.