We have restored all active alerts for the initial 10 customers identified. An exhaustive search was performed which resulted in finding two additional affected customers. Active alerts for the remaining two customers are being backfilled now.
All affected customers will receive a Root Cause Analysis early next week detailing our findings. Preventative measures are being put in place to avoid recurrence. We will close this incident once all active alerts have been restored.
Posted 2 years ago. Apr 27, 2023 - 20:00 CDT
Update
LogicMonitor engineers have uncovered the root cause which stemmed from our capacity management tools encountering an unknown issue. These actions have been stopped while we fully debug and restore services. We have confirmed that only 10 customers are affected by this issue. We have also begun a process to recover alerts for the past 30+ days, but are still currently working on the full restoration of all alerts. Alert generation, alert delivery, and data ingestion remain fully operational and unaffected, as this is still only affecting the ability to see alerts in the Alert page. Further updates expected as we make progress towards recovering to 30 days and full history of alerts, ~4 hours from now.
Posted 2 years ago. Apr 27, 2023 - 11:33 CDT
Identified
LogicMonitor has identified an issue that is affecting a small subset of customers' ability to see historic Alerts greater than 15 days old. We are still investigating the root cause, but alert generation and delivery are unaffected and remain 100% operational. We will have a more detailed update once our Development team can further diagnose the issue. Updates expected ~8 hrs from now.
Posted 2 years ago. Apr 26, 2023 - 20:10 CDT
Investigating
LogicMonitor is currently investigating historical alerts not displaying on the Alerts page. This issue is only affecting a small subset of accounts.
Posted 2 years ago. Apr 26, 2023 - 17:54 CDT
This incident affected: EMEA - West (Alert Log and Access), AMER - EAST Region (Alert Log and Access), and AMER - WEST Region (Alert Log and Access).