What happened

During the incident window, our US Saas Defense environment experienced:

• Slow or delayed access to scan results and email-related features
• Intermittent timeouts and partial unavailability of parts of the UI
• Periods when customers could not reliably interact with their scan results

The service did not experience a complete outage, but performance and responsiveness were significantly impacted over multiple days.

Root cause

The incident was triggered by an issue in our database cluster for the US region:

1. A missing database index prevented the normal cleanup of data older than 100 days.

2. This caused unexpected data growth and resource exhaustion (disk and CPU) on the US database cluster.

3. To remediate, we:

* Added capacity \(disk and nodes\) and
* Created the missing index to enable proper cleanup.

1. Because the database was slow to respond, our message-processing system accumulated a large backlog of messages, which further impacted responsiveness and led to the partial service interruption.

In summary, a combination of data cleanup not functioning as intended, index creation under load, and automatic database rebalancing caused prolonged high load on our US database cluster and degraded service performance.

How we resolved the issue

To restore normal service, our engineering teams:

1. Increased capacity in the US database cluster (additional disk space and nodes).
2. Created and optimized database indexes to resolve the underlying data growth issue and improve query performance.
3. Temporarily disabled non-essential background tasks (such as backups that added extra load) during the incident.
4. Actively managed message queues in our processing system (including splitting and redistributing queues) to reduce pressure on the database and unblock the UI.
5. Closely monitored the database rebalancing process until it completed successfully on January 15, 2026, at which point CPU usage normalized and all application functionality was fully restored.

What we are doing to prevent this in the future

We are implementing several improvements to reduce the likelihood and impact of similar issues:

• Stronger monitoring around high-risk changes
  We are enhancing observability and adding focused monitoring around database changes and high-impact operations to detect performance regressions earlier and respond faster.
• Improved data lifecycle and index management
  We are reviewing and tightening controls around data retention and index configuration to ensure cleanup jobs and maintenance operations run efficiently and safely.
• Refinements to capacity and load management
  We are updating our scaling and maintenance procedures—particularly around adding/removing database nodes and triggering rebalancing—to minimize customer impact when capacity changes are required.

These actions are aimed at detecting similar issues earlier, limiting customer impact, and shortening recovery time.

Closing

We understand the importance of Saas Defense to your operations and regret the disruption this incident caused. We are committed to learning from this event and strengthening the reliability and resilience of our platform.