Summary
Customers hosted on node aue1-bfyii-2746 experienced a service interruption that prevented access to their Datto SaaS Protection accounts. During the outage, affected customers were unable to perform backup, restore, or export operations.
The interruption occurred after the storage infrastructure supporting the node reached full capacity, causing services hosted on the node to become unavailable. Service was restored after additional storage capacity was provisioned and the environment was validated to be operating normally.
Root Cause
This incident was caused by the storage infrastructure supporting node aue1-bfyii-2746 reaching maximum capacity. Once available storage was exhausted, services hosted on the node became unavailable, preventing customers from accessing backup and recovery data or initiating backup, restore, and export operations.
Monitoring systems identified that the node was approaching critical storage utilization levels on 2026-06-29 18:02 UTC, providing advance indication of a capacity risk. However, the planned storage expansion was not completed before available capacity was exhausted due to process gaps between monitoring, alerting, tracking, and remediation activities which delayed the infrastructure capacity expansion required to prevent the disruption resulting in a service interruption for customers hosted on the affected node.
Following identification of the outage, additional storage capacity was provisioned and validated. Service was restored after verification confirmed normal operation of the affected environment.
Incident Timeline
Preventative Measures
To reduce the likelihood and impact of similar incidents, we are implementing the following improvements:
Monitoring and Alerting
Review and enhance storage-capacity thresholds to provide additional lead time before critical utilization levels are reached.
Implement automated escalation procedures when projected capacity consumption exceeds defined risk thresholds.
Increase visibility into storage growth trends and capacity forecasting across the platform.
Incident Management and Response
Establish a standardized end-to-end process for capacity expansion activities with clearly defined ownership and accountability.
Improve coordination and tracking of infrastructure remediation efforts across teams.
Introduce governance checkpoints to ensure capacity-related risks are addressed within established timelines.
Infrastructure Operations
Strengthen processes for planning and executing storage-capacity expansions.
Improve tracking and visibility of infrastructure upgrade activities from initiation through completion.
Conduct regular reviews of capacity-planning practices to better align infrastructure growth with platform demand.
Infrastructure Resiliency
Perform a proactive review of storage utilization across the Datto SaaS Protection environment.
Prioritize capacity-expansion activities for systems approaching critical utilization thresholds.
Continue investing in platform resiliency and scalable infrastructure to support future growth.