Datto RMM - Activity Log delays

Incident Report for Kaseya Inc

Postmortem

On the 11th February 2025 at 7:51pm UTC (2:51pm EST), Datto RMM partners on the Zinfandel platform started experiencing a delay with Activity Logs of recent activities displaying in the Web UI.

The issue also impacted the Merlot platform as well, albeit to a much smaller extent.

The R&D team pinpointed the root cause as a compatibility issue with the Activity Logging service, which caused the activity logs to enter a processing queue and be processed at a much slower rate.

The issue was remediated by a hotfix for the Activity Logging Service to resolve the compatibility issue. Following the fix, any new Activity Log entries started processing with the expected pace in a separate stream.

In order to avoid losing activity data, the Activity Log entries created prior to the hotfix remained to be processed with the delay they were experiencing. The maximum delay recorded by the time the time the entire queue successfully processed was around 3 days.

Posted Feb 18, 2025 - 13:00 EST

Resolved

This incident has been resolved.
Posted Feb 17, 2025 - 06:44 EST

Update

Activity Logs that were subject to the delay prior to the hotfix are still being processed by the platform and we estimate this to be complete by the end of day today.
Posted Feb 14, 2025 - 08:44 EST

Update

We are continuing to monitor for any further issues.
Posted Feb 13, 2025 - 05:41 EST

Monitoring

A fix has been implemented and we are monitoring the results.

Activity Log entries created after the fix should appear in a timely manner, while entries subject to the delay prior to this fix are being processed by the platform and will arrive with a delay.
Posted Feb 12, 2025 - 09:59 EST

Update

We are continuing to work on a fix for this issue.
Posted Feb 12, 2025 - 06:53 EST

Identified

We are aware of and working on a problem where the Activity Logs are adding to Devices' Activity Cards and the Activity Log with a long delay on the Zinfandel platform.

The Kaseya R&D Team is preparing a fix for this issue which is intended to be deployed later today or in the early hours of tomorrow.

While the fix will prevent delays of new activities being logged in the Web UI, activities prior to the fix will continue to be processed and will arrive subject to the current delays.

Subscribe to the Kaseya Status Page for up-to-date information at https://status.kaseya.com/
Posted Feb 12, 2025 - 06:07 EST
This incident affected: Datto RMM (Merlot (EU2), Zinfandel (US West)).