Description

This article explains why some files on the Datto Workplace v10 virtual drive (commonly mapped as M:\) fail to sync to the cloud, and how to protect and recover that data safely.

Please be advised that this article was written as a response to a number of data loss incidents. It is not a guaranteed fix for incorrect usage of Datto Workplace v10, it is a best effort list of possible steps to work through to rescue files from deletion.

Files on the Workplace drive are not ordinary files on the local disk. They live in an encrypted local cache and are streamed to and from the cloud on demand. Two behaviours follow from this, and they cause most of the data-loss cases we see:

  • Files that stay open and constantly change do not sync. They sit in volatile cache only, and can be lost on reboot or wiped by the next sync.
  • If the agent is uninstalled or reset before cached data has safely reached the cloud, that data is lost. Backing up the cache folder alone does not reliably protect it, because the cache is encrypted and needs a specific key file to be decrypted afterwards.

Workplace is a file sync product for files that are saved, closed, and then synced. It does not handle files that an application keeps open and changing while they are being worked on. The safe exception is documents opened through the Workplace plugin or their online editors (for example Microsoft Word and Excel, or Google Docs and Sheets opened via the plugin), which are handled correctly.

File types that will not sync reliably include, but are not limited to:

  • CAD files
  • Blender, Maya and other 3D or rendering project files
  • Adobe Photoshop .psd files
  • Adobe Premiere .prproj files
  • Large database or working files that an application keeps locked open

While an application holds one of these files open, it stays in volatile cache and does not reach the cloud. On reboot that cached copy can be lost. If a sync runs while the cloud copy is empty or behind, the cloud can push its empty or older directory structure down and wipe the local working copy.

You would typically use this article when a user reports that a file or folder is present on the M:\ drive but missing or empty in the Workplace portal, that recent work has disappeared after a reboot, that a directory has vanished after a sync or a reinstall, or that sync appears stuck.

Requirements

  • An administrator-level login on the affected endpoint
  • Access to the Datto Workplace web portal for the affected user or team
  • Somewhere safe to copy rescued data to, off the Workplace drive (a local disk, network share or external drive)
  • Ideally, a live "hot" imaging tool that runs inside Windows without a restart (for example Disk2vhd or Veeam Agent for Windows Free), for a belt-and-braces image before any change. Do not use offline or bootable-media imaging here, as the reboot it needs can lose the data.
  • For any decryption and recovery, the agent.<deviceID>.db key file (see the Process below)

The Process

Important: read before any uninstall or reset

There is a reset procedure in circulation that reads roughly as: back up the local Workplace CloudConnect folder, then fully uninstall the agent, remove all user data, delete the Datto folders under ProgramData and AppData\Local, clear the registry keys, and reinstall.

Do not run this on a machine that still has un-synced data in the local cache. It is the single most common cause of permanent data loss on this product. Any work that has not been fully committed to the cloud lives only in the encrypted local cache, and once the agent is reset and reinstalled there is no supported tool that ingests that pending data. A copy of the Workplace CloudConnect folder on its own is not enough to get the data back, because it cannot be decrypted without the key file described in step 4.

If un-synced data is at risk, treat the machine as fragile and complete the steps below before touching the agent.

1. Check for long file and folder names

Very long file paths and directory names can block sync. Check for these on the affected drive and rename them to something shorter before troubleshooting further.

2. Move volatile working files off the Workplace drive (prevention)

This is the step that stops the problem recurring.

If the end user works with CAD, Adobe, 3D or any application that keeps files open and changing, those files should live on the machine's real SSD or HDD while they are being worked on. Once a file is finished and closed (at rest), it can be copied into the Workplace drive to sync to the cloud. Workplace is fine as the resting place for these files. It is not fine as the live working location.

3. Protect the data before any reset

If there is un-synced data on the endpoint, assume it exists only in volatile cache and can be lost. Do not simply back up the local AppData Workplace cache folder and reinstall.

Be aware of one important trap before you start: a reboot can lose the data. Un-synced work sits in volatile cache, so anything that restarts the machine (including shutting down to image the disk offline, or booting from rescue media) can wipe it before it is safe. For that reason, do not use offline or bootable-media imaging tools here, and do not reboot until the data is confirmed safe elsewhere.

Work through this in order, with the machine left running the whole way:

  1. Copy the affected files out first, while the machine is still up. Drag or copy the files from the Workplace drive to a safe location off that drive (a local disk, network share or external drive). This is the quickest way to get the user's actual data to safety and needs no reboot.
  2. Confirm the rescued data is intact and ring-fenced before going any further.
  3. As a second layer of insurance for any pending or open data, take a live "hot" image of the system disk while Windows is still running. Live imaging uses Microsoft's Volume Shadow Copy Service (VSS) to take a consistent snapshot without a restart, and it captures files that are locked open by the running agent (the encrypted cache and the key file), which a plain copy may skip. Note that you are imaging the real system volume (usually C:), where the cache lives, not the virtual M:\ drive.
  4. Only once the data is confirmed safe should you remove the files from the Workplace drive and re-establish sync, with everything already backed up.

Do not reset or reinstall the agent until the data is confirmed safe elsewhere.

Tools for live imaging without a reboot (Windows): (NB, these tools are not endorsed by Manage Protect, they are suggestions. If you can manually copy the files to a safe place before using these tools, you will increase your chances of saving data dramatically)

  • Disk2vhd (Microsoft Sysinternals) is the simplest option for a one-off rescue. It is free, portable, needs no installation, and takes a VSS block-level snapshot of the running system into a .vhd/.vhdx file. Note it needs the volume to be unlocked if BitLocker is in use.
  • Veeam Agent for Microsoft Windows (Free) is a fuller option that also images live via VSS and is free for business use on your own machines.
  • Paid VSS imagers such as Macrium Reflect, Acronis or AOMEI Backupper also do live imaging if you already license one. (Macrium no longer offers a free edition.)

Avoid anything that requires booting from rescue media or shutting the machine down, as that is exactly the reboot that can cost the data.

On macOS there is no direct equivalent to VSS. The safe approach is the same in spirit: copy the working files off the Workplace drive to safe storage while the machine is running, and do not reboot until the data is secured.

4. Capture the decryption key if a cache needs recovering

If a CloudConnect cache does need to be decrypted and recovered, one specific file is required: a database key file whose name starts with agent. followed by the device ID and ends with .db, for example agent.1771297284291.db.

Without this file, the cache can still be decrypted, but the recovered files come back with numeric names only, no original filenames and no file extensions. With it, the original filenames and folder structure can be rebuilt.

Its location varies by operating system and build:

  • On Windows it is typically within the Workplace CloudConnect cache folder, or under AppData\Local\Datto.
  • On macOS it is typically under ~/Library/Preferences/com.datto.dwc.

Because the location varies, capture the whole cache and the whole Datto app-data area rather than hunting for one file. Backing up everything ensures the key file is captured with it.

5. If data is already stuck or lost, recover it safely

  1. Stop. Do not reboot repeatedly, do not reinstall, and do not force further syncs, as each of these can overwrite what is still recoverable.
  2. Check the Workplace web portal version history for the affected files, in case a good version is still held in the cloud.
  3. Check for application temp or autosave files on the local disk. In one case a critical Excel file was recovered from a temp file the application had left behind after an incomplete save.
  4. If the data only exists in the local cache, capture the entire Workplace CloudConnect cache folder and the Datto app-data area (this ensures the agent.<deviceID>.db key file is included), and zip it up before doing anything else to the machine.
  5. Provide the zipped cache to Manage Protect so the engineers can attempt to decrypt it and rebuild the original filenames and folder structure.

Recovery of pending cache data is best-effort. Data that was never committed to the cloud before a reset may not be recoverable, which is why the earlier steps exist to avoid ever reaching that point.

Where you are unsure whether data is safe to reset, raise it with Manage Protect before any agent reset rather than after.