SIRIS, ALTO, and NAS: Error: "Backup failed due to I/O error on write operation from the protected system during the backup process"

Follow

Issue

When you try to start a backup for a system protected by the ShadowSnap agent, you receive the error message, "Backup failed due to I/O error on write operation from the protected system during the backup process."

In legacy versions of the Datto IRIS environment, this error message was "STC -31 on Write."

Environment

  • ShadowSnap Agent

Cause

Causes of this error include:

  • Networking / Samba trouble
  • Server tuning variables need to be adjusted
  • Bad install of the agent
  • Protected volume is out of space
  • System writer is not present on the protected machine
  • VSS subsystem does not work
  • Hardware problems on the protected machine
  • Bad sectors on the protected disk
  • Disk errors and warnings in the event log
  • ATAPI errors and warnings in the event logs

The AppAssure application can also cause this error. It is incompatible with the Datto backup solution. When uninstalling the AppAssure program, Datto recommends following this process to ensure that you remove the AVolFlt family driver correctly. This driver has been found to remain on a protected system even after you've deleted AppAssure (also known as Rapid Recovery or Replay).

Resolution

Check disk health

1. Check the agent logs in the Protect tab of the Datto appliance to see if backups for the server are constantly failing in the same state at the same spot on the disk.

2. To further confirm the issue, attempt a ShadowProtect backup to a NAS share on the Datto appliance. Observe whether or not the backup fails in the same state. ShadowProtect backups will fail at the same point as Datto backups if there is a problem with the disk.

3. Perform a chkdsk /r on the production machine for all volumes with the filesystem unmounted. Do not attempt the chkdsk /r in read-only mode. If Windows has not identified corruption on the disk, a chkdsk with Windows mounted will not either. Running with the system unmounted ensures direct access to disk I/O and will provide a more accurate indication of the disk state. Errors may not appear in the event log if the corruption has not become apparent to the filesystem yet.

4. If these steps do not solve the problem, and you have ruled out disc errors, refer to the Further Troubleshooting section below.

Check system resources

1. Check the protected volume to make sure there is enough space to perform a backup. A minimum of 20% free space per protected volume and 1 GB of RAM need to be available on the production machine. If resources fall below these thresholds, backups will fail. Reducing system overhead during backups can also help prevent this issue.

 Volume Use

 Status

 Comments

 0 – 70%

 Good

The use is good.

 71 – 80%

 Warning 

The use is outside of optimal limits. Monitor the volume growth and look at cleaning up unnecessary files. Move non-essential data to other storage.

 81 – 100%

 CRITICAL

The volume is becoming critical. Snapshots will start to fail; disk queue lengths will increase. Create space.

Figure 1: StorageCraft disk use best-practices.

2. If the IP address of the protected agent has changed, update the protected system's address from the Datto appliance GUI. If the Datto appliance needs to follow special routing to reach the protected system, contact Datto Technical Support to assist with the changes.

Further troubleshooting

1. Ensure that a stable route from the Datto device to the ShadowSnap agent is set up on port 25566.

2. Check for any stuck file restores or virtual machines that do not show up in the web interface. You can destroy these as long as they are not in use.  

3. Check the agent version.

4. If an update is available, upgrade the agent.   

5. Check the used storage on the Datto appliance. If used space is 85% of total capacity or higher, you may find increased reliability by lowering the amount of used storage. See: Backup skipped due to not enough free space for more information.

6. Make sure that ports 25566 and 139 are open on the local Windows Firewall. 25566 will be open if the raw_agent_svc is running as part of the ShadowSnap agent. Another program might have taken the port if something was listening on it.

7. Make sure that anti-virus exceptions exist for Program Files (x86)\StorageCraft

8. Investigate possible server tuning variables as suggested by StorageCraft

9. Check the Windows Event Logs for the error.  If it does have an error, please check for and disable possible conflicting services, then uninstall and manually reinstall ShadowSnap and ShadowProtect.

10. Check network connectivity between the protected system and the Datto device. Rule out any IP address conflicts with the protected machine or the Datto device. Replace the network cable for the protected server, and try a different switch port.

11. If backup errors persist, gather the ShadowProtect/ShadowSnap Diagnostic Logs and open a case with Datto Technical Support for further assistance.

Additional Resources


Was this article helpful?

0 out of 0 found this helpful

You must sign in before voting on this article.

Want to talk about it? Have a feature request?

Head on over to our Datto Community Forum or the Datto Community Online.

For more Business Management resources, see the Datto RMM Online Help and the Autotask PSA Online Help .

Still have questions? Get live help.

Datto Homepage