This article explains best-practices and considerations for backing up a Microsoft Exchange database.
- Microsoft Exchange
A ShadowSnap backup using the VSS engine leverages Microsoft's VSS framework to quiesce the Exchange database to snapshot the database for a backup. Therefore, it is essential that the Microsoft Exchange VSS Writer is functioning for a proper backup of your exchange database as well as for proper log truncation. Additionally, Datto recommends circular logging to prevent large backups.
This article assumes you have basic knowledge of Microsoft's VSS framework. For a refresher, see the following links:
- Volume Shadow Copy Service (external link)
- Volume Shadow Copy Service Technical Reference (external link)
ShadowSnap, Microsoft's Exchange, and VSS
The Microsoft Exchange VSS Writer's first job is to tell ShadowSnap about the data needed for backup, especially the EDB file, logs, and checkpoint file for each database requested. The information about these specific Exchange data files is known as writer metadata.
For example, suppose the Exchange Writer tells ShadowSnap that there is a database located in a folder on volume E:, and that transaction logs for that database are in a folder on D:. Based on that information ShadowSnap will request snapshots of the D: and E: volumes when the job progresses.
The Exchange VSS Writer serves another critical role besides providing metadata to ShadowSnap. It also has the job of stopping writes to the databases and logs on disk, or “freezing” them, for the time it takes to create the necessary snapshots. The Exchange Writer prevents the Information Store Service, or the MS Exchange Replication Service, from writing what’s in RAM to the frozen database files. In the case of the Information Store Service, the current transaction log file (Exx.log) gets rolled and closed out before the Exchange Writer allows VSS to take the snapshot. This ensures nothing changes in the file data between the beginning of the snapshot and the completion, at which point the databases are “thawed”. When databases are thawed, write I/O held in RAM is allowed to go to disk again.
Exchange Logs and Truncation
Microsoft's Exchange does not write transactions directly to a mailbox database. Instead, the data is written to a transaction log. Transaction logs are designed to protect the data that has accumulated since your last backup. However, they can only protect you if they are stored on a separate disk (not just a separate volume) from the mailbox database.
For example, if the volume containing the database were to fail, you could restore from your ShadowSnap backup which would bring the database back to the state at which it existed at the time that the backup was created. The transaction logs are then used to retrieve any data that has accumulated between the time of the backup and the time of the crash if they were stored on a volume that did not also fail.
But, transaction data does not reside in the transaction logs forever. It has to be written to the mailbox database at some point. This is actually done as a part of the backup process. There is also a checkpoint file that keeps track of which log files have and have not been committed.
The last major responsibility of the Exchange Writer is to tell the Information Store Service (MS Exchange Replication Service in the case of a passive copy backup) that the backup was completed and, if applicable, to carry out post-backup tasks like log truncation, marking the database to indicate there was no longer a backup in progress, etc.
The Datto Windows Agent leverages the protected system's VSS writers to take a backup in the same way that the ShadowSnap agent does.As a result, VSS writers are what trigger log truncation. If backups are working correctly, log truncation should work exactly as with any other backup agent. If log truncation does not appear to be working as expected, see the Troubleshooting Log Truncation section of this article.
Troubleshooting Log Truncation
If you are troubleshooting issues with log truncation, you should first be sure that ShadowSnap is triggering a VSS backup. If you are in fact triggering a VSS backup via ShadowSnap and there is no log truncation, Datto recommends ensuring your Microsoft Exchange Writer is functioning properly. The Microsoft Exchange Writer is responsible for log truncation, NOT ShadowSnap. Beyond that, you may need to look deeper into your Exchange configuration to determine the cause.
For example, you can use the following PowerShell command to determine which type of backup Exchange has recently interpreted:
Get-MailboxDatabase -Status | ft name,last* -auto
Log truncation occurs after a successful full or incremental exchange VSS backup. These backup types are specific to Exchange and VSS and are NOT synonymous with the ShadowSnap full, incremental, and differential merge backups.
Backing up Database Availability Groups (DAGs)
What is a DAG?
Database Availability Groups (DAGs) are the primary fault-tolerant mechanism used for protecting mailbox databases in Exchange Server 2010 and 2013.
Does Datto support backing up a DAG?
Yes. Datto recommends including DAGs in your backups.
How does Datto handle multiple DAGS in a clustered environment?
The ideal configuration to back up any DAG is to install the Datto solution on the server with the primary or active DAG. You can then add the server as an agent to your Datto appliance, and set backups to that DAG.
What are the advantages of using the Datto solution over Microsoft backups?
While having redundant database copies alone would protect you against node failure or against a server volume failure, this approach does not offer protection against malware that would require you to rollback to a previous state of the database. Microsoft does offer something called a 'lagged database copy' to help prevent damage in this scenario, but the process to restore using this method can be difficult.
Does backing up with the Datto solution truncate logs?
Log truncation can be challenging when dealing with DAGs, as there are many possible configurations that can lead to differences in behavior for log truncation. In our experience, log truncation works most consistently when backing up the primary / active / mounted DAG member. But, depending on the replication policy, this may not produce the expected log truncation results, or may not be the best option altogether.
Is there a need to use circular logging?
In the case of replicated databases, Datto recommends using a log truncation method called circular logging. Circular logging uses and reuses a finite number of log files as opposed to creating and later truncating a continuous stream of log files.
Due to the way that the Datto solution works, circular logging is necessary, because our backups record data change when taking an incremental backup. When circular logging is disabled, log files can grow exponentially.
While this may not use a large amount of storage on the server, these log files are seen as data change by the incremental tracker, and are recorded as new data. This can make your usually small incremental backups become large, and take much more time and space than they would with circular logging on.
Does backing up a DAG cause a quiescing performance issue?
VSS writers freeze databases when they capture a snapshot to ensure that no data changes as the backup is being taken. The only time you should ever see a major pause will be during the agent's first backup, when a full backup of the entire server is needed. This can be done during the night to prevent impact to the database. The following backups will be incrementals, with a minimal impact to the database.
Should I use the ShadowSnap or the Datto Windows Agent?
Since log truncation is handled similarly in both installations, using ShadowSnap or the Datto Windows Agent does not matter; they should both back up the DAG correctly.
When using ShadowSnap, the VSS writers will trigger log truncation through Exchange. When using the Datto Windows Agent, Exchange will interpret backups as full VSS backups (VSS_BT_FULL).
If the VSS framework and the Exchange Writer are functioning properly, Exchange log truncation will occur. The incremental tracker will then dictate which blocks to back up.
How do I ensure VSS quiescing is happening properly for agentless backups to trigger log truncation?
Because VSS is utilized through VMWare Tools, log truncation should occur as long as the VMWare Snapshot Provider and the VSS framework and VSS writers are functioning properly. If this is not occurring you may need to troubleshoot the VSS writers, VMWare Tools, and investigate if VMWare VSS quiescing is healthy.
While Datto support has simple troubleshooting suggestions that can be offered, you may need to reach out to VMWare support VMWare tools if basic troubleshooting does not resolve any VMWare Tools VSS quiescing issues.
- Troubleshooting VMWare VSS quiesce related issues
- Disabling VSS quiesed snapshots
- Disabling specific VSS writers with VMWare Tools