Log Shipping for SQL Server

I. Introduction

Log shipping allows you to automatically send transaction log backups from a primary database on a primary server instance to one or more secondary databases on separate secondary server instances. Transaction log backups are applied to each secondary database individually. An optional third server instance, known as the monitor server, records the history and status of backup and restore operations, and optionally raises an alert if these operations do not occur as planned.

Benefits of log shipping:

• Provides a disaster recovery solution for a single primary database and one or more secondary databases, each on a separate instance of SQL Server.

• Supports limited read-only access to secondary databases (during the interval between restore jobs).

• User-specified delay between when the primary server is allowed to back up the primary database's log and when the secondary server must restore (apply) the log backup. A longer delay may be useful, for example, if data on the primary database changes unexpectedly. If unexpected changes are noticed soon, the delay allows you to retrieve still-unchanged data from the secondary database before the changes are reflected in the secondary database.

Log shipping consists of three operations:

1. Back up the transaction log on the primary server instance.

2. Copy the transaction log files to the secondary server instance.

3. Restore the log backup on the secondary server instance.

Logs can be shipped to multiple secondary server instances. In this case, actions 2 and 3 are repeated for each secondary server instance.

The log shipping configuration does not automatically fail over from the primary server to the secondary server. If the primary database becomes unavailable, any secondary database can be brought online manually.

A secondary database can be used for reporting.

Additionally, alerts can be configured for log shipping configurations.

2. Relevant terms and definitions

(1) Primary server: SQL Server instance as the production server.
Primary Database The database on the primary server to be backed up to another server. All management of the log shipping configuration through SQL Server Management Studio is performed from the primary database.

(2) Secondary server: SQL Server instance in which to keep a hot standby copy of the primary database.

(3) Auxiliary database: A warm standby copy of the primary database. A secondary database may be in a "recovering" state or a "standby" state, which makes the database available for limited read-only access.

(4) Monitor Server: An optional instance of SQL Server that tracks all details of log shipping, including:

• The last time the transaction log on the primary database was backed up.

• The last time the secondary server copied and restored the backup file.

• Information about any backup failure alerts.

Note: Once the monitoring server is configured, it cannot be changed without first removing log shipping.

(5) Backup job: A SQL Server agent job used to perform backup operations, record history to the local server and monitor server, and delete old backup files and history information. When log shipping is enabled, a job class of "log shipping backup" is created on the master server instance.

(6) Copy Job: A SQL Server Agent job that copies backup files from the primary server to a configurable target on the secondary server and records history on the secondary server and the monitoring server. When log shipping is enabled on a database, a job class of "log shipping replica" is created on each secondary server in the log shipping configuration.

(7) Restore job: Restore the copied backup file to the SQL Server Agent job of the auxiliary database. It records history on local server and monitoring server and deletes old files and old history information. When log shipping is enabled on the database, a job class of "log shipping restore" is created on the secondary server instance.

(8) Alert Job: A SQL Server Agent job that raises alerts for the primary and secondary databases when a backup or restore operation does not complete successfully within a specified threshold. When log shipping is enabled on the database, a job category of "log shipping alert" is created on the monitoring server instance.

3. Log shipping and replication

Log shipping involves two copies of a single database, usually residing on different computers. At any given time, only one copy of the database is currently available to the client. This copy is called the master database. Updates made by clients to the primary database are propagated by log shipping to other copies of the database, called secondary databases. Log shipping involves applying the transaction log from each insert, update, or delete operation on the primary database to the secondary database.

Log shipping can be used in conjunction with replication, with the following behaviors:

• Replication does not continue after a log shipping failover. In the event of a failover, the replication agent does not connect to the secondary agent, so transactions are not replicated to the Subscriber. In the event of a failback to the primary database, replication will resume. All transactions logged from the transport copy of the secondary database to the primary database are replicated to the Subscriber.

• If the primary database is permanently lost, the secondary database can be renamed so that replication can continue. The remainder of this topic describes the requirements and procedures for handling this case. The example given is a publication database, which is the most common database for log delivery, but similar procedures can be applied to subscription and distribution databases as well.

3.1. Requirements and procedures for replicating from the secondary database when the primary database is lost

• If the primary database contains multiple publication databases, ship all publication database logs to the same secondary database.

• The installation path of the secondary server instance must be the same as the primary server instance. The user database location on the secondary server must be the same as on the primary server.

• Backup the service master key on the master key. This key will be restored on the secondary key.

• Log shipping does not guarantee against data loss. A failure on the primary database can result in the loss of data that has not been backed up or loss of backups during the failure.

3.2. Log transfer using transactional replication

For transactional replication, the behavior of log shipping depends on the sync with backup option. This option can be set on both the publication database and the distribution database; in log shipping from the Publisher, only the setting on the publication database is relevant.

Setting this option on the publication database ensures that transactions are not delivered to the distribution database until they are backed up in the publication database. Then, the last publication database backup can be restored on the secondary server, and the distribution database cannot possibly have transactions that the restored publication database does not. This option guarantees consistency among the Publisher, Distributor, and Subscriber if the Publisher fails over to a secondary server. Latency and throughput suffer because transactions cannot be delivered to the distribution database until they are backed up at the Publisher; if your application can tolerate this delay. If the Synchronize with backup option is not set, Subscribers may receive changes that are no longer contained in the database restored on the secondary server.

(1) Configure transactional replication and log shipping using the sync with backup option:

1. If the sync with backup option is not set on the publication database, execute sp_replicationdboption '', 'sync with backup', 'true'.

2. Configure log shipping for the publication database.

3. If the Publisher fails, use the KEEP_REPLICATION option of RESTORE LOG to restore the last log of the database to the secondary server. This will preserve all replication settings for the database.

4. Restore the msdb database and the primary database from the primary database to the secondary database. If the primary database is also the distributor, restore the distribution database from the primary database to the secondary database. These databases must match the primary database's publication database in terms of replication configuration and settings.

5. On the secondary server, rename the computer, and then rename the SQL Server instance to match the primary server name.

6. On the secondary server, restore the service master key backed up from the primary server.

(2) Configure transactional replication and log shipping without using the sync with backup option:

1. Configure log shipping for the publication database.

2. If the Publisher fails, use the KEEP_REPLICATION option of RESTORE LOG to restore the last log of the database to the secondary server. This will preserve all replication settings for the database.

3. Restore the msdb database and the primary database from the primary database to the secondary database. If the primary database is also the distributor, restore the distribution database from the primary database to the secondary database. These databases must match the primary database's publication database in terms of replication configuration and settings.

4. On the secondary server, rename the computer, and then rename the SQL Server instance to match the primary server name. You may receive an error message from the Log Reader Agent stating that the publication and distribution databases are not synchronized.

5. On the secondary server, restore the service master key backed up from the primary server.

6. Execute sp_repl restart. This stored procedure can be used to force the Log Reader Agent to ignore all previously replicated transactions in the publication database log. Transactions applied after the stored procedure completes are handled by the Log Reader Agent.

7. Restart the Log Reader Agent after the stored procedure executes successfully.

8. Transactions that have been distributed to Subscribers may be applied at the Publisher. To ensure that the Distribution Agent does not fail with errors when attempting to reapply these transactions at the Subscriber, specify an agent configuration file titled "Continue on Data Consistency Errors".

3.3. Log shipping using merge replication

Follow the steps in the following procedure to configure merge replication and log shipping.

1. Configure log shipping for the publication database. For more information, see Configuring Log Shipping (SQL Server).

2. If the Publisher fails, rename the computer on the secondary server, then rename the SQL Server instance to match the primary server name.

3. Use the KEEP_REPLICATION option of "Restore Logs" to restore the last log of the database to the secondary server. This will preserve all replication settings for the database.

4. Restore the msdb database and the primary database from the primary database to the secondary database. These databases must match the primary database's publication database in terms of replication configuration and settings.

5. On the secondary server, restore the service master key backed up from the primary server.

6. Synchronizes a publication database with one or more subscription databases. This allows you to upload changes that were previously made in the publication database but were not represented in the restored backup.