Backup and Recovery for SAP APO (3.x) - MySAP SCM (4.x)

Backup, Recovery and High Availability

for SAP APO (3.x) / mySAP SCM (4.x) Best Practice for Solution Management

Version Date: February 2004

This version is valid for SAP APO 3.0, 3.1 and mySAP SCM 4.0, 4.1. Not valid for ICH and EM in mySAP SCM 4.x architecture.

The newest version of this Best Practice can always be obtained through SAP Solution Manager

Contents Applicability, Goals, and Requirements....................................................................................................2 Best Practice Procedure and Verification .................................................................................................4

Procedure...........................................................................................................................................4 Backup Strategy ...........................................................................................................................4 Restore and Recovery ...............................................................................................................10 High Availability ..........................................................................................................................14 Incomplete Recovery .................................................................................................................20

Verification........................................................................................................................................20 Further Information .................................................................................................................................21

Best Practice: APO Backup and Availability

2004 SAP AG

2

Applicability, Goals, and Requirements To ensure that this Best Practice is the one you need, consider the following goals and requirements.

Goal of Using this Service Use this service in the context of your mySAP SCM system landscape to guide you in creating:

An operational concept for Backup and Recovery (B&R) A suitable strategy for High Availability (HA)

Alternative Practices Your consulting partner can set up your B&R concept according to this Best Practice.

You should discuss HA options with your hardware partners.

Staff and Skills Requirements To apply this Best Practice, you need a team consisting of:

System administrators, consultants, or technical persons tasked with planning, setting up, or administering your SCM system landscape

Application developers or consultants with knowledge of your applications and business processes

SAP Active Global Support, APO CoE

APO 3.0 System

APO

liveCacheliveCache

APOApplication

Optimizer

App

Serv

er

Presentation Client

Database

Dat

abas

e Se

rver

APOApplication

BW Layer

APO DB

System Requirements The simplest example of a mySAP SCM landscape consists of one OLTP system and one APO system. This Best Practice deals with a:

mySAP SCM system landscape comprising: o An SAP R/3 System o An SAP APO server, SAP liveCache, and optionally an Optimizer


2004 SAP AG

3

Duration and Timing You should define the initial B&R and HA concept during the implementation of your mySAP SCM solution. You must finalize it before the start of production. Be sure to start early enough to leave time for procuring the required software and hardware.

Depending on the complexity of your solution landscape and business scenarios, you may need about a week to define your B&R and HA concept. You may need an additional week to verify and test it.

How to Use this Best Practice Complete the analysis described in this section. Design your operational concept using the information in section Procedure. Verify your success in implementing this Best Practice as described in section Verification.

Case Studies / Sample Scenarios For a sample backup schedule for a mySAP SCM landscape, see:

Case Study Backup Scheduling For a sample HA concept for a mySAP SCM landscape, see:

Case Study High Availability Options For a sample outline of an operations manual for B&R and HA, see:

Case Study Outline of an Operations Manual


2004 SAP AG

4

Best Practice Procedure and Verification

Procedure

Backup Strategy To determine a B&R strategy for an APO system, decide how often you need to make backups of the entire system and of the components. The two factors are data security and cost. The data security and thus your B&R strategy should be comparable for APO DB and liveCache.

Recovery is the process of bringing a database back to the point in time of a crash.

Restore is the process of retrieving data files from a backup medium.

Introduction You must create backups for the individual systems. Follow SAP standard recommendations for backup cycles and storage media.

How often should I make complete backups of the database?

The more often you make complete backups, the better. It is always faster to restore a recent backup than to restore an old backup and play back numerous logs. The backup frequency you need is directly proportional to the frequency of changes. But you should also keep in mind that creating online backups affects system performance.

Do I need to stop APO DB and liveCache to make a backup?

If you stop APO DB and liveCache to perform a backup, you get a consistent backup of the two components with a definite timestamp. In this case, no further log backups are necessary.

If instead you perform the backups during normal operation of the APO system, in a recovery you need to perform the restore and play back the logs. Only when you have completed the entire recovery procedure do you achieve a fully consistent state between APO DB and liveCache.

However, you may not need a consistent and complete backup. In any recovery, to prevent loss of data you must play back the log backups up to the last change. You only need a consistent complete backup of the APO system in offline mode if you plan to copy the APO system. Alternatively, you may wish to make such a backup to save a state of the APO system for fallback use in emergency. For example, you may wish to do this after the initial data load or before an upgrade.

In most situations, offline backups are an option only for customers who can afford increased planned downtime.

Tip If you make offline backups, always use the liveCache and APO DB tools. Never make backups by copying files, since then you cannot guarantee the consistency of the files or of the data they contain.

SAP APO server and SAP R/3 Follow the standard procedures for backup and recovery of SAP systems.

Make backups of the installation using operating system tools. You can make backups of the database using database tools, standard interfaces or the SAP

Computing Center Management System (CCMS).

The following graphic shows recommendations for an Oracle database.


2004 SAP AG

5

SAP AG 2001

28 days28 days28 days

AdditionalAdditional

Offline redo log Offline redo log file backup (x2)file backup (x2)

Offline redo log Offline redo log file backup (x2)file backup (x2)

Verify the databaseVerify the databaseVerify the backupVerify the backup

OnlineOnline

OnlineOnline

Verify the backupVerify the backupOfflineOffline

Oracle: Minimum Backup Cycle Recommendations

For details about the backup cycles, see the SAP Help Portal

or take database administration via SAP Training:

BC505 Oracle BC511 Informix ADM515 SAP Database BC520 MS SQL Server BC525 DB2 on OS/400 BC530 DB2 on OS390 BC535 DB2 on UDB

LiveCache

The following dependencies of APO releases and liveCache versions exist:

SAP APO 3.0 is delivered with liveCache 7.2 or liveCache 7.4.2, SAP APO 3.1 is delivered with liveCache 7.4.2, mySAP SCM 4.0 is delivered with liveCache 7.4.3, mySAP SCM 4.1 is delivered with liveCache 7.5,

The liveCache architecture has been improved from 7.2 to 7.4. The changes have an impact on B&R and HA as procedures become much simpler with liveCache 7.4. Therefore, this Best Practice treats the two versions separately. Operators of liveCache 7.2 should also read the 7.4 sections to facilitate later work with 7.4.

The backup & recovery procedures of liveCache versions 7.4.2, 7.4.3 and 7.5 do not differ so they are all covered by the SAP liveCache 7.4-section of this document.


2004 SAP AG

6

For either liveCache version, you need to make backups using operating system tools after installation and upgrades.

For more details, see SAP liveCache 7.4: Backup and Recovery & Administration News (E)

For liveCache, the following administration tools are available:

liveCache Administration Tools Several options are available for liveCache backup and recovery. The use of the tools or the line commands is as for SAP DB - the backup procedure for liveCache is integrated into the Computer Center Management System (CCMS) as of mySAP SCM 4.0 ( with SAP Basis Release 6.20 ).

Database Manager Graphical User Interface (DBMGUI) The Database Manager GUI is a liveCache / SAP DB tool for administration of LiveCache / SAP DB that runs on Windows platforms. It is the recommended tool for performing backups and recoveries from liveCache 7.4. onwards.

The procedure for backup administration is described in the document Database manager GUI.

For liveCache 7.2 backup and recovery other tools apply (see below).

Web Database Manager (WebDBM) The Web-based database manager (WebDBM) has the same look and feel as DBMGUI and can be used also on non-Windows clients. No local client installation is required. Access is provided over a web server. Find more about the WebDBM under SAP DB Web Tools.

Database Manager Command Line Interface (DBMCLI) The Database Manager CLI is a liveCache / SAP DB administration tool that works at command line level on the liveCache platform. For liveCache 7.4, it can be used to perform script-based backups of the database and thus also supports the backup of liveCache from an SAP system as a background job.

The backup administration procedure is described in the document:

Database Manager CLI

For liveCache 7.2 backup and recovery, other tools apply (see below).

LiveCache 7.4.x / 7.5 Performing a backup for liveCache 7.4 is basically the same as backing up the database of an APO system (APO DB).

To schedule backups and other SAP R/3 database maintenance tasks as background jobs, you can use the CCMS Database Planning Calendar. In APO 3.1, such weekly scheduling is only supported for APO DB. From mySAP SCM 4.0 (SAP Basis Release 6.20) onwards you can schedule liveCache backups from the CCMS Database Planning Calendar. You can schedule backups of liveCache in background jobs or by executing external commands.

Logging

LiveCache 7.4 features complete physical logging for all changes on persistent objects. The logging is similar to logging in RDBMS. The differences have only a minor effect on administration.

In liveCache, changed data is written to disk at regular intervals during checkpoints. This normally occurs every 10 minutes. Depending on the processed data volume, there is high I/O throughput on the log devspaces. At times of especially intensive data changes, for example during planning runs, writing the log areas can have a negative impact on performance.

The liveCache log devspaces are the disk areas with the greatest write activity. Since the data throughput for the log devspaces is so critical, disk layout should be planned thoroughly with the SAP hardware partner. For general recommendations about disk layout see the Installation Guides.


2004 SAP AG

7

Backup Cycle

A typical backup cycle for liveCache is shown in the following graphic. The term Gen specifies the generation of tapes used in the backup cycle.

SAP AG 2001, Title of Presentation, Speaker Name 26

liveCache Backup Cycle

LOG_00005

SAVEDATA (Full backup)SAVEDATA (Full backup)

SAVE PAGES (DATA incr.)SAVE PAGES (DATA incr.)

DAT_00014

AUTOLOGAUTOLOG

0101 0202 03030404

05050606

0707

28282727......

08081. Gen.1. Gen.

2. Gen.2. Gen.3. Gen.3. Gen.

4. Gen.4. Gen.

DAT_00001

DAT_000

DAT_00010

LOG_00002

PAG_00004LOG _00003

PAG _00005

060708

...

LOG_00006

LOG_00008

0909

VERIFYVERIFYLOG _00003 Check BACKUPCheck BACKUP

.. 2..3

1. Perform a backup

Backups are typically created online either as incremental backups (daily) or as full backups (weekly).

Perform backups using

liveCache administration tools

CCMS Database Planning Calendar ( as of SAP Basis Release 6.20 )

A job from within the SAP APO system that

o Starts report RSLVCBACKUP - see SAP Note 455154 o Starts an external command using DBMCLI

Third party tools using backup interfaces

Log backups can be made in parallel to online liveCache backups either

Manually by an administrator or

Periodically by CCMS Database Planning Calendar ( as of SAP Basis Release 6.20 ) or

Automatically in liveCache mode Auto Log

Make sure log backups are performed regularly to prevent liveCache standstills due to log overflow.

For details on how to perform backups, see SAP liveCache 7.4: Backup and Recovery & Administration News (E).

2. Verify the database

Verification of the database is required to detect corrupt data structures before you enter a new backup cycle. Therefore, perform verification at least once per backup cycle. Verification runs may have an impact on system performance and lock situations. For details see SAP Note 521870.

Use liveCache administration tools or schedule a job in the SAP APO system using the applicable DBMCLI command as external job step. See SAP Note 506981.


2004 SAP AG

8

3. Check the backup

Checking a backup ensures that it can be read from the backup medium. This enables you to detect hardware failures. You should perform a backup check at least once per backup cycle.

To perform backup checks, use liveCache administration tools or schedule a job in the SAP APO system using the applicable DBMCLI command as external job step.

LiveCache 7.2 In liveCache 7.2, there is no database logging, but logical logging is realized on application level. This means that changes to data in liveCache are logged at logical level in APO database. The logging does not include all the objects in the APO application module. The logging does not include changes to data in the area Demand Planning (DP), changes to inactive plan versions, or changes to some Automotive-specific data.

Checkpoint During a checkpoint, all the changes that were initially made only in the cache area of liveCache are saved to the data devspaces of liveCache in accordance with a previously defined schedule. The checkpoint is triggered by the execution of a background job on the APO system. The checkpoint ensures that data is consistent between the data cache and the data devspaces on hard disk. Also, liveCache itself needs to be in a consistent state, so write transactions are disabled for the time of the checkpoint.

How often should checkpoints be written?

Writing checkpoints can hinder work in the APO system, since no new transactions are started until all running transactions are ended. On the other hand, a checkpoint substantially increases data security. So for data security it is advantageous to trigger checkpoints as often as possible. Especially if you use DP, which is not covered by the database logging, you should write checkpoints at intervals of 2-3 hours. If you run critical long-running transactions such as planning runs, the checkpoint report can also be scheduled as directly following job steps or be triggered by those events.

The time it takes to perform a checkpoint increases when the log area is activated, since at each checkpoint data is copied from the deactivated log area into the log area. How long this takes depends mainly on how much data is copied.

Backup of liveCache 7.2

Backup of liveCache 7.2 uses the checkpoint report. Whereas in liveCache 7.4 the DBMGUI suffices for a backup, in liveCache 7.2 first a checkpoint must be written and logging switched. Therefore, when performing a backup you must always use the checkpoint report. Do not perform backups using only DBMGUI, as this falsifies the logging in the APO system and leads to inconsistencies.

For details, see SAP APO, SAP liveCache 7.2 Backup and Recovery, or SAP APO System Administration.

Optimizer The Optimizer does not contain a database. It stores data in memory only. Therefore only an operating system backup is required to secure the installation. Perform backups after any major software changes.

To enhance operational security, save optimizer logfiles regularly, preferably at least once a week.

Scheduling Online backups

Online backups affect overall system performance and should therefore be scheduled to run at times of low system workload.

Tip APO DB and liveCache can be backed up online at the same time. Simultaneous backups are easier to administer and low system workload affects both systems.


2004 SAP AG

9

In general, there is no dependency between the backups taken of the different databases forming the SCM landscape. Logs are used to recover any of the databases to the point in time of the last committed transaction. Except in the case of liveCache 7.2, database backup scheduling does not depend on the application scenario.

Offline Backups

SAP R/3 and APO offline backups may be performed at different times and independently of each other. The core interface (CIF) ensures that any requests sent are stored in one system while the partner system is down. No data gets lost.

Tip Perform offline backups of the APO DB and liveCache at the same time in order to reduce planned downtime. LiveCache is not accessed if the APO system is down.

Perform simultaneous offline backups of liveCache, APO DB, and R/3 DB if you want to create a consistent image of the SCM landscape, for example for system copies.

Monitoring of Backups Currently, backups need to be checked for each system component. It is planned to offer backup monitoring support via the SAP Solution Manager. Your operations manual has to include company-specific procedures about backup monitoring and reaction methods on backup failures.

SAP APO and SAP R/3 Use the CCMS Database Planning Calendar and the central CCMS monitoring or external backup tools.

LiveCache Use the CCMS Database Planning Calendar for SAP Basis Release >= 6.20 or the SAPDB tools and the logs of scheduled background jobs or external backup tools.

Other Components and Operating System Backups Define your own procedures in your operations manual for monitoring backups.

Consistent Landscape Backups A backup of the entire system landscape is performed component by component, for each database. If the backup is performed online, a consistent backup of the entire system landscape is not ensured.

Simultaneous Offline Backups A consistent backup of the entire system landscape is only necessary in exceptional cases, such as before a system upgrade or to create a regular system copy. The simplest way to create a consistent backup is to make simultaneous offline backups of all components during a sufficiently large maintenance window, such as over a weekend.

Technical Realization There are various methods that are recommended by storage system manufacturers. These methods allow you to make a consistent backup of several databases without shutting down the databases or scheduling a downtime.

One such method is to make a split mirror backup together with suspend write.

For details, see the Best Practice Backup and Restore for mySAP.com.


2004 SAP AG

10

Restore and Recovery Because of the architecture of an APO system, a crash can affect various system components. The following cases can be distinguished:

Crash of APO DB

In this case, a recovery with the help of the relevant RDBMS is required. The procedure is familiar from the SAP R/3 System.

Crash of liveCache

APO DB is not affected, but there is little or no availability of the APO system as a whole. In this case, liveCache needs to be recovered. Note that the recovery procedure is different for liveCache 7.2 and liveCache 7.4.

Crash of APO DB and liveCache

The procedure depends on the liveCache release.

For liveCache 7.4, the recoveries of APO DB and liveCache can run in parallel.

For liveCache 7.2, the recovery of APO DB can run in parallel with the restore of liveCache. The liveCache recovery can only be performed after successful recovery of APO DB.

Preparation In an APO system crash, the connected OLTP systems are also indirectly affected. Therefore, it is advisable first to interrupt the data transfer from these systems. Find out which SAP systems are transferring data to the APO system through the CIF interface. Stop the outbound queues going to the unavailable APO system. If the APO system is unavailable, the queues will stop anyway with an error message. However, the source systems will make multiple attempts to send the data. Prevent this if possible, as it puts an extra load on the source systems.

Analyze the crash situation. If an APO DB recovery is required, the technical procedure corresponds to the DB recovery of SAP R/3 Systems. Use the relevant database tools to perform a complete recovery. The APO system can then be reconnected for data transfer by restarting reports to reactivate the inbound und outbound queues. This can also be scheduled to occur periodically. See SAP Note 442478.

If a liveCache recovery is required, the situation is different. In this case, the APO instance remains available, so user actions can lead to runtime errors. Therefore, it is advisable to lock the APO system against further activities. These include:

User dialog activities

Background jobs

Processing of inbound queues or incoming requests from the outbound queues of dedicated systems

SAP APO and SAP R/3 If an SAP R/3 or APO system crashes, first stop the outbound queues from the APO system to the crashed R/3 System or the outbound queues from the R/3 System to the crashed APO system. The recovery procedure itself is as usual.

LiveCache

LiveCache 7.4 Since liveCache 7.4 uses full database logging, recovery is performed in a similar way as for SAP DB using SAP DB tools.

The following graphic shows what is performed during liveCache instance recovery in a case where there is hardware failure where no datafiles are lost.


2004 SAP AG

11

The technical terms of data and log devspaces, which are included in this document, correspond to the terms data and log volumes and will be replaced by them in the future.

SAP AG 2001, Global Support 1

Recovery Process

Restart to latest consistent data state data and log are still available on liveCache devices

No action: for transactions T1 and T5Redo: apply after images stored in the archive log for T4 starting at S3

and for T6 completelyUndo: apply before images stored in the undo files for T3 and T2,

both actions starting at S3

T1 Commit

T2

t

T3 Rollback

T4 Commit

T6 Commit

Checkpoint S1 CrashCheckpoint S2 Checkpoint S3

T5 Rollback

Instance recovery during restart of liveCache:

The restart ensures that transactions that were open at the time of the last checkpoint and committed at the time of the crash are redone. Transactions that were open at the time of the crash are only rolled back if they were open at the last checkpoint.

The starting point for the redo/undo operations is always the last checkpoint. Any data that was written later to data devspaces is ignored.

In the above graphic: Transactions 1 and 5 are irrelevant for redo/undo operations. Transaction 1 was committed at

the last checkpoint. The changes were written to the data devspaces. Changes from transaction 5 are not in the data area of the last checkpoint.

Transactions 2, 3, and 4 were not completed at the last checkpoint. Restart of liveCache redoes transaction 4.

Transactions 2 and 3 are undone in a rollback to the last checkpoint. Transaction 6 is completely redone during the restart. Changes from transaction 6 are not in

the data area of the last checkpoint.

Procedure in case data files need to be restored

1. Restore liveCache data files

This occurs in liveCache mode Cold. As of liveCache version 7.5 this mode is called Admin.

2. Recover the logs

If all required log data is still available in the log devspaces, liveCache can be started (instance recovery). LiveCache is automatically rolled forward with the data from the log. If you need to play back log data, do not start liveCache yet, and do not set it to mode Warm (or Online as of liveCache version 7.5). First, play back the log backups and/or restore the incremental backups.

3. Restart liveCache

Once the required data and log backups are played back, liveCache automatically restarts into Warm (Online as of liveCache version 7.5) mode. Please consider that it is recommended that the liveCache


2004 SAP AG

12

be restarted with the APO-liveCache administration transaction LC10 to ensure that post processing procedures are scheduled.

4. Resuming work in the APO system

The APO system can now be used again. Users can work and background jobs can run. Since many items may have accumulated in the queues, they should be reactivated in stages if possible.

Several options are available for the backup and recovery of a liveCache instance. The use of the tools or the line commands is as for SAP DB - the backup procedure for liveCache is integrated into the Computer Center Management System (CCMS) as of mySAP SCM 4.0 (with SAP Basis Release 6.20).

Factors influencing the time needed for liveCache restore:

Case 1. Data files not damaged

Amount of changed data

Number and length of transactions that must be rolled back

Hardware (CPUs)

Case 2. Data files must be restored

Backup infrastructure used

How much can be done in parallel

Whether logs must be played back

Plus case 1 factors

For more information about B&R, see:

SAP Training ADM55

www.sapdb.org

LiveCache 7.2 Since liveCache 7.2 has no physical logging, the recovery scenario is different from liveCache 7.4.

Do not use DBMGUI to set liveCache in mode Warm (Online as of liveCache version 7.5). To continue a recovery, use the APO system.

Logging with Checkpoints

There is no database log written but checkpoints are written at discrete intervals. If a recovery is necessary, data loss can be expected. In this case, a roll forward of liveCache is impossible. After the restore of a backup, the next step is to restore consistency between liveCache and APO DB and between the APO system and the dedicated R/3 Systems. The duration of this process depends on the data volume involved.

Alternatively, reinitialize liveCache and make an initial data transfer from the connected R/3 Systems. This limits the data loss to APO-specific data such as planning versions. To decide which procedure to use, consider the time required and the possible data loss. If you wish to offer the initialization strategy as an alternative, it must be developed and tested and then documented in the operations handbook. Key factors are the duration of the initial data transfer and the resulting data quality. Your choice of strategy also depends on whether your business processes use data that cannot be restored from an initial data transfer, such as Automotive data or orders and requests created only in APO.

Recovery with Synchronous Logging

In synchronous logging, besides checkpoints, each change between two checkpoints is recorded at object level in APO DB. Recovery of liveCache is performed using a report at APO system level. You can choose to use the logs in the recovery process. On completion of the recovery, liveCache is automatically restarted. The administrator is informed of completion by e-mail. The duration of the process depends mainly on the data volume to be restored.


2004 SAP AG

13

Tip To start liveCache, always use the APO-liveCache administration transaction LC10. Otherwise there is no synchronization between APO system and liveCache. Use other tools (such as DBMGUI) only to start liveCache in write-protected mode.

When the recovery is completed, reschedule checkpoints.

In APO 3.0 with liveCache 7.2, there is no logging for the component DP. In the case of a failure, data loss is to be expected. Therefore, if you use DP, a full recovery must include treatment of the DP data. For this purpose, you can do the following.

If a recent backup of the DP data is available in InfoCubes, you can reload the data. This process is described in detail in the APO documentation. The process can take a long time, at least as long as the creation of the DP data in the InfoCubes.

For further details, see the document SAP APO, SAP liveCache 7.2 Backup and Recovery.

Optimizer To restore the Optimizer, restore the latest operating system backup.

Test Cases When you set up a company-specific operations manual, you must test backup and recovery of the whole system landscape. It is recommended that test procedures be established for each database involved.

SAP APO and SAP R/3 Depending on the database type and release, the test procedure should include the following:

Hardware crash (instance recovery)

Crash of a disk with data files/devices

Test example for Oracle database:

Simulate a disk crash by deleting directory sapdata1

Recover your database completely using one of the backups you performed. Use the SAPDBA function Partial restore and complete recovery. After you have eliminated the error, again choose Partial restore and complete recovery.

Core Interface between SAP APO and SAP R/3 Test procedures should include volume tests to verify that during planned or unplanned downtime, CIF queues

Do not increase in size beyond the limitations of the database (to avoid tablespace overflow). See also SAP Note 505304.

Can be restarted correctly after recovery

Can be processed fully and with acceptable system performance even if queues are long

The tests presuppose that downtimes are not so long that too many items accumulate in the queues. If the queues are too long, it may be difficult to process them in the receiving system.

LiveCache 7.4 The test procedures from SAP DB apply. Test

Hardware crash (instance recovery)

Crash of a disk with data devspaces (restore and recovery)

Crash of a disk with log devspace. Note that loosing log information must be avoided by means of hardware redundancy and always causes data loss in the system. For more information see section Incomplete Recovery.


2004 SAP AG

14

LiveCache 7.2 Test restore and application recovery using the document SAP APO, SAP liveCache 7.2 Backup and Recovery.

High Availability This section lists HA options and considers their impact on level of availability and cost of ownership.

If you need an HA solution for your mySAP SCM landscape, you must consider:

The SAP R/3 System

APO DB

liveCache

APO optimizer

Since APO DB is based on the RDBMS familiar from R/3 Systems, you have the same HA options.

As for liveCache:

LiveCache 7.2 supports cluster solutions. However a cluster solution should only be used in case used applications are included in synchronous logging. In particular, when using the DP component you cannot use automatic switching but need manual action.

LiveCache as of 7.4 has familiar RDBMS behavior. Cluster solutions are fully supported.

One of the most important aspects of HA is the backup and recovery strategy for the APO system. This involves:

B&R of the SAP R/3 DB

B&R of APO DB

B&R of liveCache

Backup and restore of APO optimizer

Main Factors In a simple environment without an HA solution, unplanned downtime depends on the following main factors:

Detection of system crash

Replacement of failed hardware (service level contract with hardware partner)

Restoration of operating system

Restoration of database (and thus speed of tapes and degree of parallelization)

Recovery of database (number of transactions to recover, CPUs, )

The time it takes to restore an SCM landscape is set by the longest component restore time.

For information about disk technology, see the SAP documentation on Disk Technology and the Installation Guides.


2004 SAP AG

15

Clustering

Hardware clusters protect your system against hardware failure but not against disk failure.

SAP R/3 and SAP APO System The standard HA solutions are in place. See http://service.sap.com/ha

Clusters protect against hardware failure but not against disk failure.

LiveCache 7.4 A cluster solution is provided for liveCache 7.4 by SAP hardware partners.

For more information contact your SAP hardware partner.

Cluster support is based on the standard recovery process with the prerequisite that data and logs are stored on secure disk systems. At switchover there are two liveCache instances that access the same data resource (generally a disk subsystem). Of the two instances, only one is active at any time. If the active instance fails, for example due to a hardware failure, the second instance is activated on a different machine and receives authorization to write to the same data resource. The switchover is performed by an SAP partner product.

- Recovery time in case of a server failure includes the time normally required for the restart of a liveCache instance.

- Logical errors cannot be corrected or prevented by cluster support.

- Standard cluster solutions are offered by SAP hardware partners.

The following graphic shows liveCache in a cluster environment during normal operation.

SAP AG 2002

Cluster

Cluster Support

ArchiveLogData

l iv e C a c h el iv e C a c h el iv e C a c h el iv e C a c h e l iv e C a c h el iv e C a c h el iv e C a c h el iv e C a c h e

APO ABAP Application

primary backup


2004 SAP AG

16

The following graphic shows a liveCache cluster in which the primary server fails and a backup server takes over.

SAP AG 2001, Global Support 16

Cluster

Cluster Support

ArchiveLogData

l iv e C a c h el iv e C a c h el iv e C a c h el iv e C a c h e l iv e C a c h el iv e C a c h el iv e C a c h el iv e C a c h e

APO ABAP Application

IP SWITCH

AUTO RESTART

primary

RECONNECT

backup

A liveCache cluster scenario

When liveCache is switched over after hardware failure, it is started by means of cluster scripts. Once liveCache has reached status Cold (as of liveCache version 7.5: Admin), instance recovery begins.

If the instance recovery was performed successfully, liveCache is switched to status Warm (as of liveCache version 7.5: Online). At this point in time, liveCache can be accessed by application programs but the data is not yet in the data cache. In prefetching, data pages are read by liveCache processes into main memory. After prefetching, all data from the data cache is in main memory. This improves performance.

The following graphic shows the typical switchover process for liveCache.

The following liveCache statuses are displayed:

LC red circle: liveCache is in status Offline

LC yellow circle: liveCache is in status Cold (as of liveCache version 7.5: Admin) and performs instance recovery

LC green circle: liveCache is in status Warm (as of liveCache version 7.5: Online) and starts with prefetching


2004 SAP AG

17


Measurements on liveCache 7.4.2.003

TimeFailover InstanceRecovery Production Operation

liveCacheliveCache

CrashliveCacheliveCache

Ava

ilabi

lity

1

0LC

LCLC

Pref

etch

Fill

grad

e

Timing of liveCache cluster switchover

The following graphic shows what happens over time when liveCache is switched to the backup node. At the time of switchover, no transactions are running.

After a few minutes, liveCache is in status Warm (as of liveCache version 7.5: Online). Once liveCache is in status Warm data can be accessed. However, performance is restricted until all data pages are read into main memory.

The graphic shows a switchover process with subsequent prefetching. The times are only typical. Actual times depend strongly on the installed hardware and system configuration.


Switchover from Node D1 to Node D2: Timeline

Data 4 GB

Data cache 5 GB

Switch liveCache at simulated hardware failure

0

100000

200000

300000

400000

500000

600000

1 4 7 10 13 16 19 22

time [mins]

8K b

lock

s re

ad

D1offline

D2primary

LC8 s 60 s

LC57 s

~ 20 min

LC LCliveCache COLD liveCache WARM


2004 SAP AG

18

LiveCache 7.2 For most applications (except e.g. Demand Planning), liveCache 7.2 uses synchronous logging.

Clustering makes sense if all APO applications used by the customer are fully included in synchronous logging. However, expectations as to unplanned downtime must be limited by the time needed for application recovery and checkpoint intervals.

Clustering makes no sense if a core business process of the customer is not included in synchronous logging, since manual action is required after a failure.

Optimizer High availability of the SAP Optimizer can be achieved through Installation of the APO optimizer programs on several servers. An Optimizer availability check can be activated in APO which is executed before each Optimization run. If one Server fails it is being recognized by the Application and the Optimization run is started on the alternative Host. Optimizer Runs can also be distributed to both machines during normal productive use to provide load balancing.

The high availability concept can not rescue the actual state of the aborted optimizer run the optimizer run has to be repeated completely but it can ensure immediate availability of the optimizer software on a backup location.

For details see the Installation Guides

Note that there is no need to use cluster software.

SAP AG 20012

Optimizer High Availability

APO Application

APO Application

Optimizer 1Optimizer 1

APO-DB

liveCacheliveCacheliveCache

Optimizer 2(standby)Optimizer 2(standby)

Fast Restores Instead of backing up the database to tape, backups can be performed to fast disks or using a split mirror backup method. Compared to tapes, this allows much faster restoration times.

SAP R/3 and SAP APO System Depending on the size of the database, use of fast backup media can ensure that restoration times meet your maximum unplanned downtime requirements.


2004 SAP AG

19

LiveCache The database size of liveCache is typically much smaller than that of R/3 and APO systems. Therefore, restoring liveCache from disk or a similar medium may be fast enough.

Standby Database Standby databases are replicated databases. They offer protection against:

Disk failures (the shadow database uses its own dataset)

Errors in the restore/recovery process (the shadow database is always in recovery mode)

Logical errors (if the shadow database is run with a delay)

The process of shipping logs from one database to the standby database is performed by SAP partner products.

SAP R/3 and SAP APO System The standard HA solutions are in place. See also http://service.sap.com/ha

LiveCache 7.4 The Standby Database is available with liveCache 7.4. For details see http://help.sap.com -> SAP Netweaver -> SAP Web Application Server -> SAP Web Application Server 6.30 -> English -> SAP Netweaver Components -> SAP DB -> Basic Information -> Background Knowledge SAP DB 7.4 -> Standby Databases with SAP DB -> Setting up a Standby Database.

Hot Standby Database In a hot standby system, data is replicated on a transactional basis. Intelligent storage systems are required to support this solution. Main features:

Only committed transactions are replicated.

There is minimal delay for replication: in the worst case only some transactions has to be redone upon switchover.

The replicated database is already in memory, so startup time is minimized and the data cache is already filled.

LiveCache The hot standby database scenario is available as of mySAP SCM 4.1 with liveCache 7.5. For details see http://service.sap.com/scm >> Technology.

Use of liveCache hot standby is appropriate if extremely high availability is required and extreme high data volume needs to be processed. Since the hot standby is in memory, it offers good performance immediately after switchover. In the hot standby, there are no uncommitted transactions to be undone.

Twin Sites The above concepts may be applied to twin sites to protect system operation against the breakdown of an IT center. If you are interested, talk to your hardware partners.


2004 SAP AG

20

Incomplete Recovery In an incomplete recovery, the system landscape or one of its databases is reset to an earlier point in time before a crash. An incomplete recovery may be caused by logical errors or loss of log data. To prevent loss of log data, use redundant disks.

Logical Errors In most cases, logical errors are the reported reasons for resetting a database. In such situations, there is usually an alternative. For a detailed discussion, see SAP Note 434645.

Symptom Corrective measure

Table entries or entire tables corrupt or deleted

Recreate data using redundant data from other tables or recreate field values partly from redundant table entries in other tables of the same system

Table indexes corrupt Delete index on DB level and recreate using table data.

Corrupt blocks on file system level prevents successful database recovery

If corrupt block belongs to table index, SAP or database vendor support may be able to restore database

The index must be recreated afterwards

Objects destroyed by accidental import

Create and apply correction request

Other user errors resulting in data inconsistencies

Use specific report to repair inconsistencies in the application

Client deleted accidentally See SAP Note 31496

Consistency Once a database of a mySAP SCM landscape is recovered incompletely, data inconsistencies or losses may rise between the SAP APO and SAP R/3 and/or the SAP APO system and liveCache.

This applies specifically to data imported by external interfaces.

SAP provides tools to detect and repair inconsistencies. However, repairing inconsistencies can lead to data loss.

Tip The SAP tools for checking and repairing data inconsistencies are not a required part of backup and recovery procedures. Consistency checks might be performed after a recovery if the planned downtime allows execution and evaluation by qualified application staff. It is recommended to perform consistency checks periodically to improve overall system operation stability.

For information about consistency check tools, see the document Consistency Checks.

Verification After you executed this Best Practice, check whether you produced the desired results as follows:

1. Perform B&R tests with your operations team using your operations manual.

2. Perform tests to check that your HA environment and configuration work and meet your time constraints.


2004 SAP AG

21

Further Information

Case Study Backup Scheduling The following graphic shows an example of a backup schedule for liveCache 7.2. A daily planning process starts with the inventory load in an SAP R/3 System (available stock). After this background step is finished in the R/3 System, the results are sent to the APO via CIF. All irrelevant orders (such as orders from the previous day that were not fulfilled) are deleted at this time. The operation planning scheme is described below.


APO

Supply Network Planning

R/3 System

SNP Interactive Planning

Purchasing

Create/Update purchase requisition

Daily Operation: Production Planning Example in APO

Load inventory

Deployment / TLB

Deployment run (stock, released process orders)

Create/change Truck orders

Release to PP/DS

PP/DS

Detailed scheduling

Adjust/Change Final plan

Convert planned orders in process orders

Release process orders (Manufacturing)

Transfer orders

Final Deployment run (stock, confirmed

process orders)

TLB run

The first deployment run has to be finished by 11 a.m. A checkpoint is scheduled explicitly (using synchronous logging) to run at that time. This checkpoint can either be scheduled for that time or triggered by the event of completion of the application job. Depending on the data volume load, further checkpoints with intervals varying from 4 to 6 hours may be set. The deployment plan is based on daily demand and available-to-deploy (ATD) quantity. With the results of this run, the trucks are ordered via telephone for the afternoon (4 p.m. the same day and 5 a.m. the next day) to deploy the ATD quantity from the production plants to the distribution centers.

These daily requests are illustrated below.


2004 SAP AG

22


Daily Requests

1. Deployment run for each of 2 plants

1 Day

Truck Ordering

Planned transport Orders into Confirmed Transport Order

Final Deployment run

TLB for filling trucks

11 a.m. 4 p.m.

Checkpoint Complete online backup

(incl. Checkpoint)

The final deployment run takes the confirmed production orders of the current day into account. After the deployment run, the system automatically converts planned transport orders into confirmed transport orders, which are used for transportation planning in the transport load builder (TLB). The TLB is executed to ensure that trucks are filled at least to minimum capacity and if possible to maximum capacity. Both steps must be finished at 4 p.m. to ensure that the trucks can be loaded in time. A complete daily backup is then scheduled, for example as a background job.

Case Study High Availability Options The following customer example illustrates availability and cost considerations. The options are based on liveCache 7.4.

Simple Environment A simple environment does not have any redundancy in place. Therefore, recovering from the crash of a single component may take up to one day.

Type of fault: Disk crash or server failure


2004 SAP AG

23

SAP R/3R/3

Database

SAP APOAPO

Database

LivecacheLivecacheDatabaseCOM

Routines

covers 1 faultwithin next day

Spare Server Environment In this environment, a spare server is added. The spare server includes a cluster installation for one or more of the following:

liveCache

SAP APO central instance

SAP APO database

SAP R/3 central instance

SAP R/3 database

Type of fault: disk failure or server failure

Impact on availability

A downtime of 4 hours is expected for disk failure. Downtimes for server failures depend on the cluster product and database recovery.

Impact on costs of operation

Assumption: no performance loss after switchover

Additional server is required. Sizing of this backup server is the same as the sizing for the biggest of all the primary servers.

License fees for cluster software


2004 SAP AG

24

Cluster

SAP R/3R/3

Database

SAP APOAPO

Database

LivecacheLivecacheDatabase

Failover

COMRoutines

covers 1 faultwithin 4 hours

Spare Server and Fast Recovery Environment In this scenario, the Spare Server Environment is extended by fast disks to allow a restore from disk instead of tape.



This scenario reduces the recovery process after a disk failure to 2 hours. Downtimes for hardware failures depend on the cluster product and database recovery.


Additional disk space is required. Disk space is sum of space required by:

SAP APO DB

SAP R/3 DB

liveCache

SAP R/3R/3

Database

SAP APOAPO

Database


Failover

COMRoutines

3rdmirror

3rdmirror

3rdmirror

covers 1 faultwithin 2 hours

Cluster


2004 SAP AG

25

Standby Databases In this scenario, the Spare Server and Fast Recovery Environment is extended by shadow databases for:

SAP APO DB

SAP R/3 DB

liveCache

Type of fault: disk crash or server failure


This scenario accelerates the recovery process after a disk failure to 1 hour, since no restore is required. Downtimes for server failures depend on the cluster product and database recovery.


Additional disk space is required. Disk space is sum of space required by:

SAP APO DB

SAP R/3 DB

liveCache

License fees for shadow database software (automatic log shipping)

SAP R/3OLTP

Database

SAP APOOLAP

Database


Cluster

COMRoutines

3rdmirror

ShadowLiveCacheDatabase

ShadowOLAP

Database

ShadowOLTP

Database

3rdmirror

3rdmirror

Extended Service Environment In this scenario, theStandby Databases scenario is extended by 0.5 FTE of service staff for permanent monitoring of the landscape.



With permanent monitoring, tasks that cannot be automated can be performed by service personnel and therefore after a disk failure an expected downtime of 30 minutes can be achieved. Downtimes for server failures depend on the cluster product and database recovery.


0.5 FTE of service personnel


2004 SAP AG

26

Cluster

SAP R/3R/3

Database

SAP APOAPO

Database


Failover

COMRoutines

3rdmirror


ShadowAPO

Database

ShadowR/3

Database

3rdmirror

3rdmirror

covers 1 faultwithin 0,5 hours

0,5 x

Second Spare Server In this scenario, the Extended Service Environment is extended by another backup server.

Type of fault: Disk crash or server failure


Now two failures can be tolerated at the same time. A server failure is covered by a cluster and a disk failure by a shadow database.


Assumption: no performance loss after switchover

Additional server is required. Sizing of this backup server is the same as the sizing for the biggest of all the primary servers.

Cluster

SAP R/3R/3

Database

SAP APOAPO

Database


Failover

Failover

COMRoutines

3rdmirror


ShadowAPO

Database

ShadowR/3

Database

3rdmirror

3rdmirror

covers 2 faultswithin 0,5 hours

0,5 x


2004 SAP AG

27

Twin Site Environment In this scenario, the Second Spare Server scenario is extended by an additional remote IT center.


Failure of an IT center can be tolerated.


Setup of an additional IT center is required.

Case Study Outline of an Operations Manual The following is a sample outline of an operations manual: General

Object, Purpose, Scope Technical Prerequisites Roles & Responsibilities System Monitoring

System Landscape Backup Strategy

R/3 APO DB liveCache Optimizer Flow Chart

High Availability Concept Detailed Procedures

Problem Discovery Problem Identification and Assessment Actions in the Application / Information Issued Problem Resolution

Failure of R/3 Server Failure Disk Failure

Failure of APO Server Failure Disk Failure

Failure of liveCache Server Failure Disk Failure

Failure of Optimizer Server Failure Disk Failure

Logical System Error Validation and Communication Lessons Learned: Failure Mitigation & Adjustment of Procedures

Appendix Process Flows Checklists Organization Charts Glossary


2004 SAP AG

28

Background Information and References

Database Manager GUI www.sapdb.org >> Documentation http://www.sapdb.org/pdf/dbmgui_73eng.pdf

Database Manager CLI www.sapdb.org >> Documentation http://www.sapdb.org/pdf/dbmcli_73eng.pdf

SAP DB Web Tools www.sapdb.org >> Software >> Choose Platform >> SAP DB Web Tools

External Backup Tools www.sapdb.org >> Documentation >> External Backup Tools http://www.sapdb.org/pdf/extsich_73eng.pdf

SAP APO, SAP liveCache 7.2 Backup and Recovery SAP Service Marketplace alias scm >> Technology http://service.sap.com/~form/sapnet?_FRAME=CONTAINER&_OBJECT=011000358700006126142001

SAP APO System Administration Author: Will, Liane, ISBN 3-89842-248-8 currently available only in German

Backup and Restore for mySAP.com SAP Service Marketplace alias bestpractices >> Solution Management http://service.sap.com/~sapidb/011000358700004447112001E

SAP Training SAP Service Marketplace alias education http://service.sap.com/education

Disk technology Help Portal http://help.sap.com/saphelp_46c/helpdata/en/08/57437e4ae611d1894f0000e829fbbd/frameset.htm

Installation Guides SAP Service Marketplace alias instguides http://service.sap.com/instguides

Consistency Checks Service Marketplace alias scm >> Technology http://service.sap.com/~form/sapnet?_SHORTKEY=01100035870000331500&

SAP liveCache 7.4: Backup and Recovery & Administration News (E) SAP Service Marketplace alias scm >> Technology


2004 SAP AG

29

http://service.sap.com/~form/sapnet?_FRAME=CONTAINER&_OBJECT=011000358700001330932002

SAP Help Portal http://help.sap.com

Feedback and Questions Send any feedback by formulating an SAP customer message to component SV-GST-SMC. You can do this at: http://service.sap.com/message


2004 SAP AG

30

Copyright 2004 SAP AG. All rights reserved. No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changed without prior notice. Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors. Microsoft, Windows, Outlook, and PowerPoint are registered trademarks of Microsoft Corporation. IBM, DB2, DB2 Universal Database, OS/2, Parallel Sysplex, MVS/ESA, AIX, S/390, AS/400, OS/390, OS/400, iSeries, pSeries, xSeries, zSeries, z/OS, AFP, Intelligent Miner, WebSphere, Netfinity, Tivoli, and Informix are trademarks or registered trademarks of IBM Corporation in the United States and/or other countries. Oracle is a registered trademark of Oracle Corporation. UNIX, X/Open, OSF/1, and Motif are registered trademarks of the Open Group. Citrix, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, and MultiWin are trademarks or registered trademarks of Citrix Systems, Inc. HTML, XML, XHTML and W3C are trademarks or registered trademarks of W3C, World Wide Web Consortium, Massachusetts Institute of Technology. Java is a registered trademark of Sun Microsystems, Inc. JavaScript is a registered trademark of Sun Microsystems, Inc., used under license for technology invented and implemented by Netscape. MaxDB is a trademark of MySQL AB, Sweden. SAP, R/3, mySAP, mySAP.com, xApps, xApp, and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. All other product and service names mentioned are the trademarks of their respective companies. Data contained in this document serves informational purposes only. National product specifications may vary. These materials are subject to change without notice. These materials are provided by SAP AG and its affiliated companies ("SAP Group") for informational purposes only, without representation or warranty of any kind, and SAP Group shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP Group products and services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional warranty. Disclaimer: SAP AG assumes no responsibility for errors or omissions in these materials. These materials are provided as is without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. SAP shall not be liable for damages of any kind including without limitation direct, special, indirect, or consequential damages that may result from the use of these materials. SAP does not warrant the accuracy or completeness of the information, text, graphics, links or other items contained within these materials. SAP has no control over the information that you may access through the use of hot links contained in these materials and does not endorse your use of third party Web pages nor provide any warranty whatsoever relating to third party Web pages.

Backup and Recovery for SAP APO (3.x) - MySAP SCM (4.x)

Documents

Transcript of Backup and Recovery for SAP APO (3.x) - MySAP SCM (4.x)