Database Migration System Unicode Conversion

Front cover

DB2 Optimization Techniques for SAP Database Migration and Unicode ConversionAdvanced migration techniques

Optimization strategies

Best practices

Andreas Christian Waldemar Gaida Hans-Jrgen Moldowan Thomas Rech

ibm.com/redbooks

International Technical Support Organization DB2 Optimization Techniques for SAP Database Migration and Unicode Conversion August 2009

SG24-7774-00

Note: Before using this information and the product it supports, read the information in Notices on page ix.

First Edition (August 2009) This edition applies to DB2 UDB Versions 9.1 and 9.5 and SAP Kernel Release 6.40 and 7.x. Copyright International Business Machines Corporation 2009. All rights reserved. Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

ContentsNotices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi The team that wrote this book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii Become a published author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv Comments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv Chapter 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 Methods outside the scope of this book . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2 Heterogeneous system copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.3 Unicode conversion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Chapter 2. Migration essentials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.1 Database export . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.1.1 Unsorted export. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.1.2 Package splitter. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.1.3 Export server scenarios: Local or remote . . . . . . . . . . . . . . . . . . . . . . 8 2.2 Advanced migration techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.2.1 Socket option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.2.2 Table splitting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.3 DB2 layout and configuration options . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.3.1 Table space principles and configuration . . . . . . . . . . . . . . . . . . . . . 11 2.3.2 Logging configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.4 Database import . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.4.1 Import optimization techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.4.2 DB2 row compression based on R3load 6.40 . . . . . . . . . . . . . . . . . . 14 2.4.3 DB2 row compression based on R3load 7.00 . . . . . . . . . . . . . . . . . . 14 2.4.4 Collecting statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.5 SAP NetWeaver BW migrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 Chapter 3. Tools overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 3.1 SAPInst and R3SETUP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 3.2 R3load, R3szchk, and R3ldctl . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 3.2.1 R3load. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 3.2.2 R3ldctl . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.2.3 R3szchk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.3 Migration monitor: MigMon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Copyright IBM Corp. 2009. All rights reserved.

iii

3.4 Time analyzer: MIGTIME . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 3.4.1 Export times analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 3.4.2 Import times analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 3.4.3 Time join analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 3.5 Distribution monitor: DistMon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 3.6 Tools for package and table splitting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Chapter 4. Export optimization techniques . . . . . . . . . . . . . . . . . . . . . . . . 35 4.1 A closer look at cluster tables and table clusters . . . . . . . . . . . . . . . . . . . 36 4.2 Unsorted versus sorted export . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 4.2.1 Enabling unsorted export . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 4.2.2 Runtime comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 4.2.3 Resource consumption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 4.2.4 DB2 snapshot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 4.2.5 Conclusion and recommendation . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 4.3 Export file size considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 4.4 Package splitting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 4.5 Export server scenarios: Local or remote . . . . . . . . . . . . . . . . . . . . . . . . . 54 4.5.1 Test setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 4.5.2 Test results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 4.5.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 Chapter 5. Advanced optimization techniques. . . . . . . . . . . . . . . . . . . . . . 61 5.1 Table splitting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 5.1.1 How to split tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 5.1.2 Testing table split . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 5.1.3 Results of export tests with table splitting: CDCLS . . . . . . . . . . . . . . 69 5.1.4 Results of export tests with table splitting: GLPCA . . . . . . . . . . . . . . 75 5.1.5 Results of import tests with table splitting . . . . . . . . . . . . . . . . . . . . . 77 5.1.6 Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 5.2 Socket option. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 5.2.1 Comparing disk to socket migration method . . . . . . . . . . . . . . . . . . . 88 5.2.2 Comparison of resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 5.2.3 Migration monitor (MigMon) configuration. . . . . . . . . . . . . . . . . . . . . 91 5.2.4 Export log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 5.2.5 Import log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 5.2.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 Chapter 6. Basic DB2 layout and configuration options . . . . . . . . . . . . . . 95 6.1 Table space principles and configuration . . . . . . . . . . . . . . . . . . . . . . . . . 96 6.1.1 Data placement and SAP data classes . . . . . . . . . . . . . . . . . . . . . . . 99 6.1.2 File system caching. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 6.1.3 Page size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 6.1.4 Extent size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

iv

DB2 Optimization Techniques for SAP Database Migration and Unicode Conversion

6.1.5 NUM_IOCLEANERS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 6.1.6 Prefetch size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 6.1.7 DB2 parallel I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 6.2 Logging configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 6.3 Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 Chapter 7. Import optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 7.1 DB2 LOAD versus INSERT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 7.1.1 Data processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 7.1.2 Performance comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 7.1.3 R3load options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 7.2 Using and optimizing DB2 LOAD with R3load . . . . . . . . . . . . . . . . . . . . 114 7.2.1 Configuring the DB2 LOAD API . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 7.2.2 CPU_PARALLELISM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 7.2.3 DATA_BUFFER_SIZE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 7.2.4 DISK_PARALLELISM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 7.2.5 INDEXING MODE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 7.2.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 7.2.7 Clean up a failed DB2 LOAD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 7.3 UTIL_HEAP_SZ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 7.4 Order and configuration of index creation . . . . . . . . . . . . . . . . . . . . . . . . 122 7.5 LOCKTIMEOUT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 7.6 DB2 buffer pools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 7.7 Import using DB server versus remote client . . . . . . . . . . . . . . . . . . . . . 128 7.8 Optimal number of R3load processes . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 7.9 Using DB2 self tuning memory management . . . . . . . . . . . . . . . . . . . . . 139 7.10 Unicode influence on LOAD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 7.11 Temporary table spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146 7.12 CHNGPGS_THRESH . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 7.13 SORTHEAP. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150 7.14 SHEAPTHRES_SHR and SHEAPTHRES . . . . . . . . . . . . . . . . . . . . . . 150 7.15 INTRA_PARALLEL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 7.16 Disable database logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152 7.17 Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 7.18 R3load with DB2 compression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 7.18.1 Introduction to DB2 row compression . . . . . . . . . . . . . . . . . . . . . . 155 7.18.2 Automatic dictionary creation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 7.18.3 R3load options to compress data during import into DB2 . . . . . . . 156 7.18.4 DB2 row compression based on R3load 6.40 . . . . . . . . . . . . . . . . 157 7.18.5 DB2 row compression based on R3load 7.00 . . . . . . . . . . . . . . . . 163 7.18.6 Bernoulli sampling method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182 7.19 Deferred table creation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191 7.19.1 Enabling deferred table creation . . . . . . . . . . . . . . . . . . . . . . . . . . 192

Contents

v

7.19.2 Enabling deferred table creation in conjunction with compression 193 7.20 Optimizing statistics collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 7.20.1 Sampled statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196 7.20.2 Parallel runstats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196 7.20.3 db2look . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 7.20.4 Exclude the normal statistics collection from the process flow . . . 201 7.20.5 Influence statistics collection using table DBSTATC. . . . . . . . . . . 202 7.20.6 Choosing the correct method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202 Chapter 8. Tips for monitoring the system copy process . . . . . . . . . . . . 205 8.1 db2top . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206 8.1.1 Applications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206 8.1.2 Utilities. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207 8.2 Detailed monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207 8.2.1 Monitoring a single R3load process . . . . . . . . . . . . . . . . . . . . . . . . 208 8.2.2 Lock wait situations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210 8.2.3 Reorganization status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211 8.2.4 Log space usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212 8.2.5 Sorts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213 8.2.6 Utility heap size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213 8.2.7 Measuring table size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215 8.3 NMON . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215 Chapter 9. Considerations for migrating SAP NetWeaver BW-based systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219 9.1 Pre-migration tasks on the source system . . . . . . . . . . . . . . . . . . . . . . . 221 9.2 Post-migration tasks on the target system . . . . . . . . . . . . . . . . . . . . . . . 223 Chapter 10. Unicode details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225 10.1 Flow of Unicode conversion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226 10.2 Introduction to code page and character notation . . . . . . . . . . . . . . . . . 227 10.3 Unicode character encoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230 10.4 Big and little endian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233 10.5 Hardware requirements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234 10.6 SAP tables, languages, code pages . . . . . . . . . . . . . . . . . . . . . . . . . . . 234 Chapter 11. Setup for a Unicode migration. . . . . . . . . . . . . . . . . . . . . . . . 237 11.1 Architecture overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238 11.2 MigMon configuration for host A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239 11.3 MigMon configuration for host B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241 11.4 MigMon configuration for host C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243 11.4.1 Configuration for MigMon C.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244 11.4.2 Configuration for MigMon C.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245 11.5 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246

vi


11.6 Time join diagram for MigMon B + C.2 . . . . . . . . . . . . . . . . . . . . . . . . . 247 11.7 Time join diagram for MigMon A + C.1 . . . . . . . . . . . . . . . . . . . . . . . . . 248 11.8 Compression rates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249 Appendix A. Background information . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251 Test environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252 IBM System p with network-attached storage . . . . . . . . . . . . . . . . . . . . . . 252 IBM System x with local attached storage . . . . . . . . . . . . . . . . . . . . . . . . 253 DB2 row compression based on R3load 6.40 . . . . . . . . . . . . . . . . . . . . . . . . 254 Script for compressed import for a single table . . . . . . . . . . . . . . . . . . . . . 255 Migration monitor properties file for compressed import . . . . . . . . . . . . . . 255 DDL template mapping file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257 DDL file for uncompressed import. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258 DDL file for compressed import. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259 DB2 row compression based on R3load 7.00 . . . . . . . . . . . . . . . . . . . . . . . . 260 Migration monitor: import_monitor_cmd.properties. . . . . . . . . . . . . . . . . . 260 Definitions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265 STMM script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266 Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269 Publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269 How to get IBM Redbooks publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270 Help from IBM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270 Abbreviations and acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273

Contents

vii

viii


NoticesThis information was developed for products and services offered in the U.S.A. IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information about the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-IBM product, program, or service. IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to: IBM Director of Licensing, IBM Corporation, North Castle Drive, Armonk, NY 10504-1785 U.S.A. The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you. This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice. Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk. IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you. Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to the names and addresses used by an actual business enterprise is entirely coincidental. COPYRIGHT LICENSE: This information contains sample application programs in source language, which illustrate programming techniques on various operating platforms. You may copy, modify, and distribute these sample programs in any form without payment to IBM, for the purposes of developing, using, marketing or distributing application programs conforming to the application programming interface for the operating platform for which the sample programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs.


ix

TrademarksIBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. These and other IBM trademarked terms are marked on their first occurrence in this information with the appropriate symbol ( or ), indicating US registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at http://www.ibm.com/legal/copytrade.shtml The following terms are trademarks of the International Business Machines Corporation in the United States, other countries, or both: AIX AS/400 DB2 Universal Database DB2 IBM PowerPC Redbooks Redbooks (logo) System p System x

The following terms are trademarks of other companies: NetApp, and the NetApp logo are trademarks or registered trademarks of NetApp, Inc. in the U.S. and other countries. Novell, the Novell logo, and the N logo are registered trademarks of Novell, Inc. in the United States and other countries. Oracle, JD Edwards, PeopleSoft, Siebel, and TopLink are registered trademarks of Oracle Corporation and/or its affiliates. ABAP, SAP NetWeaver, SAP R/2, SAP R/3, SAP SEM, SAP, and SAP logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries. Java, Solaris, and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. SQL Server, Windows, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. Intel Xeon, Intel, Itanium, Intel logo, Intel Inside logo, and Intel Centrino logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States, other countries, or both. UNIX is a registered trademark of The Open Group in the United States and other countries. Linux is a trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others.

x


PrefaceSAP migrations are a standard process nowadays. We see an increasing number of customers changing their database software to IBM DB2 for UNIX, Linux, and Windows together with SAP upgrades, Unicode conversions, hardware or operating system changes, and system consolidations. This IBM Redbooks publication describes optimization strategies and best practices for migrating SAP systems towards IBM DB2 for Linux, UNIX, and Windows, as well as for performing Unicode conversions. We discuss DB2-specific recommendations and best practices for the migration process as well as for Unicode conversions. This book is intended for experienced SAP migration consultants involved in operating system and database (OS/DB) migrations or Unicode conversions, choosing IBM DB2 as a target database platform. It addresses the advanced SAP migration techniques, considerations for database layout and tuning, and the unique DB2 capabilities, such as compressing the target database while loading data. All techniques discussed within this book are based on extensive tests, as well as experiences collected on various migration projects. However, it is important to understand that the optimizations described in this document may have side effects, introduce risks to the overall process, or require changes to the production system. Therefore, the features discussed must be chosen wisely. They should be used only if the migration downtime window or compression requirements make it necessary to use these optimizations. Chapter 1, Introduction on page 1, introduces into SAP migrations and Unicode conversions. Chapter 2, Migration essentials on page 7, summarizes our recommendations and findings. It can be used as a quick reference for experienced migration consultants. Readers interested in more details can find the in-depth information beginning with Chapter 3, Tools overview on page 17. The detailed sections are divided into six main areas: Best practices and recommendations for the source system database export Advanced migration techniques (such as table splitting) Database layout and configuration Database import recommendations SAP NetWeaver Business Warehouse migration Background information about Unicode


xi

The team that wrote this bookThis book was produced by a team of specialists working at IBM Germany. Andreas Christian worked for the SAP BW/DB2 porting team at SAP in Walldorf before he joined IBM SAP DB2 Center of Excellence in 2006. Since 1999, he has helped port SAP BW to DB2, conducted SAP BW/DB2 performance and scalability studies, and supported SAP BW/DB2 customers with the implementation and operation of SAP Business Intelligence on DB2 LUW. He is coauthor of the IBM Redbooks publication Building and Scaling SAP Business Information Warehouse on DB2 UDB ESE, SG24-7094. Andreas is driving IBM DB2 sales by performing DB2 proof of concepts for IBM/SAP customers. His focus areas are SAP performance optimization and tuning (especially in SAP BW environments), as well as SAP migration topics. Waldemar Gaida is a certified SAP Technology and OS/DB Migrations consultant. Before Waldemar joined IBM, he worked as a leader of a data center for 10 years, where he worked with SAP R/2 and SAP R/3. He joined IBM in 1998, and until 2006 was involved in customer projects helping customers covering topics related to SAP Technology. Since 2006 Waldemar has supported the team of the IBM SAP DB2 Center of Excellence. His focus areas are SAP performance monitoring and tuning, investigations on new DB2 features (for example, row compression), and SAP migration topics. Waldemar holds a degree in Geodetic Engineering from Rheinische Friedrich-Wilhelms University in Bonn, Germany. Hans-Jrgen Moldowan is a certified SAP Technology and OS/DB Migrations Consultant, and has been working for IBM since 1997. Before joining the IBM SAP DB2 Center of Excellence in 2008, he provided SAP DB2 technical sales support within IBM to customers and partners. During this time, Hans-Jrgen was involved in numerous proof-of-concept and migration projects, as well as first-of-a-kind technology implementations. Hans-Jrgen holds degrees in Electrical Engineering from the University of Cooperative Education in Mosbach, Germany, and in Industrial Engineering from the Pforzheim University, Germany

xii


Thomas Rech has worked in the DB2 area since 1996. Begining as a trainer for DB2, he soon moved to the SAP arena in 1997. He started in the SAP/IBM DB2 Porting team as a Developer, followed by a role as a Technical Sales Consultant. He lead the project for the internal SAP implementation of DB2 and took over the role as Technical Lead in the largest SAP/IBM proof of concept in 2006. Thomas is now an Architect and Team Lead in the IBM SAP DB2 Center of Excellence out of Research and Development in Boeblingen, Germany. Thomas holds a masters degree in Computer Science from the University of Applied Sciences in Worms.

AcknowledgementsIn addition to the authors, many other people contributed to this book by providing helpful information about various aspects of the subjects that are presented in this book. We would like to thank the following people for their support: Christian Brockhaus Olaf Depper Jochen Donner Sergiy Malikov Thomas Matth Malte Schnemann Alexander Seelert Jens Seifert Rainer Staib IBM Germany Jeremy Broughton IBM Canada Britta Bachert Arndt Effern Frank-Martin Haas Johannes Heinrich Michael Schnaubelt Ralf Stauffer Bettina Weissinger-Stelzel Torsten Ziegler Andreas Zimmermann SAP Germany

Preface

xiii

Edgar Maniago Joachim Pfefferle SAP Canada Emma Jacobs Whei-Jen Chen International Technical Support Organization, San Jose Center

Become a published authorJoin us for a two- to six-week residency program! Help write a book dealing with specific products or solutions, while getting hands-on experience with leading-edge technologies. You will have the opportunity to team with IBM technical professionals, Business Partners, and Clients. Your efforts will help increase product acceptance and customer satisfaction. As a bonus, you will develop a network of contacts in IBM development labs, and increase your productivity and marketability. Find out more about the residency program, browse the residency index, and apply online at: ibm.com/redbooks/residencies.html

Comments welcomeYour comments are important to us! We want our books to be as helpful as possible. Send us your comments about this book or other IBM Redbooks publications in one of the following ways: Use the online Contact us review Redbooks form found at: ibm.com/redbooks Send your comments in an e-mail to: [email protected] Mail your comments to: IBM Corporation, International Technical Support Organization Dept. HYTD Mail Station P099 2455 South Road Poughkeepsie, NY 12601-5400

xiv


1

Chapter 1.

IntroductionSAP offers software for various hardware platforms and database systems. The procedure to migrate an existing SAP system from one database system or operating system (or both) to another is known as a heterogeneous system copy or operating system and database (OS/DB) migration. SAP has developed a set of tools that allows customers to export their source database in a database-independent format and import it into the target database. The same set of tools allows converting a non-Unicode SAP system into a Unicode one. The process of migrating an SAP system to another platform (for example, changing the database vendor) or converting an SAP system to Unicode is basically the same procedure regarding the export and import from a technical point of view. Therefore, OS/DB migrations and Unicode conversions easily can be combined into one single project. In this IBM Redbooks publication we describe various optimizations and best practices for converting a DB2 database to Unicode or migrating an SAP system from a non-DB2 database to DB2.


1

1.1 Methods outside the scope of this bookWe do not discuss SAP Minimized Downtime Service (MDS) in this book. MDS is a special migration approach that allows the source system to be online while exporting the largest tables. Smaller tables are exported during an offline window. This leads into a minimized overall system downtime during a migration or Unicode conversion. The MDS option is exclusively available as a service offer from SAP and is therefore not evaluated in this book. For changing the operating system only, you can use database-specific methods, for example, DB2 backup and restore between UNIX operating systems of the same endianess. For more information see the article Copying Your SAP/R3 System Across Platforms Using DB2 Universal Database V8 Redirected Restore that is available at:http://www.ibm.com/developerworks/db2/library/techarticle/0308nesiem/0308nesiem .html

These special approaches are also not described in this book.

1.2 Heterogeneous system copyThe process of copying an SAP system while changing the operating system or the database platform is known as heterogeneous system copy. Briefly speaking, a heterogeneous system copy works as follows: 1. The database of the source system is exported into a database and operating-system-independent format using SAP tools. 2. A new SAP system is installed, using the export from step 1 to load the database.

2


Figure 1-1 illustrates this process, which is supported by SAP and has been performed with many customer systems.

DB + OS s pecific SAP Source System SAP Target System

Source DB

IBM DB2

R3load

R3load

Exporttransfer

Import

DB + OS independent

Figure 1-1 SAP heterogeneous system copy overview

While exporting or importing, the SAP system must be offline. No SAP user activity is allowed. Usually, customers allow a weekends time frame for performing a heterogeneous system copy. If the system is large or the time frame is tight, special optimizations to the export and import process must take place.

1.3 Unicode conversionAll new SAP product releases from 2007 on are based on Unicode. The usage of multi-display multi-processing (MDMP) systems in new releases such as SAP ERP 6.0 (previously named SAP ERP 2005), which is based on SAP ECC 6.0, is no longer supported. SAP ERP 6.0 will be the release of choice for most customers. This implies that those systems must be converted to Unicode. The technique of a Unicode conversion is very similar to that of a database migration, which consists of an export and an import phase. They are both based on the use of R3load. The Unicode conversion itself is normally executed during the export phase. It is, therefore, very easy to change the database for the target system without additional effort. Restrictions for the migration procedure due to

Chapter 1. Introduction

3

the Unicode conversion are mentioned and we provide you with optimization hints concerning export and import steps. To minimize downtime for those customers who must perform an upgrade together with a Unicode conversion, SAP has developed the process Combined Upgrade & Unicode Conversion (CU&UC). This procedure is applicable to clients that have a 4.6C ERP system with multi-display multi-processing. Depending on the target release (SAP NetWeaver 7.0 SR1 and later), this procedure is also applicable for the following start releases (see SAP Note 928729): R/3 Enterprise (4.70) with Extension set 1.10 or 2.0 SAP ERP 2004 (SAP ECC 5.0) Figure 1-2 shows the system landscape setup and the steps required for a combined upgrade and Unicode conversion of a source release SAP R/3 4.6C to a target release SAP ERP 2005.

Figure 1-2 CU&UC process flow

Let us assume that the goal is to do a CU&UC to an SAP 4.6C production system. When your target is a Unicode system, you must be Unicode-compliant with your ABAP reports. Since the check and adjustment for this can only be done in a system that supports Unicode, you must provide such a system. This can, for example, be done by copying your 4.6C system to a sandbox system. On this system, you perform a CU&UC. At the end you have a system

4


that is able to run the Unicode checks (UCCHECK). The results of these checks and also the results of the preparation steps done for the Unicode conversion (SPUM4, resulting in a vocabulary for Unicode conversion) can be moved to the production system during the CU&UC process, so you are not required to repeat these steps manually for every system. For customers whose systems are running on older releases, SAP developed the Twin Upgrade & Unicode Conversion (TU&UC) procedure.

Chapter 1. Introduction

5

6


2

Chapter 2.

Migration essentialsIn this chapter we summarize the overall findings and the essential recommendations of our study in the database migration and Unicode conversion. We explain the details in the subsequent chapters. As a migration always means a planned downtime and is a very special phase in the life of an SAP system, ease of use and a save process are more important than reducing the downtime to the absolute minimum. The most important recommendation we would like to provide is to use only the optimizations that are required to meet the downtime target. A prerequisite for applying any of the optimization techniques described in this book is a sound knowledge of the Heterogeneous SAP System Copy process and the corresponding SAP Technology and Migration Consultant Certification.


7

2.1 Database exportThe recommendations provided in this section apply to the optimization of the source system database export.

2.1.1 Unsorted exportWhen exporting data from the source database you should choose unsorted export whenever applicable. We recommend that you: Use the migration monitor with the Data Definition Language (DDL) mapping file when unsorted and sorted export should be used in parallel. Use R3load with the latest patch level. Obey SAP Note 954268, which states the prerequisites for unsorted unload if you are using R3load prior to 6.x or before compile date February 10, 2006.

2.1.2 Package splitterYou can use the package splitter to split STR-files according to different criteria. This allows you to increase parallelism and ensure better granularity of the packages, which results in better resource usage during the migration. We recommend that you: Use the JAVA-Package splitter. Choose the options for top tables (-top 50-200), table size limit (-tableLimit 500-1000), and package size limit (-packageLimit 500-2000) for standard migrations running with SAPInst. Determine the tables that have the longest runtimes for more complex migrations and put their names into a file (for example, table_names.txt) that is used by the package splitter (tableFile). All these tables will be split out. You can use this option together with the options mentioned above.

2.1.3 Export server scenarios: Local or remoteIf CPU resources become a bottleneck on the exporting machine, it makes sense to move the workload generated by R3load onto one or more dedicated SAP application servers. With running R3load from the application server, more CPU resources could be utilized by DB2 on the SAP database server. In our tests, this remote setup showed reduced export runtimes. Nevertheless, keep in mind that when exporting from a remote application server, data must be transferred over

8


the network. You should ensure that the network connection will not turn into a bottleneck.

2.2 Advanced migration techniquesAdvanced migration techniques improve both the source system database export and the target system database import.

2.2.1 Socket optionIf you have large tables that define the overall runtime of the migration, you can save time using the socket option. As a prerequisite, make sure that you have a fast and stable network connection when you use different servers for export and import. This is important to ensure optimal performance and to minimize the risk of failure. You can also use this option within one server. When you want to combine this option with other data transport techniques, you must use multiple MigMon instances due to the configuration differences in the properties file. Be sure that you understand the setup of the socket option and its influence on other optimizations, in particular sampled compression. You can also use the socket option for Unicode conversions if you obey the restrictions in SAP Note 971646.

2.2.2 Table splittingIf a single table or a few tables determine the export runtime of a migration, which is likely for table clusters such as CDCLS in the case of a Unicode conversion, you can use the table-splitting technique to cut these tables into pieces and export these pieces in parallel. This is done by defining WHERE clauses that select only a subset of the table. You can use the tool R3ta to determine the WHERE clauses that are stored in a single WHR file. To split the WHERE clauses into separate WHR files that contain one WHERE clause each, you can use the Where Splitter.

Chapter 2. Migration essentials

9

To speed up the export and import using table splitting you can do the following: Check whether you can create an index that refers to the table column that is used in the WHERE clause. Be aware that creation of additional indexes can impact production operation, as this requires additional hardware resources and may also affect the access plans of queries. If you have created an additional index to support table splitting, check whether you can reorganize the table to be clustered by this index. In some cases, this results in a better export performance. If some of the table pieces show a longer export time compared with the others, you can introduce additional WHERE conditions to further split this table part. Be sure to understand the advantages of using sequential DB2 LOAD versus parallel inserts and plan the migration based on your needs. Although the usage of DB2 LOAD forces all R3load processes to import data sequentially into the corresponding table, performance may be in some cases as fast as with concurrent R3load processes using DB2 INSERT. However, the decision to use LOAD or INSERT varies with the infrastructure and must be tested. If you have a table with many indexes, it seems to be more effective to create the indexes before the data is loaded and with more significant effect if parallel insert is used. We recommend that you validate the most efficient way of index creation as part of a migration test. If you use DB2 LOAD and an R3load process is interrupted, the load of all corresponding data packages must be restarted from scratch including the index build. Therefore, if you want to minimize the impact of an interrupted DB2 LOAD, the table should be first loaded and then the indexes should be built. To force the use of DB2 LOAD for the import, set the appropriate R3load parameter or environment variable. If you use DB2 LOAD for the import, enable the incremental index build. Set the environment variable DB6LOAD_INDEXING_MODE=2. If you use the DB2 LOAD for the import, serialize the affected table using the orderBy entry in the import monitor properties file or use dedicated Migration Monitor instances. With respect to ease of use, the parallel insert into a single table is a good starter.

10


2.3 DB2 layout and configuration optionsBefore you start the import into the DB2 target system, you should carefully plan the database layout and database configuration.

2.3.1 Table space principles and configurationTable spaces can be created as either automatic storage table spaces managed by DB2s automatic storage management (also referred to as autostorage) or as database managed space (DMS) file table spaces in autoresize mode. The later option offers more possibilities of intervention by a database administrator and offers full control of the placement of table space containers. On the other hand, the automatic storage feature incorporates a better ease of use. We recommend that you: Use enough spindles for your I/O. Separate logging I/O from data, index, and temporary I/O. Try to avoid container sizes larger than 20 GB. Switch the file system caching off to avoid additional operating system (OS) buffering. Place large tables into separate table spaces and introduce new data classes. Avoid table spaces that are significantly larger than the others, as this may impact backup performance.

2.3.2 Logging configurationWe recommend that you: Switch off archival logging and use circular logging during the migration. Provide enough log space. A good starter is to use the planned configuration for production. More space may be required if using many parallel R3load with Inserts.

2.4 Database importThe data import phase is a long-running task. Together with the export, it influences the overall downtime phase of a heterogeneous system copy or Unicode migration. Therefore, it is important to optimize the import using SAP tools and DB2 configuration settings.


11

2.4.1 Import optimization techniquesThere are several measures that you can take to optimize the import process: Default optimizations Optional optimizations Advanced optimizations The advanced optimizations measures can be very complex, require significantly more testing effort, or possibly have side effects that can influence the overall migration process.

Default optimizationsOur default recommendations for database import are: Use the DB2 LOAD API with default settings for R3load. Specify enough utility heap to ensure parallelism of the DB2 LOAD API. use 200,000 as a starter. Configure SORTHEAP and SHEAPTHRES_SHR to accommodate the large sorts during index build. A good starting point is 50,000 pages for SORTHEAP and 2 * (SORTHEAP) * (number of R3load processes) for SHEAPTHRES_SHR. Be sure that the file system caching is disabled on database level during table space creation or use the ALTER TABLESPACE statement for existing table spaces. Do not use STMM during the migration final test and cutover to ensure stable runtimes. To avoid failing R3load processes due to wait situations, set LOCKTIMEOUT to -1. Define one buffer pool using the remaining memory after utility heap and sort memory are configured. Leave all other configuration settings according to the SAP Notes for DB2 standard parameter settings. Create primary and secondary indexes before you load data. Allocate enough temporary space for the index build to avoid temp space overflow. Be aware that the amount of data may have increased since the last successful test migration. Therefore, increase temporary space to provide enough reserve for the final productive migration. As a starting point, use as many R3load processes as you have CPU cores on your server. Do not use many more R3load processes than available CPU cores to avoid hardware (CPU, memory, I/O) resource bottlenecks.

12


Optional optimizationsThe following optimizations require more testing effort, as they may not show full effect in all environments: To determine optimized values, use Self Tuning Memory Management (STMM) during a test migration. Avoid many parallel index build processes by optimizing the package import order. The goal is to prevent I/O resource overload. Monitor the import process and adjust the configuration for the utility heap, buffer pool, and sorting. Monitor and optimize the configuration for I/O (for example, NUM_IOCLEANERS, DB2_PARALLEL_IO, or disable file system caching for logging). To optimize the number of parallel processes with respect to evenly distributed resource usage, analyze the resource usage of the system (CPU, I/O, and memory).

Advanced optimizationsThe following optimization topics can be very complex and might have side effects that can influence the overall migration process. Test them in detail. Use the application server for import if the I/O subsystem can handle additional workload, the target system is short on CPU, and a fast network connection between the application server and the database server is available. You can use a DMS temporary table space to optimize the index rebuild phase, but be aware of the side effects (for example, no parallel REORG processes and the need to create the temporary table space manually). After a successful migration make sure to switch back to SMS temporary table spaces. Change the order of index creation for selected tables if the index build phase takes significantly longer compared with other large tables, which is hard to determine. Optimize the CHNGPGS_THRESH parameter together with buffer pool and index creation to optimize the I/O in this area. Create dedicated buffer pools for one or more table spaces (for example, for temporary table spaces).


13

2.4.2 DB2 row compression based on R3load 6.40With R3load Release 6.40, you can compress table data during import in combination with the usage of the DB2 LOAD API by using the R3load parameter LOADCOMPRESS. The compression dictionary is built up by importing a defined number of rows and issuing a REORG on the table after this. Normally, a sample (which is about 1% of the number of rows of the table) is able to achieve a compression dictionary of a good quality. However, some tables might need a higher sample rate.

2.4.3 DB2 row compression based on R3load 7.00R3load Version 7.00 and later offers a new SAMPLED compression option. This option loads a representative sample of data into a table, builds a compression dictionary, and then loads the complete set of data. We recommend using the new R3load sampling method for compressing data while importing, as it shows an optimal combination of maximum compression ratio and minimized load runtime. Nevertheless, keep in mind that this approach is not fully automatic. That is, it consists of two phases, and you must restart R3load manually after successfully finishing the first phase. Import into a compressed table can be faster than into an uncompressed table. Therefore, you might actually reduce the overall runtime of the import and may accept this manual process. In addition, the two phases can be separated. You can perform phase one before a productive migration with data obtained from a test migration. This sample is still valid even after some time and the data will be replaced during to the final migration that is performed in phase two. If you perform phase one up front in a productive migration, you must ensure that no structural change to the related tables takes place.

2.4.4 Collecting statisticsAfter a migration, all non-volatile tables should have valid statistics. For large tables, the statistics collection using the RUNSTATS command can take very long. To extract statistics for such big tables after the final test migration, you can use the tool db2look. This generates update statements on the statistics tables of DB2 in an SQL script. This script can be executed to update the statistics for big tables much faster than a RUNSTATS.

14


Be aware that the definition of some tables may change after running db2look and before the production migration. Those tables may require RUNSTATS after the data is loaded. For small tables, performing a RUNSTATS is faster than running this script. To shorten the RUNSTATS runtime, you should consider parallel execution. To achieve this, a script with the RUNSTATS commands can be generated and split into several pieces. Therefore, we recommend a mixed procedure for the statistics update. Use parallel RUNSTATS for small tables and db2look for big tables.

2.5 SAP NetWeaver BW migrationsSAP NetWeaver Business Warehouse (SAP NetWeaver BW) migrations follow the same rules and optimizations mentioned in this book. However, there is a major difference, as SAP NetWeaver BW implements more database-specific optimizations that cannot all be handled by R3load. This is true in particular for DB2 database partitioning feature (DPF) and multi-dimensional clustering (MDC). Both require special DDL statements and, therefore, you must do the following to succeed in migrations of SAP NetWeaver BW-based systems: Read the white paper Heterogeneous SAP NetWeaver Business Intelligence (BI) System Copy to IBM DB2 for Linux, UNIX and Windows, which is available in the SAP Community Network:https://www.sdn.sap.com/irj/sdn/db6

Execute report SMIGR_CREATE_DDL on the source system. Execute report SAP_DROP_TMPTABLES on the source system. Execute report RS_BW_POST_MIGRATION on the target system.


15

16


3

Chapter 3.

Tools overviewIn this chapter we provide an overview of available SAP system copy tools. You can use these tools both for homogeneous and heterogeneous system copies. Tools that are used for heterogeneous system copies are often also referred to as migration tools. This chapter is not intended to replace the SAP training or documentation available. This chapter introduces the tools required for the DB2 optimizations and points our hints and tricks found during our tests.


17

3.1 SAPInst and R3SETUPThe SAP installation tools SAPInst and R3SETUP are used to install SAP systems or to unload or load systems during a system copy procedure. They invoke other tools (for example, R3ldctl, R3szchk, and R3load) and control the entire installation and migration process. Some of the tasks that the tools perform include: Creating users or groups on the operating system level Adapting file system rights Installing SAP binaries (Kernel) Triggering the database unload and load processes. Triggering post processing such as collecting database statistics SAPInst is used as of SAP basis Release 6.10, whereas R3SETUP supports basis Releases 3.1I up to 4.6D. To simplify the migration process, the SAP installation tool SAPInst provides software life-cycle options. For example, the following options are available for a source system during a heterogeneous system migration: Export preparation Table splitting preparation Database Instance export

3.2 R3load, R3szchk, and R3ldctlR3load, R3szchk, and R3ldctl are the main SAP tools used in SAP operating system and database (OS/DB) migrations. In this section we describe their purposes.

3.2.1 R3loadR3load is the core migration tool. It exports SAP ABAP table data from thesource database and imports it into the target database. Besides this base functionality, it offers advanced options, for example, using DB2 row compression while importing data.

18


Figure 3-1 illustrates the function of R3load.

DDL.TPL R3load Read .STR .WHR read Write .TOC .nnn DB Read / Write .logFiles with exported table data

.TSK

Figure 3-1 Tasks performed by R3load during the export

3.2.2 R3ldctlR3ldctl unloads SAP ABAP data dictionary structures from the source system. It generates structure files (*.STR files) that describe the definition of tables, indexes, and views. In addition, it generates database-specific template files (*.TPL files) with definitions of DDL statements, for example, statements to create a table, to drop a table, and so on.

3.2.3 R3szchkR3szchk computes the size of tables and indexes for the target database.

Chapter 3. Tools overview

19

Figure 3-2 illustrates the tasks performed by R3ldctl and R3szchk.

TablesDDLOADD DDLOADHad Re rit e W

R3szchk R3ldctl

Takes up to several hours

Takes some minutes

FilesDDL.TPL SAP.STR SAP.EXT

W ri t e

DB

Figure 3-2 Tasks performed by R3ldctl and R3szchk

R3szchk may run for a long time in some cases, as it uses the SELECT COUNT(*) statement against all tables to calculate the target database size information. This is true in particular if the database management system is changed during the migration. If you experience a long runtime of the R3szchk, check whether R3szchk is using the SELECT COUNT(*) statement for the tables. If so, there are two optimizations available to overcome this bottleneck: You may run the R3szchk while the system is still up and running and used for production. Note: R3szchk requires that the STR files created by R3ldctl calculate the target size. If running R3szchk and R3ldctl while the system is up and running, no change of the data definition is allowed afterwards (for example, by new transports). These changes will not be reflected in the STR files, so the export will not be consistent or will fail. Another option for reducing the runtime is to use the option r together with an input file to avoid the expensive statements. This file contains the name of the table and the number of records for this table. Running R3szchk with the option r and the file containing the number of records will reduce the runtime by factors.

Read

20


Example 3-1 shows a sample input file for R3szchk.Example 3-1 File with table names and number of records /B28/FCDIOEPE /TST/FCDIOEPE /BB1/FCDIOEPE /BI0/B0001012000 WBCROSSGT /BI0/AQP_DS3100 /BIC/B0000183009 /BBL/B0000183009 ZRSDRDHELP 12708488 12704927 12704554 11275128 10946626 10697826 10463648 10463648 10000000

You can determine the appropriate number of records up front by extracting the data from database statistics. You may use the DBACOCKPIT within the SAPGUI or the appropriate SQL statement. With DB2, you can use the following statement to retrieve the number of records for the largest 50 tables:SELECT tabname, card FROM syscat.tables ORDER BY CARD DESC FETCH FIRST 50 ROWS ONLY

The usage of the -r parameter is not supported by the SAPInst program at the time that the book was written. You must run the export preparation phase manually, which requires detailed knowledge of the migration procedure and a lot of experience in migrations.

3.3 Migration monitor: MigMonThe migration monitor, MigMon, is described in SAP Note 784118. Its parameters, functions, and control files are explained in more detail in the Migration Monitor User's Guide. This guide can be extracted from the DISTMON.SAR file, which is attached to SAP note 855772. The main aspects and attributes of the MigMon are: Allow advanced control of R3load export and import. Automate dump shipping between source and target system. Support parallel unload and load processing. MigMon is controlled by properties files. Properties files are constantly reread after a defined time period, so some attributes, such as the number of parallel R3load processes, can be adjusted dynamically.


21

As of SAP NetWeaver 7.0, MigMon has been fully integrated into SAPInst. With older SAP releases, manual intervention may be necessary.

3.4 Time analyzer: MIGTIMETo have a baseline for optimization, you should have an overview of the import and export runtimes. For this analysis, SAP provides a toolset that is called migration time analyzer (MIGTIME). It is delivered as software archive file MIGTIME.SAR. You can find this archive on the installation master DVD or in SAP Service Marketplace (Support Packages and Patches). The archive comprises the content shown in Example 3-2.Example 3-2 Content of MIGTIME.SAR archive SAPCAR: processing archive /sapcd2/IM_AIX_PPC64/COMMON/INSTALL/MIGTIME.SAR (version 2.01) rw-r--r-147883 17 Nov 2007 18:53 migtime.jar Java-archive rwxr-xr-x 516 17 Nov 2007 18:53 export_time.sh UNIX-script for export analysis rwxr-xr-x 516 17 Nov 2007 18:53 import_time.sh UNIX-sript for import analysis rwxr-xr-x 498 17 Nov 2007 18:53 time_join.sh UNIX-script for time joining rw-r--r-699 17 Nov 2007 18:53 export_time.bat WINDOWS-script for export analysis rw-r--r-699 17 Nov 2007 18:53 import_time.bat WINDOWS-script for import analysis rw-r--r-681 17 Nov 2007 18:53 time_join.bat WINDOWS-script for time joining rw-r-r-69828 17 Nov 2007 18:53 TimeAnalyzer.pdf Documentation in PDF-format

You can find all the necessary information about how to start the scripts and choose the parameters in the documentation. Therefore, we only present some examples for the outcome of the analysis. The result of every analysis is a text file and optionally an HTML file with the graphical representation of the runtimes. Additionally, a XML file can be created, which can be used as an input file for the time-join script.

22


3.4.1 Export times analysisExample 3-3 shows an example for an export_time.txt file as a result of the export_time script. This file has two parts. The first part relates to the package export times. The second part relates to the table export times.Example 3-3 Example of export_time.txt (part 1)----------------------------------------------------------------------------package time start date end date size MB MB/min ----------------------------------------------------------------------------REPOSRC 0:13:30 2007-09-20 20:58 2007-09-20 21:12 959.46 71.07 SAPAPPL2_1 0:04:46 2007-09-20 21:01 2007-09-20 21:06 27.48 5.76 SAPAPPL0_1 0:03:45 2007-09-20 20:59 2007-09-20 21:03 39.82 10.62 D010TAB 0:02:54 2007-09-20 20:49 2007-09-20 20:52 46.59 16.07 REPOTEXT 0:02:32 2007-09-20 20:58 2007-09-20 21:01 151.99 60.00 TRFCQDATA 0:02:29 2007-09-20 21:18 2007-09-20 21:21 155.01 62.42 SAPNTAB 0:02:02 2007-09-20 21:03 2007-09-20 21:05 41.02 20.17 SAPAPPL1 0:01:56 2007-09-20 21:00 2007-09-20 21:02 23.37 12.09 D010INC 0:01:49 2007-09-20 20:49 2007-09-20 20:51 19.55 10.76 PRCD_COND 0:01:45 2007-09-20 20:58 2007-09-20 21:00 13.52 7.73 SAPSSEXC 0:01:43 2007-09-20 21:06 2007-09-20 21:08 57.76 33.65 SMW3_BDOC2 0:01:32 2007-09-20 21:12 2007-09-20 21:13 41.81 27.27 DOKCLU 0:01:14 2007-09-20 20:53 2007-09-20 20:54 143.68 116.50 ----------------------------------------------------------------------------1:12:05 2007-09-20 20:42 2007-09-20 21:21 2795.49 -----------------------------------------------------------------------------

For every package you get the following information: Package: The name of the package that was exported (may also be only one table, if split out). Time: The runtime needed to export the package. Start date: Date and time when the export of the package starts. End date: Date and time when the export of the package ends. Size MB: Size of the exported package in MB (for example, on disk). MB/min: Export rate in MB per minute (related to the export file size). The list is sorted descending by the export time. Therefore, you get a good and fast overview of the packages that require the most time.


23

At the end of the output (see Example 3-3 on page 23), you have information about the following items: Sum of the export times Start time of the export process End time of the export process Size of the export (sum of all export files) Example 3-4 shows the table that is related to the second part of the file. The top, h or m options of the export_time script control how many tables are shown.Example 3-4 Example of export_time.txt (part 2)------------------------------------------table package time ------------------------------------------REPOSRC REPOSRC 0:13:29 D010TAB D010TAB 0:02:54 REPOTEXT REPOTEXT 0:02:31 TRFCQDATA TRFCQDATA 0:02:29 D010INC D010INC 0:01:49 PRCD_COND PRCD_COND 0:01:45 SMW3_BDOC2 SMW3_BDOC2 0:01:32 DOKCLU DOKCLU 0:01:13 MONI MONI 0:01:11 DD03L DD03L 0:01:11 DYNPSOURCE SAPSSEXC 0:01:05 BALDAT BALDAT 0:01:03 DDNTF_1B SAPNTAB 0:00:55 SMWT_TRC SMWT_TRC 0:00:50 ------------------------------------------0:57:47 -------------------------------------------

For every table listed you obtain the following information: Table: Name of the table exported Package: The name of the package containing the table (When split out, the table name and the package name are identical.) Time: The runtime needed to export the table Using this information, you can easily identify the tables that contribute mostly to the export time of a package that comprises more than one table. You obtain the graphical representation of the first part of the export_time.txt when you use the html option of the export_time script to generate a package time diagram.

24


Figure 3-3 shows a part of such a diagram. The diagram is sorted by the start time of the exported package and therefore provides a chronological overview of the export. The length of the bars represents the export time.

Figure 3-3 Example of an export package time diagram

3.4.2 Import times analysisExample 3-5 shows the partial content of an import_time.txt file as a result of the import_time script. This file consists of two parts. The first part relates to the package import times. The second part relates to the table import times.Example 3-5 Example for Import_Time.txt (part 1)-----------------------------------------------------------------------------package time start date end date size MB MB/min -----------------------------------------------------------------------------SAPAPPL2_1 0:33:19 2007-09-20 23:42 2007-09-21 00:16 27.48 0.82 SAPAPPL0_1 0:33:04 2007-09-20 23:39 2007-09-21 00:12 39.82 1.20 SAPAPPL1 0:18:37 2007-09-20 23:43 2007-09-21 00:01 23.37 1.26 SAPAPPL0_2 0:07:47 2007-09-20 23:55 2007-09-21 00:03 4.77 0.61 SAPSSEXC 0:07:36 2007-09-20 23:36 2007-09-20 23:43 57.76 7.60 REPOSRC 0:07:01 2007-09-20 23:35 2007-09-20 23:42 959.46 136.74 MONI 0:06:11 2007-09-20 23:36 2007-09-20 23:42 104.46 16.89 SAPAPPL2_2 0:03:14 2007-09-21 00:02 2007-09-21 00:06 3.92 1.21 D010TAB 0:02:26 2007-09-20 23:37 2007-09-21 00:35 46.59 19.15 SAPSDIC 0:02:21 2007-09-21 00:00 2007-09-21 00:02 4.14 1.76


25

SAPSSRC 0:01:33 2007-09-20 23:46 2007-09-20 23:48 11.11 7.17 SAPNTAB 0:01:07 2007-09-20 23:38 2007-09-20 23:39 41.02 36.74 DD03L 0:01:00 2007-09-20 23:40 2007-09-20 23:41 31.03 31.03 D010INC 0:00:48 2007-09-20 23:43 2007-09-20 23:44 19.55 24.44 DOKCLU 0:00:48 2007-09-20 23:35 2007-09-20 23:36 143.68 179.60 -----------------------------------------------------------------------------2:27:27 2007-09-20 23:35 2007-09-21 00:36 2795.49 ------------------------------------------------------------------------------

For every package you receive the following information: Package: The name of the package that was imported (may also be only one table, if split out) Time: The runtime required to import the package (including index creation) Start date: Date and time when the import of the package starts End date: Date and time when the import of the package ends Size MB: Size of the imported package in MB (on disk) MB/min: Import rate in MB per minute (related to the export file size) The list is sorted descending by the import time. Thus, you get a good and fast overview of the packages that require the most time. At the end of the report you receive the following information: Sum of the import times Start time of the import process End time of the import process Size of the import (sum of all export files) Example 3-6 shows the table that is related to the second section of the file. The top, h, or m options of the Import_time script control how many tables are shown.Example 3-6 Example of Import_Time.txt (part 2)------------------------------------------table package time ------------------------------------------REPOSRC REPOSRC 0:07:01 MONI MONI 0:06:11 DSYST SAPSSEXC 0:05:12 D010TAB D010TAB 0:01:11 DD03L DD03L 0:01:00 D010INC D010INC 0:00:48 DOKCLU DOKCLU 0:00:48 DDNTT SAPNTAB 0:00:47 PRCD_COND PRCD_COND 0:00:43

26


TRFCQDATA TRFCQDATA 0:00:40 STERM_LINK STERM_LINK 0:00:32 REPOTEXT REPOTEXT 0:00:30 DYNPSOURCE SAPSSEXC 0:00:29 ------------------------------------------0:43:17 -------------------------------------------

For every table listed you receive the following information: Table: Name of the table imported Package: The name of the package containing the table (When split out, the table name and the package name are identical.) Time: The runtime required to import the table Using this information you can easily identify the tables that contribute mostly to the import time of a package that comprises more than one table. You obtain the graphical representation of the first part of the import_time.txt when you use the html option of the import_time script to generate a package time diagram.


27

Figure 3-4 shows a part of package time diagram. The diagram is sorted by the start time of the imported package and, therefore, provides a chronological overview of the import. The length of the bar represents the import time.

Figure 3-4 Example of an import package time diagram

The dashed lines for table D010TAB mean that the import was interrupted and continued later.

3.4.3 Time join analysisThe third option of the time analysis tool is to join the export and import times. This produces the file time_join.txt. Choosing the html option also produces the file time_join.html with a graphical representation of the text version. Example 3-7 shows the text versions. The records are on one line inside the text file. They are split into two lines here because of the document format and for readability.Example 3-7 Example of time_join.txt-------------------------------------------------------------------------------------package total time export time start date end date import time start date end date size MB -------------------------------------------------------------------------------------SAPAPPL2_1 0:38:05 0:04:46 2007-09-20 21:01 2007-09-20 21:06 0:33:19 2007-09-20 23:42 2007-09-21 00:16 27.48

28


SAPAPPL0_1 SAPAPPL1 REPOSRC SAPSSEXC SAPAPPL0_2 MONI D010TAB SAPAPPL2_2 SAPNTAB TRFCQDATA REPOTEXT SAPSDIC D010INC PRCD_COND DD03L DOKCLU

0:36:49 0:20:33 0:20:31 0:09:19 0:08:15 0:07:23 0:05:20 0:04:01 0:03:09 0:03:09 0:03:02 0:03:00 0:02:37 0:02:28 0:02:11 0:02:02

143.68 -------------------------------------------------------------------------------------3:39:32 1:12:05 2007-09-20 20:42 2007-09-20 21:21 2:27:27 2007-09-20 23:35 2007-09-21 00:36 2795.49 --------------------------------------------------------------------------------------

0:03:45 0:33:04 0:01:56 0:18:37 0:13:30 0:07:01 0:01:43 0:07:36 0:00:28 0:07:47 0:01:12 0:06:11 0:02:54 0:02:26 0:00:47 0:03:14 0:02:02 0:01:07 0:02:29 0:00:40 0:02:32 0:00:30 0:00:39 0:02:21 0:01:49 0:00:48 0:01:45 0:00:43 0:01:11 0:01:00 0:01:14 0:00:48

2007-09-20 2007-09-20 2007-09-20 2007-09-20 2007-09-20 2007-09-20 2007-09-20 2007-09-20 2007-09-20 2007-09-20 2007-09-20 2007-09-20 2007-09-20 2007-09-20 2007-09-20 2007-09-21 2007-09-20 2007-09-20 2007-09-20 2007-09-20 2007-09-20 2007-09-20 2007-09-20 2007-09-21 2007-09-20 2007-09-20 2007-09-20 2007-09-20 2007-09-20 2007-09-20 2007-09-20 2007-09-20

20:59 23:39 21:00 23:43 20:58 23:35 21:06 23:36 21:00 23:55 20:57 23:36 20:49 23:37 21:02 00:02 21:03 23:38 21:18 23:35 20:58 23:35 21:04 00:00 20:49 23:43 20:58 23:45 20:51 23:40 20:53 23:35

2007-09-20 2007-09-21 2007-09-20 2007-09-21 2007-09-20 2007-09-20 2007-09-20 2007-09-20 2007-09-20 2007-09-21 2007-09-20 2007-09-20 2007-09-20 2007-09-21 2007-09-20 2007-09-21 2007-09-20 2007-09-20 2007-09-20 2007-09-20 2007-09-20 2007-09-20 2007-09-20 2007-09-21 2007-09-20 2007-09-20 2007-09-20 2007-09-20 2007-09-20 2007-09-20 2007-09-20 2007-09-20

21:03 00:12 21:02 00:01 21:12 23:42 21:08 23:43 21:00 00:03 20:58 23:42 20:52 00:35 21:03 00:06 21:05 23:39 21:21 23:36 21:01 23:35 21:04 00:02 20:51 23:44 21:00 23:46 20:52 23:41 20:54 23:36

39.82 23.37 959.46 57.76 4.77 104.46 46.59 3.92 41.02 155.01 151.99 4.14 19.55 13.52 31.03


29

Figure 3-5 shows a graphical representation of the package join time diagram.

Figure 3-5 Example of a package join time diagram

3.5 Distribution monitor: DistMonMigMon controls the export and import processing limited to a single server.

30


Using DistMon, the export and import processes can be performed by multiple servers, which is illustrated in Figure 3-6.Server 1

Export

Import

R3load R3load R3load


Source DB

Shared Directory

IBM DB2


Server n


Export

Import

Figure 3-6 Function scheme of the distribution monitor

For a detailed description of the DistMon, see the SAP Notes 855772, 989116, and 1001383. The attachment of SAP Note 855772 contains the software archive DISTMON.SAR and the user guide Distribution Monitor User's Guide, which is stored in the zip-archive file DistributionMonitorUserGuide.zip. The user guide is also part of the SAR-archive DISTMON.SAR. It is named DistributionMonitorUserGuide.doc.


31

3.6 Tools for package and table splittingTo improve the speed of the SAP system copy, multiple R3load processes can export and import data in parallel. Different options are available to package the data during export and import, as shown Figure 3-7.

Source DB

Tables A,B,C

Table D

Table ERecords 1..n Records n+1..m

R3load

R3load

R3load

R3load

.nnn

.nnn

.nnn

.nnn

R3load

R3load

R3load

R3load

Target DB

Figure 3-7 Package and table splitting options

The options for package and table splitting include: A package may contain data of multiple tables (for example, tables A, B, and C in Figure 3-7). A package may contain all data of a single table (for example, table D in Figure 3-7). A package may contain only a subset of a table (for example, as shown for table E in Figure 3-7). To export a single table into multiple data packages, the R3load process requires a WHERE file that defines the range of records that should be extracted from the table. In most cases, the source database contains a small set of large tables. You can export these tables in parallel to multiple package files where each file contains the data of one table. For very large tables, performance can be improved even further by exporting and importing a single table with multiple R3load processes in parallel.

32


SAP provides several tools for splitting packages or tables: Package Splitter This splits tables from existing structure files. You can explicitly specify the tables that should be split into separate structure files. You can also split tables into separate packages if their size exceeds a configurable threshold. Other split options are also available. R3ta This tool can generate multiple WHERE conditions for a table, which can then be used to export the data of this table with multiple R3load processes in parallel. Each R3load process requires a WHERE condition to select only a subset of the data in the table. SAPInst This tool can be used for the table splitting preparation. It invokes R3ta and the Package Splitter, and also automates some of the tasks that would otherwise need to be performed manually.


33

34


4

Chapter 4.

Export optimization techniquesThis chapter describes different techniques to improve export performance during a heterogeneous system copy or Unicode conversion. We performed tests with these techniques and different options that are available for the corresponding tools. The test results are presented together with our recommendations.


35

4.1 A closer look at cluster tables and table clustersTable clusters in many cases determine the overall migration time. This is due to the fact that they must be exported in a sorted manner when code page conversions are performed together with the fact that some of the table clusters are very large (for example, typical candidates are CDCLS, EDI40, and RFBLG). In this section we discuss cluster tables and table clusters. The basics in this section can help you understand why table clusters must generally not be unloaded unsorted in certain situations, in particular when a Unicode conversion is performed. This fact leads to a major pain point from the export point of view. The reason is that the sorted unload of table clusters can have runtimes that are by far higher than those of an unsorted export. This is especially true if the source table and indexes are highly fragmented. With a Unicode conversion added, the export time can be long enough to consume a large portion of the entire downtime window. Table 4-1 shows the effect of an unsorted export for a large table.Table 4-1 Example from a customer migration (non-Unicode) Table name Number of rows Sorted unload [hh:mm] Unsorted unload [hh:mm] CDCLS 63,422,261 34:40 03:28

Now we take a closer look at how cluster tables and table clusters are defined and what the implications are for Unicode conversions.

36


A SAP system stores table definitions in the data dictionary table DD02L. The structure of this table is shown in Figure 4-1.

Figure 4-1 Structure of table DD02L

One important attribute of a table is the table class (field: TABCLASS). If you want to know how many tables are associated with a specific table class run the following SQL statement as the SAP database connect user (for example, sap).SELECT tabclass, count(tabname) AS No_of_TABs_in_class FROM DD02L GROUP BY tabclass ORDER BY 2 DESC

For an SAP ERP 2004 IDES system the result looks like that shown in Example 4-1.Example 4-1 Table class distribution for ERP 2004 IDES TABCLASS ------INTTAB TRANSP NO_OF_TABS_IN_CLASS ------------------127468 47078

=> Structures => transparent tables (*)

Chapter 4. Export optimization techniques

37

VIEW POOL APPEND CLUSTER (*) contains data

30649 2022 1114 79

=> => => =>

views pool tables (*) append structures cluster tables (*)

Only the table classes TRANSP, POOL, and CLUSTER store data within database tables. Tables that have a table class CLUSTER are called cluster tables. They are physically stored in a table that is called a table cluster. The definitions of these table clusters are stored in the data dictionary table DD06L. To get a list of the cluster tables and their associated table cluster, use the following SQL statement:SELECT tabname AS Cluster_Table, sqltab AS Table_Cluster from dd02l where tabclass='CLUSTER' ORDER BY sqltab, tabname

The resulting list, shown in Example 4-2, is based on an SAP ERP 2004 IDES system.Example 4-2 List of cluster tables with associated table clusterCLUSTER_TABLE ----------------------------AUAA AUAB AUAO AUAS AUAT AUAV AUAW AUAY CDPOS PCDPOS TACOPC TACOPCA TAB1 TAB2 CVEP11 CVEP12 CVEP13 CVEP14 TACOPA TACOPAB TACOPAC TACOPAD TABLE_CLUSTER -----------------------------AABLG AABLG AABLG AABLG AABLG AABLG AABLG AABLG CDCLS CDCLS CLU4 CLU4 CLUTAB CLUTAB CVEP1 CVEP1 CVEP1 CVEP1 CVEP1 CVEP1 CVEP1 CVEP1

38


CVEP21 CVEP22 CVEP23 CVEP24 CVER11 CVER12 CVER13 CVER14 CVER15 CVER21 CVER22 CVER23 CVER24 CVER25 CVER31 CVER32 CVER33 TACOPB TACOPBA CVT1 CVT2 CVT3 CVER50 CVER51 DOKTL DSYOL DSYOT DSYGH DSYGI DSYGL DSYXN DSYXO EDID2 EDID4 EDIDD_OLD GLIDXB GLIDXC GLS2IDX MMIM_PRED KONV REGUP BSEC BSED BSEG BSES BSET BSSEG MHND SFHOT T512U TERMC1 TERMC2 TERMC3 UMG_TEST_B

CVEP2 CVEP2 CVEP2 CVEP2 CVER1 CVER1 CVER1 CVER1 CVER1 CVER2 CVER2 CVER2 CVER2 CVER2 CVER3 CVER3 CVER3 CVER3 CVER3 CVER4 CVER4 CVER4 CVER5 CVER5 DOKCLU DSYO1 DSYO1 DSYO2 DSYO2 DSYO2 DSYO3 DSYO3 EDI30C EDI40 EDIDOC EPIDXB EPIDXC GLS2CLUS IMPREDOC KOCLU REGUC RFBLG RFBLG RFBLG RFBLG RFBLG RFBLG RFMHN SFHOA T512CLU TERCL TERCL2 TERCL3 UMG_TEST_C


39

UMG_TEST_D UMG_TEST_G CVERI_CLNT 79 record(s) selected.

UMG_TEST_C UMG_TEST_F VER_CLUSTR

To check the number of table clusters and the associated number of cluster tables, use the following SQL statement:SELECT sqltab AS Table_Cluster, count(tabname) AS Cluster_Tables FROM dd02l WHERE tabclass='CLUSTER' GROUP BY sqltab ORDER BY 2 DESC

You can associate a table cluster to one or more cluster tables. The way the cluster tables and table clusters relate according to physical storage of data is that a logical row in a cluster table is mapped to one or more physical rows in the table cluster. Let us have a closer look at the definition of the table cluster DOKCLU and its contents. See Figure 4-2.

Figure 4-2 SE11 for table cluster DOKCLU

You can identify every logical row of a cluster table by the key fields of the table cluster. The data for the logical rows of a cluster table are stored in the column VARDATA. If the data is longer than the defined length of VARDATA, additional records for the same key are stored with an increased number of PAGENO. The field PAGELG informs you of the length of the data stored inside the VARDATA field.

40


The example in Figure 4-3 shows four records for the key DT RSMD_CONVERSION D E 0038 with an increasing PAGENO. The first three records are completely filled up with data in the field VARDATA. This can be seen from the PAGELG with a value of 3800, which matches the length of the VARDATA field. The fourth record with PAGENO 3 has a PAGELG of 60, meaning that this record is not filled up to the end.

Figure 4-3 Records of table cluster DOKCLU for one logical row

To handle cluster tables and the associated table clusters, SAP has created a cluster interface. If we look into the mechanism that this interface is using when changing logical rows of a cluster table, we understand why the associated table clusters that are storing the data must not be unloaded unsorted when doing a Unicode conversion. The following list shows the procedure that R3load uses when exporting a table cluster with Unicode conversion: 1. Reads all database rows belonging to one logical row (PAGENO is increased row by row. When PAGENO starts with 0 again, a new logical row starts.) 2. Concatenates RAW fields (for example, VARDATA) 3. Decompresses RAW fields 4. Identifies start and end offsets of character information 5. Identifies code page for character information. 6. Converts characters to Unicode. 7. Builds new RAW field content 8. Compresses RAW fields 9. Builds several database rows from the RAW field. 10.Writes database rows belonging to one logical row Due to the Unicode conversion (or other code page conversions, for example, EBCDIC to ASCII), the contents and the length of the records may change. Even the number of the physical records belonging to a logical record may change.


41

Because the physical records are built together to a logical record, the data must be read in a sorted way to find all physical records that belong to a logical record. Therefore, an unsorted unload is not supported. If no changes to the contents of the records are made (no code page conversion), the logical record does not have to be constructed and the table cluster can be unloaded unsorted.

4.2 Unsorted versus sorted exportThere are two different ways to export the data from the source database: Sorted export Unsorted export With a sorted export, the pages of a table are read in the sequence of the primary key. If the cluster ratio is not optimal, data pages will not be read continuously, but there will be mechanical movements of the disk heads on the spindles. In addition, database sort operations may occur, which will also waste time, and the export runtime will not be optimal. Note: The unsorted export is one of the most powerful optimizations to improve the export time. However, the data is also imported unsorted and thus the database performance may not be optimal. You may see inefficient access to the disks, and in the worst case the DB2 optimizer may choose a wrong access plan. Use the unsorted export wisely and plan for a subsequent reorganization of unsorted tables. With an unsorted export, the pages are read continuously and export time will decrease in most cases compared with sorted export. This improvement can be dramatic, so we recommend unsorted export for most of the tables. However, not all tables can be exported in this way. For exceptions, see SAP Note 954268. Note: If a code page conversion is performed, table clusters must be unloaded sorted. As of R3load 6.40 patch level 55 (compile date February 10, 2006) and R3load 7.00 patch level 10 (compile date February 10, 2006), R3load automatically ensures that table clusters are exported in sorted order during a code page conversion.

42


4.2.1 Enabling unsorted exportThe mode of export is controlled by the ORDER_BY_PKEY keyword in the prikey section of the DDL.TPL file (Figure 4-4).

Figure 4-4 Example of DDL.TPL file

If the keyword ORDER_BY_PKEY is deleted, the unloading is done unsorted. When using the MigMon, you may use the ddlMap option in the export properties file, which names a file that contains the mapping between the package names and the DDL template files. Example 4-3 shows an example of a mapping file. The DDL_LRG.TPL template file that does not contain the ORDER_BY_PKEY keyword in the prikey section is generated by R3ldctl in addition to the DDL.TPL file. The packages SAPCLUST and SAPSDIC are using the DDL.TPL template file. The table MSEG (which has a package of its own) and the package SAPAPPL1 are unloaded by using the DDL_LRG.TPL template file.Example 4-3 Example of a MigMon DDL template mapping file [ SORTED UNLOAD ] # DDL file for sorted unload ddlFile = /DDL.TPL #package names SAPCLUST SAPSDIC [ UNSORTED UNLOAD ] # DDL file for unsorted unload ddlFile = /DDL_LRG.TPL # package names (may also contain only on

Database Migration System Unicode Conversion

Documents

Transcript of Database Migration System Unicode Conversion