BDLS IN LESS THAN 2 HOURSIMPORTANT: The methods used here are not supported by SAP. However, they are not very complex to understand and execute.ScenarioThe BDLS run takes more than 20 hours in our quality systems post system refresh.Our system details:-Database size 2 TB

SAP versionECC 6.0 patch level 21

Database VersionOracle 11.2.0.2.0

OS VersionRHEL 5.6

No. of Clients to run BDLSTwo (450 & 500)

The objective of this blog is to analyze and find out if there are faster ways to perform BDLS operation, if we cross boundaries of SAP standard methods.Step - 1Based on SAP Note. 1547980 we decided to use older version of BDLS program RBDLS2LS as it has capacity to run in parallel.We were thinking that the program RBDLS2LS itself has some inbuilt capability to run in parallel. We were expecting it to use some RFC groups like parallel_generators, etc and submit conversion for each table in seperate task/ process like client copy.Whereas we found out that this is not the case. The RBDLS2LS does not do anything in parallel, neither does it use use any RFC groups not it uses more than one BGD/ DIA processes.Parallel in this case means that multilpe executions of the report RBDLS2LS is possible. The report do not come back complaining with an error that there is already a running BDLS process like the new BDLS tcode.Based on the tips from blog http://www.sdn.sap.com/irj/scn/weblogs?blog=/pub/wlg/4796 , we decided to run multiple jobs.We increased number of background processes to 26.Database was put in noarchivelog mode.No tables were excluded from BDLS.In client 450 > SE38 > RBDLS2LS > Generated RBDLS450Again SE38 > RBDLS450 > Filled details as below > Executed in BackgroundContinued like this for B*, C*,D*...and so on till ...Z* and finally by excluding A* to Z*.We also started execution on client 500 similarly in parallel.AchievementBDLS completed in 11 hours

ObservationsWe were able to find the jobs & tables taking longer times. For examples; Jobs converting tables C*, E*, M*, V* and S* took more times than others.

It was time consuming to schedule 26 jobs per client

Step - 2Taking help from an ABAPer, we could automate a BDC session to run report RBDLS450 with A*, B*, C*...Z* as inputs to selection screen.

PART-2Again Based on the tips from blog http://www.sdn.sap.com/irj/scn/weblogs?blog=/pub/wlg/4796 , we wanted to optimize it more by building indexes.Before building indexes, its benificial to understand how BDLS works.BDLS runs in phases:- 1. It will first determinies all tables that are using logical system in one of its fields. 2. It will then find out all tables that need conversion, by running a SELECT SINGLE command on all tables. For example; SELECT /*+ FIRST_ROWS(1) */ LOGSYSP FROM COEP WHERE MANDT=500 AND LOGSYSP = 'S1Q500' AND ROWNUM 10000 Identifying client filed (like MANDT, CLIENT, RCLIENT, etc) Identifying logical system field (like LOGSYS, LOGSYSP, SENDPRN, RECVPRN, etc) Building 80 indexes on client and logical system field in parallel taking now 2 hours.Step - 5We tried with building REVERSE key index; with no significant benifit. The run times remained more or less the same.CREATE INDEX COEP_BDLS_IDX_12345 ON COEP(MANDT, LOGSYSO) REVERSE NOLOGGING PARALLEL TABLESAPCE PSAPBDLS COMPUTE STATISTICS;Step - 6With further studies we found that Oracle Bitmap indexes works better than B-tree indexes for columns with only a few distinct values (just MALE, FEMALE for example). They are compressed, very compact and fast to scan.MANDT contains very few distinct values and also LOGSYS realted fields usually contains SPACE or very few distinct values. This characterstic makes them ideal candidate for bitmap indexes. We created bitmap indexes instead of regular indexes.An comparions of size and access time is shown below:-Activity on COEPRegular IndexBitmap Index

Create index in parallel, nologging2 Min 25 Sec1 Min 32 Sec

Size of Index3425 MB53 MB

Select query run time16 Seconds1.25 Seconds

Below is an example statement used to build bitmap index.CREATE BITMAP INDEX COEP_BDLS_IDX_12345 ON COEP(MANDT, LOGSYSO) NOLOGGING PARALLEL TABLESAPCE PSAPBDLS COMPUTE STATISTICS;Achievement80 indexes took only 1 GB of space

Building 80 indexes now took 30 minutes only with automation using sql scripts

PART-3UPDATE phase will be slower with indexes; particularly those indexes which are built on the the same fields that are being updated.In other words, mass updates on the indexed fileds causes additional overhead for maintainance and updating of the those indexes. Such indexes are referred to as evil indexes.Possible solution was to drop/ hide indexes having these fields before performing mass updates.Step - 6After creating bitmap indexes; first we ran RBDLS450 in test mode (check box - Test Run) in parallel as explained earlier in Part -1, Step - 1.We then dropped the indexes and ran RBDLS450 in Conversion Mode, i.e. by removing above check boxes.

AchievementBDLS completed in 8 hours. Not a great benifit though.

ObservationsUPDATE phase used FULL TABLE SCAN

From previous BDLS runs, we were able to gather below statistics: -TableBDLS run time in hoursTable size in MB

MKPF7.783253.19

SWW_CONTOB6.284837.54

EDIDC6.116766.91

MLHD5.522786.75

RBKP5.422972.29

VBRK2.551029.07

VBAP2.253333.66

ETXDCH1.57266.32

SMMAIN1.4496.08

SMPARAM1.4255.27

SMSELKRIT1.41234.27

We observed that these tables are not very big in size; the largest one is EDIDC with size 6.6 GB and it take surprisingly more than 6 hours to update.Step - 7With further googling around and reading several articles, we realized that CTAS (Create table as select) is the best method to perform mass updates. We decided to test it; we executed below commad: -

CREATE TABLE EDIDC_BDLS_BKUPPARALLEL COMPRESS FOR OLTP NOLOGGINGSTORAGE ( INITIAL 3253M NEXT 50M MINEXTENTS 1 MAXEXTENTS UNLIMITED)TABLESPACE PSAPNXP AS(SELECTMANDT, DOCNUM, DOCREL, STATUS, DOCTYP, DIRECT, RCVPOR, RCVPRT,decode(MANDT||RCVPRN,'500S1P500','S1Q500',RCVPRN) RCVPRN,RCVSAD, RCVSMN, RCVSNA, RCVSCA, RCVSDF, RCVSLF, RCVLAD, STD,STDVRS, STDMES, MESCOD, MESFCT, OUTMOD, TEST, SNDPOR, SNDPRT,decode(MANDT||SNDPRN,'500S1P500','S1Q500',SNDPRN) SNDPRN,SNDSAD, SNDSMN, SNDSNA, SNDSCA, SNDSDF, SNDSLF, SNDLAD, REFINT,REFGRP, REFMES, ARCKEY, CREDAT, CRETIM, MESTYP, IDOCTP, CIMTYP,RCVPFC, SNDPFC, SERIAL, EXPRSS, UPDDAT, UPDTIM, MAXSEGNUMFrom EDIDC);You may notice the lines highlighted above in the command. The oracle's decode function performs the conversion (UPDATE). The newly created table EDIDC_BDLS_BKUP will have the S1Q500 replaced wherever S1P500 was found for fields RCVPRN and SNDPRN.Another way of doing it is using CASE statement. It is more easier to build CASE statement than DECODE.For example, In a scenario we had to convert 4 logical systems, we use CASE statement as below.CREATE TABLE EDIDC_BDLS_BKUPPARALLEL COMPRESS FOR OLTP NOLOGGINGSTORAGE ( INITIAL 3253M NEXT 50M MINEXTENTS 1 MAXEXTENTS UNLIMITED)TABLESPACE PSAPNXPAS(SELECTMANDT, DOCNUM, DOCREL, STATUS, DOCTYP, DIRECT, RCVPOR, RCVPRT, CASE LOGSYS WHEN 'S1P100' THEN 'S1Q100' WHEN 'S1P500' THEN 'S1Q500' WHEN 'BWP200' THEN 'BWQ200' WHEN 'PIP300' THEN 'PIQ300' ELSE LOGSYS END LOGSYS,RCVSAD, RCVSMN, RCVSNA, RCVSCA, RCVSDF, RCVSLF, RCVLAD, STD,STDVRS, STDMES, MESCOD, MESFCT, OUTMOD, TEST, SNDPOR, SNDPRT,decode(MANDT||SNDPRN,'500S1P500','S1Q500',SNDPRN) SNDPRN,SNDSAD, SNDSMN, SNDSNA, SNDSCA, SNDSDF, SNDSLF, SNDLAD, REFINT,REFGRP, REFMES, ARCKEY, CREDAT, CRETIM, MESTYP, IDOCTP, CIMTYP,RCVPFC, SNDPFC, SERIAL, EXPRSS, UPDDAT, UPDTIM, MAXSEGNUMFrom EDIDC);To our surprise this CTAS operation on EDIDC of 6.7 GB took only 12 seconds. Then we dropped indexes on original table EDIDC.Renamed original table EDIDC to EDIDC_ORIG.

rename EDIDC to EDIDC_ORIG;Renamed newly created table EDIDC_BDLS_BKUP to EDIDC.rename EDIDC_BDLS_BKUP to EDIDC;Rebuild indexes from SE14 > EDIDC > Indexes > Create in Background The total operation took less than 5 minutes for EDIDC , whereas BDLS conversion took 6 hours.

AchievementWe were able to perform CTAS and conversion of 15 tables in 3 hours because CTAS command need to be manually constructed

BDLS run on remaining tables took less than 2 hours

Total conversion time is 5 hours

ObservationsConstructing CTAS command is time consuming and prone to human error

Building indexes from SE14 needs to be triggered manually

CTAS removed NOT NULL constrains from converted (DECODED) fields

Looking at the significant benefit (6 hours vs. 5 minutes), we decided to adopt CTAS for all tables where rows to be udpate > 1000.Step - 8The rebuilding of indexes from SE14 do not use parallel, nologging options.Some time can be saved if you build indexes from database level using parallel clause and then turning off the parallelism.Before dropping any index, you can get the DDL (SQL) Command to build the index as below:-set timi onset echo offset head offset long 5000set pagesize 0set linesize 150column DDL format A150select DBMS_METADATA.GET_DDL('INDEX','EDIDC~0') DDL from dual;Add clauses NOLOGGING PARALLEL COMPUTE STATISTICS the command;Once Index is built; turn off parallelism ALTER INDEX "EDIDC~0" NOPARALLEL;NOT NULL constraints had to be added to table EDIDC for fields RCVPRN and SNDPRN.ALTER TABLE "EDIDC" MODIFY RCVPRN NOT NULL;ALTER TABLE "EDIDC" MODIFY SNDPRN NOT NULL;Step - 9We left the update statistics running in the background with below commands:-brconnect -u / -c -f stats -t system_statsbrconnect -u / -c -f stats -t oradict_statsbrconnect -u / -c -f stats -t all -p 15 -f nocheck -f collectPart - 4Putting together all possible optimization and automation; below are the brief steps we articulated:-1Put database in NOARCHIVELOG mode

2Create PSAPBDLS tablespace of 5 GB

3Enable parallelism for all tables invloved in BDLS

4Enable parallellism for all indexes of the tables invloved in BDLS

5Create bitmap indexes with PARALLEL NOLOGGING COMPUTE STATITICS for tables with total number of rows > 10000

6Calculate rows to be updated by BDLS

7Build CTAS command lines with decode funtions

8Perform coversion of tables where rows to update > 1000 using CTAS

9Save original index definitions

10Drop original indexes

11Rename original table to old

12Rename newly created tables to original

13Recreate indexes

14Verify and Drop old table

15Run BDLS in SAP using report RBDLS450 in parallel (A*....Z*, etc) using BDC sessions.

16Clean up

17Drop bitmap indexes

18Revert parallelism to original state for all tables and indexes

19Drop function created, if any

20Drop extra tables created, if any

21Run update statistics

22Put database back on ARCHIVELOG mode

With automation and combining above optimization approach, BDLS was achieved in 1 hour and 25 minutes.

ActivityTime

Create PSAPBDLS tablespace of 5 GB5 minutes

Bitmap indexes30 Minutes

CTAS for 15 tables15 Minutes

BDLS run in parallel30 Minutes

Cleanup Bitmap indexes, tablespace, etc5 minutes

TOTAL1:25 hours

You may also plan to create bitmap indexes on source system so that these indexes are already available on target system. This will save 30 minutes.