“This presentation is for informational purposes only and may not - 1
“This presentation is for informational purposes only and ... · “This presentation is for...
Transcript of “This presentation is for informational purposes only and ... · “This presentation is for...
“This presentation is for informational purposes only and may not be incorporated into a contract or agreement.”
<Insert Picture Here>
Partitioning and Purging Best Practices for the E-Business SuiteAhmed Alomari & Mohsin SameenApplications Performance Group
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be
incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in
making purchasing decision. The development, release, and timing of any features or functionality described for Oracle’s products
remains at the sole discretion of Oracle.
Agenda
• Partitioning Best Practices• Partitioning Concepts• Supportability• Partitioning Methods• Customer Case Studies
• Purging• Business Processes• Purging Strategies• New Purge Portal
• Q & A
<Insert Picture Here>
Partitioning in the E-Business Suite
Partitioning Continues to Evolve
Database Release Partitioning Functionality
Oracle 8Range partitioned tablesLocal partitioned indexesGlobal range partitioned indexesHash and Composite; Range-hash partitioned tables, Range partitioned index-organized tablesList partitioned tables, Hash partitioned index-organized tablesComposite range-list partitioned tablesList partitioned index-organized tables
Global hash partitioned indexes
Oracle 8i
Oracle 9i
Oracle 9iR2
Oracle 10g
Partitioning Concepts
• Data Partitioning: • Process of logically and/or physically segmenting data (and
it’s associated storage, tables and indexes) into more manageable pieces, in order to improve manageability and accessibility.
• Example• Divide large Applications tables and their indexes such as
GL_BALANCES into smaller segments.
Partitioning in the eBusiness Suite
• Oracle Applications utilizes partitioning in the standard product in many modules:• Advanced Planning and Scheduling• Payables (Trial Balances)• Projects Resources• Workflow
• Directory Services• Runtime tables
• Daily Business Intelligence• HR (Employee Directory)• Engineering
Support Policy: Custom Partitioning
• Oracle Applications fully supports the use of custom partitioning of either Applications standard or custom tables.
• Definition of custom partitioning: Changing the partitioning definition of an existing Applications table as delivered (out-of-the-box).
• If custom partitioning is used, you must license the partitioning option.
• Refer to the write-up on the Technology Stack blog: http://blogs.oracle.com/schan/2006/09/06#a663
<Insert Picture Here>
Why Use Partitioning?
Partitioning Advantages
• Table availability• Significantly reduce recovery times of key transaction tables by
recovering the current partitions first.
• Table manageability• Backup, restore, and rebuild at the partition level.• Index rebuilds can be performed at the partition level.• Partition aware operations such as MOVE, EXCHANGE,
REBUILD can be used without affecting active partitions.
Partitioning Advantages
• Performance• Improves access path of most queries since the majority of the
access involves current data as opposed to historical data. • Optimizer automatically prunes unnecessary partitions.
• Analytical reports or period close jobs/reports improve by scanning the current partition as opposed to all the partitions.
• Improves purge performance.• Significantly improves upgrade performance
• Minimizes upgrade downtime.
Partitioning Advantages
• CBO is partition aware and employs partition pruning at runtime.• Eliminates partitions which are not needed by the SQL
statement.• Joins are also partition aware and benefit from pruning.
• Parallel operations can occur at the partition level.• Utilizes the Information Lifecycle Management (ILM)
infrastructure to allow partitions containing historical Applications data to be placed on lower cost storage devices.
<Insert Picture Here>
Partitioning Methods
Rangepartitioning
Hashpartitioning
Compositepartitioning
Listpartitioning
• The following partitioning methods are available:
Partitioning Methods
Partitioning Methods - Range
• Range partitioning requires that rows identified by a "partition key" fall into a predefined range of values.
• Value of the partition key determines the partition to which a row belongs.
• Range partitioning was introduced in Oracle8.• Range partitioning is useful for Applications tables for
which the partition key matches the access path key.
Partitioning Methods - Range
GL_BALANCES476,720,210 rows
JUN-044,886,548
rows
JUL-044,949,399
rows
AUG-043,833,931
rows
SEP-04464,417
rows
OCT-0434,212rows
Partition the GL runtime tables by period_name
CREATE TABLE GL_BALANCES(SET_OF_BOOKS_ID NUMBER(15) NOT NULL,CODE_COMBINATION_ID NUMBER(15) NOT NULL,CURRENCY_CODE VARCHAR2(15) NOT NULL,PERIOD_NAME VARCHAR2(15) NOT NULL,ACTUAL_FLAG VARCHAR2(1) NOT NULL,BUDGET_VERSION_ID NUMBER(15),LAST_UPDATE_DATE DATE NOT NULL,. . . . . . )PARTITION BY RANGE (PERIOD_NAME)(PARTITION jan04_per VALUES LESS THAN (‘JAN-2005’),PARTITION feb04_per VALUES LESS THAN (‘FEB-2005’). . . . . .);
• Range Partitioning Example:
Partitioning Methods
Partitioning Methods- List
• Partitions are mapped to an enumerated list. • Allows data to be mapped to partitions by specifying a
list of discrete values for the partitioning key.• List partitioning does not support multi-column
partition keys.• List partitioning was introduced in Oracle9i.
Partitioning Methods - List
OE_ORDER_LINES_ALL
(44,519,880 rows)
Open Orders(192,403 rows)
Closed Orders(40,169,899 rows)
Partition the OM runtime tables by open_flag
CREATE TABLE OE_ORDER_LINES_ALL(LINE_ID NUMBER NOT NULL,ORG_ID NUMBER,HEADER_ID NUMBER NOT NULL,LINE_TYPE_ID NUMBER NOT NULL,LINE_NUMBER NOT NULL NUMBERORDERED_ITEM VARCHAR2(2000),OPEN_FLAG VARCHAR2(1) NOT NULL,. . . . . . )PARTITION BY LIST (open_flag)(PARTITION open_par VALUES (‘Y’),PARTITION close_par VALUES (‘N’)
);
• List Partitioning Example:
Partitioning Methods
Partitioning Methods - Hash
Applies a hash function to the partitioning key to stripe data into partitions. Controls the physical placement of data across a fixed number of partitions.Hash partitioning was introduced in Oracle8i.Generally useful for Applications batch programs which use parallel workers for which block contention is significant.
Order ImportInvoice Import
Partitioning Methods - Hash
Hash15,045,430
rows
Hash25,045,402
rows
Hash45,042,929
rows
Partition the OM runtime tables by the primary key
OE_ORDER_LINES_ALL
(44,519,880 rows)
Hash35,043,712
rows
Hash55,046,608
rows
Hash65,045,024
rows
Hash75.048.923
rows
Hash85,044,312
rows
CREATE TABLE OE_ORDER_LINES_ALL(LINE_ID NUMBER NOT NULL,ORG_ID NUMBER,HEADER_ID NUMBER NOT NULL,LINE_TYPE_ID NUMBER NOT NULL,LINE_NUMBER NOT NULL NUMBERORDERED_ITEM VARCHAR2(2000),OPEN_FLAG VARCHAR2(1) NOT NULL,. . . . . . )PARTITION BY HASH (LINE_ID)PARTITIONS 8. . .. . .;
• Hash Partitioning Example:
Partitioning Methods
Partitioning Methods - Composite
Data is partitioned by using both the range method and then sub-partitioned by the hash method. Composite partitioning combines the best of both worlds (range and hash) by allowing logical groupings at the partition level and handling data skew within the sub-partitions.Composite partitioning was introduced in Oracle8i.
hash1 hash2 hash4
The Workflow Runtime tables use composite partitioning.
WF_ITEM_ACTIVITY_STATUSES
(82,585,860 rows)
hash3 hash5 hash6 hash7 hash8
Order Item Type(3,402,130 rows)
hash1 hash2 hash4hash3 hash5 hash6 hash7 hash8
Purchase OrderItem Type
(13,319,867 rows)
Item KeyHash
Partitioning Methods – Composite
create table wf_item_activity_statusespartition by range (item_type) subpartition by hash (item_key) subpartitions 8 (partition wf_item1 values less than ('A1'), partition wf_item2 values less than ('AM'), partition wf_item3 values less than ('AP'), partition wf_item4 values less than ('AR'), partition wf_item5 values less than ('AZ'), . . .partition wf_item48 values less than ('OE'), partition wf_item49 values less than ('OF'), partition wf_item50 values less than ('OK'), partition wf_item51 values less than ('OL'), . . .partition wf_item56 values less than ('PO'), partition wf_item57 values less than ('PQ'), partition wf_item58 values less than ('PR'), partition wf_item59 values less than ('QA'), . . .);
• Composite Partitioning Example:
Partitioning Methods
<Insert Picture Here>
Index Partitioning
Index Partitioning
• Indexes can be partitioned like tables• Partitioned or nonpartioned indexes can be used with
partitioned or nonpartitioned tables• Partioned indexes can be
• Global or local• Prefixed or nonprefixed
Normal Partitioned
Table Index Table Index
• Index Partitioning Methods• Global Non-partitioned (prefixed)• Global Partitioned (prefixed)• Local Partitioned Prefixed• Local Partitioned Non-Prefix
Index Partitioning
• Index Partitioning Methods• Global non-partitioned index• Global partitioned prefixed index• Local prefixed index• Local non-prefixed index
Global nonpartitioned index
Global partitioned Index
Table partitions
Local prefixed index
Local nonprefixed index
Global & Local Index Partitioning
• Index Partitioning Methods• Global Non-partitioned (prefixed)
• Use Global Non-partitioned indexes for all indexes which are not prefixed by the table partition key.
• Global Range Partitioned (prefixed)• Generally not useful for Applications.
• Global Hash Partitioned (prefixed) – new in 10g• Extremely useful for right growing indexes experiencing contention
due to high levels of concurrency.• Allows the index to be partitioned without affecting the table.
Global Indexes
• Index Partitioning Methods• Local Partitioned Prefixed
• Use in place of indexes which already contain the partition key as a prefix.
• Local Partitioned Non-Prefix• Generally not useful for Applications.• Should only be used when all the queries using the local
(non-prefixed) index always include the partition key filter.
Local Indexes
• Index Partitioning Example• GL_BALANCES table is partitioned by range on the
period_name
Existing IndexesNew
Index Name: Columns: Index Type:=======================================================================GL_BALANCES_N1 (CODE_COMBINATION_ID,PERIOD_NAME) GlobalGL_BALANCES_N2 (PERIOD_NAME) Not NeededGL_BALANCES_N3 (PERIOD_NUM, PERIOD_YEAR) GlobalGL_BALANCES_N4 (TEMPLATE_ID) Global
Global Index Partitioning
• Index Partitioning Example• WF_ITEM_ACTIVITY_STATUSES table is partitioned using
composite partitioning (range on item_type, sub-partitioned by hash on the item_key).
Existing IndexesNew
Index Name: Columns: Index Type:=============================================================================WF_ITEM_ACTIVITY_STATUSES_N1 (ACTIVITY_STATUS,ITEM_TYPE) GlobalWF_ITEM_ACTIVITY_STATUSES_N2 (NOTIFICATION_ID) GlobalWF_ITEM_ACTIVITY_STATUSES_N3 (DUE_DATE, ITEM_TYPE) GlobalWF_ITEM_ACTIVITY_STATUSES_N4 (ASSIGNED_USER, ITEM_TYPE) GlobalWF_ITEM_ACTIVITY_STATUSES_PK (ITEM_TYPE,ITEM_KEY,PROCESS_ACTIVITY) Local
Local & Global Indexes
Useful Data Dictionary Views Table Partitioning
View Name Purpose L
DBA_TABLES Table structure, Partition Y/N
Partition type, default values
Partitions detail
Partition keys
Histogram Statistics
Column and Histogram Stats.
DBA_PART_TABLES
T
T
P
P
P
DBA_TAB_PARTITIONS
DBA_PART_KEY_COLUMNS
DBA_PART_HISTOGRAMS
DBA_PART_COL_STATISTICS P
T = per Table P = per Partition
T = per Table I = per index P = per Partition
Useful Data Dictionary Views Index Partitioning
Name Purpose L
DBA_SUBPART_KEY_COLUMNSSub-partition key Information
Index Partition details
Index Sub-partition details
Partitioned Index Definition
DBA_IND_PARTITIONS
T
P
PDBA_IND_SUBPARTITIONS
DBA_PART_INDEXES I
<Insert Picture Here>
Customer Partitioning Case Studies
Partitioning & Table Compression
• Example OE_ORDER_LINES_ALL
• Before table compression
• After table compression
create table oe_order_lines_all_comp parallel compress tablespaceA_TXN_DATA_01 nologging;
select segment_name, blocks from dba_segments
where segment_name = 'OE_ORDER_LINES_ALL';
SEGMENT_NAME BLOCKS
---------------------------------------------------- -------
OE_ORDER_LINES_ALL 4322352
select segment_name, blocks from dba_segments
where segment_name = 'OE_ORDER_LINES_ALL_COMP';
SEGMENT_NAME BLOCKS
---------------------------------------------------- -------
OE_ORDER_LINES_ALL_COMP 1234957 (~3X)
• Customer Case Study• International Bank• Data Volumes
• GL_JE_LINES (1.1 Billion rows)• GL_CODE_COMBINATIONS (203 Million rows)• GL_BALANCES (1.3 Billion Rows)
• Partitioning Method: Range• Partition Key: set_of_books_id• # of Partitions: 34• Achieved Throughput:
• 11.4 million journal lines imported and posted per hour
Case Study 1: General Ledger (GL) Partitioning
• Customer Case Study• Australian Bank• Data Volumes
• GL_JE_LINES (650 million rows)• GL_CODE_COMBINATIONS (8.5 Million rows)• GL_BALANCES (200 million Rows)
• Partitioning Method: Range• Partition Key: period_name• # of Partitions: 109• Achieved Throughput:
• 7.5 million journal lines imported and posted per hour
Case Study 2: General Ledger (GL) Partitioning
• Partitioning Case Study• Payables Trial Balances• Data Volumes
• AP_LIABILITY_BALANCE (70,567,250 rows)• Partitioning Method: Hash• Partition Key: ORG_ID• # of Partitions: 32• Achieved Throughput:
• Trial Balance report runtime reduced from 2 hours to 10 minutes.
Case Study 3:Accounts Payable (AP) Partitioning
<Insert Picture Here>
Information Lifecycle Management (ILM)
The Data Lifecycle
• Tables continue to grow• Data comes from many sources such as
• Transactional (Fiscal/Non-Fiscal) (e.g. General Ledger)• Transient data (i.e. login, concurrent requests etc..)• Reference/seed data
Active Less Active Historical Archive
This Month This Year Previous Years
Is all this data needed or required to be available online? Will require more storage and memoryIncreased data will impact performance for both online and batch processing
Go-live
What is ILM?
• Information Lifecycle Management encompasses the following:• Policies which define how to manage the data• Processes which actually manage the data
• Including Archiving & Purging and Partitioning• Software which performs the policies & processes
• RDBMS Partitioning functionality • (ILM Advisor)• Purge & Archive concurrent programs
• Hardware where the data is stored• Partitioning, Purging and Archiving are all integral
parts of the ILM lifecycle
Implementing ILM
Active LessActive Historical
High PerformanceStorage Tier
Low CostStorage Tier
HistoricalStorage Tier
1. Define Data Classes
3. Manage Access and Migration of Data by Class2. Create Storage Tiers
for the Data Classes
D E M O N S T R A T I O N
ILM Assistant
<Insert Picture Here>
Archiving & Purgingin the E-Business Suite
Purging & Archiving
Archive
Periodically saving data (with reduced attributes) e.g. transaction data
Purge
Deleting data that you no longer need e.g. transient data or useless data such as dead leads or closed orders.
Improves Upgrade performance
(Less Data.. Less Work)
After Archive
Performance
Periodically
SpaceStatutory Requirements
HistoricalRequirements
Purging & Archiving
• Fiscal Data - Financial Data that can be used to represent a profit or a loss e.g. GL_JOURNALS
• Typically, you should purge and archive non-fiscal data such as sales leads, sales opportunities
Purging & Archiving
• Utilize the standard Oracle supplied programs• Purge/Archive programs provided at the module level in the
E-Business Suite• Many documented standard programs available
• ~214 purge programs in 11.5.10• ~260 purge programs in R12
• Financials• GL, AR, AP, • Costing, • Cash Management, • HR, FA, PA
• Manufacturing• Order Management, • INV, BOM, • WIP, QP, • Shipping, Purchasing
Purge and Archive Programs
Product Concurrent Program Name
Order Management Order Purge
AOL Purge Concurrent Request /Manager DataPurge "Sign on" Audit data
General Ledger GL Archive and PurgeAssets Mass Additions Purge Report
Purge accounting tablesPayables Payables PurgeReceivables Archive and Purge
Purchasing Purge and Archive POs
Customer Example:Typical Purge Programs run
Concurrent Program Name Concurrent Program
Purge Debug Log FNDLGPRG
Purge Cost Information CSTCSPCT
Purge Invoice Extract Output Table RAXINVPG
Catalog Data Purge POXCDXPG
Purge Concurrent Request/Manager Data FNDCPPUR
BEE Batch Process (Purge) PAYLINK(PURGE)
Purge Signon Audit data FNDSCPRG
Payables Open Interface Purge APXIIPRG
Purge System Saved Requisition POXSSPG
Purge Interface Tables PNVPURGE
Purge Obsolete Workflow Runtime Data FNDWFPR
Delete/Purge Timecards (OTL) DELPURTC
<Insert Picture Here>
DemonstrationPurge Portal
Navigation
• Accessing the Purge Portal• System Administrator > Oracle Applications Manager
>Purging/Critical Activities
• Purge Portal introduced in 11i10• Single purge/archive management console• Purge programs can be configured, initiated and monitored• Set the execution frequency as well view history of purge
programs.
• Further enhanced in Release 12• Purge programs tagged with the “purge” concurrent program
type.
Purge Portal Features
Purging – Purge Portal
More Purge Programs in Release 12
• Service Request Purge• Faster WF Purge
• Rewritten to use bulk processing• Email Fulfillment Purge• Export and Purge All GL Interface Data• Approvals Management Transaction Data Purge• Purge Secured Payment Data• Service Contracts Purge Credit Card• Shipping Purge• Task Purge Program• Web Analytics: Purge Tracking Data• Many others
Purging & Archiving
• Oracle is committed to providing Purge/Archive solutions for the key transactional entities
• Oracle continues to enhance its purge and archive offering
• The native E-Business Suite purge and archive programs are synchronized with the data model• Avoids having to re-apply the same patch to multiple schemas
when using non-native purge/archive solutions.
Further InformationMetaLink & Product User Guides
• Please refer to MetaLink (examples) • 138264.1 – General Ledger Archive/Purge FAQ• 144431.1 – Fixed Assets Archive/Purge FAQ• 136919.1 – General Ledger Archive/Purge Setup and Usage
GL 11.5.10User Guide Chapter: 8
Maintenance
AP 11.5.10User Guide Chapter: 10Resource Management
AR 11.5.10User Guide Chapter:1
Archive & Purge
<Insert Picture Here>
Summary
Options Available
Data StorageArchive
Purge
Partitioning Data Access
ILM
Partitioning in Oracle ApplicationsWhat you need to consider
• Functional use of the table (i.e. what exactly does the table store?).
• Which modules or business flows use this table?• Growth rates and access patterns of this table.
• Review the Statspack or AWR Repository for the relevant SQL statements.
• Determine the optimal table partitioning method based on your implementation and based on the data points above.
• Evaluate different indexing methods, global vs. local partitioned indexes.
• Leverage the ILM Assistant to implement partitioning.
Implementing Purging & Archiving
• Consider• Data Retention Requirements• Recovery Requirements• Data access methods and security• Storage topology and costs
• Define• Your archive and purge policy
<Insert Picture Here>
We would like to hear from you
If you have any questions or experiences you would like to share regarding partitioning and/or purging and archiving in the E-Business Suite, please send an e-mail:
MetaLink References
Note #: Description:
138264.1 General Ledger Archive and Purge - Frequently Asked Questions
144431.1 Oracle Assets Archive, Purge and Restore Processes
136919.1 General Ledger Archive/Purge Setup and Usage
248857.1 Oracle Applications Tablespace Model Release 11i - Tablespace Migration Utility
<Insert Picture Here>
Questions & Answers