11g Partitioning Ins and Outs

<Insert Picture Here>

Getting to know the ins and outs of Oracle Partitioning in Oracle Database 11g

Ananth RaghavanSenior Director, Database Development

3

Agenda

• Partitioning in a nutshell• Oracle 11g .. a quite old release• What’s new in 11.2.0.2?• Some things less known• Q&A

4

The Concept of PartitioningSimple Yet Powerful

Large Table

Difficult to Manage

Partition

Divide and Conquer

Easier to Manage

Improve Performance

Composite Partition

Better Performance

More flexibility to match business needs

JAN FEB

USA

EUROPE

ORDERS

Transparent to applications

ORDERS

JAN FEB

ORDERS

5

<Insert Picture Here>

“Partitioning is a key enabler for the performance of our system. We just could not process our data volumes

without Partitioning.”

Arup NandaSenior Director

Database Engineering and Operations Starwood Hotels

6

Partitioning Benefits

Partitioning is• Faster– By touching and maintaining only the relevant data

• Cheaper– By storing the data appropriately and most cost-effective

• Flexible– By managing partitions autonomously

6

7

• Only relevant partitions will be accessed– Static pruning with known values in advance– Dynamic pruning uses internal recursive SQL to

find the relevant partitions

• Minimizes I/O operations– Provides massive performance gains

SELECT sum(sales_amount) FROM salesWHERE sales_date BETWEEN ‘01-MAR-2006’ AND ‘31-MAY-2006’;

SELECT sum(sales_amount) FROM salesWHERE sales_date BETWEEN ‘01-MAR-2006’ AND ‘31-MAY-2006’;

Partition for Performance--

-Sa

les

Ta

ble

---

06-Apr

06-Mar

06-May

06-Feb

06-Jan

06-Jun

8

5% Active5% Active95% Less Active95% Less Active

ORDERS TABLE (7 years)

High End Storage Tier

2003

Partition for Tiered Storage

2009

Low End Storage Tier

2-3x less per terabyte

2008

© 2009 Oracle Corporation

9

FlexiblePartition for Manageability/Availability

Q4’05 Q1’06 Q2’06 Q3’06

Order Table(partitioned by quarter)

Other data & queries not affected

10

Agenda


11

XX Days of Oracle Database 11g

• days since Oracle 11g Release 1 – Released on


As of 09/21/2010, inclusive

12


• days since Oracle 11g Release 1 – Released on 07/11/2007



1168

13





1168

385

• Time for you to upgrade• .. and for me to rehash Oracle Database 11g

14





1168

385

• Time for you to upgrade• .. and for me to rehash Oracle Database 11g

15

Core functionality Performance Manageability

Oracle 8.0 Range partitioning

Global Range indexes

Static partition pruning Basic maintenance: ADD, DROP, EXCHANGE

Oracle 8i Hash partitioningRange-Hash partitioning

Partition-wise joins

Dynamic partition pruning

Expanded maintenance:MERGE

Oracle 9i List partitioning Global index maintenance

Oracle 9i R2 Range-List partitioning Fast partition SPLIT

Oracle 10g Global Hash indexes Local Index maintenance

Oracle 10g R2 1M partitions per table Multi-dimensional pruning

Fast DROP TABLE

Oracle 11g Virtual column based partitioningMore composite choicesREF partitioning

Interval partitioningPartition AdvisorIncremental stats mgmt

Oracle 11g R2 Hash-Hash partitioning “AND” pruning Multi-branch executionSegment creation on demand*

15

Oracle PartitioningOver a decade of development

* available with 11.2.0.2

16

Interval Partitioning

• Partitions are created automatically as data arrives

17


• Interval Partitioning– Extension to Range Partitioning– Full automation for equi-sized range partitions

• Partitions are created as metadata information only– Start Partition is made persistent

• Segments are allocated as soon as new data arrives– No need to create new partitions– Local indexes are created and maintained as well

No need for any partition managementNo need for any partition management

18


• As easy as One, Two, Three ..

Table SALES

Jan 2006

... ...

Feb 2006 Mar 2006 Jan 2007 Oct 2009 Nov 2009

...

CREATE TABLE sales (order_date DATE, ...)PARTITON BY RANGE (order_date)INTERVAL(NUMTOYMINTERVAL(1,'month')(PARTITION p_first VALUES LESS THAN ('01-JAN-2006');

CREATE TABLE sales (order_date DATE, ...)PARTITON BY RANGE (order_date)INTERVAL(NUMTOYMINTERVAL(1,'month')(PARTITION p_first VALUES LESS THAN ('01-JAN-2006');

• First segment is created– Mandatory to have a defined lower bound

• Other segments only meta-data– Will be allocated when data is inserted

19

• Data is organized in along two dimensions– Record placement is deterministically identified by dimensions• Example RANGE-LIST

Composite Partitioning

…Jan 2009 Feb 2009 Mar 2009 Apr2009 Nov 2009 Dec 2009

…

… … … … … …

USA

EMEA

20

Composite Partitioning

• Composite partitioning available since Oracle 8i– Range-Hash– Range-List (9i)

• Extensions in Oracle Database 11g– List-Range – Range-Range– List-Hash– List-List

• Extensions in Oracle Database 11g Release 2– Hash-Hash

21

REF Partitioning

• Inherit partitioning strategy

ORDERSORDERS

LineItems

PickLists

StockHolds

BackOrders

PartitionORDERSby Date

PartitionORDERSby Date

JAN

ORDERSORDERS

LineItems

PickLists

StockHolds

BackOrders FEB

ORDERSORDERS

LineItems

PickLists

StockHolds

BackOrders

MAR

ORDERSORDERS

LineItems

PickLists

StockHolds

BackOrders APR

ORDERSORDERS

LineItems

PickLists

StockHolds

BackOrders

22

REF Partitioning

Business problem• Related tables benefit from same partitioning strategy– Sample 3NF order entry data model

• Redundant storage of same information solves problem– Data overhead– Maintenance overhead

Solution• Oracle Database 11g introduces REF Partitioning

• Child table inherits the partitioning strategy of parent table through PK-FK relationship

• Intuitive modelling

• Enhanced Performance and Manageability

23

Table ORDERS

Jan 2006

... ...

Feb 2006

Table LINEITEMS

Jan 2006

... ...

Feb 2006

Before REF Partitioning

• RANGE(order_date)• Primary key order_id

• Redundant storage of order_date• Redundant maintenance

• Redundant storage of order_date• Redundant maintenance


24

Table ORDERS

Jan 2006

... ...

Feb 2006

Table LINEITEMS

Jan 2006

... ...

Feb 2006


PARTITION BY REFERENCE

• Partitioning key inherited through PK-FK relationship

PARTITION BY REFERENCE

• Partitioning key inherited through PK-FK relationship

• RANGE(order_date)• Foreign key order_id

With REF Partitioning

25

Virtual Column-Based Partitioning

• REGION requires no storage• Partition by ORDER_DATE, REGION

ORDERS

ORDER_ID ORDER_DATE CUSTOMER_ID...

---------- ----------- ----------- --

9834-US-14 12-JAN-2007 65920

8300-EU-97 14-FEB-2007 39654

3886-EU-02 16-JAN-2007 4529

2566-US-94 19-JAN-2007 15327

3699-US-63 02-FEB-2007 18733

JAN FEB

USA

EMEA

ORDERS

REGION AS (SUBSTR(ORDER_ID,6,2))------

US

EU

EU

US

US

26

Virtual Columns

• Base table with all attributes• Virtual (derived) column based on values of other columns• No cross-table references possible

12500 Adams12507 12Blake12666 1212875 12Smith

King

12

CREATE TABLE accounts (acc_no number(10) not null, acc_name varchar2(50) not null, ... acc_branch number(2) generated always as (to_number(substr(to_char(acc_no),1,2)))

CREATE TABLE accounts (acc_no number(10) not null, acc_name varchar2(50) not null, ... acc_branch number(2) generated always as (to_number(substr(to_char(acc_no),1,2)))

27

Virtual Columns - Example

12500 Adams12507 12Blake12666 1212875 12Smith

12

• Base table with all attributes• Virtual (derived) column based on values of other columns• No cross-table references possible• Virtual column is used as partitioning key

CREATE TABLE accounts (acc_no number(10) not null, acc_name varchar2(50) not null, ... acc_branch number(2) generated always as (to_number(substr(to_char(acc_no),1,2)))partition by list (acc_branch) ...

CREATE TABLE accounts (acc_no number(10) not null, acc_name varchar2(50) not null, ... acc_branch number(2) generated always as (to_number(substr(to_char(acc_no),1,2)))partition by list (acc_branch) ...

32320 Jones32407 32Clark32758 3232980 32Phillips

32

...King Hurd

28

Agenda


29 29

Deferred Segment CreationA.k.a Segment creation on demand

• Segment creation for partitioned tables (and indexes) is delayed until first data inserted– Support for partitioned objects beginning with 11.2.0.2

• Specifically beneficial for pre-packaged applications– Common deployments consist of thousands of tables, many

of them being empty– Reduced storage foot print– Faster initial deployment

• Leverage this functionality after database migration– API to drop segments for existing empty objects

30 30

Deferred Segment CreationA.k.a Segment creation on demand

• Enabled by DEFAULT with compatible=11.2– Init.ora: deferred_segment_creation = [TRUE | FALSE ]• Session and system level attribute

– Object level: SEGMENT CREATION [IMMEDIATE | DEFERRED}

• Indexes inherit the attribute from the table– No support for bitmap join indexes and domain indexes

31

Agenda

• Partitioning in a nutshell• XX days of Oracle 11g ..• What’s new in 11.2.0.2?• Some things less known• Q&A

32

Some Things less known

• Oracle Database 11g Release 1 (11.1)– Interval partitioning versus Range partitioning– Deferred segment creation versus Interval partitioning– Child tables of REF partitioning are “different”

• Oracle Database 11g Release 2 (11.2)– Partitioning and partially unusable indexes– Enhanced index maintenance for partition maintenance

operations– Enhanced pruning capabilities

33

Interval versus Range Partitioning

• Partition bounds– Interval partitions have lower and upper bound– Range partitions only have upper bounds• Lower bound derived by previous partition

• Partition naming– Interval partitions cannot be named in advance• Use the PARTITION FOR (<value>) clause

– Range partitions must be named

34

Interval versus Range Partitioning, cont.

• Partition merge– Multiple non-existent interval partitions are silently merged – Only two adjacent range partitions can be merged at any

point in time

• Number of partitions– Interval partitioned tables have always one million partitions• Non-existent partitions “exist” through INTERVAL clause• No MAXVALUES clause for interval partitioning– Maximum value defined through number of partitions

and INTERVAL clause– Range partitioning can have up to one million partitions• MAXVALUES clause defines most upper partition

35

Table SALES

Jan 2006

... ...

Feb 2006 Mar 2006 Jan 2007 Oct 2009 Nov 2009

...

• Interval partitioned table has classical range and automated interval section– Concept of a “transition point”– [USER|ALL|DBA]_TAB_PARTITIONS.INTERVAL

Range partition section Interval partition section

Interval versus Range Partitioning, cont.

Transition point

36

Deferred Segment Creation versus Interval Partitioning

• Interval Partitioning– Maximum number of one million partitions are pre-defined• Explicitly defined plus interval-based partitions

– No segments are allocated for partitions without data• New record insertion triggers segment creation

– Ideal for “ever-growing” tables

• “Standard” Partitioning with deferred segment creation– Only explicitly defined partitions are existent• New partitions have to be added via DDL

– No segments are allocated for partitions without data• New record insertion triggers segment creation when data

matches pre-defined partitions– Ideal for sparsely populated pre-defined tables

37

REF Partitioning

• Inheritance of partition key couples parent and child tables together– Child tables do not have a partitioning key– PK – FK relationship cannot be disabled or even dropped

• Due to the tight coupling, some things are different– [Sub]Partition names are inherited down from the parent to

the child tables• No system-generated names unless parent has them

– Child partitions are by default co-located with the parent partition• Default for user is automatically overwritten

38

REF Partitioning

• Partition maintenance operations (PMOPs)– PMOP that change the table structure are implicit for child

tables and inherited from the parent table• ADD, DROP, MERGE, and SPLIT

– PMOPs without structure changes are fully supported• MOVE, EXCHANGE

– TRUNCATE works in the presence of PK-FK relationship

• Partition-wise Joins (PWJ)– Joining parent and child tables are always eligible for PWJ,

due to the known data co-location in the joining partitions

39

Unusable Indexes

• Unusable index partitions are commonly used in environments with fast load requirements– “Safe” the time for index maintenance at data insertion– Unusable index segments do not consume any space (11.2)

• Unusable indexes are ignored by the optimizer– SKIP_UNUSABLE_INDEXES = [TRUE | FALSE ]

• Partitioned indexes can be used by the optimizer even if some partitions are unusable– Prior to 11.2, static pruning and only access of usable index

partitions mandatory– With 11.2, intelligent rewrite of queries using UNION ALL

40

Multi-Branch Execution

• Sample plan

41

Intelligent Multi-Branch Execution

• Intelligent UNION ALL expansion in the presence of partially unusable indexes– Transparent internal rewrite– Usable index partitions will be used– Full partition access for unusable index partitions

42

Enhanced Index Maintenance for PMOPs

• Set a global index UNUSABLE right at partition creation time

ALTER TABLE toto ADD PARTITION p2 VALUES LESS THAN (20)

UPDATE INDEXES (i_toto (PARTITION p2 UNUSABLE));

ALTER TABLE toto ADD PARTITION p2 VALUES LESS THAN (20)

UPDATE INDEXES (i_toto (PARTITION p2 UNUSABLE));

43 43

Enhanced Pruning CapabilitiesOracle Database 11g Release 2

• Extended modeling capabilities for better data placement and pruning– Support for virtual columns as primary and foreign key for

Reference Partitioning

• Enhanced optimizer support for Partitioning– Multi-predicate pruning– Intelligent multi-branch execution plan with unusable index

partitions

44 44

“AND” Pruning

• All predicates on partition key will used for pruning– Dynamic and static predicates will now be used combined

• Example:– Star transformation with pruning predicate on both the FACT

table and a dimensionFROM sales s, times t …WHERE s.time_id = t.time_id ..AND t.fiscal_year in (2000,1999)AND s.time_id between TO_DATE('01-JAN-1999','DD-MON-YYYY') and TO_DATE('01-JAN-2000','DD-MON-YYYY')

Dynamic pruning

Static pruning

45

“AND” Pruning

• Sample plan

46

Summary

• Partitioning in a nutshell• Proven functionality in 9th generation– Experience comes with age and customer usage

• Oracle 11g .. a quite old release• It’s time to upgrade– Comprehensive partitioning for all business problems

• One consistent way to manage all your data – Manageability and performance benefits

• Questions? Enhancement ideas?– Contact [email protected]

47

Q & A

48

For More Information

search.oracle.com

or

oracle.com

Oracle Partitioning

11g Partitioning Ins and Outs

Documents

Transcript of 11g Partitioning Ins and Outs