1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005...

54
1 juliandyke.co Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke

Transcript of 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005...

Page 1: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

1

juliandyke.com

Bitmap Index Internals

Julian Dyke

Independent Consultant

Web Version

© 2005 Julian Dyke

Page 2: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

2 © 2005 Julian Dykejuliandyke.com

Agenda

Introduction Bitmap Row Sources Internal Structures Bitmap Index DML Conclusion

Page 3: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

3 © 2005 Julian Dykejuliandyke.com

Introduction

Page 4: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

4 © 2005 Julian Dykejuliandyke.com

Bitmap Indexes Introduced in Oracle 7.3

Originally intended for columns with Low cardinality (few distinct values) Little or no DML activity

Often used in DSS environments Useful for ad-hoc queries where predicates are not known

in advance

Page 5: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

5 © 2005 Julian Dykejuliandyke.com

Bitmap Index Properties Bitmap indexes

Must be non-unique Include NULL values (B*Tree indexes do not) Maximum 30 columns (B*Tree indexes max 32 columns) Can be locally partitioned (8.1.5 and above) Cannot be globally partitioned Can be function-based indexes (built-in or user-defined) Can be IOT secondary indexes (9.2 and above) Can be created on global temporary tables Cannot be compressed Cannot be reversed

Page 6: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

6 © 2005 Julian Dykejuliandyke.com

Bitmap Indexes

CREATE TABLE property (

property_code NUMBER,bedrooms NUMBER,receptions NUMBER,garages NUMBER

);

CREATE BITMAP INDEX index1 ON property (bedrooms);CREATE BITMAP INDEX index2 ON property (receptions);CREATE BITMAP INDEX index3 ON property (garages);

SELECT property_code FROM propertyWHERE bedrooms = 4AND receptions = 3AND garages = 2

For example

Page 7: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

7 © 2005 Julian Dykejuliandyke.com

Logical Operations Bitmaps can be combined using the logical operations AND,

OR , NOT.

A B A AND B A OR B NOT A

0 0 0 0 1

0 1 0 1 1

1 0 0 1 0

1 1 1 1 0

Oracle also implements a MINUS operation internally A MINUS B is equivalent to A AND NOT B

Page 8: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

8 © 2005 Julian Dykejuliandyke.com

Multicolumn Bitmap Indexes Bitmap indexes can be defined for more than one column

CREATE BITMAP INDEX i1 ON t1 (c1,c2);

Can be used for queries such as

SELECT * FROM t1 WHERE c1 = 0;SELECT * FROM t1 WHERE c1 = 0 AND c2 = 0;

However for queries such as

SELECT * FROM t1 WHERE c2 = 0;

there is apparently no equivalent to INDEX (SKIP SCAN) Consider creating two indexes and allowing Oracle to perform join dynamically

CREATE BITMAP INDEX i1 ON t1 (c1);CREATE BITMAP INDEX i2 ON t1 (c2);

Page 9: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

9 © 2005 Julian Dykejuliandyke.com

Bitmap Row Sources

Page 10: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

10

© 2005 Julian Dykejuliandyke.com

Bitmap Row SourcesOperation Version

BITMAP INDEX (SINGLE VALUE) 7.3.2

BITMAP INDEX (RANGE SCAN) 7.3.2

BITMAP INDEX (FULL SCAN) 7.3.2

BITMAP INDEX (FAST FULL SCAN) 9.0.1

BITMAP AND 7.3.2

BITMAP OR 7.3.2

BITMAP MINUS 7.3.2

BITMAP MERGE 7.3.2

BITMAP KEY ITERATION 7.3.2

BITMAP CONVERSION TO ROWIDS 7.3.2

BITMAP CONVERSION COUNT 7.3.2

BITMAP CONVERSION FROM ROWIDS 7.3.2

BITMAP CONSTRUCTION 7.3.2

BITMAP COMPACTION 7.3.2

Page 11: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

11 © 2005 Julian Dykejuliandyke.com

Bitmap Index Operation Introduced in 7.3 Options are

BITMAP INDEX (SINGLE VALUE) Builds a single bitmap for a given value

BITMAP INDEX (RANGE SCAN) Builds a bitmap containing each value in the range

BITMAP INDEX (FULL SCAN) Builds a bitmap containing each value in the index

BITMAP INDEX (FAST FULL SCAN) Oracle 9.0.1 and above Equivalent to INDEX (FAST FULL SCAN)

Page 12: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

12

© 2005 Julian Dykejuliandyke.com

Bitmap Conversion Operation Introduced in Oracle 7.3 Options are

BITMAP CONVERSION (TO ROWIDS) Converts a bitmap into a set of ROWIDs

BITMAP CONVERSION (COUNT) Returns the number of set bits in a bitmap Used to implement COUNT() aggregates

BITMAP CONVERSION (FROM ROWIDS) Converts a set of ROWIDs into a bitmap

Page 13: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

13

© 2005 Julian Dykejuliandyke.com

Bitmap And Operation Introduced in Oracle 7.3 Performs logical AND operation between two bitmaps

CREATE TABLE t1 (c1 NUMBER, c2 NUMBER);CREATE BITMAP INDEX i1 ON t1 (c1);CREATE BITMAP INDEX i2 ON t1 (c2);

0 SELECT STATEMENT Optimizer=CHOOSE1 0 BITMAP CONVERSION (TO ROWIDS)2 1 BITMAP AND3 2 BITMAP INDEX (SINGLE VALUE) OF 'I1‘4 2 BITMAP INDEX (SINGLE VALUE) OF 'I2‘

SELECT /*+ INDEX_COMBINE (t1 i1 i2) */ c1,c2FROM t1WHERE c1 = 0 AND c2 = 0;

Page 14: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

14

© 2005 Julian Dykejuliandyke.com

Bitmap Or Operation Introduced in Oracle 7.3 Performs logical OR operation between two bitmaps

CREATE TABLE t1 (c1 NUMBER, c2 NUMBER);CREATE BITMAP INDEX i1 ON t1 (c1);CREATE BITMAP INDEX i2 ON t1 (c2);

0 SELECT STATEMENT Optimizer=CHOOSE1 0 TABLE ACCESS (BY INDEX ROWID) OF 'T1‘2 1 BITMAP CONVERSION (TO ROWIDS)3 2 BITMAP OR4 3 BITMAP INDEX (SINGLE VALUE) OF 'I1‘5 3 BITMAP INDEX (SINGLE VALUE) OF 'I2'

SELECT /*+ INDEX_COMBINE (t1 i1 i2) */ c1,c2FROM t1WHERE c1 = 0 OR c2 = 0;

Page 15: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

15

© 2005 Julian Dykejuliandyke.com

Bitmap Minus Operation Introduced in Oracle 7.3 Performs logical MINUS operation between two bitmaps

CREATE TABLE t1 (c1 NUMBER, c2 NUMBER);CREATE BITMAP INDEX i1 ON t1 (c1);CREATE BITMAP INDEX i2 ON t1 (c2);

0 SELECT STATEMENT Optimizer=CHOOSE1 0 TABLE ACCESS (BY INDEX ROWID) OF 'T1‘2 1 BITMAP CONVERSION (TO ROWIDS)3 2 BITMAP MINUS4 3 BITMAP MINUS5 4 BITMAP INDEX (SINGLE VALUE) OF 'I1‘6 4 BITMAP INDEX (SINGLE VALUE) OF 'I2‘7 3 BITMAP INDEX (SINGLE VALUE) OF 'I2'

SELECT /*+ INDEX_COMBINE (t1 i1 i2) */ c1,c2FROM t1WHERE c1 = 0 AND NOT c2 = 0;

Page 16: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

16

© 2005 Julian Dykejuliandyke.com

Bitmap Merge Operation Introduced in Oracle 7.3 Merges two or more bitmaps together

CREATE TABLE t1 (c1 NUMBER, c2 NUMBER);CREATE BITMAP INDEX i1 ON t1 (c1);CREATE BITMAP INDEX i2 ON t1 (c2);

0 SELECT STATEMENT Optimizer=CHOOSE1 0 TABLE ACCESS (BY INDEX ROWID) OF 'T1' 2 1 BITMAP CONVERSION (TO ROWIDS)3 2 BITMAP AND4 3 BITMAP INDEX (SINGLE VALUE) OF 'I2‘5 3 BITMAP MERGE6 5 BITMAP INDEX (RANGE SCAN) OF 'I1'

SELECT /*+ INDEX_COMBINE (t1 i1 i2) */ c1,c2FROM t1WHERE c1 > 0 AND c2 = 0;

Page 17: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

17

© 2005 Julian Dykejuliandyke.com

Bitmap Construction / Compaction Introduced in Oracle 7.3 Used by index creation/rebuild operations Appear in EXPLAIN PLAN Do not appear in AUTOTRACE or V$SQL_PLAN

CREATE TABLE t1 (c1 NUMBER);

0 CREATE INDEX STATEMENT Optimizer=CHOOSE1 0 INDEX BUILD (NON UNIQUE) OF 'I1‘2 1 BITMAP COMPACTION3 2 SORT (CREATE INDEX)4 3 BITMAP CONSTRUCTION5 4 TABLE ACCESS (FULL) OF 'T1'

EXPLAIN PLAN FOR CREATE BITMAP INDEX i1 ON t1 (c01)

Page 18: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

18

© 2005 Julian Dykejuliandyke.com

Bitmap Key Iteration Introduced in Oracle 7.3 Used to implement star transformations Star transformations join three or more tables together

-- Create fact table CREATE TABLE fact (factkey NUMBER,dim1key NUMBER,dim2key NUMBER);

-- Create bitmap indexes on fact columnsCREATE BITMAP INDEX index1 ON fact (dim1key);CREATE BITMAP INDEX index2 ON fact (dim2key);

-- Set the number of rows BEGIN

DBMS_STATS.SET_TABLE_STATS(USER,’FACT’,numrows=>100000);END;

-- Create dimension tables CREATE TABLE dim1 (dim1key NUMBER,dim1desc NUMBER);CREATE TABLE dim2 (dim2key NUMBER,dim2desc NUMBER);

Page 19: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

19

© 2005 Julian Dykejuliandyke.com

Bitmap Key Iteration Star transformations require the following parameter

SELECT /*+ STAR_TRANSFORMATION */ fact.factkey,dim1.dim1key,dim2.dim2key

FROM fact, dim1, dim2WHERE dim1.dim1key = fact.dim1keyAND dim2.dim2key = fact.dim2keyAND dim1.dim1desc = 100AND dim2.dim2desc = 200;

ALTER SESSION SET star_transformation_enabled = TRUE;

The statement

Page 20: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

20

© 2005 Julian Dykejuliandyke.com

Bitmap Key Iteration In Oracle 9.0.1 and above, the following plan is generated

0 SELECT STATEMENT Optimizer=CHOOSE1 0 HASH JOIN2 1 MERGE JOIN (CARTESIAN)3 2 TABLE ACCESS (FULL) OF ‘DIM1' 4 2 BUFFER (SORT)5 4 TABLE ACCESS (FULL) OF ‘DIM2' 6 1 TABLE ACCESS (BY INDEX ROWID) OF ‘FACT‘7 6 BITMAP CONVERSION (TO ROWIDS)8 7 BITMAP AND9 8 BITMAP MERGE10 9 BITMAP KEY ITERATION11 10 TABLE ACCESS (FULL) OF ‘DIM1‘12 10 BITMAP INDEX (RANGE SCAN) OF 'INDEX1‘13 8 BITMAP MERGE14 13 BITMAP KEY ITERATION15 14 TABLE ACCESS (FULL) OF ‘DIM2‘16 14 BITMAP INDEX (RANGE SCAN) OF 'INDEX2'

Page 21: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

21

© 2005 Julian Dykejuliandyke.com

B*Tree Bitmap Plans In Oracle 7.3 and above it is possible to convert B*tree indexes

into bitmaps Enabled using _B_TREE_BITMAP_PLANS parameter Default value FALSE (8.1.7 and below), TRUE (9.0.1 and above)

CREATE TABLE t1 (c1 NUMBER, c2 NUMBER);

BEGINFOR f IN 0..9999 LOOP

INSERT INTO t1 VALUES (MOD (f,100), MOD (f,200));END LOOP;

END;

COMMIT;

CREATE INDEX i1 ON t1 (c1); -- Not bitmap indexCREATE INDEX i2 ON t1 (c2); -- Not bitmap index

ANALYZE TABLE t1 COMPUTE STATISTICS;

For example (8192 byte block size)

Page 22: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

22

© 2005 Julian Dykejuliandyke.com

B*Tree Bitmap Plans In Oracle 9.2 the statement

SELECT c1, c2 FROM t1 WHERE c1 = 0 AND c2 = 0;

generates the plan

0 SELECT STATEMENT Optimizer=CHOOSE (Cost=3 Card=1 Bytes=5)1 0 BITMAP CONVERSION (TO ROWIDS)2 1 BITMAP AND3 2 BITMAP CONVERSION (FROM ROWIDS)4 3 INDEX (RANGE SCAN) OF 'I2' (NON-UNIQUE) (Cost=1)5 2 BITMAP CONVERSION (FROM ROWIDS)6 5 INDEX (RANGE SCAN) OF 'I1' (NON-UNIQUE) (Cost=1)

Page 23: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

23

© 2005 Julian Dykejuliandyke.com

Internal Structure

Page 24: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

24

© 2005 Julian Dykejuliandyke.com

Leaf Block Format Each value in the index is represented by one or more bitmaps Each bitmap represents a range of ROWIDs Each leaf row entry typically contains four columns

Column Description

1 Column Value

2 Start ROWID

3 End ROWID

4 Bitmap

A multi-column index will have additional key columns Branch block format and characteristics are identical to B*Tree

indexes

Page 25: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

25

© 2005 Julian Dykejuliandyke.com

Key

Start ROWID

End ROWID

Bitmap

Bitmaps and Leaf Blocks For example, consider a bitmap index on column BEDROOMS

Block 212Block 211

PROPERTY_CODE

BEDROOMS

RECEPTIONS

GARAGES

2 4 3 3

1 2 3 4

1 3 2 2

0 2 1 1

2

211/0

212/3

10000010

3

211/0

212/3

00110101

4

211/0

212/3

01001000

4 3 2 3

5 6 7 8

3 2 1 2

2 1 1 2

1 2 3 4

1 3 2 2

0 2 1 1

5 6 7 8

3 2 1 2

2 1 1 2

Page 26: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

26

© 2005 Julian Dykejuliandyke.com

Bitmap Block Dump Partial block dump for the previous slide

row#0[8013] flag: -----, lock: 0col 0; len 2; (2): c1 03col 1; len 6; (6): 01 00 01 82 00 00col 2; len 6; (6): 01 00 01 83 00 07col 3; len 3; (3): 00 c2 44

row#1[7988] flag: -----, lock: 0col 0; len 2; (2): c1 04col 1; len 6; (6): 01 00 01 82 00 00col 2; len 6; (6): 01 00 01 83 00 07col 3; len 5; (5): c8 0c f8 56 0a

row#2[7965] flag: -----, lock: 0col 0; len 2; (2): c1 05col 1; len 6; (6): 01 00 01 82 00 00col 2; len 6; (6): 01 00 01 83 00 07col 3; len 3; (3): 01 c0 44

Key

Start ROWID

End ROWID

Bitmap

Page 27: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

27

© 2005 Julian Dykejuliandyke.com

B*Tree Block Dump

row#0[8024] flag: -----, lock: 0col 0; len 2; (2): c1 03col 1; len 6; (6): 01 00 01 82 00 00

row#1[8012] flag: -----, lock: 0col 0; len 2; (2): c1 03col 1; len 6; (6): 01 00 01 83 00 02

row#2[8000] flag: -----, lock: 0col 0; len 2; (2): c1 04col 1; len 6; (6): 01 00 01 82 00 02

row#3[7988] flag: -----, lock: 0col 0; len 2; (2): c1 04col 1; len 6; (6): 01 00 01 82 00 03

Key

ROWID

row#4[7976] flag: -----, lock: 0col 0; len 2; (2): c1 04col 1; len 6; (6): 01 00 01 83 00 01

row#5[7964] flag: -----, lock: 0col 0; len 2; (2): c1 04col 1; len 6; (6): 01 00 01 83 00 03

row#6[7952] flag: -----, lock: 0col 0; len 2; (2): c1 05col 1; len 6; (6): 01 00 01 82 00 01

row#7[7940] flag: -----, lock: 0col 0; len 2; (2): c1 05col 1; len 6; (6): 01 00 01 83 00 00

Page 28: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

28

© 2005 Julian Dykejuliandyke.com

Bitmap Pieces Every bitmap piece has a start ROWID and an end ROWID

Start ROWID is rounded to the nearest byte boundary below End ROWID is rounded to the nearest byte boundary above

Each indexed column value may have one or more bitmap pieces

A bitmap piece covers a contiguous set of rows in one or more extents may cross extent boundaries may split within a block

Page 29: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

29

© 2005 Julian Dykejuliandyke.com

Bitmap Pieces For example the following are all valid bitmap pieces

Start 211/0 End 533/15

Start 532/0 End 533/15

End 533/7Start 211/8

Block 211 Block 532 Block 533

Start 532/8 End 532/15

Start 211/0 End 533/15Start 533/0End 532/15

Start 211/0 End 533/15Start 532/8End 532/7

Extent 1 Extent 2

Start 211/0 End 211/15

Page 30: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

30

© 2005 Julian Dykejuliandyke.com

Compression Algorithm Bitmaps consists of zeros and ones Zeros are compressed; ones are not compressed Bitmaps can be subdivided into groups of bytes First byte in each group determines length of group

First Byte Description

< 192 Single-byte group

>= 192 Multi-byte group

Page 31: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

31

© 2005 Julian Dykejuliandyke.com

Single-Byte Groups Byte represents the number of zero bits followed by a one bit Maximum of 191 zero bits

Byte Bitmap

00 10000000

01 01000000

02 00100000

03 00010000

08 00000000 10000000

0F 00000000 00000001

10 00000000 00000000 10000000

11 00000000 00000000 01000000

17 00000000 00000000 00000001

18 00000000 00000000 00000000 10000000

19 00000000 00000000 00000000 01000000

Range of byte values is 0x00 to 0xBF

Page 32: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

32

© 2005 Julian Dykejuliandyke.com

Multi-Byte Groups Multi-byte groups allow more than 192 bits to be skipped First byte is a control byte

Next three bits indicate number of zero bytes to skip If all three bits are set then number overflows to second byte If top bit of second byte is set then number of zero bytes

overflows to third byte

First two bits indicate this is a control byte (always 11)

Last three bits indicate number of bytes following control block (minimum 1, maximum 8)

1 1

Page 33: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

33

© 2005 Julian Dykejuliandyke.com

Multi-Byte Groups Examples

Code Number ofZero Bytes

Number ofZero BitsHex Binary

C8 11001000 0 0

D0 11010000 1 8

D8 11011000 2 16

E0 11100000 3 24

E8 11101000 4 32

F0 11110000 5 40

F8 00 11111000 00000000 6 48

F8 01 11111000 00000001 7 56

F8 02 11111000 00000010 8 64

F8 7F 11111000 01111111 133 1064

F8 80 01 11111000 10000000 00000001 134 1072

F8 81 01 11111000 10000001 00000001 135 1080

Page 34: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

34

© 2005 Julian Dykejuliandyke.com

Multi-Byte Groups Last three bits indicate number of bytes following control

block (minimum 1, maximum 8)

Code Number of

bytes

Example

Hex Binary

C8 11001000 1 C8 FF

C9 11001001 2 C9 FF FF

CA 11001010 3 CA FF FF FF

CB 11001011 4 CB FF FF FF FF

CC 11001100 5 CC FF FF FF FF FF

CD 11001101 6 CD FF FF FF FF FF FF

CE 11001110 7 CE FF FF FF FF FF FF FF

CF 11001111 8 CF FF FF FF FF FF FF FF FF

Page 35: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

35

© 2005 Julian Dykejuliandyke.com

Håkan Factor For each table describes maximum number of rows that each

block can theoretically hold Derived from column definition Affected by

Number of columns Column datatypes and lengths NOT NULL constraints

Stored in lower 11 (at least) bits of SYS.TAB$.SPARE1 Minimum value is 11 bytes per row to allow row to be migrated

Block Size Maximum Rows per Block

2048 178

4096 364

8192 736

16384 1481

Page 36: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

36

© 2005 Julian Dykejuliandyke.com

Håkan Factor Håkan Factor can be adjusted using

ALTER TABLE table_name MINIMIZE RECORDS_PER_BLOCK;

This command Scans entire table Counts number of rows in each block Sets Håkan Factor in SPARE1 to maximum row count Sets bit 0x8000 in SPARE1 to indicate value has been set

Subsequently modified blocks will be limited to the new Håkan Factor

Command is reversed using

ALTER TABLE table_name NOMINIMIZE RECORDS_PER_BLOCK;

Page 37: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

37

© 2005 Julian Dykejuliandyke.com

Håkan Factor Hakan Factor is used when compressing bitmaps Each bitmap represents an array

Blocks

Maximum rows per block

Minimising records per block reduces size of this array

Blocks

Maximum rows per block Reduces memory required to access/manipulate bitmap May reduce disk space required to hold bitmap

Page 38: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

38

© 2005 Julian Dykejuliandyke.com

Nulls B*Tree indexes do not include NULL values Bitmap indexes do include a leaf row for NULL values For example, if I1 is a bitmap index, the statement

SELECT COUNT(*) FROM t1WHERE c1 IS NULL;

generates the plan

0 SELECT STATEMENT Optimizer=CHOOSE (Cost=3 Card=1 Bytes=5)1 0 SORT (AGGREGATE)2 1 BITMAP CONVERSION (COUNT)3 2 BITMAP INDEX (SINGLE VALUE) OF ‘I1'

If I1 is a B*Tree index the same statement generates the plan

0 SELECT STATEMENT Optimizer=CHOOSE (Cost=3 Card=1 Bytes=5)1 0 SORT (AGGREGATE)2 1 TABLE ACCESS (FULL) OF ‘T1’

Page 39: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

39

© 2005 Julian Dykejuliandyke.com

Distinct Keys This example shows how the number of distinct keys affects

number of leaf blocks for 100000 row table 8192 byte block size

Distinct Keys B*Tree Bitmap

1 237 3

2 237 5

5 237 13

10 237 25

100 237 50

1000 237 48

10000 237 87

50000 237 210

100000 237 363

0

50

100

150

200

250

300

350

400

Distinct Keys

Le

af

Blo

ck

s

B*Tree Bitmap

Page 40: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

40

© 2005 Julian Dykejuliandyke.com

Clustering Size of bitmap indexes is affected by the clustering of the data For example 100000 row table Column1 and Column2 contain 100 distinct values

Column1 (Distributed)

Column2 (Clustered)

Column1 (Distributed)

Column2(Clustered)

Height 2 2

Blocks 24 8

Leaf Rows 100 100

Leaf Blocks 20 3

Branch Rows 19 2

Branch Blocks 1 1

Page 41: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

41

© 2005 Julian Dykejuliandyke.com

Bitmap Index DML

Page 42: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

42

© 2005 Julian Dykejuliandyke.com

Inserts In Oracle 9.2 and below, insertion is very inefficient When a row is inserted

if leaf row already exists for ROWID then bitmap is updated otherwise a new leaf row is inserted for the eight bits

around the new row

In Oracle 10g, insertion is more efficient rows are inserted into existing leaf rows where possible if necessary start ROWID or end ROWID is adjusted

Page 43: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

43

© 2005 Julian Dykejuliandyke.com

Inserts Insertion behaviour in Oracle 9.2 and below

Bitmap Index Leaf Rows

Block 573 Block 574

Table Rows

2

573/0

574/7

Key

Start

End

Block 575

2

574/8

574/15

Key

Start

End

2

575/0

575/7

Key

Start

End

2

575/8

575/15

Key

Start

End

Key = 2

Key <> 2

Page 44: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

44

© 2005 Julian Dykejuliandyke.com

Inserts Insertion behaviour in Oracle 10g

Bitmap Index Leaf Rows

Block 573 Block 574

Table Rows

2

573/0

574/7

Key

Start

End

Block 575

574/15575/7575/15

Key = 2

Key <> 2

Page 45: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

45

© 2005 Julian Dykejuliandyke.com

Updates As with B*Tree indexes updates consist of a delete followed

by an insert delete old value from bitmap piece add new value from bitmap piece

At least two bitmap pieces are affected by every update

Also applies to bitmap indexes containing NULL values

When bitmaps are updated on a block Must have sufficient space on block for old and new rows Otherwise block is split Can lead to long branch block rows

Page 46: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

46

© 2005 Julian Dykejuliandyke.com

Locking When a bitmap index is updated, a row lock is taken for the

index leaf entry

The index leaf entry contains a bitmap which may cover many rows across a number of blocks

No other transaction can update an indexed column in any row covered by that bitmap piece until the original transaction commits or rolls back

Page 47: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

47

© 2005 Julian Dykejuliandyke.com

LockingBitmap Index Leaf Rows

2

10499 100 107

43 3 2

515 20 10

9792 94 98

34 2 3

1525 10 20

8986 88 90

43 3 2

1510 20 5

8077 78 83

43 3 2

1025 5 5

Block 211 Block 212 Block 213 Block 214

c1

c2

c3

333 4 4 44

Table Rows

2

211/0

214/3

Key

Start

End

Lock 0

3

211/0

212/3

Key

Start

End

Lock 0

3

211/0

212/3

Key

Start

End

Lock 1

3

213/0

214/3

Key

Start

End

Lock 0

4

211/0

214/3

Key

Start

End

Lock 0

4

211/0

214/3

Key

Start

End

Lock 1

4

20

88

CREATE BITMAP INDEX i1 ON t1 (c2);

UpdatedUPDATE t1SET c2 = 4WHERE c1 = 88;

Locked

Page 48: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

48

© 2005 Julian Dykejuliandyke.com

Leaf Block Splits When bitmaps indexes are updated

Old row is retained until transaction is committed New row is written to free space in block If no free space in existing block, new block is used Leaf rows stored in ascending order Key, start and end ROWIDs are unchanged therefore leaf

row ordered by bitmap

If last bitmap in block is updated then block is split Branch blocks contain unique leading edge for columns If change is at end of bitmap, entire bitmap may be copied

up into branch block

Page 49: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

49

© 2005 Julian Dykejuliandyke.com

Leaf Block Splits Example of leaf block split Oracle 9.2 – 8192 byte block size 8 distinct values – approximately 3000 rows each

1 2START

END

100000010000001000000100000010

010000001000000100000010000001

STARTEND

3 4START

END

001000000100000010000001100000

000010000001100000010000001000

STARTEND

5 6START

END

100000010000001000000100000010

010000001000000100000010000001

STARTEND

7 8START

END

001000000100000010000001100000

000100000010000001100000010000

STARTEND

3 51 7BranchBlock

LeafBlocks

Initial state. Branch rows initial contain index key values only

Page 50: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

50

© 2005 Julian Dykejuliandyke.com

Leaf Block Splits

1 2START

END

100000010000001000000100000010

010000001000000100000010000001

STARTEND

2START

END

100000010000001000000100000010

3 4START

END

001000000100000010000001100000

000010000001100000010000001000

STARTEND

4START

END

100000010000001000000100000010

5 6START

END

100000010000001000000100000010

010000001000000100000010000001

STARTEND

7 8START

END

001000000100000010000001100000

000100000010000001100000010000

STARTEND

UPDATE t1 SET c2 = 2 /* was 4 */WHERE c1 = 23988;

2 3STARTEND

100000010000001000000100000010

41STARTEND

010000001000000100000010000001

75

Updated

Deleted

BranchBlock

LeafBlocks

Deleted row is retained until end of transaction. Leaf block splits between updated and deleted rows. Branch row must contain bitmap

Page 51: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

51

© 2005 Julian Dykejuliandyke.com

Leaf Block Splits

1 2START

END

100000010000001000000100000010

010000001000000100000010000001

STARTEND

2START

END

100000010000001000000100000010

3 4START

END

001000000100000010000001100000

000010000001100000010000001000

STARTEND

4START

END

100000010000001000000100000010

5 6START

END

100000010000001000000100000010

010000001000000100000010000001

STARTEND

6START

END

100000010000001000000100000010

7 8START

END

001000000100000010000001100000

000100000010000001100000010000

STARTEND

8START

END

000010000001100000010000001000

51

2 3START

END

100000010000001000000100000010

41STARTEND

010000001000000100000010000001

6 7START

END

100000010000001000000100000010

85STARTEND

010000001000000100000010000001

UPDATE t1 SET c2 = 6 /* was 8 */WHERE c1 = 23992;

Updated

Deleted

Insufficient space for new branch rows in branch blockBranch block splits. New root block created. Index height increases

Page 52: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

52

© 2005 Julian Dykejuliandyke.com

Conclusions Columns with relatively high cardinality may be suitable for

bitmap indexes

Clustering of data significantly affects bitmap index size

DML is very inefficient in Oracle 9i and below; in Oracle 10g it is much more efficient

Be aware of implications of locking when indexes are updated by multiple sessions

In Oracle 10g bitmap indexes are viable for volatile tables if all updates are made by one session

Page 53: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

53

© 2005 Julian Dykejuliandyke.com

References Jonathan Lewis

Understanding Bitmap Indexes (http://www.dbazine.com) Optimising Oracle – Performance by Design 2001-2003

Steve Adams Miracle Master Class 2003

Julian Dyke (http://www.juliandyke.com)

Page 54: 1 juliandyke.com Bitmap Index Internals Julian Dyke Independent Consultant Web Version © 2005 Julian Dyke.

54

© 2005 Julian Dykejuliandyke.com

Thank you for your interest

For more information and to provide feedback

please contact me

My e-mail address is:[email protected]

My website address is:

www.juliandyke.com