Post on 18-May-2015
description
Patrick Barel , AMIS, The Netherlands
Wednesday, June 27, 2012
ODTUG KScope 12
San Antonio, Texas, USA
Developers Toolbox - Coding Can collections speed up your PL/SQL?
Agenda
Records
Collections
Bulk Processing
Table Functions
Record
Records
Table
View
Cursor
User Defined
} %rowtype
Records
Record type definition
Field definition
TYPE record_type IS RECORD field_definition (
,
) ;
field datatype
NOT NULL :=
DEFAULT
expression
Collections
Three types available
PL/SQL Tables
Index by Tables
Associative Arrays
Collections
Three types available
Associative Arrays
Nested Tables
Varrays
Collections
Three types available
Associative Arrays
Nested Tables
Varrays
PL/SQL Only
SQL and PL/SQL
SQL and PL/SQL
PL/SQL Collections (3 Types)
Associative Arrays
Varrays
Nested Tables
Number of elements is unbounded, practically speaking.
Valid row numbers range from -231+1 to 231-1. (i.e. -2,147,483,647 to 2,147,483,647)
This range allows you to employ the row number as an intelligent key, such as the primary key or unique index value, because…
AAs also:
Can be sparse.
Data does not have to be stored in consecutive rows, as is required in traditional 3GL arrays and VARRAYs.
Can have index values of integers or strings (Oracle9i R2 and above).
assoc_array_example.sql
collection_of_records.sql
count_vs_rowcount.sql
About Associative Arrays
Name Changes:
7: PL/SQL Tables
8i: Index By Tables
9i: Associative Arrays
Single dimensioned, unbounded, sparse collection of
homogeneous elements
PL/SQL Only value key
Associative Arrays
First
Last
Next (n)
Prior (n)
Count
Exists (n)
Delete [(n[,m])]
idx := emp_t.first;
idx := emp_t.last;
idx := emp_t.next (idx);
idx := emp_t.prior (idx);
idx := emp_t.count;
if emp_t.exists (idx)
then
...
end if;
emp_t.delete (1);
emp_t.delete (2, 4);
emp_t.delete;
Methods
declare
type num_tbl is table of number
index by binary_integer;
l_num_tbl num_tbl;
l_idx integer;
begin
...
value key
Associative Array
declare
type num_tbl is table of number
index by binary_integer;
l_num_tbl num_tbl;
l_idx integer;
begin
l_num_tbl(1):= 12;
l_num_tbl(54):= 5;
...
value
12
5
key
1
54
Associative Array
declare
type num_tbl is table of number
index by binary_integer;
l_num_tbl num_tbl;
l_idx integer;
begin
l_num_tbl(1):= 12;
l_num_tbl(54):= 5;
l_num_tbl(3):= 98;
l_num_tbl(5):= l_num_tbl.count;
l_idx:= l_num_tbl.first;
...
end;
value
12
98
3
key
1
3
5
5 54
Associative Array
value
12
5
key
1
54
2
10
declare
type num_tbl is table of number
index by binary_integer;
l_num_tbl num_tbl;
l_idx integer;
begin
l_num_tbl(1):= 12;
l_num_tbl(54):= 5;
l_num_tbl(3):= 98;
l_num_tbl(5):= l_num_tbl.count;
l_idx:= l_num_tbl.first;
loop
dbms_output.put_line(l_num_tbl(l_idx));
l_idx:= l_num_tbl.next(l_idx);
exit when l_idx is null;
end loop;
l_num_tbl.delete(2,10);
dbms_output.put_line(l_num_tbl.count);
end;
value
12
98
3
key
1
3
5
5 54
Associative Array
declare
type str_tbl is table of varchar2(40)
index by varchar2(40);
l_str_tbl str_tbl;
l_idx varchar2(40);
begin
...
value key
Associative Array VARCHAR2 Key
declare
type str_tbl is table of varchar2(40)
index by varchar2(40);
l_str_tbl str_tbl;
l_idx varchar2(40);
begin
l_str_tbl('one'):= 'een';
l_str_tbl('two'):= 'twee';
...
value
een
twee
key
one
two
Associative Array VARCHAR2 Key
declare
type str_tbl is table of varchar2(40)
index by varchar2(40);
l_str_tbl str_tbl;
l_idx varchar2(40);
begin
l_str_tbl('one'):= 'een';
l_str_tbl('two'):= 'twee';
l_str_tbl('three'):= 'drie';
l_str_tbl('four'):= 'vier';
l_idx:= l_str_tbl.first;
...
end;
value
vier
een
drie
key
four
one
three
twee two
Associative Array VARCHAR2 Key
declare
type str_tbl is table of varchar2(40)
index by varchar2(40);
l_str_tbl str_tbl;
l_idx varchar2(40);
begin
l_str_tbl('one'):= ‘een';
l_str_tbl('two'):= ‘twee';
l_str_tbl('three'):= 'drie';
l_str_tbl('four'):= 'vier';
l_idx:= l_str_tbl.first;
loop
dbms_output.put_line(l_str_tbl(l_idx));
l_idx:= l_str_tbl.next(l_idx);
exit when l_idx is null;
end loop;
l_str_tbl.delete('o', 'tr');
dbms_output.put_line(l_str_tbl.count);
end;
o
tr
value
vier
een
drie
key
four
one
three
twee two
value
vier
twee
key
four
two
Associative Array VARCHAR2 Key
for idx in emp_t.first .. emp_t.last
loop
...
end loop;
for idx in 1 .. emp_t.count
loop
...
end loop;
idx := emp_t.first;
while idx is not null
loop
...
idx := emp_t.next (idx);
end loop;
• Dense
• Count > 0
• Dense
• …
Retrieval
PL/SQL Collections (3 Types)
Associative Arrays
Varrays
Nested Tables
varray_example.sql
About Varrays
Has a maximum size, associated with its type.
Can adjust the size in Oracle10g R2.
Part of object model, requiring initialization.
Is always dense; you can only remove elements
from the end of a varray.
Can be defined as a schema level type and used
as a relational table column type.
Single dimensioned, always bounded, never
sparse collection of homogeneous elements
SQL and PL/SQL
Can be used as column datatype in a table
Stored “in-line” in same table
Retains its ordering
Needs to be initialized and extended
value key
Varray
First
Last
Next (n)
Prior (n)
Count
Exists (n)
Delete
Limit
Extend [(n[,m])]
Trim [(n)]
idx := ename_t.limit;
ename_t.extend (1);
ename_t.trim (1);
ename_t.trim;
Methods
declare
type ename_vt is varray (10) of varchar2(10);
ename_t ename_vt;
begin
ename_t := ename_vt();
ename_t.extend (1);
ename_t(1) := 'Spencer';
...
end;
Initialization
declare
type ename_vt is varray (10) of varchar2(10);
ename_t ename_vt := ename_vt ('Davis');
begin
...
end; Initialization and
Extending
Using Varray
Pre 10gR2: VARRAY needed to be recreated.
10gr2 and up: ALTER TYPE MODIFY LIMIT
Only to increase the limit
varraylimit.sql
How Variable is the Varray?
PL/SQL Collections (3 Types)
Associative Arrays
Varrays
Nested Tables
Name reflects fact that this collection can be
"nested" inside relational table as a column.
Type can be defined at schema level.
No practical, pre-defined limit on a nested table.
Valid row numbers range from 1 to 231-1. (i.e. 1 to 2,147,483,647)
Part of object model, requiring initialization.
Is always dense initially, but can become sparse
after deletes.
nested_table_example.sql
About Nested Tables
Single dimensioned, unbounded, sparse collection
of homogeneous elements
SQL and PL/SQL
Can be used as column datatype in a table
Stored “out-of-line” in a separate table
Initially dense, can be sparse
value key
Nested Tables
First
Last
Next (n)
Prior (n)
Count
Exists (n)
Delete [(n[,m])]
Extend [(n[,m])]
Trim
(Limit)
Methods
declare
type ename_nt is table of varchar2(10);
ename_t ename_nt;
begin
ename_t := ename_nt();
ename_t.extend (1);
ename_t(1) := 'Spencer';
...
end;
Initialization
declare
type ename_nt is table of varchar2(10);
ename_t ename_nt := ename_nt ('Davis');
begin
...
end; Initialization and
Extending
Using Nested Tables
Differences
Feature Associative Array Nested Table VArray
SQL – PL/SQL PL/SQL only SQL and PL/SQL SQL and PL/SQL
Dense - Sparse Sparse Initially Dense
Can become sparse
Dense
Size ‘Unlimited’ ‘Unlimited’ Limited
Order Unordered Unordered Ordered
Usage Any set of data Any set of data Small sets of data
Use in Table No Yes Yes
FORALL Use with inserts, updates and deletes. Move data from collections to tables.
BULK COLLECT Use with implicit and explicit queries. Move data from tables into collections.
In both cases, the "back back" end processing in the SQL engine is unchanged. Same transaction and rollback segment management Same number of individual SQL statements will be
executed. But BEFORE and AFTER statement-level triggers only
fire once per FORALL INSERT statements.
statement_trigger_and_forall.sql
Bulk Processing in PL/SQL
Fetch one or more rows into a collection.
Collection is always filled sequentially from index value 1.
Query does not raise NO_DATA_FOUND if no rows are
fetched.
Instead, the collection is empty.
Use FETCH with LIMIT to manage memory.
SELECT * BULK COLLECT INTO collection FROM table;
FETCH cur BULK COLLECT INTO collection;
BULK COLLECT for multi-row querying
Fetch one or more rows into a collection.
Collection is always filled sequentially from index value 1.
Query does not raise NO_DATA_FOUND if no rows are
fetched.
Instead, the collection is empty.
Use FETCH with LIMIT to manage memory.
SELECT * BULK COLLECT INTO collection FROM table;
FETCH cur BULK COLLECT INTO collection;
BULK COLLECT for multi-row querying
If you are certain that your table will never have
more than N rows, use a VARRAY (N) to hold the
fetched data.
If that limit is exceeded, Oracle will raise an
error.
If you do not know in advance how many rows
you might retrieve, you should:
Declare an explicit cursor.
Fetch BULK COLLECT with the LIMIT clause.
Limiting retrieval with BULK COLLECT
The limit value can be a literal or a variable.
Use a variable for the limit to give you
maximum flexibility.
With very large volumes of data and small
numbers of batch processes, however, a larger
LIMIT could help.
Details on that LIMIT clause
You will need to break the habit of checking
%NOTFOUND right after the fetch.
You might skip processing some of your data.
Instead, do one of the following:
At the end of the loop, check %NOTFOUND.
Right after fetch, exit when collection.COUNT = 0.
At end of loop, exit when collection.COUNT < limit.
LOOP
FETCH my_cursor BULK COLLECT INTO l_collection LIMIT 100;
EXIT WHEN my_cursor%NOTFOUND; BAD IDEA
bulklimit_stop.sql
Terminating loops containing BULK COLLECT
Prior to Oracle10g, you should convert all multiple row
fetch logic, including cursor for loops, to BULK
COLLECTs.
For Oracle10g and above, leave your cursor for loops in
place if they...
contain no DML operations.
seem to be running fast enough.
Explicit BULK COLLECTs will usually run faster than
cursor for loops optimized to Bulk Collect.
When to convert to BULK COLLECT
Convert loops that contain inserts, updates or deletes to FORALL statements.
Header looks identical to a numeric FOR loop. Implicitly declared integer iterator At least one "bind array" that uses this iterator as its
index value.
PROCEDURE upd_for_dept (...) IS
BEGIN
FORALL indx IN low_value .. high_value
UPDATE employee
SET salary = newsal_in
WHERE employee_id = list_of_emps (indx);
END; Binding array
Use FORALL for multi-row DML operations
Use any type of collection with FORALL. One DML statement is allowed per FORALL. Each FORALL is its own "extended" DML statement.
The collection must be indexed by integer. The binding array must be sequentially filled. Unless you use the INDICES OF or VALUES OF clause.
SQL%ROWCOUNT returns total number of rows modified by entire FORALL. Unreliable when used with LOG ERRORS.
Use the SQL%BULK_ROWCOUNT cursor attribute to determine how many rows are modified by each statement.
bulktiming.sql
bulk_rowcount.sql
More on FORALL
Prior to 11g, you cannot reference a field of a record in
FORALL.
You must instead break data into separate collections, or...
You can also perform record-level inserts and updates.
In 11g, this restriction is lifted (but it is an undocumented
feature). http://technology.amis.nl/blog/2367/implementation-restricted-relaxed-in-oracle-11g
11g_field_of_record.sql
FORALL and collections of records
Prior to Oracle10g R2, the binding arrays in a FORALL
statement must be sequentially filled.
Now, however, you can bind sparse collections by using
INDICES OF and VALUES OF in the FORALL header.
10g_indices_of*.sql
10g_values_of*.sql
PROCEDURE upd_for_dept (...) IS
BEGIN
FORALL indx IN INDICES OF list_of_emps
UPDATE employee
SET salary = newsal_in
WHERE employee_id = list_of_emps (indx);
INDICES OF and VALUES OF
When an exception occurs in a DML statement....
That statement is rolled back and the FORALL stops.
All (previous) successful statements are not rolled
back.
Use the SAVE EXCEPTIONS clause to tell Oracle to
continue past exceptions, and save the error information
for later.
Then check the contents of the pseudo-collection of
records, SQL%BULK_EXCEPTIONS.
Two fields: ERROR_INDEX and ERROR_CODE
Exception handling and FORALL
slow-by-slow
Change from integrated, row-by-row approach to
a phased approach.
Phase 1: get the data with BULK COLLECT.
Filling those collections
Phase 2: massage collections so they are ready
for DML operations.
Phase 3: push the data to the database with
FORALL.
cfl_to_bulk_0.sql
cfl_to_bulk_5.sql
10g_indices_of.sql
10g_values_of.sql
Converting old-fashioned code to bulk
Most important performance tuning feature in PL/SQL. Almost always the fastest way to execute multi-row
SQL operations in PL/SQL. You trade off increased complexity of code for
dramatically faster execution. But in Oracle Database 10g and above, the compiler
will automatically optimize cursor FOR loops to BULK COLLECT efficiency.
No need to convert unless the loop contains DML or you want to maximally optimize your code.
Watch out for the impact on PGA memory!
Bulk Processing Conclusions
Oracle server
PL/SQL Runtime Engine SQL Engine
PL/SQL block Procedural
statement
executor SQL
statement
executor
OPEN cur; FETCH cur INTO rec; WHILE cur%found LOOP <<Do Stuff>> FETCH cur INTO rec; END LOOP; CLOSE cur;
Performance penalty
for many “context
switches”
Row by row processing of data in PL/SQL
© Steven Feuerstein/Patrick Barel
Oracle server
PL/SQL Runtime Engine SQL Engine
PL/SQL block Procedural
statement
executor SQL
statement
executor
OPEN cur; FETCH cur BULK COLLECT INTO col; FOR indx IN col.first .. col.last LOOP <<Do Stuff>> END LOOP; CLOSE cur;
Fewer context switches,
same SQL behavior
Row
Row
Row
Row
Row
Row
Row
Row
Row
Row
Row
Row
Bulk processing with BULK COLLECT
© Steven Feuerstein/Patrick Barel
Oracle server
PL/SQL Runtime Engine SQL Engine
PL/SQL block Procedural
statement
executor SQL
statement
executor
FOR rec IN emp_cur LOOP UPDATE employee SET salary = ... WHERE employee_id = rec.employee_id; END LOOP;
Performance penalty
for many “context
switches”
Row by row processing of DML in PL/SQL
© Steven Feuerstein
Oracle server
PL/SQL Runtime Engine SQL Engine
PL/SQL block Procedural
statement
executor SQL
statement
executor
FORALL indx IN list_of_emps.FIRST.. list_of_emps.LAST UPDATE employee SET salary = ... WHERE employee_id = list_of_emps(indx);
Fewer context switches,
same SQL behavior
Update...
Update...
Update...
Update...
Update...
Update...
Update...
Update...
Update...
Update...
Update...
Update...
Bulk processing with FORALL
© Steven Feuerstein
Table Functions
Table functions are functions that produce
a collection of rows (either a nested table or a varray)
that can be queried like a physical database table.
You use a table function like the name of a
database table, in the FROM clause of a query.
Table Functions
Table functions are based on collections
Must be available in the SQL layer
Nested tables and Varray
Table Functions
Create a function in PL/SQL
Make sure it returns a collection
Query it using the TABLE() operator
Table functions can be pipelined (return results as they are produced)
Table functions can be paralellized
Table Functions
You can use Table Functions when
Calculations cannot (easily) be done in SQL
You want to take advantage of PL/SQL e.g.
caching or package variables
You want to leverage the power of PL/SQL in
SQL
Make your views more dynamic
Online
tahiti.oracle.com
For all documentation online
www.allthingsoracle.com o http://allthingsoracle.com/collections-in-oracle-pt-1/
o http://allthingsoracle.com/collections-in-oracle-part-2/
o http://allthingsoracle.com/bulk-processing-in-oracle-part-1/
o http://allthingsoracle.com/bulk-processing-in-oracle-part-2/
Books
Oracle PL/SQL Programming
Chapter 12 (collections) and
chapter 21 (bulk processing)
Oracle PL/SQL for DBAs
Chapter 1 (collections and bulk processing)
Resources