Oracle PL/SQL Programming - Toad World
Transcript of Oracle PL/SQL Programming - Toad World
Oracle PL/SQL Programming
Oracle PL/SQL Programming
Steven Feuerstein [email protected]
Making the Most
of the Best
of Oracle PL/SQL
Things to change: • Overloading: for use in SQL, int vs bool • dbms_trace in tracing • API is a contract/backward compatibility
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 2
How to benefit most from this training
• Watch, listen, ask questions, focus on concepts and principles.
• Download and use any of my training materials:
You have my permission to use all these materials to do internal trainings and build your own applications. – But remember: they are not production ready.
– You must test them and modify them to fit your needs.
filename_from_demo_zip.sql
Download and use any of my scripts (examples, performance scripts, reusable code) from the same location: the demo.zip file.
http://www.ToadWorld.com/SF PL/SQL Obsession
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 3
Websites for PL/SQL Developers
www.plsqlchallenge.com Daily PL/SQL quiz with weekly and monthly prizes
www.plsqlchannel.com 27+ hours of detailed video training on Oracle PL/SQL
www.stevenfeuerstein.com Monthly PL/SQL newsletter
www.toadworld.com/SF Quest Software-sponsored portal for PL/SQL developers
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 4
Making the Most of the Best of Oracle PL/SQL - What's the Best?
• Packages - the fundamental building block for PL/SQL apps • Compiler optimization • CASE statement and expression • Autonomous Transactions • Collections and set operations • BULK COLLECT and FORALL • Function Result Cache • Table Functions • NOCOPY hint • Subtypes • Nested subprograms
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 5
How do we make THE MOST of all that?
• Declare First, Program Second
– Lift heavy only when necessary
• Craft Excellent APIs
– Best form of communication
• Never Repeat Anything
– For low maintenance code
• Program Socially
– Never code alone. Note: some of the "best"
items will be explored in
the "most" section.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 6
Here's the Plan
• First, cover many of the "best" features, so that everyone knows what they are and what they do for you.
– More an overview than in-depth training
• Next, tackle each "most", bringing in specific PL/SQL features as is relevant.
• Finally, a quiz! And prizes!
• And now… on to our "best" features.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 7
Compiler Optimization
• As of Oracle Database 10g, the compiler will automatically optimize your code.
– Default setting of 2 is best, but 11g also offers a new level 3 "inlining" optimization.
• The optimizer takes advantage of "freedoms" to re-order the execution of statements.
– In essence, changing the route that the runtime engine takes to get from point A to point B in your code.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 8
Some Optimizer Examples
... A + B ...
...
... A + B ...
T := A + B; ... T ... ... ... T ...
T is a generated variable. We never see
it. And one operation is saved.
for i in 1 .. 10 loop A := B + C; ... end loop;
A := B + C; for i in 1 .. 10 loop ... end loop;
Automatic relocation of a loop invariant.
Avoid repetitive computations.
10g_optimize_cfl.sql
FOR rec in (SELECT ...) LOOP ... do stuff END LOOP;
SELECT ... BULK COLLECT INTO ... FROM ...
Execute cursor FOR loop
at BULK COLLECT
levels of performance.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 9
Optimizer Bottom Line
• Use the default setting.
– Check ALL_PLSQL_OBJECT_SETTINGS for violations.
• Apply the inlining pragma selectively if needed.
• Forget all about it and enjoy the benefits!
• OTN offers several whitepapers on the optimizer for those who want more.
all_plsql_object_settings.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 10
CASE
• Added to PL/SQL in 9i (earlier in SQL), you can use CASE statements and expressions to replace IF statements.
• CASE expressions are especially good at replacing multiple IF statements with a single expression.
– When building a complex string for example.
• Most lengthy ELSIFs should be replaced with a CASE statement.
case*.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 11
Autonomous Transactions
• Default transaction behavior in PL/SQL is at the session level – A commit saves all outstanding changes in your session. – A rollback erases all outstanding changes in your session.
• Define a PL/SQL block as an "autonomous transaction" to control the scope of commit/rollback. – Any changes made within that block will be saved or reversed
without affecting the outer or main transaction.
• Two rules for autonomous transactions: – Must include the autonomous transaction pragma. – Must commit or rollback before exiting the block if any DML
statements were executed.
• Most common application: error logging
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 12
Logging with Autonomous Transactions
logger.sp
log81.pkg
log81*.tst
CREATE OR REPLACE PACKAGE BODY log IS PROCEDURE putline ( code_in IN INTEGER, text_in IN VARCHAR2 ) IS PRAGMA AUTONOMOUS_TRANSACTION; BEGIN INSERT INTO logtab VALUES (code_in, text_in, SYSDATE, USER, SYSDATE, USER, rec.machine, rec.program ); COMMIT; EXCEPTION WHEN OTHERS THEN ROLLBACK; END; END; retry.pkg
retry.tst
Save on
successful exit
Avoid inter-
dependencies with
the main
transaction.
Don't forget to
rollback on error!
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 13
Collections
• Collections are PL/SQL's implementation of arrays.
• All collections in PL/SQL are single dimensional lists or sets.
• They provide the foundation for many performance optimization features.
– Bulk processing, table functions and more.
• They can consume lots of PGA memory.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 14
Memory Management and Collections
• Memory for collections (and almost all PL/SQL data structures) is allocated from the PGA (Process Global Area).
• Accessing PGA memory is quicker than accessing SGA memory. – Sometimes much, much faster.
• Collections represent a very clear tradeoff: use more memory (per session) to improve performance. – But you definitely need to keep an eye on your
PGA memory consumption.
plsql_memory*.*
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 15
Different Types of Collections
• Three types of collections, with different characteristics and use cases.
• Associative array
– Use in PL/SQL only. Most flexible in that scope.
• Nested table
– Use in PL/SQL and SQL. Lots of set-related operations.
• Varray (varying arrays)
– Use in PL/SQL and SQL, but unlikely to ever use them.
• Collections come "with" methods.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 16
Collection Methods • The term method is used to describe
procedures and functions that defined in a class or object type. – You invoke a method by attaching it, using dot
notation, to the name of the type/class or to an instance of the class.
• Collection methods are procedures and functions that are attached to a collection variable. – First introduction of object-oriented syntax in
PL/SQL – way back in Oracle 7.3.4!
method_vs_proc.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 17
Collections Methods
• COUNT – number of elements currently defined in collection.
• EXISTS – TRUE if the specified index values is defined.
• FIRST/LAST – lowest/highest index values of defined rows.
• NEXT/PRIOR – defined index value after/before the specified index value .
• LIMIT – max. number of elements allowed in a VARRAY.
• DELETE – remove elements from a collection
• EXTEND – add elements to a nested table or varray
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 18
About Associative Arrays
• TABLE OF datatypes can be almost any valid PL/SQL type (details to follow).
• INDEX BY type can be integer or string.
– This means you can essentially index by anything!
– But index values can never be NULL.
• Associative arrays can be sparse.
– Can populate elements in non-consecutive index values.
– Easily used to emulate primary keys and unique indexes.
DECLARE TYPE list_of_names_t IS TABLE OF employees.last_name%TYPE INDEX BY PLS_INTEGER;
assoc_array_example.sql
emplu.pkg, emplu.tst
string_tracker*.*
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 19
About Nested Tables
• A nested table is a type of collection, which, according to Oracle documentation, "models an unordered set of elements." – It is a "multiset": like a relational table, there is
no inherent order to its elements, and duplicates are allowed/significant.
• From a practical standpoint, you can access nested table elements through an integer index.
• MULTISET operators allow set-level operations on nested tables.
1 Apple
2 Pear
3 Orange
4 Apricot
CREATE OR REPLACE TYPE list_of_names_t IS TABLE OF NUMBER;
5 Pear
Unordered set
of elements
Integer index
also available
nested_table_example.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 20
Manipulating Nested Tables as Multisets
• Nested tables are, from a theoretical standpoint, "multisets." – There is no inherent order to the elements.
– Duplicates are allowed and are significant.
– Relational tables are multisets as well.
• If a set has no order, then it has no index, so it must be manipulated as a set.
• In Oracle Database 10g, Oracle added MULTISET set operators to manipulate the contents of nested tables (only). – Use in both PL/SQL blocks and SQL statements.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 21
Set-Oriented Features for Nested Tables • Determine if...
– two nested tables are equal/unequal
– a nested table has duplicates, and remove duplicates
– one nested table contains another
– an element is a member of a nested table
• Perform set operations. – Join contents of two nested tables: MULTISET UNION.
– Return common elements of two nested tables with MULTISET INTERSECT.
– Take away the elements of one nested table from another with MULTISET EXCEPT (oddly, not MINUS).
10g_compare*.sql
10g_set.sql
10g_member_of.sql
10g_union.sql
10g_intersect.sql
10g_minus.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 22
Choosing the best type of collection
• Use associative arrays when you need to... – Work within PL/SQL code only
– Sparsely fill and manipulate the collection
– Take advantage of negative index values or string indexing
• Use nested tables when you need to... – Access the collection inside SQL (table functions, columns in
tables, or utilize SQL operations)
– Want or need to perform high level set operations (MULTISET)
• Use varrays when you need to... – If you need to specify a maximum size to your collection
– Optimize performance of storing collection as column
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 23
Bulk Processing of SQL in PL/SQL (BULK COLLECT and FORALL)
• The central purpose of PL/SQL is to provide a portable, fast, easy way to write and execute SQL against an Oracle database.
• Unfortunately, this means that most developers take SQL for granted when writing SQL...and just assume Oracle has fully (automagically) optimized how SQL will run from within PL/SQL.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 24
The Problem with SQL in PL/SQL • Many PL/SQL blocks execute the same SQL statement
repeatedly with different bind values. – Retrieve data one row at a time. – Performs same DML operation for each row retrieved.
CREATE OR REPLACE PROCEDURE upd_for_dept ( dept_in IN employee.department_id%TYPE ,newsal_in IN employee.salary%TYPE) IS CURSOR emp_cur IS SELECT employee_id,salary,hire_date FROM employee WHERE department_id = dept_in; BEGIN FOR rec IN emp_cur LOOP adjust_compensation (rec, newsal_in); UPDATE employee SET salary = rec.salary WHERE employee_id = rec.employee_id; END LOOP; END upd_for_dept;
The result? Simple and
elegant but inefficient...
Why is this?
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 25
Oracle server
PL/SQL Runtime Engine SQL Engine
PL/SQL block Procedural
statement
executor SQL
statement
executor
FOR rec IN emp_cur LOOP UPDATE employee SET salary = ... WHERE employee_id = rec.employee_id; END LOOP;
Performance penalty
for many “context
switches”
Repetitive statement processing from PL/SQL
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 26
Bulk Processing in PL/SQL
• The goal is straightforward: reduce the number of context switches and you improver performance.
• To do this, Oracle "bundles up" the requests for data (or to change data) and then passes them with a single context switch.
• FORALL speeds up DML. – Use with inserts, updates, deletes and merges. – Move data from collections to tables.
• BULK COLLECT speeds up queries. – Can be used with all kinds of queries: implicit, explicit,
static and dynamic. – Move data from tables into collections.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 27
Bulk processing with FORALL
Oracle server
PL/SQL Runtime Engine SQL Engine
PL/SQL block Procedural
statement
executor SQL
statement
executor
FORALL indx IN list_of_emps.FIRST.. list_of_emps.LAST UPDATE employee SET salary = ... WHERE employee_id = list_of_emps(indx);
Fewer context switches,
same SQL behavior
Update...
Update...
Update...
Update...
Update...
Update...
Update...
Update...
Update...
Update...
Update...
Update...
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 28
Impact of Bulk Processing in SQL layer
• The bulk processing features of PL/SQL change the way the PL/SQL engine communicates with the SQL layer.
• For both FORALL and BULK COLLECT, the processing in the SQL engine is almost completely unchanged. – Same transaction and rollback segment management – Same number of individual SQL statements will be
executed.
• Only one difference: BEFORE and AFTER statement-level triggers only fire once per FORALL INSERT statements. – Not for each INSERT statement passed to the SQL engine
from the FORALL statement.
statement_trigger_and_forall.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 29
BULK COLLECT for multi-row querying
• Retrieve multiple rows into collections with a single context switch to the SQL engine. – Collection filled sequentially from 1.
• Use the LIMIT clause to constrain PGA memory consumption. – 100 is good default, but experiment with higher values.
• When populating nested tables and varrays, Oracle automatically initializes and extends.
SELECT * BULK COLLECT INTO collection(s) FROM table; FETCH cur BULK COLLECT INTO collection(s); EXECUTE IMMEDIATE query BULK COLLECT INTO collection(s);
bulkcoll.sql
bulkcollect.tst
bulklimit.sql, cfl_to_bulk0.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 30
Use FORALL for repeated DML operations
• Convert loops that contain inserts, updates, deletes or merges to FORALL statements.
• Header looks identical to a numeric FOR loop. – Implicitly declared integer iterator – At least one "bind array" that uses this iterator as its
index value. – You can also use a different header "style" with INDICES
OF and VALUES OF (covered later)
PROCEDURE upd_for_dept (...) IS BEGIN FORALL indx IN low_value .. high_value UPDATE employee SET salary = newsal_in WHERE employee_id = list_of_emps (indx); END; Bind array
forall_timing.sql
forall_examples.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 31
More on FORALL
• Use the SQL%BULK_ROWCOUNT cursor attribute to determine how many rows are modified by each statement. – SQL%ROWCOUNT returns total for FORALL.
• Use SAVE EXCEPTIONS and SQL%BULK_EXCEPTIONS to execute all statements, saving errors for later. – You will need to handle ORA-24381.
• When collections may be sparse, use INDICES OF or VALUES OF.
bulk_rowcount.sql
bulkexc.sql
10g_indices_of*.sql
10g_values_of.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 32
Converting to Bulk Processing
• Let's take a look at the process by which you go from "old-fashioned" code to a bulk processing-based solution.
• From integrated row-by-row to phased processing
• With multiple DML statements in loop, how do you "communicate" from one to the other?
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 33
The "Old Fashioned" Approach • Cursor FOR loop with two DML statements, trap
exception, and keep on going.
CREATE OR REPLACE PROCEDURE upd_for_dept ( dept_in IN employees.department_id%TYPE , newsal_in IN employees.salary%TYPE) IS CURSOR emp_cur ...; BEGIN FOR rec IN emp_cur LOOP BEGIN INSERT INTO employee_history ... adjust_compensation (rec.employee_id, rec.salary); UPDATE employees SET salary = rec.salary ... EXCEPTION WHEN OTHERS THEN log_error; END; END LOOP; END upd_for_dept;
cfl_to_bulk_0.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 34
A phased approach with bulk processing • Change from integrated, row-by-row approach to
a phased approach.
Relational Table
Relational Table
Phase 1: Bulk collect from table(s) to collection
Phase 3: FORALL from collection to table
Phase 2: Modify contents of collection according to requirements
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 35
Translating phases into code
• The cfl_to_bulk_5.sql file contains the converted program, following the phased approach.
cfl_to_bulk_0.sql
cfl_to_bulk_5.sql
BEGIN OPEN employees_cur; LOOP fetch_next_set_of_rows ( bulk_limit_in, employee_ids, salaries, hire_dates); EXIT WHEN employee_ids.COUNT = 0; insert_history; adj_comp_for_arrays (employee_ids, salaries); update_employee; END LOOP; END upd_for_dept;
Phase 1: Get Data
Phase 3: Push Data
Phase 2: Massage Data
Phase 3: Push Data
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 36
Conclusions – Bulk Processing
• FORALL is the most important performance tuning feature in PL/SQL. – Almost always the fastest way to execute repeated SQL
operations in PL/SQL.
• You trade off increased complexity of code for dramatically faster execution. – But remember that Oracle will automatically optimize
cursor FOR loops to BULK COLLECT efficiency.
– No need to convert unless the loop contains DML or you want to maximally optimize your code.
• Watch out for the impact on PGA memory!
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 37
The Oracle 11g Function Result Cache
• Many of our queries fetch the same data many times, even if it hasn't changed. – Paying the price of executing SQL, no matter how
optimized.
• Use the function result cache to tell Oracle to return unchanged data, without hitting the SQL layer.
• This cache is... – stored in the SGA
– shared across sessions
– purged of dirty data automatically
• You can use and should use it to retrieve data from any table that is queried more frequently than updated.
11g
11g_frc_demo.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 38
Performance Impact of Result Cache
• The result cache is stored in the SGA. • So we should expect it be slower than a PGA-
based cache. • But accessing result cache data does not
require going through the SQL engine. • So it should be much faster than executing a
query. – Even if the statement is parsed and the data
blocks are already in the SGA.
• Let's find out!
11g_emplu*.*
11g
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 39
Result Cache – Things to Keep in Mind • If you have uncommitted changes in your session,
dependent caches are ignored. – The cache will not override your own changed data.
• You can't cache everything. – Oracle has to be able to do an "=" comparison.
• Functions with session-specific dependencies must be "result-cached" with great care. – Virtual private database configurations, references to
SYSDATE, reliance on NLS_DATE_FORMAT, time zone changes, Application contexts (calls to SYS_CONTEXT)
– Solution: move all dependencies into parameter list.
• Work with your DBA to identify the "sweet spots" for applying this feature.
11g
11g_frc_demo.sql
11g_frc_vpd.sql
11g_frc_vpd2.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 40
Table Functions
• A table function is a function that you can call in the FROM clause of a query, and have it be treated as if it were a relational table.
• Perform arbitrarily complex transformations of data and then make that data available through a query: "just" rows and columns! – After all, not everything can be done in SQL.
• Improve performance for…. – Parallel queries – User perceptions of elapsed time
SELECT COLUMN_VALUE FROM TABLE (my_function (. . .))
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 41
Building a table function
• A table function must return a nested table or varray based on a schema-defined type. – Types defined in a PL/SQL package can only be
used with pipelined table functions.
• The function header and the way it is called must be SQL-compatible: all parameters use SQL types; no named notation allowed until 11g. – In some cases (streaming and pipelined
functions), the IN parameter must be a cursor variable -- a query result set.
tabfunc_scalar.sql
tabfunc_streaming.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 42
Pipelined functions enhance performance.
• Pipelined functions allow you to return data iteratively, asynchronous to termination of the function. – As data is produced within the function, it is passed
back to the calling process/query.
• Pipelined functions can only be called within a SQL statement. – They make no sense within non-multi-threaded
PL/SQL blocks.
CREATE FUNCTION StockPivot (p refcur_pkg.refcur_t) RETURN TickerTypeSet PIPELINED
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 43
Applications for pipelined functions
• Execution functions in parallel. – Use the PARALLEL_ENABLE clause to allow your pipelined
function to participate fully in a parallelized query.
– Critical in data warehouse applications.
• Improve speed of delivery of data to web pages. – Use a pipelined function to "serve up" data to the webpage
and allow users to begin viewing and browsing, even before the function has finished retrieving all of the data.
• And pipelined functions use less PGA memory than non-pipelined functions!
tabfunc_pipelined.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 44
Optimizing Table Functions in SQL
• A function called in SQL is a "black box" to the optimizer - unless you provide statistics on the costs and selectivities of that function. – Use ASSOCIATE STATISTICS or hints or both.
http://www.oracle-developer.net/display.php?id=426 http://www.oracle-developer.net/display.php?id=427
http://www.dbprof.com/index.php?option=com_jdownloads&Itemid=57&view=viewcategory&catid=3
SQL> SELECT /*+ OPT_ESTIMATE(table, e, scale_rows=2.62) */ 2 * 3 FROM departments d 4 , TABLE(employees_piped) e 5 WHERE d.department_id = e.department_id; SQL> SELECT /*+ DYNAMIC_SAMPLING(e, 2) */ * 2 2 FROM TABLE(employees_piped) e;
SQL> ASSOCIATE STATISTICS WITH FUNCTIONS high_cpu_io DEFAULT COST (6747773, 21, 0);
Resources
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 45
Table functions - Summary
• Table functions offer significant new flexibility for PL/SQL developers.
• Consider using them when you...
– Need to pass back complex result sets of data through the SQL layer (a query);
– Want to call a user defined function inside a query and execute it as part of a parallel query.
• Use pipelined table functions for performance improvement and reduced PGA consumption.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 46
The NOCOPY hint
• By default, Oracle passes all IN OUT and OUT arguments by value, not reference.
– This means that OUT and IN OUT arguments always involve some copying of data.
• With NOCOPY, you turn off the copy process.
– But it comes with a risk: Oracle will not automatically "rollback" or reverse changes made to your variables if the NOCOPY-ed program raises an exception.
nocopy*.*
string_nocopy.*
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 47
That's a LOT of BEST… Now it's Time for the MOST
• Whew.
• That's a lot of functionality.
• And even if you know all about all of it, you still have to figure out how to use it so that your application is easy to:
– Understand
– Maintain (fix and enhance)
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 48
The Big Four
• Declare First, Program Second
– Lift heavy (write code) only when necessary
• Craft Excellent APIs
– It's the best form of communication
• Never Repeat Anything
– For low maintenance code
• Program Socially
– Never code alone.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 49
Declare First, Program Second
• Maximize the declarative aspects of your technology before you start writing algorithms.
• Get the data model right.
– Use constraints and triggers
• Then maximize the SQL language and all its latest features.
• Only then should you start writing programs/algorithms.
DDL
DML
PL/SQL
Java/.Net
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 50
Maximize the SQL Language • Courtesy of Lucas Jellema of AMIS Consulting, Netherlands • Analytical Functions
– Especially LAG and LEAD; these allow you to look to previous and following rows to calculate differences. But also RANK, PIVOT, etc.
• WITH clause (subquery factoring) – Allows the definition of 'views' inside a query that can be used and reused; they
allow procedural top-down logic inside a query
• Flashback query – No more need for journal tables, history tables, etc.
• ANSI JOIN syntax – Replaces the (+) operator and introduces FULL OUTER JOIN
• SYS_CONNECT_BY_PATH and CONNECT_BY_ROOT for hierarchical queries
• Scalar subquery – Adds a subquery to a query like a function call.
• And soon…Oracle12c new SQL features!
select d.deptno , (select count(*) from emp e where e.deptno = d.deptno) number_staff from dept
Check out the SQL quizzes at the PL/SQL Challenge!
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 51
Leverage Declarative Statements in PL/SQL
• SQL, first and foremost - inside PL/SQL
• Cursor FOR loop
– Automatic optimization demonstrates the benefit
• FORALL statement
• More generally, don't reinvent the wheel when built-in functions can do the heavy lifting.
– Regular expressions
– All nuances of string functions
– TRUNC and TO_CHAR
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 52
Craft Excellent APIs
• An API is an application program interface. – A set of procedures and functions (and
more) that can be used as "building blocks" for application construction.
• Clean, well-designed APIs hide details, reduce overall code volume, improve productivity, and the lower the frequency and severity of bugs.
• In PL/SQL, packages are the best way to build APIs, though you can also do so in object types.
do_X
do_Y
return_Z
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 53
Building with Packages • Employ object-oriented design principles
– Build at higher levels of abstraction – Enforce information hiding - control what people see and
do – Call packaged code from object types and triggers
• Encourages top-down design and bottom-up construction – TD: Design the interfaces required by the different
components of your application without addressing implementation details
– BU: packages contain building blocks for new code
• Organize your stored code more effectively • Implements session-persistent data
dbms_errlog_helper.sql
errpkg.pkg
loop_killer.pkg
sf_timer.*
plsql_memory.*
assert.pkg
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 54
Package Data: Useful and (but?) Sticky • The scope of a package is your session, and any data
defined at the "package level" also has session scope. – If defined in the package specification, any program can
directly read/write the data. – Ideal for program-specific caching.
• Attention must be paid: – Package cursors must be explicitly closed. – Collection contents must be explicitly deleted. – Clean-up will not occur automatically on close of block.
• Note that with connection pools and stateless applications, you should not rely on package state - between server calls.
thisuser.*
emplu.pkg
emplu.tst
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 55
More on Package-level Data • Hide your package data in the body so that you can
control access to it. – Only constants and "scratch" variables should be placed in
the specification. – Build "get and set" subprograms
• Use the SERIALLY_REUSABLE pragma to move data to SGA and have memory released after each usage. – This also means that the package is re-initialized with each
server call.
• A package with at least one variable has state, and that can greatly complicate recompilation. – Watch out for the ORA-04068 errors! – If you have these, check out Edition-Based Redefinition.
serial.sql
valerr.pkg
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 56
Package Initialization
• The initialization section: – Is defined after and outside of any
programs in the package.
– Is not required. In fact, most packages you build won't have one.
– Can have its own exception handling section.
• Useful for: – Performing complex setting of default or
initial values.
– Setting up package data which does not change for the duration of a session.
– Confirming that package is properly instantiated.
PACKAGE BODY pkg IS PROCEDURE proc IS BEGIN END; FUNCTION func RETURN BEGIN END; BEGIN ...initialize... END pkg;
BEGIN after/outside
of any program
defined in the pkg.
init.pkg
init.tst
datemgr.pkg
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 57
Overloading in Packages: key API/usability technique
• Overloading (static polymorphism): two or more programs with the same name, but different signature. – You can overload in the declaration section of any
PL/SQL block, including the package body (most common).
• Overloading is a critical feature when building comprehensive programmatic interfaces (APIs) or components using packages. – If you want others to use your code, you need to make
that code as smart and as easy to use as possible.
– Overloading transfers the "need to know" from the user to the overloaded program.
myproc
myfunc
myproc
Compare: DBMS_OUTPUT and p packages dynamic_polymorphism.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 58
How Overloading Works
• For two or more modules to be overloaded, the compiler must be able to distinguish between the two calls at compile-time. – Another name for overloading is "static
polymorphism."
• There are two different "compile times": – 1. When you compile the package or block containing
the overloaded code.
– 2. When you compile programs that use the overloaded code.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 59
How Overloading Works, continued
• Distinguishing characteristics: – The formal parameters of overloaded modules must
differ in number, order or datatype family (CHAR vs. VARCHAR2 is not different enough).
– The programs are of different types: procedure and function.
• Undistinguishing characteristics: – Functions differ only in their RETURN datatype.
– Arguments differ only in their mode (IN, OUT, IN OUT).
– Their formal parameters differ only in datatype and the datatypes are in the same family.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 60
Quiz! Nuances of Overloading
• Will these specifications compile? If so, can I call the subprograms?
CREATE OR REPLACE PACKAGE sales IS PROCEDURE calc_total (zone_in IN VARCHAR2); PROCEDURE calc_total (reg_in IN VARCHAR2); END sales;
ambig_overloading.sql
CREATE OR REPLACE PACKAGE sales IS PROCEDURE calc_total (zone_in IN CHAR); PROCEDURE calc_total (zone_in IN VARCHAR2); END sales;
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 61
Tips for Optimal Overloading
• Don't repeat code across implementations.
– Single point of definition!
• Most overloadings involve doing mostly the "same thing", but with different combinations of data.
• So make sure that in the package body, all overloadings are based on the same "core."
• Don't overload "just in case".
– Avoid hypothetically useful code.
lazy_overloading.sql
dynamic_polymorphism.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 62
Tips for Writing Package Bodies
• A well-constructed API (interface) is determined by the package specification.
• But the way we implement that API in the package body has enormous ramifications on maintainability.
• You must make very careful decisions about how to modularize your code. – Avoid spaghetti code
– Expose only the functionality that is needed "out there"
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 63
Modularization Choices
• You can choose from: – Schema-level procedure or function – Public packaged subprogram – Private packaged subprogram – Nested subprogram
• Avoid schema-level programs; put all your code in packages. – Entire package is loaded into memory – Each package provides a "namespace" in which to
organize related code.
• Use nested subprograms to improve readability and maintainability of your bodies.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 64
Extreme Modularization (Write tiny chunks of code)
• Spaghetti code is the bane of a programmer's existence.
• It is impossible to understand and therefore debug or maintain code that has long, twisted executable sections.
• Fortunately, it is really easy to make spaghetti code a thing of the past.
Organize your code so that the
executable section has no more than fifty lines of code.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 65
Fifty lines of code? That's ridiculous!
• Of course you write lots more than 50 lines of code in your applications.
• The question is: how will you organize all that code?
• Turns out, it is actually quite straightforward to organize your code so that it is transparent in meaning, with a minimal need for comments.
• Key technique: local or nested subprograms.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 66
Let’s write some code!
• My team is building a support application. Customers call with problems, and we put their call in a queue if it cannot be handled immediately.
– I must now write a program that distributes unhandled calls out to members of the support team.
• Fifty pages of doc, complicated program!
While there are still unhandled calls in the queue, assign them to employees who are under-utilized (have fewer calls assigned to
them then the average for their department).
But there is an "executive
summary"
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 67
First: Translate the summary into code.
• A more or less direct translation. No need for comments, the subprogram names "tell the story" – but those subprograms don't yet exist!
PROCEDURE distribute_calls ( department_id_in IN departments.department_id%TYPE) IS BEGIN WHILE ( calls_are_unhandled ( ) ) LOOP FOR emp_rec IN emps_in_dept_cur (department_id_in) LOOP IF current_caseload (emp_rec.employee_id) < avg_caseload_for_dept (department_id_in) THEN assign_next_open_call (emp_rec.employee_id); END IF; END LOOP; END LOOP; END distribute_calls;
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 68
Explanation of Subprograms
• Function calls_are_unhandled: takes no arguments, returns TRUE if there is still at least one unhandled call, FALSE otherwise.
• Function current_caseload: returns the number of calls (case load) assigned to that employee.
• Function avg_caseload_for_dept: returns the average number of calls assigned to employees in that department.
• Procedure assign_next_open_call: assigns the employee to the call, making it handled, as opposed to unhandled.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 69
Next: Implement stubs for subprograms
• These are all defined locally in the procedure.
PROCEDURE call_manager.distribute_calls ( department_id_in IN departments.department_id%TYPE) IS FUNCTION calls_are_handled RETURN BOOLEAN IS BEGIN ... END calls_are_handled; FUNCTION current_caseload ( employee_id_in IN employees.employee_id%TYPE) RETURN PLS_INTEGER IS BEGIN ... END current_caseload; FUNCTION avg_caseload_for_dept ( employee_id_in IN employees.employee_id%TYPE) RETURN PLS_INTEGER IS BEGIN ... END current_caseload; PROCEDURE assign_next_open_call ( employee_id_in IN employees.employee_id%TYPE) IS BEGIN ... END assign_next_open_call; BEGIN
locmod_step_by_step.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 70
About Nested Subprograms
• They can be called only from within the block in which they are defined. – They can reference any variables defined in the parent
block.
– Watch out for "global" references.
• Only procedures and functions can be nested. – No packages within packages
– No object types
– No triggers
• Use these instead of nested blocks. – You replace code with a name – tell the story!
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 71
Next: Think about implementation of just this level.
• Think about what the programs need to do.
• Think about if you or someone has already done it. Don’t reinvent the wheel!
Hey! Just last week I wrote another function that is very similar to
current_caseload. It is now "buried" inside a procedure named
show_caseload. I can’t call it from distribute_calls, though. It is local,
private, hidden.
Should I copy and paste? No! I should extract the program and
expand its scope.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 72
Next: Isolate and refactor common code.
• Now current_caseload is at the package level and can be called by any program in the package.
CREATE OR REPLACE PACKAGE BODY call_manager IS FUNCTION current_caseload ( employee_id_in IN employees.employee_id%TYPE , use_in_show_in IN BOOLEAN DEFAULT TRUE) RETURN PLS_INTEGER IS BEGIN ... END current_caseload; PROCEDURE show_caseload ( department_id_in IN departments.department_id%TYPE) IS BEGIN ... END show_caseload; PROCEDURE distribute_calls ( department_id_in IN departments.department_id%TYPE ) IS BEGIN ... END distribute_calls; END;
current_ caseload
distribute _calls
show_ caseload
Note the increased complexity,
needed to ensure backward compatibility.
locmod_step_by_step.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 73
Next: Reuse existing code whenever possible.
• Just last week, Sally emailed all of us with news of her call_util package. – Returns average workload of employee and much more.
– Just what I need! Don’t have to build it myself, just call it.
BEGIN WHILE ( calls_are_unhandled ( ) ) LOOP FOR emp_rec IN emps_in_dept_cur (department_id_in) LOOP IF current_caseload (emp_rec. employee_id) < call_util.dept_avg_caseload (department_id_in) THEN assign_next_open_call (emp_rec.employee_id); END IF; END LOOP; END LOOP; END distribute_calls;
This program has the widest scope possible: it can be executed by any schema with execute authority on the call_util package, and by any program within the owning schema.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 74
Next: Implement what’s left.
• Now I am left only with program-specific, nested subprograms.
• So I move down to the next level of detail and apply the same process. – Write the “executive summary” first.
– Keep the executable section small.
– Use local modules to hide the details.
• Eventually, you get down to the “real code” and can deal with the actual data structures and algorithms without being overwhelmed.
locmod_step_by_step.sql
topdown*.*
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 75
Challenges of Nested Subprograms
• Requires discipline: always be on the lookout for opportunities to refactor.
• Need to read from the bottom, up. – Takes some getting used to.
• Sometimes can feel like a "wild goose chase". – Where is the darned thing actually implemented? – Your IDE should help you understand the internal
structure of the program.
• You cannot directly test nested subprogams. • But how do you decide when a module should be
local or defined at a “higher” level?
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 76
Rule: Define subprograms close to usage.
• When should the program be nested? Private to the package? Publicly accessible?
• The best rule to follow is: Define your subprograms as close as possible to their usage(s).
• The shorter the distance from usage to definition, the easier it is to find, understand and maintain that code.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 77
Craft Excellent APIs - Conclusion
• The more clearly you define and control access to underlying functionality, the easier it will be to use and maintain your code.
• Ideally, a developer never needs to look at the package body to use that code.
• It's a real joy to be able to construct new, complex programs by pulling out pre-tested units from your "set of blocks."
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 78
Never Repeat Anything
• The single most important guideline for writing high quality code.
• Repetition => hard coding => exposed implementation => maintenance nightmare.
• Instead, aim for a Single Point of Definition (SPOD) for every aspect of your application.
• The hardest part of doing this can be recognizing the different ways that hard-coding can creep into your code.
hardcoding2.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 79
Never Repeat Anything - Specifically….
• Hide magic values
• Hides constrained declarations
• Use a shared error logging utility
• Use a shared execution tracer
• Build a data encapsulation layer for SQL
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 80
Hide Magical Values (Literals)
• The most commonly recognized form of hard-coding.
• The only place a literal should appear in your code is in its SPOD.
• Hide literals behind constants or functions.
• Consider soft coding values in tables.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 81
Hide Behind Constant
• Instead of exposing the literal value, and referencing it in multiple places, declare a constant and reference that name.
• Best to put such constants in a package specification. – Can share across entire code base.
• Constants are simple and quick, but they expose the value in the package specification. – If the value needs to change, all programs that
depend on that package must be recompiled.
constant_vs_function.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 82
Hide Behind Function
• You can also define a function whose body returns the value.
– Best done in a package
• Advantages over constants include
– When the value changes, only the package body must be recompiled.
– Developers cannot "lazily" see/use value.
– You can call the function in an SQL statement
• But this is less efficient than a constant.
constant_vs_function.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 83
Soft-Code Values in Table
• You can make things really flexible by putting all literals in a table, associating them with a name, and retrieving them as needed from the table.
• Downsides are:
– More complex code
– More overhead, but caching can avoid this problem.
soft_code_literals.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 84
Hide error codes with EXCEPTION_INIT
• Oracle doesn't provide a name for every error code, but you can do this.
• Best place to put exception declarations is a package, so they can be shared across the application.
WHEN OTHERS THEN IF SQLCODE = -24381 THEN ... ELSIF SQLCODE = -1855 THEN ... ELSE RAISE; END;
e_forall_failure EXCEPTION; PRAGMA EXCEPTION_INIT ( e_forall_failure, -24381); BEGIN .... EXCEPTION WHEN e_forall_failure THEN ... END;
errpkg.pkg
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 85
Hide Constrained Declarations
• Every declaration requires a datatype.
• If you are not careful, the way you specify that datatype could be a hard-coding. – Generally, any declaration that relies on a constrained
datatype is a hard-coding.
– VARCHAR2(n) is constrained; BOOLEAN and DATE are unconstrained.
• Two problems with hard-coding the datatype: – Constraints can lead to errors in future.
– The datatype does not explain the application significance of the element declared.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 86
"SPODification" for Datatypes
• Whenever possible, anchor the datatype of your declaration to an already-existing type. – That way, if the existing type or SPOD ever changes,
then your code will be marked INVALID and automatically recompiled to pick up the latest version of the anchoring type.
• Use %TYPE and %ROWTYPE whenever possible – Fetch into record, never list of variables
• Use SUBTYPEs when anchoring is not possible.
Consider every VARCHAR2(N) declaration to be a bug – unless it's a SPOD.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 87
%TYPE and %ROWTYPE
• Use %TYPE for declarations based on columns in tables.
• Use %ROWTYPE for records based on tables, views or cursors.
• The lookup of the datatype from these attributes occurs at compile-time.
– There is no run-time overhead.
no_more_hardcoding.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 88
Fetch into record, not list of variables
• If your FETCH statement contains a list of individual variables, you are hard-coding the number of elements in the SELECT list.
– When the cursor changes, you must change the FETCH as well.
• Solution: always fetch into a record, defined with %ROWTYPE against the cursor.
fetch_into_record.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 89
SUBTYPEs
• You can't always use %TYPE or %ROWTYPE in your declaration.
• You can, however, always define a "subtype" or subset of an existing type with the SUBTYPE statement. SUBTYPE benefits:
– Avoid exposing and repeating constraints.
– Give application-specific names to types. Critical when working with complex structures like collections of records, and nested collections.
– Apply constraints, such as numeric ranges, to the variable declared with the subtype.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 90
SUBTYPE Details and Examples
• Define a subtype based on any pre-defined type or other, already-defined subtype.
• If the base type can be constrained, then you can constrain the subtype.
– (precision,scale) or RANGE
• You can also, always specify NOT NULL.
– Even if the base type could be NULL.
SUBTYPE type_name IS data_type [ constraint ] [ NOT NULL ]
subtype_examples.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 91
Applying SUBTYPEs
• Two key scenarios:
– Whenever you are about to write a VARCHAR2(N) or other constrained declaration, define a subtype instead, preferably in a package specification.
– Instead of writing a comment explaining a declaration, put the explanation into a subtype.
fullname.pks plsql_limits.pks
string_tracker3.*
DECLARE l_full_name VARCHAR2(100); l_big_string VARCHAR2(32767);
DECLARE l_full_name employees_rp.full_name_t; l_big_string plsql_limits.maxvarchar2;
Instead of this:
Write this:
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 92
Conclusions
• Declarations offer a danger of hard-coding of both datatype and constraint on that type.
• Assume that over time everything will change.
• Apply the same "single point of definition" principle to your declarations.
– Use %TYPE and %ROWTYPE whenever possible.
– Fall back on subtypes to define application specific types and PL/SQL limitations.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 93
Error Management and Logging
• Handling problems gracefully and effectively is critical.
• You need to know what Oracle offers to help you diagnose issues.
• And you need to make sure error handling and logging is not a mess of hard-codings.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 94
Oracle Built-ins For Handling Exceptions
• In addition to the application-specific information you may want to log, Oracle built-ins provide you with answers to the following questions:
– How did I get here?
– What is the error code?
– What is the error message and/or stack?
– On what line was the error raised?
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 95
SQLCODE and SQLERRM
• SQLCODE returns the error code of the most recently-raised exception in your session.
• SQLERRM returns the error message associated with SQLCODE – but it also a generic error message lookup function.
• Neither SQLCODE nor SQLERRM can be called from within a SQL statement. – You must assign them to local variables to use their values
in SQL statements (like writing to an error log).
sqlcode.sql sqlcode_test.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 96
SQLERRM Details
• If you don't pass an argument to SQLERRM, it returns the error message for the SQLCODE value. – When called outside of an exception handler, always
returns "success" message – no error.
• You can also pass an error code to SQLERRM and it will return the generic error message.
• The maximum size of a string returned by SQLERRM is 512 bytes. – When there is a stack of errors, Oracle may truncate the
string returned by SQLERRM.
– Oracle recommends you use DBMS_UTILITY.FORMAT_ERROR_STACK instead.
sqlerrm.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 97
DBMS_UTILITY error functions
• Answer the question "How did I get here?" with DBMS_UTILITY.FORMAT_CALL_STACK.
• Get a more complete error message with DBMS_UTILITY.FORMAT_ERROR_STACK.
• Find line number on which error was raised with DBMS_UTILITY.FORMAT_ERROR_BACKTRACE.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 98
DBMS_UTILITY.FORMAT_CALL_STACK
• The "call stack" reveals the path taken through your application code to get to that point.
• Very useful whenever tracing or logging errors.
• The string is formatted to show line number and program unit name.
– But it does not reveal the names of subprograms in packages.
callstack.sql callstack.pkg
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 99
DBMS_UTILITY.FORMAT_ERROR_STACK
• This built-in returns the error stack in the current session.
– Possibly more than one error in stack.
• Returns NULL when there is no error.
• Returns a string of maximum size 2000 bytes (according to the documentation).
• Oracle recommends you use this instead of SQLERRM, to reduce the chance of truncation.
errorstack.sql big_error_stack.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 100
DBMS_UTILITY.FORMAT_ERROR_BACKTRACE
• The backtrace function (new to 10.2) answers the question: "Where was my error raised?
– Prior to 10.2, you could not get this information from within PL/SQL.
• Call it whenever you are logging an error.
• When you re-raise your exception (RAISE;) or raise a different exception, subsequent BACKTRACE calls will point to that line.
– So before a re-raise, call BACKTRACE and store that information to avoid losing the original line number.
backtrace.sql bt.pkg
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 101
Logging Errors
• We usually, but not always, want to write error information out to a log table. How's this?
WHEN NO_DATA_FOUND THEN l_code := SQLCODE; INSERT INTO errlog VALUES ( l_code , 'No company for id ' || TO_CHAR ( v_id ) , 'fixdebt', SYSDATE, USER ); WHEN OTHERS THEN l_code := SQLCODE; l_errm := SQLERRM; INSERT INTO errlog VALUES (l_code, l_errm, 'fixdebt', SYSDATE, USER ); RAISE; END;
It's easy to "read" but only because it exposes the logging mechanism.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 102
Hide how and what you log
• Don't call RAISE_APPLICATION_ERROR.
• Don't explicitly insert into log table or write to file.
• Don't call all those useful built-in functions in each handler.
• Do use a generic and shared error management utility. – Check out Quest Error Manager at PL/SQL Obsession for an example.
WHEN NO_DATA_FOUND THEN q$error_manager.register_error ( text_in => 'No company for id ' || TO_CHAR ( v_id )); WHEN OTHERS THEN q$error_manager.raise_unanticipated ( name1_in => 'COMPANY_ID', value1_in => v_id); END;
qem_demo.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 103
Execution Tracing (Instrumentation)
• Tracing, also known as instrumentation, is an important technique for diagnosing production issues and quickly resolving them.
– Also helpful as you build your code.
• Sadly, too many developers do not sufficiently instrument their code and too many use the wrong instrument.
– You know what I'm talking about: DBMS_OUTPUT.PUT_LINE!
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 104
Doesn't Oracle do tracing for us?
• Sure, but we often need to retrieve additional, application-specific information from our code while running. – Especially in production.
• DBMS_OUTPUT.PUT_LINE is the "default" tracing mechanism – and should never appear in your application code. – You are exposing the trace/display mechanism
(hard-coding).
– Too many drawbacks, too little flexibility.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 105
Tips on Instrumenting Code
• Don't remove trace calls when you are "done" writing your program.
– You will need it for production problem diagnoses.
• Minimize overhead when tracing is disabled.
– Put calls to the trace program inside a faster Boolean check.
• Make it possible for users to enable tracing to gather production data.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 106
Some Tracing Options
• Trace packages in demo.zip include:
– sf_trace: my latest, simple and sufficient tracer
– watch.pkg: "watch" the action, ability to direct trace
information to a variety of targets.
• logger utility by Tyler Muth
– Very popular with APEX developers; check
http://goo.gl/5jrkT for update on project.
• DBMS_APPLICATION_INFO
– Writes to V$ views, good for long-running operations
sf_trace*.*
watch.pkg
dbms_application_info_demo.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 107
Writing SQL in PL/SQL • The most critical aspect of our programs.
• SQL statements directly reflect our business models. – And those models are always changing.
• SQL statements cause most of the performance problems in our applications. – Tuning SQL and the way that SQL is called in
PL/SQL overwhelms all other considerations.
• Many runtime errors in applications result from integrity and check constraints on tables.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 108
The fundamental problem with SQL in PL/SQL • We take it entirely for
granted. – Why not? It's so easy to
write SQL in PL/SQL!
• We don't set rules on how, when and where SQL should be written in PL/SQL.
The Backend
Order Table Item
Table
Order Entry Application
Customer Table
The result? Slow, buggy code that is difficult to optimize and maintain.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 109
Set Standards for Writing SQL
• Check Bryn Llewellyn's "Doing SQL in PL/SQL" whitepaper on OTN for many ideas.
• Use a data encapsulation layer. – Hide SQL statements behind an interface.
• Hide all tables in schemas users cannot access.
• Qualify every identifier in SQL statements.
• Dot-qualify references to Oracle-supplied objects.
• And some best practices for dynamic SQL
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 110
Data Encapsulation: hide all tables in schemas users cannot access. • A fundamental issue of control and security.
• Do not allow users to connect to any schema that contains tables.
– Simply too risky.
• Define tables in other schemas.
• Grant access via privileges, mostly via EXECUTE on packages that maintain the tables (data encapsulation/API).
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 111
Architecture with "inaccessible schema" • The OE Data schemas own all tables.
• The OE Code schema owns the code and has directly granted privileges on the tables.
• User schemas have execute authority granted on the code.
Page 111
Orders
OE Data
OE Code
Order_Mgt
Cancel
Sam_Sales
Place Close Old
Orders
X
Cannot access table directly.
But we also need to extend these controls
to developers. Because every SQL
statement you write is a hard-coding!
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 112 Page 112
SQL statements as hard-codings • I suggest that every SQL statement you will ever
write is a hard-coding. Consider....
• I need to write a complex query to return HR data for a report.
SELECT . . . FROM employees, departments, locations WHERE . . . (a page full of complex conditions)
• And then the three way join turns into a four way join – and we have to find all occurrences of this query. A very tough thing to do!
• And Joe needs to use that same query in his business rule procedure. And so on...
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 113
What to do about SQL hard coding
• Of course, you have to (and should) write SQL statements in your PL/SQL code.
– PL/SQL is, in fact, the best place for SQL.
• But we should be very careful about where, when and how we write these statements.
– Follow the principles; they are your guide.
– Don't repeat anything!
• The best approach: hide SQL statements inside a data access layer.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 114
SQL as a Service
• Think of SQL as a service that is provided to you, not something you write.
– Or if you write it, you put it somewhere so that it can be easily found, reused, and maintained.
This service consists of views and programs defined in the data access layer.
– Views hide complex query construction
– Packaged APIs – for tables, transactions and business entities
Order
Table
Item
Table
Application
Code
Intermediate Layer
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 115
With a data access layer, I can...
• Change/improve my implementation with minimal impact on my application code. – The underlying data model is constantly changing.
– We can depend on Oracle to add new features.
– We learn new ways to take advantage of PL/SQL.
• Vastly improve my SQL-related error handling. – Do you handle dup_val_on_index for INSERTs,
too_many_rows for SELECT INTOs, etc?
• Greatly increase my productivity – I want to spend as much time as possible implementing
business requirements.
11g_frc_demo.sql 11g_emplu.*
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 116
How to implement data encapsulation • It must be very consistent, well-designed and
efficient - or it will not be used.
• Best solution: generate as much of the code as possible.
– This includes products like APEX and Hibernate that generate lots of their own SQL for you.
• Any custom SQL statements should be written once and placed in a shareable container (usually a package, but also views).
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 117
Qualify every column and identifier in the SQL statement. • Improves readability.
• Avoids potential bugs when variable names match column names.
• Minimizes invalidation of dependent program units in Oracle11g.
PROCEDURE abc (...) IS BEGIN SELECT last_name INTO l_name FROM employees WHERE employee_id = employee_id_in;
PROCEDURE abc (...) IS BEGIN SELECT e.last_name INTO l_name FROM employees e WHERE e.employee_id = abc.employee_id_in;
Instead of this.... Write this....
11g_fgd*.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 118
Dot-qualify all references to Oracle-supplied objects with "SYS." • Another annoying, but incontestable
recommendation.
• If you don't prefix calls to all supplied packages with "SYS.", you are more vulnerable to injection.
– More details available in "Best Practices for Dynamic SQL" in the "Dynamic SQL in PL/SQL" series.
BEGIN run_dynamic_plsql_block (append_this_in => 'employee_id=101; EXECUTE IMMEDIATE ''CREATE OR REPLACE PACKAGE DBMS_OUTPUT ... ''' ); END;
code_injection.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 119
Best Practices for Dynamic SQL
• Stored programs with dynamic SQL should be defined as AUTHID CURRENT_USER.
• Remember that dynamic DDL causes an implicit commit. – Consider making all DDL programs autonomous
transactions.
• Always EXECUTE IMMEDIATE a variable, so that you can then display/log/view that variable's value in case of an error.
• Avoid concatenation; bind whenever possible. dropwhatever.sp
usebinding.sp
toomuchbinding.sp
useconcat*.*
ultrabind.*
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 120
Conclusions • SQL statements are among the most critical
parts of your application. – They change frequently, they consume lots of
resources, result in many errors.
• You should have a clearly defined set of guidelines about when, where and how to write SQL.
• Most important: Don't repeat SQL statements. – Most good practices will follow more easily if you
manage to avoid SQL hard-coding.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 121
Never Repeat Anything - Conclusions
• Repeat after me: Everything is going to change.
• When you hide the mechanics, how you get things done, behind a procedure or function, you are "liberated."
– Change the implementation, and you don't need to change all the places in which it is used.
• Back to that same principle:
Never Repeat Anything.
Aim for Single Point of Definition.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 122
Program Socially
• Never code alone.
– Writing code by yourself results in much buggier code than if you had someone with whom to consult; to ask questions; to help, in turn.
– It's OK to ask for help.
– Follow the 30 minute rule.
• Automated code review in PL/SQL
– Compile-time warnings
– Leverage the data dictionary views
– PL/Scope
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 123
Don't be afraid to ask for help.
• Our evolved survival instinct urges us to hide weakness.
• On top of that, we software developers are supposed to be really smart. – We are the wizards of modern society.
• Unfortunately, ignorance leads directly to bugs and sub-optimal code.
"Predators look for signs of illness or weakness when choosing their prey, so a prey animal needs to appear healthy, or it will be a sure target. By the time they are showing signs of disease, in many instances, the birds have become too weak to be able to disguise it." - From peteducation.com
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 124
It's OK to say "I don't know. Help me!"
• Just thinking about asking for help will often do the trick.
• Most people like to be asked to help. – It makes them feel valued. – It strengthens the team as a whole.
• It may not really matter who you ask for help. – If there are no programmers handy, ask your spouse
or parent or child to be a sounding board. – Or write an email. By the time you finish writing it,
you will likely have found the answer to your problem.
– The important thing is to get the issue out of your head.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 125
Code Review - Automated and Otherwise
• Even if you write your code mostly by yourself, it's extremely important for someone or something to check that code. – Does it make sense? Does it follow standards? Is
there a better way to do it?
• Peer code review - in a group or one-on-one is very helpful, but often intimidating.
• Automated code review is less personal, might catch many issues the "eyeball" might miss. – Compile time warnings, DD views, PL/Scope
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 126
Follow the Thirty Minute Rule
• We are usually too deeply inside (and part of) the problem to step back and take a fresh look.
• If you can't fix a bug in 30 minutes, ask for help.
– For "trivial" bugs, "give up" after just a few minutes!
• Senior developers and managers must take the lead.
– Ask more junior members for help. Show that you are fallible, that you can learn from anyone and everyone.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 127
Warnings help you build better code
• Your code compiles without errors. Great, you can run that program!
• But does it use the PL/SQL language optimally?
• In Oracle 10g, Oracle added a compile-time warnings framework. – Automatically informs you of ways to improve the
quality or performance of your code.
• All warnings shown in Error Messages manual, with the PLW prefix.
http://docs.oracle.com
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 128
Enable and Disable Warnings
• To use compiler warnings, you must turn them on for session or for a particular program unit.
– By default, warnings are disabled.
• Can specify individual warnings or categories.
ALTER SESSION [ENABLE | DISABLE |ERROR]: [ALL|SEVERE|INFORMATIONAL|PERFORMANCE|warning_number] REM To enable all warnings in your session: ALTER SESSION SET plsql_warnings = 'enable:all‘; REM If you want to enable warning message number 06002 and all warnings in REM the performance category, and treat 5005 as a "hard" compile error: ALTER PROCEDURE my_procedure SET plsql_warnings = 'enable:06002', 'enable:performance', 'ERROR:05005';
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 129
Checking for Warnings
• The USER_ERRORS data dictionary view shows both "hard" errors and compilation warnings.
• Use the SHOW ERRORS command in SQL*Plus.
• IDEs will usually display warnings within the edit window.
• Or run your own query against USER_ERRORS.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 130
Example: check for unreachable code • There may be lines of code that could never, ever
execute.
SQL> CREATE OR REPLACE PROCEDURE unreachable_code IS 2 x NUMBER := 10; 3 BEGIN 4 IF x = 10 THEN 5 x := 20; 6 ELSE 7 x := 100; -- unreachable code 8 END IF; 9 END unreachable_code; 10 / SP2-0804: Procedure created with compilation warnings SQL> show err Errors for PROCEDURE UNREACHABLE_CODE: LINE/COL ERROR -------- ------------------------------------- 7/7 PLW-06002: Unreachable code
plw6002.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 131
Finally, Oracle warns me of too-large value
• One big frustration I have had with compile-time warnings is that it did not flag code like you see above. What could be more basic?
• This is finally addressed – sort of – in Oracle11g with the PLW-06017 warning.
CREATE OR REPLACE PROCEDURE plw6017 IS c VARCHAR2 (1) := 'abc'; BEGIN
plw6017.sql
PLW-06017: an operation will raise an exception
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 132
New compile-time warnings in Oracle11g
• PLW-6009: Exception handler does not re-raise an exception. – Doesn't recognize when a subprogram does the raise.
• PLW-7205: warning on mixed use of integer types – Namely, SIMPLE_INTEGER mixed with PLS_INTEGER
and BINARY_INTEGER
• PLW-7206: unnecessary assignments – My own warning: I can't get this warning to "fire"!
• Lots of PRAGMA INLINE-related warnings • More feedback on impact of optimization
– PLW-6007: Notification that entire subprograms were removed
plw*.sql files
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 133
Treating a warning as "hard" compile error
• You might identify a warning that reflects such bad coding practices, that you want to ensure it never makes its way into production code.
– Just set the warning as an error and stop the use of that program "in its tracks."
• "Function does not return value" is a prime example.
plw5005.sql
ALTER SESSION SET PLSQL_WARNINGS='ERROR:5005' /
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 134
Watch out for "false negatives" and "nuisances" warnings
• The check for unreachable code is not very useful prior to Oracle11g.
– Shows as unreachable code that is removed by the optimizer.
• You might be overwhelmed by warnings about which you don't really care.
– Show us "missing AUTHID clause".
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 135
Code analysis with data dictionary views
• Oracle's data dictionary provides access to many views containing information about our stored program units. With them we can...
– Analyze objects defined in the database
– Analyze source code for contents and patterns
– Analyze program unit structure and header
– Check compile-time settings of program units
• With a good set of scripts you can easily and productively analyze your code.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 136
Analyzing source code
• ALL_SOURCE – Write queries against source code to identify
violations of coding standards.
– Which programs contain/exclude particular strings?
• Use with other data dictionary views and utilities that reference source code. – DBMS_UTILITY.FORMAT_CALL_STACK
– Profiler data
• ALL_IDENTIFIERS (Oracle11g) –PL/Scope – Analyze all references to identifiers (named
elements) – covered in separate lesson.
all_source.sql valstds.pks/pkb
package_analyzer.pks/pkb
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 137
Analyzing program unit structure/header
• Source code is handy, but also "freeform" text. – The more structured the data, the better.
• ALL_PROCEDURES – Information about every subprogram you can execute
– Missing some information (the type of subprogram)
• ALL_ARGUMENTS – Information about every argument of every subprogram you can
execute
– Rich resource of information, not that well designed.
– Can use it to figure out type of subprogram
– DBMS_DESCRIBE offers another access path to more or less the same data
all_arguments.sql
show_all_arguments*.*
show_procs_with_parm_types.sql
is_function.sf
show_authid.sql
show_deterministic.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 138
Compile time settings for program units
• ALL_PLSQL_OBJECT_SETTINGS
• Stores information about compile-time characteristics of program units.
– Optimization level
– Code type: NATIVE or INTERPRETED
– Debug settings
– Compile-time warnings
– Conditional compilation flags
– PL/Scope settings
whats_not_optimal.sql show_non_default_object_settings.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 139
PL/Scope: powerful code analysis tool
• A compiler-driven tool that collects information about identifiers and stores it in data dictionary views. – Introduced in Oracle Database 11g
• Use PL/Scope to answer questions like: – Where is a variable assigned a value in a program?
– What variables are declared inside a given program?
– Which programs call another program (that is, you can get down to a subprogram in a package)?
– Find the type of a variable from its declaration.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 140
Getting Started with PL/Scope
• PL/Scope must be enabled; it is off by default.
• When your program is compiled, information about all identifiers are written to the ALL_IDENTIFIERS view.
• You then query the contents of the view to get information about your code.
• Check the ALL_PLSQL_OBJECT_SETTINGS view for the PL/Scope setting of a particular program unit.
ALTER SESSION SET plscope_settings='IDENTIFIERS:ALL'
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 141
Key Columns in ALL_IDENTIFIERS • TYPE
– The type of identifier (VARIABLE, CONSTANT, etc.)
• USAGE – The way the identifier is used (DECLARATION,
ASSIGNMENT, etc.)
• LINE and COL – Line and column within line in which the identifier is found
• SIGNATURE – Unique value for an identifier. Especially helpful when
distinguishing between overloadings of a subprogram or "connecting" subprogram declarations in package with definition in package body.
• USAGE_ID and USAGE_CONTEXT_ID – Reveal hierarchy of identifiers in a program unit
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 142
Start with some simple examples
• Show all the identifiers in a program unit
• Show all variables declared in a subprogram (not at package level)
• Show all variables declared in the package specifications
• Show the locations where a variable could be modified
plscope_demo_setup.sql plscope_all_idents.sql
plscope_var_declares.sql plscope_gvar_declares.sql
plscope_var_changes.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 143
More advanced examples
• Find exceptions that are defined but never raised
• Show the hierarchy of identifiers in a program unit
• Validate naming conventions with PL/Scope
plscope_unused_exceptions.sql plscope_hierarchy.sql
plscope_naming_conventions.sql
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 144
PL/Scope Helper Utilities
• Clearly, "data mining" in ALL_IDENTIFIERS can get very complicated.
• Suggestions for putting PL/Scope to use:
– Build views to hide some of the complexity.
– Build packages to provide high-level subprograms to perform specific actions.
plscope_helper_setup.sql plscope_helper.pkg
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 145
Break Down Coding Isolation
• Ask for help, and offer help to others.
• Participate willingly in code review.
– Your code and your skills will both benefit.
• Automate code evaluation as much as possible.
– It's always been possible with the DD views, but now with compile-time warnings and PL/Scope, the quality of feedback is much higher.
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 146
Make the Most of the Best of Oracle PL/SQL!
• This language is not evolving very rapidly these days (less change than in SQL).
• Make sure that you are aware of key new (and existing) features, and put them to use.
• Always prioritize the maintainability of your code.
– It's going to be around for YEARS to come!
Oracle PL/SQL Programming
Copyright 2013 Feuerstein and Associates Page 147
Websites for PL/SQL Developers
www.plsqlchallenge.com Daily PL/SQL quiz with weekly and monthly prizes
www.plsqlchannel.com 27+ hours of detailed video training on Oracle PL/SQL
www.stevenfeuerstein.com Monthly PL/SQL newsletter
www.toadworld.com/SF Quest Software-sponsored portal for PL/SQL developers