Date Ranges:Time for Oracle 12c SQL

"Ask not what Oracle can do for you,ask what you can do with Oracle."

Stew Ashton 2.0 UKOUG Tech 15 Stew ASHTONUKOUG Tech 14

2

Agenda

• Who am I?• Date/time data types• Date range concepts• 12c Temporal Validity• Date range DDL• Date range queries

3

Who am I?• 35 years as Developer / Technical Architect

– Aeronautics, IBM, Finance– Mainframe, client-server, Web apps

• 27 years as an American in Paris• 10 years using Oracle database

– Performance analysis– Replace Java with SQL

• 3 years as internal "Oracle Development Expert"• "Career 2.0" started last week

4

Review of date/time datatypes

• There is no "date" type: always a time element– Closest thing to a date: dte DATE CHECK(dte = TRUNC(dte))

• There are no "formats"• Values from 4712-01-01 00:00:00 BC through 9999-12-31 23:59:59 AD• Any of the above can be used in ranges

Datatype Year Month Day Hour Minute Second Fraction Time zoneDate Y Y Y Y Y Y

Timestamp Y Y Y Y Y Y Y Timestamp with local time zone Y Y Y Y Y Y Y implicit

Timestamp with time zone Y Y Y Y Y Y Y explicit

5

Date/time Ranges• Use same data_type for start and end (duh!)• To NULL or not to NULL?

– SQL:2011 and 12c allow– Makes NULL mean something– Accounting for NULLs complicates queries and index design– Alternative: extreme values such as DATE '9999-21-31'

• "Start" is included in the range, "end" is not– Works for date/time ranges, not just dates– See SQL:2011 standard, 12c Temporal Validity– Allows ranges to "meet"– SQL can't use BETWEEN "start" and "end"

6

Allen’sTime Interval

Arithmetic

Day of month 1 2 3 4A precedes B 1 2

B preceded by A 3 4A meets B 1 2 B met by A 2 3

A overlaps B 1 3B overlapped by A 2 4

A finished by B 1 3B finishes A 2 3A contains B 1 4B during A 2 3A starts B 1 2

B started by A 1 3A equals B 1 2 B equals A 1 2

Meet

Gap

"Overlap"

Day of month 1 2 3 4Day of month 1 2 3 4A precedes B 1 2

B preceded by A 3 4

Day of month 1 2 3 4A precedes B 1 2

B preceded by A 3 4A meets B 1 2 B met by A 2 3

7

Temporal Validity: SQL:2011, 12c • Two temporal concepts

– "Transaction time": when data committed (flashback)• "Oracle Flashback" Tues. 11:20 (Connor McDonald)

– "Valid time":• start / end times determined by users• Past, present or future

• "Period": date/time range– Closed-Open: includes start time but not end time– Constraint: start time < end time– NULL start time = valid any time before end– NULL end time = valid from start time on

8

What Oracle can do for you

create table valid_emp( ename varchar2(64), PERIOD FOR PRESENCE);

9

select column_name, data_type, nullable,hidden_column, virtual_columnfrom user_tab_cols where table_name = 'VALID_EMP'order by internal_column_id;

Name Data Type Nullable Hidden VirtualPRESENCE_START TIMESTAMP(6) WITH TIME ZONE Y YES NOPRESENCE_END TIMESTAMP(6) WITH TIME ZONE Y YES NO

PRESENCE NUMBER Y YES YESENAME VARCHAR2 Y NO NO

select search_condition from user_constraintswhere table_name = 'VALID_EMP';

SEARCH_CONDITIONPRESENCE_START < PRESENCE_END

10

insert allinto valid_emp (ename, presence_start, presence_end) values('Stew', date '1998-03-01', date '2015-12-01')into valid_emp (ename, presence_start, presence_end) values('Stew', date '2016-01-01', date '2050-12-01')select null from dual;

select * from valid_emp;

ENAME

Stew

StewOops! Hidden columns…

11

select ename, presence_start, presence_endfrom valid_emp;

ENAME PRESENCE_START PRESENCE_END

Stew 1998-03-01 2015-12-01

Stew 2016-01-01 2050-12-01

12

select ename, presence_start, presence_endfrom valid_empas of period for presence systimestamp;

ENAME PRESENCE_START PRESENCE_END

select * from table(dbms_xplan.display_cursor(format=>'+PREDICATE'));

filter(((T.PRESENCE_START IS NULL OR SYS_EXTRACT_UTC(T.PRESENCE_START)<=SYS_EXTRACT_UTC(SYSTIMESTAMP(6))) AND(T.PRESENCE_END IS NULL OR SYS_EXTRACT_UTC(T.PRESENCE_END) > SYS_EXTRACT_UTC(SYSTIMESTAMP(6)))))

13

select ename, presence_start, presence_endfrom valid_empas of period for presence to_timestamp_tz('2015-11-01');

exec DBMS_FLASHBACK_ARCHIVE.ENABLE_AT_VALID_TIME ('CURRENT');select ename, presence_start, presence_endfrom valid_emp;

ENAME PRESENCE_START PRESENCE_END

Stew 1998-03-01 2015-12-01

ENAME PRESENCE_START PRESENCE_END

14

Valid Time Rangeselect ename, presence_start, presence_endfrom valid_empversions period for presence between to_timestamp_tz('2015-11-01') and to_timestamp_tz('2016-01-01');

ENAME PRESENCE_START PRESENCE_END

Stew 1998-03-01 2015-12-01

Stew 2016-01-01 2050-12-01

15

• So what did Oracle do for us?– Notion of "period"– Automatic, hidden column definitions– New query syntax with automatic rewrite

• And what did Oracle not do for us?– Query performance?– Temporal constraints?– Query for gaps or overlaps?– Temporal DML?

• Now: what can we do with Oracle?

16

What we need to do:

• Temporal constraints• Performance: indexing• Temporal queries• Temporal DML

17

Temporal Constraints• No gaps, no overlaps

– Don't use ranges!– "effective date"

• "start" = EFF_DATE , "end" = lead(EFF_DATE) over (EFF_DATE)

• Temporal primary keys– Use ID + start_date (usually)– ORA-02329: …TIMESTAMP WITH TIME ZONE cannot be unique or primary key

• Temporal foreign keys– Child range must be within parent range– "On commit" materialized views (hard to scale)– Triggers (hard to write correctly)

18

Temporal DDL & Indexingcreate table valid_emp( ename varchar2(64), presence_start date, presence_end date not null, CHECK(presence_start < presence_end), period for presence(presence_start, presence_end), CONSTRAINT valid_emp_pk primary key (ename, presence_start) USING INDEX ( CREATE INDEX valid_emp_pk ON valid_emp (ename, presence_start, presence_end) ));

19

Temporal Queries• Use Analytic functions

or 12c MATCH_RECOGNIZE• Typical use cases– find gaps: "free time" in calendars– Merge ranges that "meet"– Merge ranges that "meet" or "overlap"– Join on intersecting date ranges

• For simplicity, I assume "not NULL"

20

Find GapsSTART_DATE END_DATE ---------- ----------2007-01-12 2007-01-252007-01-20 2007-02-012007-02-05 2007-02-102007-02-05 2007-02-282007-02-10 2007-02-152007-03-01 2007-03-022007-03-03 2007-03-16

SELECT end_date start_gap, LEAD(start_date) OVER (ORDER BY start_date) end_gap FROM T;

START_GAP END_GAP ---------- ----------2007-01-25 2007-01-202007-02-01 2007-02-052007-02-10 2007-02-052007-02-28 2007-02-102007-02-15 2007-03-012007-03-02 2007-03-032007-03-16

21

Find GapsSTART_DATE END_DATE ---------- ----------2007-01-12 2007-01-252007-01-20 2007-02-012007-02-05 2007-02-102007-02-05 2007-02-282007-02-10 2007-02-152007-03-01 2007-03-022007-03-03 2007-03-16

SELECT * FROM ( SELECT MAX(end_date) OVER (ORDER BY start_date) start_gap, LEAD(start_date) OVER (ORDER BY start_date) end_gap FROM T)WHERE start_gap < end_gap;

START_GAP END_GAP ---------- ----------2007-01-25 2007-01-202007-02-01 2007-02-052007-02-28 2007-02-052007-02-28 2007-02-102007-02-28 2007-03-012007-03-02 2007-03-032007-03-16

22

PACK: merge ranges that meetID START END1 01-01 01-032 01-03 01-053 02-05 02-284 02-10 02-155 03-01 03-116 03-10 03-127 03-11 03-21

select * from tmatch_recognize( order by start_date, end_date measures first(id) "first", last(id) "last", first(start_date) "start", last(end_date) "end" pattern(A* B) define A as end_date = next(start_date));

first last start end ----- ---- ----- ----- 1 2 01-01 01-05 3 3 02-05 02-28 4 4 02-10 02-15 5 5 03-01 03-11 6 6 03-10 03-12 7 7 03-11 03-21

overlap

intervening

23

Merge ranges that meet or overlapID START END1 01-01 01-032 01-03 01-053 02-05 02-284 02-10 02-155 03-01 03-116 03-10 03-127 03-11 03-21

select * from tmatch_recognize( order by start_date, end_date measures first(id) "first", last(id) "last", first(start_date) "start", MAX(end_date) "end" pattern(A* B) define A as MAX(end_date) >= next(start_date));

first last start end ----- ---- ----- ----- 1 2 01-01 01-05 3 4 02-05 02-28 5 7 03-01 03-21

gap

gap

24

Join on intersecting ranges

• Prerequisite for temporal DML– Join input to table to find affected rows

• Use UNION ALL to gather input and affected rows• Use UNPIVOT and DISTINCT to get range boundaries• Use LEAD() to create base ranges• DO LEFT JOIN from base ranges to table + input.

1 3 5 7

2 4

St12345

StEnd1 22 33 44 5

St End OldNew

1 2 1-32 3 1-3

2-43 4 3-5

2-44 5 3-5

25

Questions?

More details at:stewashton.wordpress.com