Supporting Reporting projects with...

2012-2019 - Rick Flagler IT Consulting, LLC

Supporting Reporting projects with Db2

1

Rick Flagler IT Consulting, LLC Richmond, NH (M) 603-831-6873 (H) 603-239-6330 [email protected]

This session will review DB2 capabilities we have to massage data, create reports and support Business Intelligence (BI). We’ll look at various Db2/SQL techniques, with advantages of each and with an eye to performance impacts. On the BI front, a consideration of many vendors with focus on IBM’s Db2 Web Query for i as a presentation layer. Differentiating the Express and Standard Edition offerings.

This presentation illustrates SQL features commonly used to facilitate reporting, data warehousing and Business Intelligence. Attendees will see numerous examples of features that can reduce HLL programming or could be used in conjunction with any BI tool of choice. Some examples demonstrated will utilize IBM Db2 Web Query for i - Express Edition (the less expensive version) for presentation of query results.

Db2 Web Query for i is a product developed by IBM in collaboration with Information Builders Inc. and targets the IBM i platform for BI, DW and ETL.


• Understand WHAT you are reporting (key metrics)

• Determine HOW to gather data

• Decide HOW to summarize

• PLAN to achieve high-performance

• RECOMMENDATION - Obtain Executive Sponsorship when launching a BI or DW initiative.

Important aspects of Business Intelligence & Reporting

Db2 contains multiple techniques to facilitate your efforts

Regardless of tooling, better

answer these

!2


• Views - rearranging data to present

• CTEs - Common Table expressions - make life easier When creating views. Reduce complexity. Also can be used recursively, when processing tables containing Bill-of-materials or other hierarchies.

• Use of SUBSTR, CHAR, DEC, Calculations to create or modify database fields

• Indexes - keys to performance

• OLAP - on-line analytical processing

• Ranking, summarizing

•MQTs - summarizing data to present

• An interesting cross between an index and a view

• Procedures - Encapsulating logic

• utilize parameters or apply important logic to your data

• returning result(s)

Tools in Db2 to facilitate reporting

!3


Views - What they enable•Rearrange or select data

•columns - pick only columns that matter

•where/having clauses - limit which records you return

•Convert 6-digit date numbers to DATE data types

•Extract YEAR, MONTH,DAY from dates

•Create/Extend values that are NOT present in underlying tables

•SALES = QTY * PRICE

•DAYSTILDUE = DAYS(DueDate) - DAYS(InvoiceDate)

•Summarize data into logic groupings

•Group by SUM, AVG, MIN, MAX, more…

•Join files to denormalize for presentation and reporting

•Add descriptions for CODE fields

Reduce number of

records / fields to improve

performance

!4


Basic View Create Syntax Create View {library}.MYTEST as ( SELECT Country, Region, state, storename, productnumber, linetotal AS Revenue FROM qwqcent . orders oJOIN qwqcent . stores s ON s.storecode = o.storecode WHERE country = 'United States' and region in ('New England','North' )ORDER BY Country, Region, state, storename) rcdfmt MYTESTR;

5

Test your SELECT then wrap with Create View. Views have NO

overhead. View can’t contain an ORDER BY clause, but you can add

when using.

select * from MYTEST order by Country, region, state, store name

Possible View uses:

select Region, sum(Revenue) from MYTEST group by Region order by Region DESC

RUNQRY MYTEST

Use in RPG program, where RCDFMT clause is important


Simple CTE example Break logic apart using CTEs, to make complex queries easier to understand

WITH base_data AS ( SELECT Country, Region, state, storename, productnumber, linetotal AS Revenue FROM qwqcent . orders oJOIN qwqcent . stores s ON s.storecode = o.storecode WHERE country = 'United States' and region in ('New England','North' )ORDER BY Country, Region, state, storename) select * from base_data;

Several temporary results can be created with format:

With T1 as ( statement1 ), T2 as ( statement2 ), Tn as ( statements ) select * from T1, T2, …Tn …..;

32 statements maximum

?

6

Intermediate CTE results (Temp Tables) may be used in combination with others or with actual database

Tables or Views. More complex examples to follow….


Db2 Index Advice Easy way to improve performance

1

2 3

Db2 says you might benefit from

an index and number of times advised so far

7

1. locate table 2. Select index advisor 3. Show SQL or Create


Index Advice details for a specific Table Sorted by Times Advised Descending

Sort by any column with a click. Number of times advised in

DESCENDING sequence may help identify repeat offenders!

!8


Want the Index Creation syntax specifics? use Show SQL

Right-Click an advised Table line to create or show the SQL needed to

create the index.

!9


Index recommendations•Don’t simply create any index that is advised

•Understand the size of files and number of times advised before taking action

•Watch advised indexes and look for repeat offenders, sort by various columns

•When creating indexes recommend you save the SQL code in an SQL source member for later rebuilds, if required.

•Beware of too many indexes - similar to LFs, more isn’t always better, since system has to maintain any index you create..

•Yes, you can use a CREATE INDEX before the beginning of a batch run, and DROP INDEX after completion, to get benefits of the index without it being permanent. Beware of file size when doing this.

•Db2 will create some temporary indexes for you, over time, but when you IPL, these may go away. If you like it, create it permanently.

More isn’t always better

!10


OLAPOn-line Analytical

Processing

Online analytical processing (OLAP) specifications provide the ability to return ranking, row numbering, and other aggregate function information as a scalar value in a query result.

OK, but what does that really mean?

!11


Commonly used OLAP functions for IBM i

• RANK( )

• DENSE_RANK ( )

• ROW_NUMBER ( )

!12


SQL Syntax for OLAP functions for IBM iwith base_data as ( select Region, state, storename, sum(linetotal) as Revenue from qwqcent . orders o join qwqcent . stores s on s.storecode=o.storecode group by Region, state, storenameunion all select Region, state, storename, sum(linetotal) as Revenue from qwqcent . orders o join qwqcent . stores s on s.storecode=o.storecode where Region = 'New England' group by Region, state, storenameorder by Region, state, storename)SELECT REGION, Storename, Revenue, RANK() OVER(ORDER BY Revenue DESC) as RANK, DENSE_RANK() OVER(ORDER BY Revenue DESC)as D_RANK, ROW_NUMBER() OVER(ORDER BY Revenue DESC) as ROW_NUM FROM base_data FETCH FIRST 10 ROWS ONLY; Useful for Top-N

reporting

!13


Rank, Dense_Rank, Row_Number

Two lines the same; Differing function

behaviorRow_Number - Numbered 1

to 10

Dense_Rank - no skipsRank -

duplicate causes two 4’s,

skipped 5

!14


SQL Syntax for DENSE_RANK with Partition -- Ranking within a group (region) with base_data as ( select Region, state, storename, sum(linetotal) as Revenue from qwqcent . orders o join qwqcent . stores s on s.storecode=o.storecode group by Region, state, storename order by Region, state, storename)SELECT region, state, storename, revenue, DENSE_RANK() OVER(PARTITION BY REGION ORDER BY REVENUE DESC) AS REV_RANK_IN_REG— DENSE_RANK() OVER(PARTITION BY State ORDER BY REVENUE DESC) AS REV_RANK_IN_St FROM base_datawhere Region = 'New England' ;

Partition for Ranking within a group

Might switch between one and the other? In a procedure, a

parameter could control whether ST or REG is used? Or, return both and

populate two metrics…

!15


Ranking within a Group Using Partition

Region all the same, ranked 1 to

6.

States differ, so ranked within each.

}}

}

States differ, so ranked within each.

!16


Adding Subtotals and Totals Grouping and With Rollup

-- Adding subtotals and totals using Grouping and with Rollup WITH base_data AS (SELECT Country, Region, state, storename, linetotal AS Revenue FROM qwqcent . orders o JOIN qwqcent . stores s ON s.storecode = o.storecode where country in ('Canada' , 'United States') ORDER BY Country, Region, state, storename) SELECT ROW_NUMBER() OVER(ORDER BY Country, Region, state, storename) AS ROW_ID, Country, Region, state, storename, Sum(Revenue) as Rev, GROUPING(Country) AS GrandLvl from base_data GROUP BY Country, Region, state, storename WITH ROLLUP;

WITH ROLLUP for Reporting Subtotals

Column GrandLvl will be 0 or 1 depending upon

whether on a grand total line

WITH ROLLUP adds subtotal and total lines

!17


GROUPING and WITH ROLLUP

Subtotals inline

Subtotal or Total indicator field ….

……Some lines omitted

!18


Materialized Query Table (MQT)• Useful for:

• Data Warehouses (DW)

• Business Intelligence (BI)

• Staging data for some transfer

• Facilitates performance improvements (improved response) for complex queries. Particularly those featuring:

• Aggregated (summarized) data

• Joined data with several tables

• Commonly accessed data

!19


Materialized Query Table (MQT) • Rick’s definition - IBM method to define a table often based on other table(s),

with indexes or views over it, to potentially summarize, refreshed periodically.

• Excerpt of IBM’s definition - A materialized query table (MQT) is a table whose definition is based upon the result of a query. The data that is contained in an MQT is derived from one or more tables on which the materialized query table definition is based. You can think of an MQT as a kind of materialized view. Both views and MQTs are defined on the basis of a query. The query on which a view is based is run whenever the view is referenced; however, an MQT actually stores the query results as data, and you can work with the data that is in the MQT instead of the data that is in the underlying tables. Materialized query tables can significantly improve the performance of queries, especially complex queries. If the optimizer determines that a query or part of a query could be resolved using an MQT, the query might be rewritten to take advantage of the MQT. Source IBM Developer Article 2005

!20


Though MQTs can be directly specified in a user’s query, their real power comes from the query optimizer’s ability to recognize the existence of an appropriate MQT implicitly, and to rewrite the user’s query to use that MQT. The query accesses the MQT (instead of accessing one or more of the specified tables). This shortcut can drastically minimize the amount of data read and processed (see Figure 1).

!21

Source IBM - MQT documentation


Cool things about MQTs• Created once, used repeatedly

• Maintained by System or by user

• Can be refreshed (or rebuilt) by the SQL command

• REFRESH TABLE{table name}

• Indexes over MQTs are not removed when refreshed

• Views over MQTs are not destroyed when table is refreshed

• Performance of MQTs vs. other Query techniques can produce significant improvement (i.e. reduce response time)

!22


A full source MQT Example

!23

CREATE TABLE {library} . ORDMQT AS ( WITH base_data AS ( SELECT ordernumber, orderdate, requestedshipdate, shipdate, invoicedate, o.storecode, storename, plantcode, salesrep, quantity, linetotal, costofgoodssold, returns, warrantyexp, shippingcost, country, region, state, city, o.productnumber, producttype, productcategory, productname, model FROM qwqcent . orders o JOIN qwqcent . stores s ON s.storecode = o.storecode JOIN qwqcent . inventory p on o.productnumber = p.productnumber where country in ('Canada' , 'United States') --and region in (/*'Mountain',*/'New England', 'North') ORDER BY Country, Region, state, storename ) SELECT bd.*, quantity*linetotal as ExtRev,quantity*costofgoodssold as Extcost, (quantity*linetotal) - (quantity*costofgoodssold) as Extprofit, days(shipdate) - days(requestedshipdate) as OTD, case when days(shipdate) - days(requestedshipdate) < -9 then 'Very Early' when days(shipdate) - days(requestedshipdate) between -9 and -2 then '1Wk Early' when days (shipdate) - days(requestedshipdate) between -1 and +1 then 'On-time' when days (shipdate) - days(requestedshipdate) between +2 and +9 then '1Wk Late' else 'Very Late' end as OTD_Sts, case when days(shipdate) - days(requestedshipdate) < -9 then '1' when days(shipdate) - days(requestedshipdate) between -9 and -2 then '2' when days (shipdate) - days(requestedshipdate) between -1 and +1 then '3' when days (shipdate) - days(requestedshipdate) between +2 and +9 then '4' else '5' end as OTD_PER from base_data bd ORDER BY Country, Region, state, storename ) DATA INITIALLY DEFERRED

REFRESH DEFERRED MAINTAINED BY USER ENABLE QUERY OPTIMIZATION RCDFMT OrdMQTR ;

REFRESH used after 1st CREATE to load data


Refreshing the MQT to pick up data changes to underlying

tables

!24

REFRESH table {library}. ordmqt;

REFRESH causes the MQT query to be run to refresh data in the table

Might be employed where IMMEDIATELY current data is

not required so, for example, a DAILY refresh is OK.


Example-1 Query using MQT

!25

with a as ( select year(shipdate) as TOTYR, count(*) as totcnt from Ordmqt group by year(shipdate) ) select year(shipdate) as YR, OTD_PER, otd_sts, count(*) NbrShips, totcnt, -- decimal percent rounded to 2-places makes 100% dec(round(((count(*)*100.00)/totcnt),2),5,2) as OTD_PCT from Ordmqt join a on year(shipdate)=TOTyr group by TOTCNT, otd_sts, otd_per, year(shipdate) ORDER BY YEAR(SHIPDATE) DESC, OTD_PER;


Example-2 Query using MQT

!26

with a as ( select year(shipdate) as YR, productname, count(*) as totcnt from Ordmqt group by productname,year(shipdate) ) select year(om.shipdate) as YR, om.productname, OTD_PER, otd_sts, count(*)Nbrships, totcnt, -- decimal percent rounded to 2-places makes 100% dec(round(((count(*)*100.00)/totcnt),2),5,2) as OTD_PCT from Ordmqt om join a on om.productname=a.productname and year(om.shipdate)=a.YR group by TOTCNT, otd_sts, otd_per, om.productname,year(om.shipdate) ORDER BY YR,productname, OTD_PER;


MQT creation actions• Create the MQT

• Initial Refresh, then schedule Refresh actions as desired

• run queries against the MQT data

• Add indexes or Views to support various Queries you run or anticipate

• Examine Index Advisor for table periodically

!27


Business Intelligence

!28

Every vendor will say: • Their product is best • Try it for free • Get your BI/DW project started in 90-minutes • We can do it in the Cloud, etc. etc.

Not sure ALL claims are true but reporting clearly isn’t as hard as it used to be!!!

Probably two big players with “Native” products on IBM i: IBM - Db2 Web Query for i - $ or $$ Information Builders - Webfocus - $$$ or $$$$

Many vendors, Many price points


Business Intelligence

!29

Chart at right -> Sourced from:


Why use BI tools on top of Db2 for i?• Db2 is best in class database • Your data is already there in Db2, why do BI elsewhere? • BI tools offer more sophistication than QUERY/400 product• Present data visually - “A picture is worth 1000 words!”• Modern tools are designed for Self-service by users rather than by

programmers• Typically “data setup” is required by a DBA, analyst or QPGMRs• The tools CODE many features for you, often automating the

SQL and creating intermediate or final totals• Enables easy Pivot and Drill-Down of data spanning Time,

Geography or Family• OLAP features enable DRILL-DOWN from one level to another

• Free up programmers for high-value tasks rather than saddle with report programming

!30

2012-2019 - Rick Flagler IT Consulting, LLC2012-2019 - Rick Flagler IT Consulting, LLC

TYPICAL BI ENVIRONMENT

31

IBM POWER System Hardware

Db2 DatabaseWeb/Http Server

AR SALES IT PURCH FIN MFG

Enterprise Data

Reports written using BI

Other BI tool

OPER

IBM Db2 Web Query for i

Business Functional Areas

For IBM i


Features most BI tools offer•Widgets in the form of Pie, Bar, Surface, Area, Line Charts,

Geographic Mapping and other report types

•Development using WSYIWYG designer tools, Browser or Client based

•MetaData which enables you to enhance what Db2 tables have without touching the Db2 tables. Examples:

• Rename fields to more meaningful names

• Convert legacy formatted dates to DATE data types

• Join files so users don’t have to worry about bad joins

•Ability to pass parameters between reports

•Drill-down linkages between related subject matter

•A repository of stored reports with analysis about usage

•Security features to implement allowed viewing !32


Report typesLine - Trend Pie - current breakdown Metric - Indicator

Bar - ComparisonReport - Expandable

Report - Traffic Lighting

Map - Geographic penetration


TYPICAL CHARTS/GRAPHS

34

Bars - Compare one group to anotherPie - Compare size/magnitudeLine or Area - Plot values/progress over time

You need to understand your data and sometimes limit the data or reformat in SQL or in BI tools with calculations.

Did you know? If you study Lean-6-Sigma, statistics experts DISLIKE pie charts, saying they do not convey enough information.


EXAMPLE OF PICKING WRONG

35

Chart axis is plotting every date in a many record file - too much data spoils the chart.Solutions? - Plot by month, quarter, year or prompt for a limited range to plot

2012-2019 - Rick Flagler IT Consulting, LLC 36

PIES AND GAUGES

In context of a Visualization report for analysis


CAVEATS ABOUT BROWSER-BASED REPORTING

• Most BI tools allow you to quickly create reports in Web Browser formats

• But BI is not about putting today’s 300 page report in a Web Browser (or on paper) from your BI tool

• Business Intelligence is more about:

• Metrics• Drill-down• Data Exploration• Analysis• Trend spotting

37

Recommendations:• Define Key Performance Indicators• Capture metrics• Chart trends• Create dashboards, OLAP, Drill-down


IBM Db2 Web Query for i

!38

Express Edition Standard Edition

Express Edition

Report Scheduleing and Delivery

Commands

More DB Adapters

Three offerings currently

Repository Security

Named Users

Data Migrsator - ETL tool

Report Design Report Execution

OLAP Drill-down linkages

Charts, Maps, Visualization

Run-time Users

You can give ALL of this a try on your IBM i system with a FREE 70-day trial

of IBM Db2 Web Query for i

1 2

3


Stored Procedures• Programs written in SQL language (or in your favorite HLL)

• Doesn’t run from IBM i Command line, but is executed like other SQL from:

• STRSQL

• Client Access for Windows - Run SQL Scripts

• RUNSQL command

• RUNSQLSTM command

• ACS - Run SQL Scripts

• QSHELL - QSH Db2utility

• From PHP or other open source Languages with DB2, ODBC or JDBC connectivity

!39


USING STORED PROCEDURES

40

• Advantages• Powerful• Drives business rules into database (Db2)• Provide customized parameter-driven queries• Return one or more Results• Performance benefits• One line of SQL = 50-100 lines of RPG or other HLL• Hides complexity

• Disadvantages (if any?)• Requires additional SQL knowledge and programming skills

Programming in the SQL language

Also, can “wrap” existing RPG logic in SQL to make RPG logic reusable in your reporting Applications


Where to build/test your SQL• Green screen STRSQL - POSSIBLE BUT HARD

• Client Access for Windows - NOT any more - WITHDRAWN by IBM this year for Win10 and higher

• Access Client Solutions (ACS) - YES = The new Client

• Runs on Mac, Linux, Windows 8 and above

• IBM Data Studio - MAYBE = If you have it

• Rational Developer for i - YES = at least to code and view

!41


Steps to construct a SP to drive reporting•Identify file(s) used

•Create individual SQL statements to verify data extraction and summarization from ERP system data

•Create SP using your favorite IBM i SQL tool

•Declare SQL statements with individual result sets

•Test execute SP in script to verify it generates one, two, …n result sets

•In Web Query or other BI tool, configure your report to consume the result data

•Repeat - Creating a number of charts, reports, graphs from various results

•Save and Run!42


A few Caveats• The following examples:

• Were developed using ACS - Run SQL Scripts

• SQL techniques should work on IBM I 7.1 and above

• SQL techniques should work for any presentation technique with any BI tool or open-source charting method

• Assumption here is results will be consumed with IBM Db2 Web Query for i

• Using the IBM-Supplied WQ sample data in library QWQCENT

• Provides YOU with a framework for similar reporting

!43


SQL example selectWITH dt as (

select LAST_DAY(max(orderdate))-12 months as begin_date FROM qwqcent.orders )

SELECT STORECODE, SUM(LINETOTAL * QUANTITY) AS REVENUE FROM qwqcent.orders , dt where orderdate > begin_date GROUP BY STORECODE ORDER BY REVENUE DESC FETCH FIRST 10 ROWS ONLY;

With denotes use of a Common Table Expression (CTE)DT is a temporary result table for this query, containing a BEGIN_DATE computed to be 12 months prior to last day of the month of the most recent order date in the subject file..

The second select joins the temp table DT to the original ORDERS table to provide 1-year of data while also summarizing to yield the top-10 stores by revenue descending. This result can be returned rapidly to facilitate a chart or table of information.

** Depending on your data, the date calculation in DT might require CURRENT DATE to be used, from which you could back off a set period or days, months, years with SQL date math.

!44


SQL example selectWITH a AS ( SELECT YEAR(orderdate) AS ordYY, MONTH(orderdate) AS ordM, (LINETOTAL * QUANTITY) AS Period_REVENUE, (COSTOFGOODSSOLD * QUANTITY) AS Period_COST FROM qwqcent.orders ) SELECT ordYY || '-' || SUBSTR(DIGITS(ordM), 9, 2) as Period , sum(Period_REVENUE - Period_COST) AS PerProfit FROM a GROUP BY ordYY || '-' || SUBSTR(DIGITS(ordM), 9, 2) ORDER BY ordYY || '-' || SUBSTR(DIGITS(ordM), 9, 2) DESC FETCH FIRST 12 ROWS ONLY

With denotes use of a Common Table Expression (CTE)A is a temporary result table for this query, extracting the year and month from a date, computing revenue and cost of goods.

The second select uses the temp table A to summarize by Period, descending fetching the most recent 12 months which can be charted as a line or bar chart, to provide 1-year of historical data. Yes, I could have constructed column Period in the first select.

!45


SQL to DECLARE a named cursor for select in an SP

DECLARE C1 CURSOR FOR WITH dt as ( select LAST_DAY(max(orderdate))-12 months as begin_date FROM qwqcent.orders ) SELECT STORECODE, SUM(LINETOTAL * QUANTITY) AS REVENUE FROM qwqcent.orders , dt where orderdate > begin_date GROUP BY STORECODE ORDER BY REVENUE DESC FETCH FIRST 10 ROWS ONLY;

DECLARE is only allowed in SQL program (function or Procedure).

Cursor C1 is defined by the DECLARE and can later be used to fetch records or to pass data back to the calling application.

!46


SQL to Create a procedure -- Define procedureCREATE PROCEDURE Library.ProcName () DYNAMIC RESULT SETS 1 — or more than 1 LANGUAGE SQL BEGIN—Declare the Statement(s) DECLARE C1 CURSOR FOR WITH dt as (

select LAST_DAY(max(orderdate))-12 months as begin_date FROM qwqcent.orders )

SELECT STORECODE, SUM(LINETOTAL * QUANTITY) AS REVENUE FROM qwqcent.orders , dt where orderdate > begin_date GROUP BY STORECODE ORDER BY REVENUE DESC FETCH FIRST 10 ROWS ONLY;--Open each Result set(s) OPEN C1;— Set results and return (END) to caller SET RESULT SETS CURSOR C1; END; One or more DECLARE statements enables you to return several results to the calling application. If parameters are required, they would be inserted after ProcName ( specifying parameter & datatype i.e. P1 CHAR(5) , P2 DEC(9,0) ) and subsequently those variables could be utilized within the procedure's logic.

!47


Typical development Approach for Stored Procedure

• Test SELECT statement independently to verify it returns the proper information for your application

• Wrap the select with DECLARE {name} CURSOR statement

• Wrap DECLARE with CREATE PROCEDURE statement

• Add OPEN, SET RESULTS and END

• RUN Create Procedure to use SQL source to generate a program

• Run Procedure via CALL [Procedure name}

• BOOM! - you have data results

!48


Full SQL for Example procedure with 5 results to drive BI example-- Define procedure with 5 results CREATE OR REPLACE PROCEDURE Dash2() DYNAMIC RESULT SETS 5 LANGUAGE SQL BEGIN

DECLARE C1 CURSOR FOR WITH dt as ( select LAST_DAY(max(orderdate))-12 months as begin_date FROM qwqcent.orders ) SELECT STORECODE, SUM(LINETOTAL * QUANTITY) AS REVENUE FROM qwqcent.orders , dt where orderdate > begin_date GROUP BY STORECODE ORDER BY REVENUE DESC FETCH FIRST 10 ROWS ONLY; DECLARE C2 CURSOR FOR WITH dt as ( select LAST_DAY(max(orderdate))-12 months as begin_date FROM qwqcent.orders ) SELECT PLANTCODE, SUM(LINETOTAL * QUANTITY) AS REVENUE FROM qwqcent.orders, dt GROUP BY PLANTCODE ORDER BY REVENUE DESC FETCH FIRST 10 ROWS ONLY; DECLARE C3 CURSOR FOR WITH dt as ( select LAST_DAY(max(orderdate))-12 months as begin_date FROM qwqcent.orders ) SELECT SALESREP, SUM(LINETOTAL * QUANTITY) AS REVENUE FROM qwqcent.orders ,dt GROUP BY SALESREP ORDER BY REVENUE DESC FETCH FIRST 10 ROWS ONLY; DECLARE C4 CURSOR FOR WITH a AS (SELECT YEAR(orderdate) AS ordYY, MONTH(orderdate) AS ordM, (LINETOTAL * QUANTITY) AS Period_REVENUE, (COSTOFGOODSSOLD * QUANTITY) AS Period_COST FROM qwqcent.orders) SELECT ordYY || '-' || SUBSTR(DIGITS(ordM), 9, 2) as Period , sum(Period_REVENUE - Period_COST) AS PerProfit FROM a GROUP BY ordYY || '-' || SUBSTR(DIGITS(ordM), 9, 2) ORDER BY ordYY || '-' || SUBSTR(DIGITS(ordM), 9, 2) DESC

FETCH FIRST 12 ROWS ONLY ;

DECLARE C5 CURSOR FOR WITH dt as ( select LAST_DAY(max(orderdate))-12 months as begin_date FROM qwqcent.orders ), a AS ( SELECT YEAR(orderdate) AS ordYY, MONTH(orderdate) AS ordM, case when days( shipdate ) - days(requestedshipdate) between -2 and +2 then 'On-time' when days( shipdate ) - days(requestedshipdate) < -2 then 'Early' when days( shipdate ) - days(requestedshipdate) > +2 then 'Late' end as status , 1 as total FROM qwqcent.orders,dt where orderdate > begin_date ) SELECT ordYY || '-' || SUBSTR(DIGITS(ordM), 9, 2) as Period , sum(case status when 'Early' then 1 else 0 end ) as Early, sum(case status when 'On-time' then 1 else 0 end ) as ontime, sum(case status when 'Late' then 1 else 0 end ) as Late, sum(total) as total, dec((sum(case status when 'On-time' then 1 else 0 end ) *100.0) / sum(total),9,2) as OTDPct FROM a GROUP BY ordYY || '-' || SUBSTR(DIGITS(ordM), 9, 2) ORDER BY ordYY || '-' || SUBSTR(DIGITS(ordM), 9, 2) desc; --Open each of the Result sets and return them OPEN C1; OPEN C2; OPEN C3; OPEN C4; OPEN C5; SET RESULT SETS CURSOR C1, CURSOR C2, CURSOR C3, CURSOR C4, CURSOR C5; END;

!49


SAMPLE “DASHBOARD” DRIVEN BY STORED PROCEDURE

50

Run many metrics simultaneously, Improve dashboard performance

SP gives several results, each returns recent data, plotting reports or charts.


Reasons you might take the SP approach

• All logic is located in one place

• Runs fast because of FETCH limiting number of records

• DB2 optimizer probably “remembers” after SP runs several times, creates “plan”

• Easier to document and explain to others

• Easy to clone for future applications

!51


How the SP data is used in Web Query1.Select data source - DASH2 in

this case, Using procedure name yields available fields

2.See various fields from each ANSWERSET which correspond to the SP result sets

3.Drag fields to report definition(s)

4.Press RUN to see data summarized in report or chart

1

2

3

4

!52


Web Query InfoAssist Designer window with multiple reports on single canvas

As you design each “report” you can see different fields used. When Run, all SP values

are returned to populate the canvas areas.

Available Fields Report

DefinitionCanvas

workspace!53


Capturing Enterprise Data for BI reporting

• Copy data from IBM i to Excel

• Copy data off IBM i to a Windows/Linux Server

• Copy data to another LPAR where BI tool resides

• Utilize Db2 objects and programming to reduce burden of reporting on Transactional machines. Make summary data available quickly to enable responsive applications, with drill down and analysis capabilities.

My rankings of approaches …. your opinion may vary!!

!54


New Web Query Visualizations offer another option with or without Stored Procedures

!55

This example uses tables joined by the WQ metadata Redraws when user modifies selection filters

Filters for What-if analysis


Auto Drill down exampleLinkage - Develop single report, get automatic drill-down to levels in a

pre-defined hierarchy

Click to drill-down, Click to return


Other Reporting Techniques • IBM provides SERVICES to access various system information via SQL

• Jobs, Subsystems, Users

• Spool Files

• Consider creating your own “services” for use in business reporting or to support BI projects, using:

• User-Defined table functions (UDTF)

• Views

• MQTs

• Such “Services” might expose data so users can make their own reports

!57


One last approach to Enterprise metrics

!58


Creating an Enterprise metric database•Requires some brain-storming and discussion to determine what Management wants

•Metric “data points” are harvested and stored for reporting

•Small batch jobs on your Job Scheduler will populate metrics by doing summarize and insert for recent time period

•Possible examples:

•Sales $ vs. Forecast $

•On-Time Delivery % attainment

•Year-over-year Value comparisons

•Quality objective attainment (sales vs. scrap)

•Lost-time Accident metrics vs. Days since last accident

•Inventory turns

•Inventory Days on Hand!59


Example metric database design

Example data exposed by View

Records inserted into detail by night-time jobs, reference definitions for metric and report


Example of individual metrics plotted

Create reports that present last X months of


SUMMARY ADVICE• Drive Work into Db2

whenever possible

• Present data visually when possible

• Drill down to dimension details in order to reduce results presented

• Pick the right chart type for presentations

• Don’t try to print the world to your web browser

• Use Color to differentiate values

• via Traffic-lighting• via data bars• Charts

• Don’t over-use color in Themes and Styles

• Use Stored procedures or User Defined Table functions if they fit your requirements

• Build a metric database to provide data points for frequently used KPIs

62


END OF PRESENTATION

63

QUESTIONS AND DISCUSSION

Supporting Reporting projects with...

Documents

Transcript of Supporting Reporting projects with...