Copyright 2007, Information Builders. Slide 1 Relational Efficiencies: Part II Overview and JOIN /...
-
Upload
trent-warder -
Category
Documents
-
view
218 -
download
5
Transcript of Copyright 2007, Information Builders. Slide 1 Relational Efficiencies: Part II Overview and JOIN /...
Copyright 2007, Information Builders. Slide 1
Relational Efficiencies: Part IIOverview and JOIN / Sort Scenarios
Renee Teatro
Information Builders
Optimization Overview Definition Command syntax and settings Optimization Hierarchy Projection and Selection Tracing
JOIN ScenariosSort Scenarios
Relational Efficiencies Agenda
Interface optimization is the degree to which a TABLE request is translated to SQL
In other words, the process in which the interface translates projection, selection, JOIN, sort, and aggregation operations of a report request into its SQL equivalent and passes it to the RDBMS for processing
TABLE, MODIFY, MAINTAIN SQL Direct SQL Passthru Passing JOINs
Relational Efficiencies: Overview Definition of Optimization
Copyright 2007, Information Builders. Slide 4
TSOCMSSQL
DB2SQLDSSQLDBCSQLORASQLMSSSQLIDMS
SETOPTIMIZATIONSQLJOIN
ONOFFSQLFOCUS
*
Components Description
Target Database Engine DB2 for DB2SQLDS for SQL/DSSQLDBC for TeradataSQLORA for Oracle
SQLMSS for MS SQL Server SQLIDMS for IDMS SQL
OUTER
Relational Efficiencies: OverviewOptimization Command
SQL DB2 SET OPTIMIZATION ON
SQL SQLORA SET OPTIMIZATION ON
Relational Efficiencies: OverviewOptimization Hierarchy
JOIN
SORT
AGGREGATION
If the JOIN optimization fails, so will the SORT and
AGGREGATION.
If the SORT optimization fails, so will AGGREGATION.
AGGREGATION will occur only ifJOIN and SORT operations wereoptimized (and AGGREGATION
requested).
Copyright 2007, Information Builders. Slide 6
Relational Efficiencies: Overview Selection and Projection
What is selection and projection?
Regardless of the optimization setting, the data adapter will always try and translate selection and projection.
Why are selection and projection so important?
They are both processes that reduce the volume of data being returned from the RDBMS, which helps to improve performance, efficiency and report response time.
Selection The process of retrieving the table rows that meet the request criteria.
Projection The process of retrieving the table columns that meet the request criteria.
SET TRACEON=component//destination Component
SQLDI - FSTRACE - All Interface-RDBMS activitySQLAGGR - FSTRACE3 - Optimization messagesSTMTRACE - FSTRACE4 - SQL onlySQLCALL - commands and data exchange between
the physical and the logical layers of the data adapter Destination
FSTRACE - allocation for the ddname of FSTRACE CLIENT - displays client session to the screen
NOTE: Make sure to SET TRACEUSER=OFF afterwards
Relational Efficiencies: OverviewData Adapter TRACE Facility
Relational Efficiencies: OverviewData Adapter TRACE Facility
SET TRACEOFF=ALL SET TRACEUSER=CLIENTSET TRACEON=SQLAGGR//CLIENTSET TRACEON=STMTRACE//CLIENTSET XRETRIEVAL=OFFTABLE FILE …
SET XRETRIEVAL=[ON | OFF]ON – the data adapter sends the request to the
RDBMS and it processes the requestOFF – the data adapter attempts to optimize the
request and trace output is generated but no RDBMS processing is done
Copyright 2007, Information Builders. Slide 9
JOIN Scenarios
Relational Efficiencies JOIN: Performance related to KEYS attribute
JOINing 3 ORACLE tables results in very poor performance.
These are newly created MFDs and AFDs via CREATE SYNONYM
and the first time any requests / JOINs are being issued.
Scenario
Enable data adapter tracing to see if the JOIN is being
passed over and if not, why.
Suggestion
SET TRACEOFF=ALLSET TRACEUSER=CLIENTSET TRACEON=STMTRACE//CLIENTSET TRACEON=SQLAGGR//CLIENTSET XRETRIEVAL=OFFJOIN CLEAR *JOIN COURSE_NO IN COURSE TO ALL COURSE_NO IN SECTION AS J1JOIN SECTION_ID IN COURSE TO ALL SECTION_ID IN ENROLLMENT AS J2TABLE FILE COURSE …
Relational Efficiencies JOIN: Performance related to KEYS attribute
(FOC2510) FOCUS-MANAGED JOIN SELECTED FOR FOLLOWING REASON(S):
(FOC2505) A SEGMENT IN THE STRUCTURE IS NON-KEYED: : SECTION
(FOC2517) FST. OR LST. WHERE SORT FIELDS DO NOT COVER KEY
(FOC2590) AGGREGATION NOT DONE FOR THE FOLLOWING REASON:
(FOC2592) RDBMS-MANAGED JOIN HAS BEEN DISABLED
SELECT T1."COURSE_NO",T1."DESCRIPTION" FROM STUDENT.COURSE T1;
SELECT T2."SECTION_ID",T2."SECTION_NO",T2."START_DATE_TIME" FROM STUDENT.SECTION T2 WHERE (T2."COURSE_NO" = :0001);
SELECT T3."STUDENT_ID" FROM STUDENT.ENROLLMENT T3 WHERE (T3."SECTION_ID" = :0001);
Output
The .acx (ACCESS FILE DESCRIPTION) should have KEYS= with a
NON-ZERO value to indicate the PRIMARY KEY for the table in
question. The PRIMARY KEY is made up of the FIRST n fields
in the .mas (MASTER FILE DESCRIPTION).
Correction
Relational Efficiencies JOIN: Performance related to KEYS attribute
SEGNAME=COURSE, TABLENAME=STUDENT.COURSE,
CONNECTION=STUDENTA.IBI.COM, KEYS=1,$
SEGNAME=SECTION, TABLENAME=STUDENT.SECTION,
CONNECTION=STUDENTA.IBI.COM, KEYS=0,$ *** Change to KEYS=1
SEGNAME=ENROLLMENT, TABLENAME=STUDENT.ENROLLMENT,
CONNECTION=STUDENTA.IBI.COM, KEYS=2,$
Correction
(FOC2590) AGGREGATION NOT DONE FOR THE FOLLOWING REASON: (FOC2594) AGGREGATION IS NOT APPLICABLE TO THE VERB USED SELECT T1."COURSE_NO",T1."DESCRIPTION",T2."SECTION_ID", T2."SECTION_NO",T2."START_DATE_TIME",T3."STUDENT_ID", T3."SECTION_ID" FROM STUDENT.COURSE T1,STUDENT.SECTION T2, STUDENT.ENROLLMENT T3 WHERE (T2."COURSE_NO" = T1."COURSE_NO") AND (T3."SECTION_ID" = T2."SECTION_ID") ORDER BY T1."COURSE_NO", T2."SECTION_ID",T3."STUDENT_ID",T3."SECTION_ID";
Relational Efficiencies JOIN: Performance related to MULTI-PATH
JOINing 3 ORACLE tables results in very poor performance.
Certain JOIN combinations using these same tables do not
have this issue.
Scenario
Suggestion
SET TRACEOFF=ALLSET TRACEUSER=CLIENTSET TRACEON=STMTRACE//CLIENTSET TRACEON=SQLAGGR//CLIENTSET XRETRIEVAL=OFFJOIN CLEAR *JOIN STUDENT_ID IN STUDENT TO ALL STUDENT_ID IN ENROLLMENT AS J1JOIN STUDENT_ID IN STUDENT TO ALL STUDENT_ID IN GRADE AS J2TABLE FILE STUDENTPRINT SECTION_ID ENROLL_DATE GRADE_TYPE_CODEBY STUDENT_ID BY LAST_NAME BY FIRST_NAMEEND
Relational Efficiencies JOIN: Performance related to MULTI-PATH
(FOC2510) FOCUS-MANAGED JOIN SELECTED FOR FOLLOWING REASON(S):
(FOC2512) MULTI-PATH REQUEST
(FOC2590) AGGREGATION NOT DONE FOR THE FOLLOWING REASON:
(FOC2592) RDBMS-MANAGED JOIN HAS BEEN DISABLED
SELECT T1."STUDENT_ID",T1."FIRST_NAME",T1."LAST_NAME" FROM
STUDENT.STUDENT T1;
SELECT T2."GRADE_TYPE_CODE" FROM STUDENT.GRADE T2 WHERE
(T2."STUDENT_ID" = :0001);
SELECT T3."SECTION_ID",T3."ENROLL_DATE" FROM
STUDENT.ENROLLMENT T3 WHERE (T3."STUDENT_ID" = :0001);
Output
Adjust JOIN field(s) to create a SINGLE PATH structure SQL Passthru Create relational VIEW on RDBMS – single segment MFD Create HOLD files & JOIN HOLD files vs. FOCUS-managed JOIN
Correction
Relational Efficiencies JOIN: Performance related to MULTI-PATH
CHECK FILE STUDENT PICTURE
Relational Efficiencies JOIN: Performance related to MULTI-PATH
JOIN CLEAR *
JOIN STUDENT_ID IN STUDENT TO ALL STUDENT_ID IN ENROLLMENT AS J1
JOIN ENROLLMENT.STUDENT_ID AND SECTION_ID IN STUDENT TO ALL
STUDENT_ID AND SECTION_ID IN GRADE AS J2
-*CHECK FILE STUDENT PICTURE
Correction
(FOC2590) AGGREGATION NOT DONE FOR THE FOLLOWING REASON: (FOC2594) AGGREGATION IS NOT APPLICABLE TO THE VERB USED SELECT T1."STUDENT_ID",T1."FIRST_NAME",T1."LAST_NAME", T2."STUDENT_ID",T2."SECTION_ID",T2."ENROLL_DATE", T3."GRADE_TYPE_CODE" FROM STUDENT.STUDENT T1,STUDENT.ENROLLMENT T2,STUDENT.GRADE T3 WHERE (T2."STUDENT_ID" = T1."STUDENT_ID") AND (T3."STUDENT_ID" = T2."STUDENT_ID") AND (T3."SECTION_ID" = T2."SECTION_ID") ORDER BY T1."STUDENT_ID",T1."LAST_NAME", T1."FIRST_NAME";
Relational Efficiencies JOIN: Performance related to MULTI-PATH
Relational Efficiencies JOIN: Separate Node / Subsystem
JOINing 3 ORACLE tables results in very poor performance.
Certain JOIN combinations using these same tables do not
have this issue.
Scenario
Suggestion
SET TRACEOFF=ALLSET TRACEUSER=CLIENTSET TRACEON=STMTRACE//CLIENTSET TRACEON=SQLAGGR//CLIENTSET XRETRIEVAL=OFFJOIN CLEAR *JOIN STUDENT_ID IN STUDENT TO ALL STUDENT_ID IN ENROLLMENT AS J1JOIN ENROLLMENT.STUDENT_ID AND SECTION_ID IN STUDENT TO ALL STUDENT_ID AND SECTION_ID IN GRADE AS J2TABLE FILE STUDENT…
Relational Efficiencies JOIN: Separate Node / Subsystem
(FOC2510) FOCUS-MANAGED JOIN SELECTED FOR FOLLOWING REASON(S):
(FOC2519) THE JOINED SEGMENTS RESIDE IN MORE THAN ONE NODE OR SUBS
(FOC2590) AGGREGATION NOT DONE FOR THE FOLLOWING REASON:
(FOC2592) RDBMS-MANAGED JOIN HAS BEEN DISABLED
SELECT T1."STUDENT_ID",T1."FIRST_NAME",T1."LAST_NAME" FROM
STUDENT.STUDENT T1;
SELECT T2."STUDENT_ID",T2."SECTION_ID",T2."ENROLL_DATE" FROM
STUDENT.ENROLLMENT T2 WHERE (T2."STUDENT_ID" = :0001);
SELECT T3."GRADE_TYPE_CODE" FROM STUDENT.GRADE T3 WHERE
(T3."STUDENT_ID" = :0001) AND (T3."SECTION_ID" = :0002);
Output
CONNECTION / HOST values in ACX file is not the same for all tables involved in JOIN structure
Another MFD/AFD by the same name higher in the application path list with the SAME name as one of the JOINed tables that did indeed point to a different node.
Correction
Relational Efficiencies JOIN: Separate Node / Subsystem
SEGNAME=STUDENT, TABLENAME=STUDENT.STUDENT,
CONNECTION=STUDENTA.IBI.COM, KEYS=2,$
SEGNAME=GRADE, TABLENAME=STUDENT.GRADE,
CONNECTION=STUDENTB.IBI.COM, KEYS=4,$ *** Change to STUDENTA.IBI.COM
SEGNAME=ENROLLMENT, TABLENAME=STUDENT.ENROLLMENT,
CONNECTION=STUDENTA.IBI.COM, KEYS=2,$
Correction
(FOC2590) AGGREGATION NOT DONE FOR THE FOLLOWING REASON: (FOC2594) AGGREGATION IS NOT APPLICABLE TO THE VERB USED SELECT T1."STUDENT_ID",T1."FIRST_NAME",T1."LAST_NAME", T2."STUDENT_ID",T2."SECTION_ID",T2."ENROLL_DATE", T3."GRADE_TYPE_CODE" FROM STUDENT.STUDENT T1,STUDENT.ENROLLMENT T2,STUDENT.GRADE T3 WHERE (T2."STUDENT_ID" = T1."STUDENT_ID") AND (T3."STUDENT_ID" = T2."STUDENT_ID") AND (T3."SECTION_ID" = T2."SECTION_ID") ORDER BY T1."STUDENT_ID",T1."LAST_NAME", T1."FIRST_NAME";
Copyright 2007, Information Builders. Slide 21
Sort Scenarios
Relational Efficiencies Sort Management by the RDBMS
Relational Efficiencies Sort Management by the RDBMSBY and ACROSS equate to SQL ORDER BY clauseBY / FOR ROWS OVER syntax will translateJOIN COURSE_NO IN COURSE TO ALL COURSE_NO IN SECTION AS J1TABLE FILE COURSEPRINT COSTBY SECTION_IDFOR COURSE_NO103 OVER 100 OVER150 OVER136 OVER121END
(FOC2590) AGGREGATION NOT DONE FOR THE FOLLOWING REASON: (FOC2594) AGGREGATION IS NOT APPLICABLE TO THE VERB USED SELECT T1."COURSE_NO",T1."COST",T2."SECTION_ID" FROM STUDENT.COURSE T1,STUDENT.SECTION T2 WHERE (T2."COURSE_NO" = T1."COURSE_NO") AND (T1."COURSE_NO" IN(100, 103, 121, 136, 150)) ORDER BY T2."SECTION_ID",T1."COURSE_NO";
Relational Efficiencies Sort Management by FOCUS / WebFOCUS
Relational Efficiencies Sort Management by FOCUS / WebFOCUSDEFINE FILE COURSESHORT_DESC/A5=EDIT(DESCRIPTION,'99999');COST_INC/P8=COST * .10;ENDTABLE FILE COURSEPRINT COSTBY COURSE_NOBY COST_INC-*BY SHORT_DESCEND
(FOC2590) AGGREGATION NOT DONE FOR THE FOLLOWING REASON:(FOC2594) AGGREGATION IS NOT APPLICABLE TO THE VERB USEDSELECT T1."COURSE_NO",T1."COST" FROM STUDENT.COURSE T1;
(FOC2590) AGGREGATION NOT DONE FOR THE FOLLOWING REASON:(FOC2594) AGGREGATION IS NOT APPLICABLE TO THE VERB USEDSELECT T1."COURSE_NO",T1."DESCRIPTION",T1."COST" FROMSTUDENT.COURSE T1;
Relational Efficiencies Sort Management by the RDBMSDEFINE FILE COURSESHORT_DESC/A5=EDIT(DESCRIPTION,'99999');COST_INC/P8=COST * .10;ENDTABLE FILE COURSESUM COSTBY COURSE_NOBY COST_INC-*BY SHORT_DESCEND
AGGREGATION DONE ...SELECT T1."COURSE_NO",(T1."COST" * .1), SUM(T1."COST") FROMSTUDENT.COURSE T1 GROUP BY T1."COURSE_NO",(T1."COST" * .1)ORDER BY T1."COURSE_NO",(T1."COST" * .1);
AGGREGATION DONE ...SELECT T1."COURSE_NO",(SUBSTR(T1."DESCRIPTION", 1, 5)),SUM(T1."COST") FROM STUDENT.COURSE T1 GROUP BY T1."COURSE_NO",(SUBSTR(T1."DESCRIPTION", 1, 5)) ORDER BY T1."COURSE_NO",(SUBSTR(T1."DESCRIPTION", 1, 5));
Relational Efficiencies Sort Management by the RDBMSDEFINE FILE SSSALESMONTH_COMPONENT/A3=HNAME(ORD_DATE,'MONTH','A3');ENDTABLE FILE SSSALESSUM QTYBY MONTH_COMPONENT-*BY ORD_DATEEND
Tracing via DEFINE field MONTH_COMPONENT sort:
(FOC2590) AGGREGATION NOT DONE FOR THE FOLLOWING REASON:(FOC2609) CANNOT AGGREGATE BY NON-SQLIZABLE SORT KEY : MONTH_COMPSELECT T1."ord_date",T1."qty" FROM pubs.dbo.sales T1;
Tracing via REAL field ORD_DATE sort:
AGGREGATION DONE ...SELECT T1."ord_date", SUM(T1."qty") FROM pubs.dbo.sales T1GROUP BY T1."ord_date" ORDER BY T1."ord_date";
Relational Efficiencies Sort Management by the RDBMSDEFINE FILE SSSALESMONTH_COMPONENT/A3=HNAME(ORD_DATE,'MONTH','A3');ENDTABLE FILE SSSALESSUM QTYBY ORD_DATEM-*BY MONTH_COMPONENT-*BY ORD_DATEEND
AGGREGATION DONE ...SELECT T1."ord_date", SUM(T1."qty") FROM pubs.dbo.sales T1GROUP BY T1."ord_date" ORDER BY T1."ord_date";
FILENAME=SSSALES, SUFFIX=SQLMSS , $ SEGMENT=SSSALES, SEGTYPE=S0, $ FIELDNAME=STOR_ID, ALIAS=stor_id, USAGE=A4, ACTUAL=A4, $ FIELDNAME=ORD_NUM, ALIAS=ord_num, USAGE=A20, ACTUAL=A20, $ FIELDNAME=TITLE_ID, ALIAS=title_id, USAGE=A6, ACTUAL=A6, $ FIELDNAME=ORD_DATE, ALIAS=ord_date, USAGE=HYYMDs, ACTUAL=HYYMDs, $ FIELDNAME=ORD_DATEM,ALIAS=ord_date, USAGE=MT, ACTUAL=DATE,$ FIELDNAME=QTY, ALIAS=qty, USAGE=I11, ACTUAL=I4, $
From a performance standpoint, consider using TABLEF instead of TABLE for requests that you know are being returned in the requested sort order. This will free FOCUS / WebFOCUS from having to verify the sort order by loading its internal matrix. FOCUS / WebFOCUS will act directly on the data returned by the RDBMS.
ACROSS phrases are not supported. It must be verified that an ORDER BY statement has been generated in
the SQL for the correct columns prior to using TABLEF. Multi-verb requests are not supported. RETYPE is not available. TABLEF can hold locks on data pages until the data adapter issues a
COMMIT (usually when the report displays). Locks may prevent access to the data by other applications. (online core FOCUS)
Certain prefix operators are not available (TOT., PCT.) Computes that use prefix operators are not supported.
Relational Efficiencies Sort: Using TABLEF
Copyright 2007, Information Builders. Slide 30