BMC SQL Performance Solution for DB2 on z/OS

BMC SQL Performance Solution for DB2 on z/OS

Bill ArledgeConsulting DB2 Product Manager March 2009

©2008 BMC Software2

The Business Problem

› SQL performance– The most challenging problem in DB2 shops today

• Direct impact on meeting service level objectives• Direct impact on cost of ownership for the IT organization

– What makes it difficult?• Dynamic application workloads• More and more dynamic SQL coming from distributed connections• SQL coding quality and validation

› Why is this an even bigger deal today? – IT dynamics driving the problem– ERP/CRM applications using atypical SQL – Outsourced application development – Less staff

• People and experience


Most Important DB tools

› Unisphere Research survey for IDUG January ‘07


› Focus on individual SQL statements– Do they meet “best practice” coding standards?– Do they use expected/accepted DB2 access paths?– Do they deliver desired result set in acceptable time

with acceptable resource consumption?› Developed and tested in controlled environment› More predictive in nature

› Focus on workload execution metrics– How does concurrent execution affect response

time/resource consumption?– Does this SQL statement/program collide with other

transactions?– Same application

• Other applications in a shared subsystem› Real world unpredictability comes into play› More focus on measuring the workload and rapidly

reacting

Solving the Problem SQL Analysis Across the Application Life Cycle


BMC SQL Performancefor DB2

BMC SQL Performance for DB2

› Solution objectives– Control or eliminate poorly performing SQL code – SQL performance problems identified and solved quickly – Proactive identification of inefficient SQL

› Solution components– APPTUNE captures and analyzes SQL execution metrics – SQL Explorer for SQL analysis that extends beyond DB2 Explain– Performance Advisor Technology adds data analysis and

intelligence• Workload Access Path Compare prevents bad access paths• Index Advisor provides effective management of indexes• Reorg Advisor ensures that only objects that would benefit

from reorg get reorged.

BMC SQL Explorerfor DB2

BMC APPTUNEfor DB2

PerformanceAdvisor

BMC SQL Performance for DB2 BMC SQL Explorer for DB2


BMC SQL Explorer for DB2

BMC SQL EXPLORER for DB2•SQL Analysis

•Plan & Package Management•Statistics Migration

BMC Common Explain• Access path analysis• Common component across

multiple BMC BMC APPTUNE for DB2BMC MAINVIEW for DB2BMC CATALOG MANAGER for DB2

Access Path Compare

Mismatch Analysis

Impact Analysis

Statistics Migration

Declarations Generator


Explain Output

Key points• Extensive explain analysis beyond the basic EXPLAIN

Rules analysisPredicate reports‘What if’ analysisLock activity

• Customizable display allows for reordering or hiding columns• Reports toggle on and off allowing for as much detail as desired


Common Explain SQL Analysis


SQL Performance Common Explain Predicate Report

Key points• Predicate report provides details about column usage in predicates including filter factors

• Additional options including rules analysis• “What if” analysis to analyze impact of change on SQL statement access path and cost


BMC SQL Explorer for DB2 Additional Functions

BMC SQL EXPLORER for DB2•SQL Analysis

•Plan & Package Management•Statistics Migration

Access Path Compare

Mismatch Analysis

Impact Analysis

Statistics Migration

Declarations Generator

• Compare process to identify SQL access path changes

Compare on SQL text, not statement numberShows differences in statistics used at time of Explain

• Identifies predicate mismatches that can impact SQL access paths

• Finds statements impacted by a schema change

• Migrate optimizer statistics within and across DB2 subsystems

• Automatically generates DECLARE TABLE statements


Integrating SQL Explorer Into the Application Development Process

› Identifies negative access path changes before they make it to production

› Easy two-step process–1. Execute SQL Explorer Batch analysis after every bind or rebind

• IF analysis passes standards, proceed with Compare; ELSE reject –2. Prior to production promotion, execute SQL Explorer Compare of OLD and NEW package, plan or DBRM

• IF access path degradation not detected, proceed with change promotion; ELSE Reject

› Compare uses historical baseline snapshots–Detects differences in Access paths, SQL Text, and Catalog Stats

› Use as part of your V8/V9 migration plans

Customer Success Story

Large financial services organization implemented SQL Performance for DB2to automate SQL change

validation process

Benefit - Customer now avoids 90+ % of reviews on

SQL changes saving 300 man-months per month

for review meetings

BMC SQL Performance for DB2 BMC APPTUNE for DB2


BMC APPTUNE for DB2 An Introduction

SSIDSSIDDB2PDB2P

Accumulate totals for DB2P DB2PDB2P

STATEMENTSTATEMENTAA

Accumulate totals for STATEMENTA +Accumulate totals for STATEMENTA +

Capture SQL TEXT (no trace required)Capture SQL TEXT (no trace required)

STATEMENTASTATEMENTA

PLANPLANAA

Accumulate totals for PLANAAccumulate totals for PLANAPLANAPLANA

PACKAGEPACKAGEAA

Accumulate totals for PACKAGEAAccumulate totals for PACKAGEA PACKAGEAPACKAGEA

› Collects detailed performance execution data at the statement level

– Details rolled up at multiple levels– Extensive filtering – Support for dynamic and static SQL

• Including host variables

› Captures detailed performance data at the object level

– Getpage details– BP Hit ratios– I/O response times

› Linkages between SQL and object usage


BMC APPTUNE Collection Keys & Options

Important Points• Monitoring Requirements vary by application across subsystems

• Options set using filters that identify unique tuning requirements by application


BMC APPTUNE for DB2 Starting at the Top

Drilldown to more detailNumerous drilldowns will take you into detail organized to your interest

Key points• DB2 subsystems across your enterprise on a single screen

• DB2s with highest elapsed times bubble to the top (generally the best place to start)

• Results can be sorted on different field on display (getpages or CPU for example) with simple command

Performance characteristics at the subsystem level


SQL Statement Drilldown

Top three statements are using 90% of the CPU

S

“S” again to see the SQL text

Key points• Top SQL statements in terms of total elapsed time sorted to top

• Dynamic and Static SQL visible • Numerous drilldowns for additional detail


Extensive SQL Detail by Statement

“O” will take you to object detail by SQL statement

O

Key points• Provides extensive SQL statement execution detail

CPU and elapsed timeGetpage activityBP usageLock activity

• Launch directly into a full- function explain from the common Explain function

• Hyperlink directly to objects accessed by this statement


Object Details for This Statement Key points

• Provides object level access information for each SQL statement

# of getpagesI/O response timesBP hit ratiosLock activityIncludes system level objects

• Object level reports also show object access in aggregate across the DB2


Extensive SQL Detail by Statement

“E” will take us to errors for this statement

E

Key points• Provides extensive SQL statement execution detail

CPU and elapsed timeGetpage activityBP usageLock activity

• Launch directly into a full- function explain from the common Explain function

• Hyperlink directly to objects accessed by this statement


Error Analysis Report

Key points• Identifies SQL Errors

List of all occurrences Details on each occurrenceBy user/plan/program


Extensive SQL Statement Detail

“X” will invoke the common Explain function

X

Key points• Launch directly into a full- function explain from the common Explain function


Explain Output

Key points• Extensive explain analysis beyond the basic EXPLAIN

Rules analysisPredicate reports“What if” analysisLock activity

• Customizable display allows for reordering or hiding columns• Reports toggle on and off allowing for as much detail as desired


APPTUNE – the SQL Optimization Solution

› Basic SQL tuning can substantially reduce CPU cost right out of the box – Identifies and helps resolve problem SQL

• Bad statements bubble to the top–Identify object performance problems–Reduces overall CPU requirements

› Service levels improve as CPU cost goes down› Productivity Increases

–Minimal clicks to improved performance–Analysis and recommendations indicate possible problem resolutions

Customer Success Story

Large mid-west manufacturer with large IMS/DB2 application

utilizing APPTUNE for DB2 to tune their SQL

Benefit - Reduced IMS/DB2application cost by 2.1 M seconds and $340,000 on

an annual basis

BMC SQL Performance for DB2 Solution-Only Components


BMC SQL Performance for DB2 – Index Component

› Collects predicate data– Number of times a column was specified in a predicate and in what

manner (Indexable Equal, Indexable Range)› Collects I/O information for DB2 objects

– Index usage breakdown by READ, UPDATE, DELETE, INSERT– DB2 catalog statistics for indexes

› Collects application index usage detail– Percentage of getpages from indexes by SQL statement – Identifies indexes not being used for READ access– Identifies indexes not being used optimally– Identifies tables not using indexes for READ access

› Provides What-if index analysis on an SQL statement– Model a new index– Add an index………What’s the Effect?– Drop an index……What’s the Effect?


BMC SQL Performance for DB2 Index Component Reporting

Index Component Details• Indicates % of total getpages executed against the indexes

Tables with low percentage of access via index good starting pointHyperlink to the statement list for this table

• Provides synchronous I/O information including response times• Drill down to additional detail on statements and stats


• Access path analysis for SQL statement of interest

• Cloning technique that creates new objects or modifies existing objects

Clones created in a BMC database so no DBD locksObjects created with DEFINE NO Objects not created till a dynamic explain is executed Objects dropped upon exiting ExplainIndex statistics can be modified Index DDL can be saved or imported into the process

BMC SQL Performance for DB2 What-if Index Analysis Using Cloning


• We’ve created a new index in this step and modified the statistics so the optimizer will have enough information

• We can also delete existing indexes (deleted from the clone only) or update statistics data for existing indexes as well

BMC SQL Performance for DB2 What-if Index Analysis

•Here is the index we created in the previous step

•Remember the index hasn’tactually been created yet

•Next step we’ll using the XD (DynamicExplain) to create the clone


• The new index has dramatically affected the access path and caused the Cost Rate for this statement to go down

• More importantly, the access path went from a table space scan to a matching index access

BMC SQL Performance for DB2 What-if Index Analysis•With the new index the access path changes dramatically

•Meanwhile the cost of the statementhas gone down as well

•Next step we’ll using the XD (DynamicExplain) to create the clone


Workload Compare (new in v6r2)

› Access paths can change due to:– Reorg/RUNSTATs– Application migration from test to prod– New Indexes– DSNZPARM changes (e.g. Buffer/EDM Pool)– Changes to DBMS, OS, and hardware

› Compare can identify changes and analyze impact– Compare full app between 2 DB2s (across versions)– Predict changes before changes move to prod– Predict changes due to system modifications– Predict access path for dynamic SQL– Selective REBIND (to avoid letting bad paths through)– Selective BIND REPLACE (don’t BIND if no SQL change)


SQLTXT_FILE

TraceDataSet

TraceDataSet

ArchiveTraceDataSets

BMC SQL Performance for DB2 6.2 Workload Access Path Compare Feature

DBRMLIB

EXPLAIN

COMPARE & REPORT

EXTRACT & EXPLAIN

SYSSTMTSYSPACKSTMT

DB2A

PLAN_TABLE

DB2A

OBJECT_FILE

PATH_FILE

COMPARE_FILE

PLAN_TABLE

DB2B

Compare static and dynamic statement access paths over time across systems, across DB2 releases


BMC SQL Performance for DB2 6.2 Workload Access Path Compare Reporting

› Online or batch reporting options› Unique statement compare process identifies matching statements even if statement numbers

are different› Identifies potential reasons for difference in cost

Report shows cost changes between different workloads• for dynamic workloads factors in # of executions• Shows categories where characteristics are different

(PATH, STATS, INDEX)Multiple levels of detailReport formats are modifiable for online and batch


New Performance Advisor Technology

› Many types of data collected by BMC SQL Performance for DB2 can be maintained historically in DB2 tables as part of the performance database.

› Infrastructure has been created to support populating and maintaining these tables.

› These tables can be used for analysis and trend reporting.

› The first Performance Advisor solution is provided: a REORG recommendation process that uses performance data to help decide when REORGs are needed.


Performance Management Database (PMDB)

› “Performance Management Databases (PMDBs) will emerge in the next five years as a second point of data-level integration. This isn’t an alternative to a Configuration Management Database (CMDB), which focuses on the configuration of an IT service for change impact. Rather, it is an additional management database that focuses on managing data needed for the performance management workflow cycle, which will require federation and reconciliation.” Moreover, “A PMDB must be able to provide data for the entire performance management workflow cycle— real- time analytics, historical data analysis, long-term capacity planning and performance tuning.”

› Source: Gartner, Inc “Expect Performance Management Databases in the Future”, M. Govekar, W. Cappelli, D. Williams, February 2008


Performance Advisor Concept – Manage DB2 Performance

› Turn data into information, course corrections

Stats, Events

TraceDataSet

TraceDataSet

DB2

Trend reportsRecommendations

notices/alerts

ServiceLevel

PerformanceManagement

SQL W

orkload Stats

Access Path

Data

Object Statistics

DB

2 SubsystemStats/acctng

Merge/Rollup

Performance Management

Database (PMDB)

Stmt/objstats

Stmtstats

RTS

Objectstats

ReorgCndts

Indexusage

Wrkldstats

AutomatedCorrective

Actions

Analysis

Catlg stats

Sys/acct stats


Performance Advisor: REORG Overview

› REORG only the objects that are disorganized AND show degraded performance since last REORG

› REORG of object with large percentage of random (SYNC) I/O won’t benefit – why waste the time & resources?

› Two primary criteria– Physical disorganization of object (RTS)– Performance characteristics of object (APPTUNE)

› Analysis produces a table of pagesets with a column for REORG_Candidate (Y/N)


REORG Analysis Details

› Three source tables– DAILY_OBJ_STATISTICS– BMCTABLESPACESTATS– BMCINDEXSPACESTATS

› Two criteria– Physical disorganization of object

• Table spaces: FARINDREF, REORGUNCLUSTINS as percentage of total rows• Indexes: LEAFFAR as a percentage of total leaves (DB2 9)• Indexes: LEAFFAR as a percentage of LEAFNEAR (DB2 v8)

– Performance characteristics of object• Limit to objects that showed Asynchronous IO just last REORG• Synchronous IO increase since last REORG• Getpages per Synchronous IO decrease since last REORG

› End result: recommend for REORG only the objects that are disorganized AND show degraded performance since last REORG


Integration with BMC DASD MANAGER

› Seamless integration for customers using BMC DASD Manager to control the REORG process

– Object set provided by Performance Advisor technology is sent to BMCTRIG– BMCTRIG may force exceptions to REORG all objects in object set– BMCTRIG may perform additional evaluation to reorg only some objects

› Cheapest REORG is the one you don’t run because it would not benefit your applications


BMC SQL Performance for DB2 The Value Proposition

› Manages SQL performance across the application lifecycle› Identifies poorly performing SQL statements to:

– Reduce CPU cost– Improve response times– Validates SQL syntax and identifies access path selection problems– Supports the promotion process by identifying changes that will negatively impact

performance as application changes are migrated– Comprehensive Explain function with what-if analysis capabilities

› Performance Metrics– Captures detailed SQL performance details and object access information– Extensive filtering capabilities that reduce cost of monitoring – Online and batch reporting

BMC SQL Performance Solution for DB2 on z/OS

Documents

Transcript of BMC SQL Performance Solution for DB2 on z/OS