Rat overall mar2011

<Insert Picture Here>

Real Application Testing for Oracle Database 9i / 10g / 11gR1/R2

Jan van Tiggelen

Principal Sales Consultant Oracle EMEA

2

Outline

Oracle’s Testing Lifecycle for Database and Applications

SQL Performance Analyzer (SPA): Overview

Database Replay (DBReplay): Overview

New in 11g Release 2 and Best Practices

Real-world Customer Case Studies

Conclusion

3

Oracle Application Quality ManagementHigh quality testing for all tiers of application stack

• Testing Application Changes• Application Testing Suite

• Ensure quality and performance with

end-to-end application testing

• Testing Infrastructure Changes

• Real Application Testing

• Designed and optimized for testing

database tier infrastructure changes

• Test Data Management

• Data Masking Pack

• Achieve security & compliance objectives by

obfuscating sensitive data in test environments

4

Ap

plic

ati

on

Ap

plic

ati

on

Ap

plic

ati

on

Oracle’s Testing Lifecycle

Production

Application Logic

Test

Staging

Clone

Clone

Mask

Load Testing Functional

Testing

Production Test

Real Application

Testing

Replay

Capture

Database Logic

Change

Comprehensive

testing

Data Masking

5

Real Application Testing FeaturesSPA and Database Replay

• SQL Performance Analyzer

• SQL unit testing for response time

• Identify and tune regressed SQL

Replay

Workload

Capture

Workload

Deploy Replay

Clients

Create Test

System

End-to-end testing with real workloads

• Database Replay

• Load and performance testing for throughput

• Remediate application concurrency problems

6

Outline






Conclusion

7

SQL Performance Regressions: Challenges

SQL performance regressions: #1 cause of poor system performance

SQL performance can regress due to many changes - DB upgrades, patch-sets, optimizer statistics refresh, schema, parameter, hardware, etc.

Changes need to be tested thoroughly to avoid regressions

Traditional testing techniques suffer from many limitations: large workloads (100K SQL), expensive, partial workload capture & manual point solutions

As a result, no testing or sometimes limited testing done in production

• Proactively detects ALL SQL regressions BEFORE deploying actual change

• Provides integrated, comprehensive, and end-to-end solution

SQL Performance

Analyzer (SPA)

8

• Test and predict impact of system changes on SQL query performance

• Analyze performance changes for improvements and regressions

• Comprehensive performance analysis and reporting

• Re-execute SQL queries in the given environment

• End-to-end solution: STS, SQL Plan Baselines, and SQL Tuning Advisor

Oracle Real Application Testing: SPA

SQL Workload

STS

SQL Plans + Run-time Stats

Pre-Change Trial

SQL Plans + Run-time Stats

Post-Change Trial

Compare SQL

Performance

Analysis Report

9

SPA Report (Example)

1

2

3

10

When to use SPA?• Testing database upgrades and patch-set releases*

– 9.2/10.1 10.2 or 11g releases

– 10.2.0.x 10.2.0.y or 11g releases

• SPA supports testing in Oracle Database Releases 10.2 and 11g

– Optimizer statistics refresh

– Database parameter changes

– Database schema changes (e.g., add/drop indexes)

– Implementation of tuning recommendations

– I/O subsystem changes (e.g., ASM, Database Machine)

• SPA handles trials in a manner that does not change database data

– Hence can be used for testing in production/standby environments

*MOS Note: 560977.1

• Across many releases of Oracle and for upgrades

• On test, standby, and production environments

• Extended to home-grown scripts, third-party testing tools, etc.

• Supports most applications - EBS, SAP, Siebel, home-grown, etc.

SPA Provides Broad Testing

Coverage

11

Outline






Conclusion

12

• Database load and performance

testing with real production

workloads

– Production workload characteristics

such as timing, transaction

dependency, think time, etc., fully

maintained

• Test and measure transaction

throughput improvements

• Identify application scalability and

concurrency problems with new

features

• Remediate issues pre-production

for risk-free change

• Supports migrations from Oracle

9iR2 and 10gR2

Test

Capture Process ReplayAnalysis &Reporting

ProductionReplay DriverClients

Storage Storage

Oracle Real Application Testing Database Replay

• *MOS 560977.1: Real Application Testing for Earlier Releases

File 1

File 2

…

File N

Metadata

Replay Files

…Server Server Server

Replay Clients

…

Workload Replay Architecture

• Replay captured workload

• Replayed operations see the same data and perform the same work

• Preserve timing and concurrency characteristics

• Same number of user connections

• Replay Client

• Multithreaded OCI Client

• Drives multiple captured processes

• Scalable Architecture

• Interprets capture into sequence of OCI calls

• Functional replay

Background

Timing Preservation

Commit Order Synchronization

Sequence Replay

Timing Preservation

Re-mapping

14

Database Replay - Supported Changes

Changes

Unsupported

Changes Supported

•Database Upgrades, Patches

•Schema, Parameters

•RAC nodes, Interconnect

•OS Platforms, OS Upgrades

•CPU, Memory

•Storage

•Etc.

ClientClient

…Client

Middle Tier

Storage

Recording of External Client

Requests

15

Outline






Conclusion

16

New Techniques for Avoiding SQL Regressions

Create central “SPA System” for all testing

• Many database releases

• Test, production, standby databases

Minimize production impact

• Use scoped or private session testing where possible

Leverage Oracle Active Data Guard (Read-only Physical Standby) for testing

• Use idle standby resources for testing

• Provides full and current dataset

17

What is SPA system?• Remote test executes SQL workload (STS)• Performs reporting/analysis• Any 11g database, preferably latest release• Not mandatory except for testing pre-11g

upgrades or 10.2.0.x 10.2.0.y• Same or higher version than the target

database being tested

New Techniques for Avoiding SQL RegressionsCentral SPA System

Benefits• Use latest software for analysis• Can be used as repository, helps persist

results across database refreshes

Prod Standby DB

Test 1

Test N

Prod 1

Prod N

11g SPA System

Tips• Use logon triggers to set environment or

to make change on remote system

• SQL workload (STS) only

• No application schema/data

necessary

19

Database Replay EnhancementsNew in Oracle Database 11g Release 2

Earlier restrictions removed with support for

• Shared Server configuration

• Streams “apply” workload

Replay filter support to target sub-set workload

• Similar to existing capture filters – Include/Exclude

• Service, Program, Action, Module, etc.

• API support only in current release

Integration of SPA and Database Replay

• Allows SQL-centric analysis when using Database Replay

• Simultaneously captures SQL workload into two different STS during workload capture and replay

• SPA Report built from the two STS captured helps understand workload drift

• Uses STS Compare functionality to highlight new, missing, top SQL, changes in execution plans, #SQL executions etc.

20

Oracle Database Testing AcceleratorLeverage DB Workload captures

• Create synthetic test scripts for

Oracle Database testing

– Import transactions from DB

Replay capture to create scripts

– Import from custom SQL Script

– Create DB test scripts manually

• Extend scripts in OpenScript

through GUI or Java Code view

• Run scripts in Oracle Load

Testing to emulate concurrent

Virtual Users

• Leverage integration with EM

Diagnostics to analyze database

performance under load

21

Outline






Conclusion

22

“Oracle Real Application Testing reduces the time

required to test changes by as much as 80%, lowers

testing costs by as much as 70%, mitigates risks by

reducing the number of unexpected outages, and

improves the quality of service for their IT operations.”

David Mitchell

Senior Vice President, OVUM

Source: Oracle Real Application Testing – business agility through superior testing, Jan 2008

23

Case Study EM RAT Real Application Testing

• 50% cost saving in Application Testing using Enterprise Manager RAT

• Modernized Testing methods on less costs with higher reputation of IT team and best and stable support for

missions critical business processes

• „Oracle Real Application Testing provides a unique tool that enables us to significantly improve our speed in rolling

out technology, and reduces our costs associated with critical testing.” – Erzsébet Szirjákné Birkás, MOL Expert of

System Programming

BUSINESS CHALLENGE

• Leading oil company operating in 10 countries in

Central and Eastern Europe

• Business drivers for RAT

– Cost reduction in application testing via

centralization in IT and less dependency on

external developers

– Less performance problems in version

upgrades

– Accelerated db infrastructure

implementation change

• Oracle Partner Remedios supported implementation

of RAT

RESULTS

ORACLE SOLUTION

• EM Enterprise Manager RAT Real Application

Testing enabled:

• More efficiency in Testing using RAT's „db replay

feature“

• Safeguarding mission critical business process via

via better testing and avoiding negative runtime

impact in production

• Faster deployment on system enhancement

requests

• Planned additional usage of RAT's „SQL

performance Analyzing feature“ to filter out SQL-

level problems in an early stageprior to production

runtime impacts

“The Database Replay and SQL Plan Management

features alone are worth upgrading to Oracle

Database 11g” 11g "

Arup Nanda

Senior Director

Starwood Hotels & Resorts

Worldwide

“Oracle Database 11g will be welcome relief to DBAs

responsible for testing changes against a production

workload. The way that Data Guard and Real Application

Testing work together is sheer poetry, and no doubt most

shops will quickly come to rely on the combination”

Sean McCown

Contributing Editor, Infoworld

26

Outline






Conclusion

27

Conclusion

• SPA enables businesses to

safely test and deploy system

changes using real workloads

• Real Application Testing is

much more than only a one-

time upgrade toolkit

• Increases staff productivity –

less firefighting

• More focus on strategic

planning and execution

• Capital expenditure savings

• 232% ROI over 4 years*

• <12 months payback

period *

*Based on POC and business case from a Fortune 500 Insurance Company

28

Resources

• Dedicated ½ day workshops for Oracle Customers and Partners

– Contact [email protected] for more information

• Oracle Database 11g Manageability and Real Application Testing

– http://www.oracle.com/technology/products/manageability/database/

• Oracle 11g Education (http://education.oracle.com)

– Oracle 11g: New features for Administrators

– Oracle 11g: Administration Workshop I and II

– Oracle 11g: Change Management Overview Seminar

• Oracle-By-Example Series on Oracle 11g

– http://www.oracle.com/technology/obe/11gr1_db/index.htm

• Oracle Database 11g Documentation

– http://www.oracle.com/pls/db111/homepage

• Metalink Notes

– 445116.1: Using the workload capture and replay in 11g

– 443580.1: 11g New Feature Database Replay

– 443720.1: 11g Using Snapshot Standby Database

– 455889.1: SQL Performance Analyzer

– 560977.1: Real Application Testing Now Available for Earlier Releases

mailto:[email protected]

http://www.oracle.com/technology/products/manageability/database

http://education.oracle.com/

http://www.oracle.com/technology/obe/11gr1_db/index.htm

http://www.oracle.com/pls/db111/homepage

Rat overall mar2011

Documents

Transcript of Rat overall mar2011