1. Application Performance Management Virtual Classroom Mayur Palta Senior Technical Consultant

2. The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracles products remains at the sole discretion of Oracle. 2009 Oracle Corporation Proprietary and Confidential 3. Agenda Application Quality Management Application Testing Oracles Approach to Application Quality Test Data Management Application User Experience Management Managing Real User Experience Application Operations Management Oracle Enterprise Manager Next Steps 4. What is Application Testing? A set of activities conducted through scripts with the intent of finding errors in software. An activity being part of the software development process aimed at evaluating a software item (system, subsystem, unit etc.) features (functionality, performance etc.) against the given set of system requirements. Process of verifying that a program functions properly. 5. Business Impact of Application Quality Poor software quality puts your investments at risk Cost to fix an error found after product release was up to 100 times morethan one identified in the design phase [Source: National Institute of Standards & Technology,2002] About 80 percent of development costs are consumed by softwaredevelopers identifying and correcting defects [Source: IBM: The business value of softwarequality.] Testing typically accounts for 30 to 50% of a software project budget &non-reproducible or ad-hoc testing is of little or no use [Source: Pullam & Doyle, 1998][Source: W.R. Blischke, D.N.P. Murthy. "Reliability", Section 9.5.1, pp. 311.] 6. Common Testing CyclesTraditional Ways15% TestedApplication Groups Own TestingUsed Application Testing ToolsSimulation QueriesVery Scripting & Labor Intensive85% UntestedProblems Limits the Testing Not a Production Test Application Performance Problems Deploying an Untested System predominately link back to the SQLQueries in the Database Vulnerable to Poor Performance 7. A Lifecycle Approach to Quality1. Design Test Plans 2. Develop Manual TestBased on Application Cases and AutomatedRequirements Test Scripts 4. Execute Load Tests 3. Execute Functionaland Tune ApplicationTests to ValidatePerformance ApplicationRequirements 8. Application Performance Testing & ManagementReal User Experience InsightApplicationTesting Suite EnterpriseManager Application Lifecycle 9. Application Performance Testing & Management Application Testing Suite Application Lifecycle 10. Oracle Testing History Acquired e-Test Suite from Empirix in June 2008 10+ years of automated testing experience Currently Working with Internal Application Developers Specialist in testing web-based application .NET JAVA Coldfusion 11. Oracle Application Testing Suite A powerful, integrated testing solutionFunctional Testing for Web Apps Test Manager for Web ApplicationsLoad Testing for Web Apps 12. Oracle Application Testing SuiteA Lifecycle Approach to Quality Oracle Test Manager forWeb Applications 1. Design Test Plans2. Develop Manual TestBased on ApplicationCases and AutomatedRequirementsTest Scripts 4. Execute Load Tests 3. Execute Functionaland Tune ApplicationTests to ValidatePerformance ApplicationRequirements 13. Oracle Test Manager for Web Applications:Test Process Management Manage test process from centralized Web-based console Define test requirements Develop manual & automated test cases Document and track defects Create reports 14. Oracle Application Testing SuiteA Lifecycle Approach to Quality Oracle Test Manager forWeb Applications 1. Design Test Plans2. Develop Manual TestBased on ApplicationCases and AutomatedRequirementsTest Scripts Oracle FunctionalTesting for Web Applications 4. Execute Load Tests 3. Execute Functionaland Tune ApplicationTests to ValidatePerformance ApplicationRequirements 15. Oracle Functional Testing for Web Applications:Automated Functional & Regression Testing Automate transactions for Web applications & Web Services Execute rigorous functional test cases Create automated regression test suites Identify and report on functional application failures 16. Oracle Application Testing SuiteA Lifecycle Approach to Quality Oracle Test Manager forWeb Applications 1. Design Test Plans2. Develop Manual TestBased on ApplicationCases and AutomatedRequirementsTest Scripts Oracle Load Oracle Functional Testing for WebTesting for WebApplications Applications 4. Execute Load Tests 3. Execute Functionaland Tune ApplicationTests to ValidatePerformance ApplicationRequirements 17. Oracle Load Testing for Web Applications:Load and Performance Testing and Tuning Create realistic load test scenarios that simulate end-user behavior Scale to thousands of concurrent users Perform functional content validation under load Monitor server-side performance & correlate to end-user response times Isolate and resolve performance bottlenecks 18. Need for Data Masking Test Data Management Key Drivers Privacy and compliance LAST_NAMESSN SALARY HIPAA, Breach Notification LawsAGUILAR203-33-323440,000 EU Data Privacy DirectiveBENSON 323-22-294360,000 Application testing DSOUZA989-22-240380,000 Offshore application development FIORANO093-44-382345,000 Offshore / In-house software QA Key Requirements LAST_NAMESSN SALARY Support database and applicationANSKEKSL 11123-111140,000 referential integrity BKJHHEIEDK 111-34-134560,000 Minimal performance impactKDDEHLHESA 111-97-274980,000 Protect against reverse FPENZXIEK111-49-384945,000 transformation 19. Data Masking Pack Oracle Enterprise Manager Production Automates production data masking Mask data from a production database Define rules once Data relationship discovery Automates data relationshipenforcement using existing foreign keysClone/ Enter custom data relationships knownMask Stagingto the application Rules repository Format library, masking definitions Testing View sample data before maskingTestTest 20. Application Performance Testing & ManagementReal User Experience InsightApplicationTesting Suite EnterpriseManager Application Lifecycle 21. ChallengesBusiness ChallengeIT Challenge Line of Business Apps Manager /IT Operations (LOB) ManagerApps Support Low transaction Isolating problems Reactive: Issues effectiveness lost to applications andidentified when user calls revenue, highmodules No visibility into support costs No insight into each application & operational Poor customerusers experienceerrors satisfaction Providing reports Time consuming diag. No visibility into and visibility to Inability to anticipate customer trendsstakeholders impact of change 22. Analysts Agree: User Experience is Critical Consistent end-to-end application and service87%performance guaranteesUnplanned infrastructure changes resulting in 73%incidents and downtimeUnanticipated infrastructure effects from49% consolidation & new application projects Mis-configuration of network objects28% Source: Forrester Research, Top Five Challenges For Enterprise IT Infrastructure Managers And How To Resolve Them, Thomas Mendel, March 2005Real life experience shows: 74% of problem alerts come from end userscomplaining to the service desk about performance problems, not frommonitoring infrastructure components!Jean-Pierre GarbaniForrester Research 2004 23. TomTomDelights customers and recovers revenue lost because of IT issues There was a point in time when we were more or less driving in the darkNow Im convinced that for us, this product will pay back within the first six months, easily.-Oscar Diele,Global vice president of e-commerce Marketing removed user manual from the box, not telling IT!Challenge Product was a big hit last Christmas, many people went on the website to download user manuals & performance went down, including the online shop 27% of customer about to check out canceled their transaction Oracle Enterprise ManagerSolution Real User Experience Insight IT gets alerted, understands and reacts very quickly, cutting down the 27% loss. IT captures all the 27% customers emails , Marketing immediately sends an Benefitsemail with an apology and a coupon to attract those customers to shop again. TomTom recovered most of the 27% losshttp://www.oracle.com/profit/features/091908_tomtom_.html 24. Solution Overview Optimizing end-user experience through real end-user monitoringBusiness Reporting KPI Monitoring Data Export Real User Experience Insight key highlights: - Replay of poor performance Transaction Performance - Executive dashboardsExecutive dashboard - Extensive KPI and SLA monitoring - Full alerting capabilities - Quick bottle-neck analyses - Transaction performance analyses - Customized reporting - Trend analyses - Full data integration via XML 25. Real User Experience Insight For:Line of Business executivesSatisfaction Reports Business Transaction Funnel Usage Reports: Data Browser Sales by Country/State 26. Real User Experience Insight For: Application Owners and Application Support Dashboards & SLM Business Service Monitoring User Session Tracking, (e.g. Order Entry, Buy Stock)PageReplay 27. Real User Experience Insight For: IT Operations KPI configuration, monitoring, & alerting Server, network,& object performancediagnosticsData ExportError reports& analysis 28. Application Performance Testing & ManagementReal User Application Experience Insight Testing Suite EnterpriseManager Application Lifecycle 29. Application Operations ManagementTop Challenges & Risks How to identify and resolve business issues? How to manage entire application lifecycle and reduce compliance risks? How to reduce operational costs? How to prioritize issues that impact business user experience? 30. Solution:Integrated, Top-Down Application ManagementApplication Users ApplicationSiebel, PeopleSoft, EBS Java, .NET, Composite Exchange, Legacy Software Oracle ApplicationsCustom Applications Non-Oracle Applications ApplicationWeb Server, PortalDatabase AS, ESB, SOA Components Presentation Layer Management Systems MiddlewareInfrastructure ComponentsVirtual EnvironmentsServersDatabases Storage Oracle Enterprise Manager 31. Comprehensive Applications Management Oracle Enterprise Manager Manage Oracle applicationsand technology from a singleconsole Application Management Packs for E-Business Suite, Siebel, PeopleSoft, JDE Optimize User Experience forOracle Applications E-Business Suite Accelerator PeopleSoft Accelerator Siebel Accelerator 32. Oracle Application Management PacksComplement Application Tools Application Management Pack Application Management Pack Application Management Pack for PeopleSoft Enterprisefor Siebelfor Oracle E-Business Suite Server Manager PS/AdminOracle Applications Manager SARManalyzer, SARMqueryPerformance MonitoriSetupDiagnostic Data CollectorPeopleToolsRapid Install Environment Verification Tool Change Impact Manager Rapid Clone Application Deployment Mgr 33. Application Operations Management Managing Packaged Applications Oracle E-Business Suite PeopleSoft Siebel E-Common Capabilities 1-step Discovery Out-of-box transactions Reports Topologies System monitoring Configuration Management Service Level Management Application monitoring Transaction Tracing Root-cause/Impact Analysis Dashboards Out-of-box service models Embedded Application-Specific Capabilities E-Business SuitePeopleSoftSiebel Topological root-cause views 500+ config metrics & 50+ business metrics 100+ configuration metricsPSAdmin admin tasks Out-of-box processes Clone automation for test 25+ tribal knowledge- HI Component support production environments based policies SARM integration real Cross-instance comparisons Centralized Log Mgt end user metrics, Key bus flow monitoring transaction diagnostics 34. Application Management Pack for Oracle E-Business Suite 35. Application Management Pack for EBS E-Business Suite-Specific Management Full Visibility into E-Business Suite Components E-Business Suite System Parameter Change Tracking Cloning Automation Seamless Integration withvs Oracle Applications Manager Support R12 and R11i ATG RUP4 or above 36. Benjamin CabanasProgram Manager GE InfrastructureImplementing Application Management Packfor Oracle E-Business Suite has allowedGE Infrastructure to realize $200K annual savings, 84% reduction in clone cycle time, and75% reduction in resources 37. Application Change Management Pack for Oracle E-Business Suite (ACP) 38. Application Change Management Pack for Oracle E-Business Suite: Version 3.0 Simplify Change Improve UserReduce HumanOrchestration productivityErrors 39. Product Overview Summary of Key FeaturesCustomization ManagerPatch ManagerSetup Manager Create AD compliant Configurable Patch Process: Central console for all activity customization packagesStep-by-step interview process Migrate functional Central console for all custom Deploys custom and standard configuration/setup data across patches Oracle patches from a central different instances of EBS Custom patches can be applied console using Patch Manager Group extracts, loads, and Simultaneous, multi-patch,reports into projects for easier Over 200 file types supported multi-target deployment management and reuse. Creating build lists with Integration with Customization Recognize and enforce the manifests Manager dependencies between different Build or compile Java or Form Multiple Patch Options: Support selection sets and provide libraries for wide variety of patchingsequencing recommendations. Validate the custom files based options such as Hot Patching, Automated, simultaneous loads on a set of standards disable archive mode.etc across multiple instances Interface with Version Control Centralized Patch Deployment Perform loads across different systems (CVS, Subversion) Tracking: In Progress,RUP levels Capture additional Metadata Completed, and Scheduled Share the extracts, loads, and around customization Integrates with MetaLink to reports content with other Standard and Comparison download patchesusers. reports 40. Application Management Pack for Siebel 41. Application Management Pack for Siebel Siebel-Specific Management Full Visibility into Siebel Server Components SARM Integration Enterprise / Server / Server Component Parameters Siebel Business Metrics 42. Application Management Pack for PeopleSoft 43. Application Management Pack for PeoplesoftPeoplesoft-Specific Management Full Visibility into PeopleSoft Components PeopleSoft System Parameter Change Tracking Out-of-box Configuration Policies Centralized Domain Administrationvs Support PeopleTools 8.48 & above 44. Projected Cost Savings Reduce User ReportedIncidents to ApplicationAdministrators by Over 50%*Costs Reduce Problem ResolutionTimes by Over 50%* Increase AdministratorProductivity by over 25%* 10 18 months payback* * actual savings, productivity improvements, and payback period may vary 45. Summary 46. Application Quality Management Pre-DeploymentPost-DeploymentPerformance Testing Performance Monitoring Real User Monitoring Real Application Testing Capture user performance Database Testing Identify key spot in applications Database Capture/Replay Alerting/Reporting SQL Queries Capture/ReplayService Level Management Load & Stress Testing Synthetic transactions Any web application Active Monitoring Server Performance metrics Collaborative testingApplication Management Packs Functional/Regression TestingSOA Application Management Onescript creation Easy record/playback Diagnostic and Tuning Test ManagementProvisioning, ConfigurationManagement Data Masking Testing prod data securely Change Trackingand Synchronization END-TO-END PERFORMANCE MANAGEMENT 47. Application Performance Testing & ManagementReal User Experience InsightApplicationTesting Suite EnterpriseManager Application Lifecycle 48. Oracle Helps You Maximize Customer Value Deploys SOASaves 80% time andAvoids online revenueinfrastructure 92% effort for managinglosses up to 25%faster DatabasesCuts configurationImproves ITDrives asset utilizationmanagement effort byproductivity by 25%up by 70%90%Saves $1.9 million Saves $170,000 per Replaces manualwith Oracle Enterprise year with Oracle tools with automation;ManagerEnterprise Manager saves time by 50%Saves weeks onReduces Database Reduces provisioningapplication testingtesting time by 90%effort by 75%timeCuts applicationDelivers 24/7 uptime Reduces criticaltesting from weeks to with Oracle Enterprise patching time by 80%hours Manager 49. Next Steps 50. Architecture: Network Protocol Analysis Collected data:Pagerequest Client-ip, Server-ip, Gateway-ipNetwork Response Time Requested URL (Page,Object) Cookie/Session-idInternet Referrer GET & POST requests User agents (phone type) Firewall SniffingSwitch Server Response Time Response? Server error, web site WebWeb Weberror, Hit OKserver serverserver Content of page (i.e orderdetails, functional errors) Delivered?Aborted? DBDBNetwork timeout? 51. Key Customers 250+ and growing