Post on 07-Jan-2016
description
Tools for Analyzing Tools for Analyzing Problems in Oracle Problems in Oracle
ApplicationsApplications
Jeff SlavitzJeff SlavitzJeff@OracleAppsPro.comJeff@OracleAppsPro.com
(415) 388-3003(415) 388-3003
High-level overview of:High-level overview of: Oracle DiagnosticsOracle Diagnostics Oracle Application ManagerOracle Application Manager StatspackStatspack Tracing Tracing
Working with supportWorking with support Other tools and resourcesOther tools and resources Bring up questions as we go alongBring up questions as we go along
AgendaAgenda
Oracle DiagnosticsOracle Diagnostics
Users can run Diagnostics on their own!Users can run Diagnostics on their own! Diagnostics v2.x replaces/supplements Diagnostics v2.x replaces/supplements
standalone diagnostic testsstandalone diagnostic tests Use for regression testingUse for regression testing Write your own DiagnosticsWrite your own Diagnostics
Installing Oracle DiagnosticsInstalling Oracle Diagnostics
Note 179661.1 is the portal to Diagnostics Note 179661.1 is the portal to Diagnostics knowledge. knowledge. Instructions on how to install the latest Instructions on how to install the latest
Diagnostics pack (comes out monthly)Diagnostics pack (comes out monthly) Diagnostic catalog showing a list of all Diagnostic catalog showing a list of all
diagnostics and their functiondiagnostics and their function Update your Diagnostics regularly. You Update your Diagnostics regularly. You
probably have a very old version.probably have a very old version.
What are Oracle Diagnostics?What are Oracle Diagnostics?
Diagnostics are divided into three Diagnostics are divided into three categories:categories: Setup Diagnostics Setup Diagnostics
• Examine profile values, general application setupExamine profile values, general application setup Data CollectionData Collection
• Examine invoice, customer, project, …Examine invoice, customer, project, … Activity DiagnosticsActivity Diagnostics
• Examine a process – period closing, invoicing, …Examine a process – period closing, invoicing, …
Logging into DiagnosticsLogging into Diagnostics
Users with Sysadmin responsibility can Users with Sysadmin responsibility can login through Oracle Application Managerlogin through Oracle Application Manager
Everybody can login using the URL:Everybody can login using the URL:http://<web-tier-host:port>/OA_HTML/jtfqalgn.htm
Click GO …
Click the Advanced tab …
…select an Application …
… then select a test from the column on the left.
• Fill in parameters and click Run Test• Some tests require a user to have a particular responsibility
When test is complete click View Report
Review report and messages. Some ‘problems’ found are really warnings.
Users can run Setup, Activity and Data Collection tests.
You can email, print and save test output
Oracle Diagnostics SummaryOracle Diagnostics Summary
Available to end-users and DBAsAvailable to end-users and DBAs Use for proactive and reactive testingUse for proactive and reactive testing Monthly updates from SupportMonthly updates from Support Customize with your own DiagnosticsCustomize with your own Diagnostics
Oracle Application ManagerOracle Application Manager
Oracle Applications ManagerOracle Applications Manager
Variety of monitoring, analysis and Variety of monitoring, analysis and administration toolsadministration tools
Wealth of informationWealth of information Workflow setup and monitoringWorkflow setup and monitoring Downtime managementDowntime management Spawns concurrent request Spawns concurrent request
‘‘OAM Application Dashboard Collection’OAM Application Dashboard Collection’
(verify only one!)(verify only one!)
Go to Rapid Install Portal page. Click Apps Logon Links.
Click ’Oracle Applications Manager’
and login ….
Alternatively, login from Sysadmin screen
Main OAM login screen
Click ‘Site Map’. This is the Administration screen.
Click ‘License Manager’ to license new products.
Alternative to adlicmgr.sh
Click ‘Autoconfig’ to update your context file
Nicer interface than the old context editor
From the Site Map, this is the Monitoring screen.
Monitor SQL Activity
Monitor JServ usage
Monitor Database Sessions. Click on AUDSID ...
… for details of a database session.
Concurrent Processing usage
Back to Site Map – Maintenance screen
Click Applied Patches and then Timing Details to find the details of a particular patch application (useful for upgrade timing!).
Click Timing Reports to view adadmin activity
Diagnostics and Repair screen
OAM SummaryOAM Summary
Lots of toolsLots of tools Lots of dataLots of data Use site map to find tools that are useful to Use site map to find tools that are useful to
youyou Watch that you never have more than one Watch that you never have more than one
OAM Application Dashboard Collection OAM Application Dashboard Collection concurrent request runningconcurrent request running
StatspackStatspack
StatspackStatspack
Creates permanently stored database Creates permanently stored database performance statistical informationperformance statistical information
Uses ‘snapshots’ to report on performanceUses ‘snapshots’ to report on performance Adjust time between snapshots based on Adjust time between snapshots based on
reporting needsreporting needs General performance use long time (1 hour+)General performance use long time (1 hour+) Specific problem use short time (15 minutes)Specific problem use short time (15 minutes)
Installing StatspackInstalling Statspack
Note 228913.1 is your portal to Statspack Note 228913.1 is your portal to Statspack knowledgeknowledge
In init.ora set TIMED_STATISTICS=TRUEIn init.ora set TIMED_STATISTICS=TRUE Create new tablespace for Statspack userCreate new tablespace for Statspack user Install StatspackInstall Statspack
connect / as sysdbaconnect / as sysdba @$ORACLE_HOME/rdbms/admin/spcreate@$ORACLE_HOME/rdbms/admin/spcreate
Creates the user PERFSTAT which owns Creates the user PERFSTAT which owns all Statspack dataall Statspack data
Running StatspackRunning Statspack
sqlplus perfstat/perfstatsqlplus perfstat/perfstat execute statspack.snapexecute statspack.snap [wait some amount of time][wait some amount of time] execute statspack.snapexecute statspack.snap
Consider scheduling with cron or using Consider scheduling with cron or using $ORACLE_HOME/rdbms/admin/spauto.sql$ORACLE_HOME/rdbms/admin/spauto.sql
Running StatspackRunning Statspack
For more detailed data collection specify a For more detailed data collection specify a level when executing snap.level when executing snap. 0 gathers general performance data0 gathers general performance data 5 (default) additionally gathers info on high 5 (default) additionally gathers info on high
resource usage SQL statementsresource usage SQL statements 6 additionally gathers execution plan 6 additionally gathers execution plan
information for statements found in level 5information for statements found in level 5 10 additionally gathers child latches10 additionally gathers child latches
Running StatspackRunning Statspack
To specify a level:To specify a level:execute statspack.snap(i_snap_level=>10)execute statspack.snap(i_snap_level=>10)
There are other parameters:There are other parameters: Capture data for a specific session onlyCapture data for a specific session only Define threshholds for which level 5 snaps Define threshholds for which level 5 snaps
consider high usage SQL statementsconsider high usage SQL statements Set note 149121.1 for more detail on Set note 149121.1 for more detail on
running ‘snap’.running ‘snap’.
Generating a Statspack ReportGenerating a Statspack Report Now you’ve got all this great data, now what?Now you’ve got all this great data, now what?
sqlplus perfstat/pwdsqlplus perfstat/pwd @$ORACLE_HOME/rdbms/admin/spreport@$ORACLE_HOME/rdbms/admin/spreport
Previously run snapshots will be displayed. You Previously run snapshots will be displayed. You will be prompted for:will be prompted for: The beginning snapshot IdThe beginning snapshot Id The ending snapshot IdThe ending snapshot Id The name of the report text file to be created The name of the report text file to be created
Begin and End snapshots must not include an Begin and End snapshots must not include an instance shutdown during that time periodinstance shutdown during that time period
See note 149124.1 for more detail on spreportSee note 149124.1 for more detail on spreport
Statspack OutputStatspack Output
spreport generates a LOT of output with a spreport generates a LOT of output with a LOT of good information.LOT of good information. Instance cache sizeInstance cache size Load profile (reads, writes, …)Load profile (reads, writes, …) Instance efficiency ratio (buffer hit %, buffer Instance efficiency ratio (buffer hit %, buffer
nowait %, …)nowait %, …) Top 5 events and much moreTop 5 events and much more
See note 228913.1 for more detailSee note 228913.1 for more detail
Statspack SummaryStatspack Summary
Schedule on a regular basis to monitor Schedule on a regular basis to monitor general performancegeneral performance
Review data with spreport on a regular Review data with spreport on a regular basisbasis
Use when unknown system performance Use when unknown system performance problemproblem
TracingTracing
TracingTracing
Records every SQL statement executed to Records every SQL statement executed to a filea file
Records timing information if Records timing information if TIMED_STATISTICS is trueTIMED_STATISTICS is true
Allows you to see what is happening Allows you to see what is happening inside a report, concurrent program or inside a report, concurrent program or form form
The easy part is getting the trace file The easy part is getting the trace file Review note 117129.1 for more detailsReview note 117129.1 for more details
TracingTracing
Tracing concurrent programsTracing concurrent programs Tracing within FormsTracing within Forms Tracing within Self-ServiceTracing within Self-Service
Concurrent Program TracingConcurrent Program Tracing
Forms TracingForms Tracing
Enable Trace, perform activity to be traced …
… … and then disable traceand then disable trace
Self-Service TracingSelf-Service Tracing
Set profile option FND: Diagnostics to Yes Set profile option FND: Diagnostics to Yes at any level. Site allows everybody to do at any level. Site allows everybody to do this.this.
In Self-Service, click on Diagnostics iconIn Self-Service, click on Diagnostics icon Select Set Trace Level Select Set Trace Level Execute process in Self-ServiceExecute process in Self-Service Click on Diagnostics icon to turn off traceClick on Diagnostics icon to turn off trace
Now what?Now what?
Trace file located in udump directory. Trace file located in udump directory. Check init.ora for location orCheck init.ora for location or
SELECT value SELECT value
FROM v$parameterFROM v$parameter
WHERE name=‘user_dump_dest’WHERE name=‘user_dump_dest’
A single trace may result in multiple trace A single trace may result in multiple trace files, especially with self-servicefiles, especially with self-service
Use ‘tkprof’ to convert trace file to a more Use ‘tkprof’ to convert trace file to a more useful formatuseful format
TkprofTkprofUsage: tkprof tracefile outputfile [explain= ] [table= ]Usage: tkprof tracefile outputfile [explain= ] [table= ] [print= ] [insert= ] [sys= ] [sort= ][print= ] [insert= ] [sys= ] [sort= ] table=schema.tablename Use 'schema.tablename' with 'explain=' option.table=schema.tablename Use 'schema.tablename' with 'explain=' option. explain=user/password Connect to ORACLE and issue EXPLAIN PLAIN.explain=user/password Connect to ORACLE and issue EXPLAIN PLAIN. print=integer List only the first 'integer' SQL statements.print=integer List only the first 'integer' SQL statements. insert=filename List SQL and data inside INSERT statements.insert=filename List SQL and data inside INSERT statements. sys=no Do not list SQL statements run as user SYS.sys=no Do not list SQL statements run as user SYS. sort=option Sort optionssort=option Sort options
For example:For example:
tkprof inputfile outputfile explain=apps/xxx sort='(prsela,exeela,fchela)'tkprof inputfile outputfile explain=apps/xxx sort='(prsela,exeela,fchela)'
SQL Statement
SELECT A.JOB_ID, A.ORGANIZATION_IDFROM PER_ASSIGNMENT_STATUS_TYPES S , PER_ASSIGNMENTS_F A , PA_IMPLEMENTATIONS I WHERE A.JOB_ID IS NOT NULL AND A.PRIMARY_FLAG = 'Y' AND TRUNC(:B2 ) BETWEEN TRUNC( A.EFFECTIVE_START_DATE ) AND TRUNC( A.EFFECTIVE_END_DATE ) AND A.PERSON_ID = :B1 AND A.BUSINESS_GROUP_ID = I.BUSINESS_GROUP_ID AND A.ASSIGNMENT_TYPE = 'E' AND S.PER_SYSTEM_STATUS = 'ACTIVE_ASSIGN' AND S.ASSIGNMENT_STATUS_TYPE_ID = A.ASSIGNMENT_STATUS_TYPE_ID
Timing in CPU seconds
call count cpu elapsed disk query current rows------- ------ -------- ---------- ---------- ---------- ---------- ----------Parse 1 0.00 0.00 0 0 0 0Execute 406 0.03 0.02 0 0 0 0Fetch 406 509.91 503.79 106 50344 0 406------- ------ -------- ---------- ---------- ---------- ---------- ----------total 813 509.94 503.82 106 50344 0 406
Tkprof outputTkprof output
Tkprof outputTkprof output Execution PlanExecution Plan
Rows Row Source OperationRows Row Source Operation------- ---------------------------------------------------------- --------------------------------------------------- 406 NESTED LOOPS406 NESTED LOOPS 406 NESTED LOOPS406 NESTED LOOPS 406 TABLE ACCESS FULL PA_IMPLEMENTATIONS_ALL406 TABLE ACCESS FULL PA_IMPLEMENTATIONS_ALL 406 TABLE ACCESS BY INDEX ROWID PER_ALL_ASSIGNMENTS_F406 TABLE ACCESS BY INDEX ROWID PER_ALL_ASSIGNMENTS_F1500982 INDEX RANGE SCAN PER_ASSIGNMENTS_F_FK1 (object id 42980)1500982 INDEX RANGE SCAN PER_ASSIGNMENTS_F_FK1 (object id 42980) 406 TABLE ACCESS BY INDEX ROWID 406 TABLE ACCESS BY INDEX ROWID
PER_ASSIGNMENT_STATUS_TYPESPER_ASSIGNMENT_STATUS_TYPES 406 INDEX UNIQUE SCAN PER_ASSIGNMENT_STATUS_TYPE_PK (object 406 INDEX UNIQUE SCAN PER_ASSIGNMENT_STATUS_TYPE_PK (object
id 43099)id 43099)
For more details on tkprof review note 117129.1For more details on tkprof review note 117129.1 For more details on explain plan review note 46234.1For more details on explain plan review note 46234.1
Working with SupportWorking with Support
Help is just a phone call away
Working with SupportWorking with Support
Make the most out of Metalink search Make the most out of Metalink search abilityability Use Advanced Search Use Advanced Search Enter exact text of error messageEnter exact text of error message Specify form or report name, if knownSpecify form or report name, if known Search Technical ForumsSearch Technical Forums
Open Service Request in parallel with Open Service Request in parallel with trying to solve problemtrying to solve problem
Working with SupportWorking with Support
Recognize Support is busyRecognize Support is busy Some problems are easier to solve than Some problems are easier to solve than
othersothers Be nice – the Support community is smallBe nice – the Support community is small If problem is outside of analyst’s area of If problem is outside of analyst’s area of
expertise help him realize that and request expertise help him realize that and request a SR transfera SR transfer
If analyst’s time zone is an issue, request If analyst’s time zone is an issue, request a SR transfera SR transfer
Working with SupportWorking with Support
If not getting prompt response you may need to If not getting prompt response you may need to more aggressively manage your SRmore aggressively manage your SR Frequent updates to SR for statusFrequent updates to SR for status Three updates without analyst response changes SR Three updates without analyst response changes SR
to immediate response requiredto immediate response required
You need to call 800 number to see if analyst is You need to call 800 number to see if analyst is out, to ping analyst or to duty manage SRout, to ping analyst or to duty manage SR
For repeated problems have your manager talk For repeated problems have your manager talk with the Oracle customer repwith the Oracle customer rep
Other Tools and ResourcesOther Tools and Resources
Third party products such as FoglightThird party products such as Foglight googlegoogle asktom.oracle.comasktom.oracle.com www.oaug.org/listserverswww.oaug.org/listservers NorCalOAUG and OAUG eventsNorCalOAUG and OAUG events
Questions?Questions?Other tips and techniques?Other tips and techniques?
Want a copy of this presentation?Want a copy of this presentation?
Feel free to email or call me …Feel free to email or call me …
Jeff SlavitzJeff SlavitzJeff@OracleAppsPro.comJeff@OracleAppsPro.com
(415) 388 - 3003(415) 388 - 3003