Spotlight on Oracle Getting Started

Spotlight on Oracle 9.5

Getting Started Guide

2013 Quest Software, Inc.ALL RIGHTS RESERVED.

This guide contains proprietary information protected by copyright. The software described inthis guide is furnished under a software license or nondisclosure agreement. This software may beused or copied only in accordance with the terms of the applicable agreement. No part of thisguide may be reproduced or transmitted in any form or by any means, electronic or mechanical,including photocopying and recording for any purpose other than the purchasers personal usewithout the written permission of Quest Software, Inc.

The information in this document is provided in connection with Quest products. No license,express or implied, by estoppel or otherwise, to any intellectual property right is granted by thisdocument or in connection with the sale of Quest products. EXCEPT AS SET FORTH INQUEST'S TERMS AND CONDITIONS AS SPECIFIED IN THE LICENSE AGREEMENT FORTHIS PRODUCT, QUEST ASSUMES NO LIABILITY WHATSOEVER AND DISCLAIMSANY EXPRESS, IMPLIED OR STATUTORY WARRANTY RELATING TO ITS PRODUCTSINCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTY OFMERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT. IN NO EVENT SHALL QUEST BE LIABLE FOR ANY DIRECT,INDIRECT, CONSEQUENTIAL, PUNITIVE, SPECIAL OR INCIDENTAL DAMAGES(INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF PROFITS, BUSINESSINTERRUPTION OR LOSS OF INFORMATION) ARISING OUT OF THE USE ORINABILITY TO USE THIS DOCUMENT, EVEN IF QUEST HAS BEEN ADVISED OF THEPOSSIBILITY OF SUCH DAMAGES. Quest makes no representations or warranties with respectto the accuracy or completeness of the contents of this document and reserves the right to makechanges to specifications and product descriptions at any time without notice. Quest does notmake any commitment to update the information contained in this document.

If you have any questions regarding your potential use of this material, contact:

Quest Software World HeadquartersLEGAL Dept5 Polaris WayAliso Viejo, CA 92656email: [email protected]

Refer to our Web site (www.quest.com) for regional and international office information.

Patents

Protected by U.S. Patents #6,901,582, 7,784,027; 8,181,154; 8,015,454 and 8,347,148.Additional patents pending.

Trademarks

Quest, Quest Software, the Quest Software logo and Spotlight are trademarks and registeredtrademarks of Quest Software, Inc in the United States of America and other countries. For acomplete list of Quest Softwares trademarks, please seehttp://www.quest.com/legal/trademark-information.aspx. Other trademarks and registeredtrademarks are property of their respective owners.

Spotlight 9.5Getting Started GuideFriday, 31 May 2013

Table of ContentsWelcome 9

Oracle Architecture Overview 9

Spotlight on Oracle Overview 10

Home Page 10

Alarms 11

Drilldowns 11

Features of Spotlight 12

Features of Spotlight on Oracle 12

Predictive Diagnostics 12

DTrace Analysis (Unix Solaris 10 And Later) 13

Install Spotlight 14

Requirements 14

Steps To Install 15

Multiple Concurrent Installations 16

MSI Installers for Spotlight on Oracle 16

Uninstall Spotlight 16

Start Spotlight 17

Project Lucy 17

User Experience Feedback 18

Authorize Spotlight 18

Spotlight Users 19

Oracle User Wizard 19

Spotlight on Oracle 22

Connect to Oracle Instance 22

Add A New Connection 22

Spotlight on Oracle Getting Started GuideTable of Contents

5

Configuring Oracle User for Spotlight Dialog 26

Toad DBA Suite for Oracle 27

Install And Configure DTrace Scripts 28

Spotlight on Oracle Home Page 29

Sessions Panel 30

Host Panel 30

Server Processes Panel 31

SGAPanel 32

Background Processes Panel 33

Disk Storage Panel 34

Spotlight on Oracle Alarms 34

Spotlight on Oracle Drilldowns 41

Spotlight Tools 45

Tools |Explain Plan 45

Tools |SQLOptimizer 45

Tools |Space Manager 46

Tune Your Database 46

Troubleshooting 48

Connection Problems 48

Dead Sessions in Top Sessions Drilldown 49

Data That Does Not Appear in The History Browser 49

Predict Database Performance 50

Using Predictive Diagnostics 50

Predictive Diagnostics And Spotlight 51

Requirements for Predictive Diagnostics 52

Troubleshooting 53

Predictive Diagnostics Unavailable 53


6

Spotlight on Oracle RAC 54

What is Spotlight on Oracle RAC? 54

What Is Oracle RAC? 54

Connect to Spotlight on Oracle RAC 55

Spotlight on Oracle RAC Home Page 60

Interconnect and Global Status Panel 61

Instance Panels 62

IO Subsystem Panel 63

Exadata Storage Panel 63

Spotlight on Oracle RAC Alarms 63

Alarms on A Database Instance 64

Alarms on The Oracle RAC 64

Spotlight on Oracle RAC Drilldowns 67

Troubleshooting 69

Average Ping Time Chart Not Populated 69

Spotlight on Windows 71

Connect to Windows Systems 71

Add A New Connection 71

Spotlight on Windows Home Page 73

System Panel 73

Network Panel 73

Event Log Panel 74

CPUPanel 74

Memory Panel 75

Paging Files Panel 75

Disks Panel 76

Files Panel 76


7

Spotlight on Windows Alarms 76

Spotlight on Windows Drilldowns 77

Troubleshooting Spotlight on Windows 78

Issues Connecting 78

Issues Monitoring a Windows Machine 81

Spotlight on Unix 84

Configure the Unix System 84

Connect to Unix Systems 86

Add a new connection 86

Spotlight on Unix Home Page 88

System Panel 88

Network Panel 88

CPUPanel 89

Memory Panel 90

Disk Activity Panel 90

Swap Space Panel 91

Spotlight on Unix Alarms 91

Spotlight on Unix Drilldowns 93

Spotlight on MySQL 95

Connect to a MySQLDatabase 95

Spotlight on MySQL Home Page 97

Sessions Panel 97

MySQL Panel 98

InnoDB Panel 99

Storage Panel 100

Host Panel 100

Spotlight on MySQL Alarms 101


8

Spotlight on MySQL Drilldowns 102

Troubleshooting 103

Monitoring 103

Disabled Controls 103

Alarms Not Raised 104

Delay in Reporting Alarms 105

Alarm Help Is Displayed Instead of The Drilldown 105

Spotlight Home Page Hard To Read 105

NoChange to Data Flows 106

Operating System Connection / Statistics 106

I Cannot Open a Migrated Connection 108

Spotlight Consumes Too Many Resources on Either The Client Or The Server 108

Support 109

Using Trace To Add Details To Support Bundle 109

Build the Support Bundle 109

Appendix: Contact Quest 111

Contact Quest Support 111

Spotlight Support 111

Contact Quest Software 112

About Quest Software 112

Index 113

WelcomeSpotlight on Oracle is Quest Softwares powerful diagnostic and problem-resolution tool forOracle databases. Its unique user interface provides you with an intuitive, visual representation ofthe activity on your database and operating system.

Oracle Architecture OverviewThe following diagram illustrates some of the basic components of an Oracle server at thememory, database and disk layers. This architecture has been used to design the Spotlight onOracle home page.

The numbers indicate the order of the flow of information.

The numbered labels in the Oracle architecture diagram correspond to the following activities:

1. The client program (for example, SQL*PLUS, Oracle Power Objects, or some other tool)sends a SELECT statement to the server process.

2. The server process looks in the shared pool for a matching SQL statement. If none isfound, the server process parses the SQL and inserts the SQL statement into theshared pool.

3. The server process looks in the buffer cache for the data blocks required. If found, thedata block must be moved on to the most recently used end of the Least RecentlyUsed (LRU) list.

1

Spotlight on Oracle Getting Started GuideWelcome

10

4. If the block cannot be found in the buffer cache the server process must fetch it from thedisk file. This requires a disk I/O.

5. The server process returns the rows retrieved to the client process. This may involve somenetwork or communications delay.

6. When the client issues the UPDATE statement the process of parsing the SQL andretrieving the rows to be updated must occur. The update statement then changes therelevant blocks in shared memory and updates entries in the rollback segment buffers.

7. The update statement also makes an entry in the redo log buffer that records thetransaction details.

8. The database-writer background process copies modified blocks from the buffer cache tothe database files. The Oracle session performing the update does not have to wait forthis to occur.

9. When the COMMIT statement is issued the log writer process must copy the contents ofthe redo log buffer to the redo log file. The COMMIT statement does not return control tothe Oracle session issuing the commit until this write is complete.

10. If running in ARCHIVELOG mode, the archiver process copies full redo logs to thearchive destination. A redo log is not eligible for re-use until it has been archived.

11. At regular intervals, or when a redo log switch occurs, Oracle performs a checkpoint. Acheckpoint requires all modified blocks in the buffer cache to be written to disk. A redolog file cannot be re-used until the checkpoint completes.

Recommended Reading

The best information to help you tune and configure Oracle databases can be found online. Useas your first point of reference web sites such as:

l The Quest Software Oracle Solutions Web site.

l The Oracle home page.

Spotlight on Oracle OverviewSpotlight on Oracle makes it possible for you to view your database at different levels of detail.

Home PageThe Spotlight home page is a visual representation of the status of the major components of thedatabase. The detail at this level is designed to help you locate and identify bottlenecks as theydevelop. Additionally, Spotlight provides tools that allow you to identify and rectifyinefficiency in your databases configuration.

If Spotlight detects that some component of the Oracle database constitutes a performancebottleneck, the corresponding component generates a visual or auditory alarm. The exactappearance of the alarm depends on how you have configured alarm severities. When an alarm iscurrent you can press F1 to display help for the alarm. Clicking the component (on the homepage ) displays the relevant drilldown.


11

See "Spotlight on Oracle Home Page" (page 29) for more information.

The home page display is based on the Oracle database architecture. See "Oracle ArchitectureOverview" (page 9) for more information.

AlarmsSpotlight alarms indicate when the Oracle database exceeds acceptable performance thresholds.

See "Spotlight on Oracle Alarms" (page 34) for more information.

DrilldownsOnce a problem is isolated you can display a drilldown page with charts and tables that providea detailed breakdown of the underlying statistics.

For example, use the Activity drilldown to display overall efficiency and resource usage. Inparticular, the Event waits chart shows the amount of time database sessions spend waiting forvarious resources, as well as the amount of CPU being utilized. Efforts at tuning the database aremost likely to succeed when they concentrate on the resources being used most heavily and theWait activity topic contains a discussion of the meaning and implications of various wait events.

Other pages help you examine detailed activity. For instance, the Top Sessions drilldownidentifies the users who contribute most to database activity. You can display various details forthese sessions, including the SQL statements, locks and resource usage.

See "Spotlight on Oracle Drilldowns" (page 41) for more information.


12

Features of Spotlight

Features of Spotlight on OracleSpotlight is designed for ease of installation and use, and offers a range of tools forproblem diagnosis.

Some of the main features and benefits of Spotlight on Oracle are that it:

l Presents a visual representation of process flows within an Oracle instance, allowing youto observe actual database activity in real time.

l Visually identifies bottlenecks and provides extensive drilldown capabilities.

l Displays the details of problem areas, including most active sessions, inefficient SQL,locks & latches, and disk I/O, for rapid problem resolution.

l Provides visual and audible warnings to alert you when performance metrics exceedacceptable thresholds.

l Provides detailed information about a specific component allowing you to pinpoint thesource of problems.

l Learns the normal range of values for your database.

l Has a quick and simple installation process.

l Can be launched from Quest Softwares Foglight for 24x7, unattended monitoring ofsystems, databases, and applications.

l Has UTF8 support.

Contact Quest Software's support team if you need more information than is supplied in thishelp system.

Predictive DiagnosticsPredictive Diagnostics allows Spotlight to collect and analyze performance metrics for:

l SQL statements as they are executed on the database instance.

l System bottlenecks as they affect database throughput and response time.

l Database resources as they reach their full capacity and affect database throughput andconcurrency.

When enough performance data is collected, Spotlight may be able to predict the futureperformance for individual SQL statements on that database, for potential bottlenecks in thesystem, and for database CPU, memory, and disk I/O resources.

Note: When you start using Predictive Diagnostics, Spotlight will take about 10 days to collectenough data to make valid predictions on the future performance of the database.


13

DTrace Analysis (Unix Solaris 10 And Later)DTrace is a dynamic analysis tool for Sun Microsystems operating systems. It was firstintroduced in the Solaris 10 operating system, and can be used ONLY on Sun platforms fromSolaris 10 onwards.

Spotlight and DTrace

Spotlight on Oracle uses DTrace to capture details of the interaction between Oracle and theoperating system.

When Oracle sessions use CPU time, most of that elapsed time is spent executing Oracle code.However, Oracle sometimes uses system calls to perform functions such as:

l Allocating or deallocating private (PGA) memory.

l Synchronizing activities with other processes through semaphores or other IPCmechanisms.

l Allocating shared memory (SGA).

l Performing I/O operations on datafiles, redo logs, raw devices, and so on.

l Capturing CPU or elapsed timing to support the time model and wait interfaces.

Spotlight uses DTrace to investigate these system calls to calculate where that elapsed time isbeing spent.

Install and Configure DTrace scripts

Before you can capture the details of the interaction between Oracle and the Solaris operatingsystem, you need to Install And Configure DTrace Scripts (page 28).

Install Spotlight

RequirementsWindows 8 / Windows 7 / Windows Vista / Windows Server 2012 and 2008 :

Section Requirement

User Privileges Ensure the Windows user installing Spotlight hasWindows Administrator privileges.

Folder Permissions During installation, Spotlight requires write access tothe following:

l The Spotlight installation folder (by default,Program Files\Quest Software\Spotlight)

l Program Files\Common Files\Quest Shared

l ProgramData\Quest Software\Spotlight

l Users\\AppData\Roaming\QuestSoftware\Spotlight

Note:For 64-bit operating systems the Program Filesdirectory is Program Files (x86).

Permissions to Windowsregistry keys and sub-keys

During installation, Spotlight requires write access tothe following:

l HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\QuestSoftware\Products

l HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\QuestSoftware\Spotlight

l HKEY_CLASSES_ROOT

2

Spotlight on Oracle Getting Started GuideInstall Spotlight

15

Windows XP / Windows Server 2003:

Section Requirement

User Privileges Ensure the Windows user installing Spotlight hasWindows Administrator privileges.

Folder Permissions During installation, Spotlight requires write access tothe following:

l The Spotlight installation folder (by default,Program Files\Quest Software\Spotlight)

l Program Files\Common Files\Quest Shared

l Documents and Settings\All Users\ApplicationData\Quest Software\Spotlight

l Documents and Settings\\Application Data\QuestSoftware\Spotlight

Permissions to Windows registrykeys and sub-keys

During installation, Spotlight requires write access tothe following:

l HKEY_LOCAL_MACHINE\SOFTWARE\QuestSoftware\Products

l HKEY_LOCAL_MACHINE\SOFTWARE\QuestSoftware\Spotlight

l HKEY_CLASSES_ROOT

Steps To InstallTo install Spotlight from CD:

1. Close all open applications on your PC.

2. Insert the Spotlight Installer CD into your CD-ROM drive.

If the installation program does not start select Windows Start | Run

l For Spotlight delivered as part of the Toad DBA Suite, enter the commandd:\TOAD DBA Suite for Oracle vvv.exe (where d: is the CD-ROM identifier andvvv is the Spotlight version).

l For Spotlight delivered as a standalone installer, enter the commandd:\SpotlightonOracle_vvv.exe (where d: is the CD-ROM identifier and vvv is theSpotlight version).

3. Follow the instructions on the screen to complete the installation.

Spotlight on Oracle Getting Started GuideInstall Spotlight

16

To install Spotlight from the downloaded Quest Spotlight installer from the Quest Web site:

1. Close all open applications on your PC.

2. Execute the downloaded installer.

3. Follow the instructions on the screen to complete the installation.

Note:To cancel installation at any time, press Esc or click Cancel.

Multiple Concurrent InstallationsThere has been a major shift in the architecture from Spotlight on Oracle 9.5. You can installmajor and minor versions of Spotlight on Oracle side by side (or upgrade). Patch and progressivebuilds will upgrade.

Note:If Spotlight is delivered as part of the Toad DBA Suite and you require multiple Spotlightinstallations on your computer then contact Quest Software for more information.

MSI Installers for Spotlight on OracleMSI installers are available for Spotlight on Oracle. These installers can be run silently, andproduce an installation that can be captured and re-used via automatic deployment techniques.

Contact Quest Software for more information.

Uninstall SpotlightTo remove the application and all associated files from your machine, if required.

To uninstall Spotlight

1. Open Windows Control Panel.

2. Select Programs and Features. (On Windows XP select Add/Remove Programs.)

3. Click Spotlight | Remove | Yes.

Spotlight on Oracle Getting Started GuideStart Spotlight

17

Start SpotlightTo start Spotlight:

Double click on the Spotlight desktop icon

or

On the Windows task bar, click Start | Programs |Quest Software | Spotlight |Spotlight.

During operation

Spotlight requires write access to the following:

Windows Server 2012, Windows Server 2008,Windows Server 2003, Windows 8, Windows 7,Windows Vista

Users\\AppData\Roaming\QuestSoftware\Shared

Users\\AppData\Roaming\QuestSoftware\Spotlight

Windows XP Documents and Settings\\Application Data\QuestSoftware\Shared

Documents and Settings\\Application Data\QuestSoftware\Spotlight

You should also verify that you can run the required Oracle client on the Spotlight machine.

Project LucyThe first time you run Spotlight following install you will be prompted to upload data to ProjectLucy. Project Lucy is a free service provided by Quest Software that analyzes your performancedata, gives you reports and insight into your performance and allows you to compare yourexperience to everyone else that uses the service. Performance data is uploaded to Project Lucyonce a day.


18

Actions Description

To open the Project Lucy home page, in the SpotlightBrowser click Welcome to Spotlight | Spotlight on theWeb | Project Lucy.

l Learn more aboutProject Lucy and howit can benefit you.

l Login to your ProjectLucy account and useProject Lucy services.

l Create a Project Lucyaccount.

To configure uploading data from Spotlight on Oracle toProject Lucy, click View | Options | Project Lucy.

See "Project Lucy" in theonline help for moreinformation.

User Experience FeedbackYou can influence the design of future versions of Spotlight and help us improve its quality,reliability and performance. To participate or change your mind select Help | UserExperience Feedback.

Authorize SpotlightSpotlight is supplied with a time-limited trial key so you may test the product. When the trialkey expires Spotlight reverts to a preview mode, which limits access to the Spotlight home pageand one connection at a time. If you select to open a drilldown in preview mode the followingmessage is displayed:

The current license does not allow access to this functionality.

Contact your Quest Software representative to obtain the necessary authorization keys to ensureSpotlight remains fully functional when the trial key expires.

To enter a new authorization key:

While Spotlight is running

1. Click Help | About Spotlight.

2. Ensure Installed Products is to the front.

3. Select View / change product license.

4. Click Change this license.

5. Enter the Authorization key.

Notes:


19

l The Windows user that updates the authorization key must have administrator access tothe Windows registry and write access to the Console\Licenses sub-folder of theSpotlight installation folder.

l Make sure you enter the authorization key exactly as stated on your ProductAuthorization sheet or as provided by your Quest Software distributor. The site messageis case sensitive. If you enter the authorization key incorrectly, the following message isdisplayed:Invalid authorization key

l If Spotlight is in preview mode and the trial period has not expired, check your computersystem date is set correctly.

Spotlight UsersSpotlight on Oracle connects to the Oracle database system through the Spotlight user.

l You should not use any system accounts. We recommend creating a separate user forSpotlight or using a personal account.

l The Spotlight user must have sufficient privileges, views, and tables to diagnose theOracle database. You can view the privileges, views and tables required by Spotlightusers in the Database Objects and Privileges document in the Spotlight installationfolder: \Plug-ins\SoO\.

The Oracle User Wizard configures Spotlight users.

Oracle User WizardThe Oracle User Wizard configures Spotlight users. Use the Oracle User Wizard to:

l Add a new user to the Oracle database

l Convert the account of an existing user on the Oracle database to use Spotlight

l Reconfigure an existing Spotlight user to work with new versions of Spotlight onOracle and Spotlight on Oracle RAC. Use this procedure if you have upgraded yourversion of Spotlight.

Notes:

l Spotlight users must be configured for each database they are to diagnose.

l You cannot configure Spotlight users for a database while Spotlight is diagnosingthat database.

l You must have the user ID and password of the DBA that manages the database to whichyou are adding the Spotlight user.

l Select Don't show this page in future to hide the first page of the Oracle User Wizard.

To fill in the Oracle User Wizard

The Oracle User Wizard runs automatically when required or to initiate it:


20

1. Click File | Oracle User Wizard...

2. At the DBA connection credentials page:

Field Description

Enter theSQL*Netconnectionstring

Enter the database instance on which you wish to create the user.

Enter the username of aDBAaccount

Enter a DBA user name for that database instance (so that theSpotlight user and the relevant objects can be created). The username requires SYSDBA access.

Enter thepassword

Enter the DBA user password.

3. Select Create a new user

4. At the New oracle user page:

Field Description

User name Enter a name for the new user.

Password Enter a password for the new user.

Confirm password Confirm the password.

Select any table Select to allow the user to generate explain plans.

Alter system Select to allow the user to trace and terminate sessions.

5. At the Select tablespace page:

Field Description

Spotlight tables The default tablespace is USERS, if available.

temporary segments The default temporary tablespace is TEMP, if available.

6. At the Predictive Diagnostic Settings page:

Select Create/Upgrade Predictive Diagnostics schema to employ Predictive Diagnosticson this database instance.

7. Select to use an existing Predictive Diagnostics schema (recommended) or installa new schema.


21

8. Select to install a set of sample data (which you can use to experiment with the PredictiveDiagnostics feature).

9. Select the settings Spotlight will use to schedule the Oracle jobs that collect data on SQLperformance and impending bottlenecks.

Spotlight on Oracle

Connect to Oracle Instance1. From the Spotlight Browser select All Connections |Spotlight on Oracle

2. Select the database. If the database is not on the list then add a new connection.

Notes: (Now the database is listed in the Spotlight Browser |Live connections. .)

l Click Set timed statistics now if you are prompted to do so. You see this promptwhen for the Spotlight user the timed_statistics parameter is set to false in theinit.ora file. Spotlight requires this parameter be true for the Spotlight user toaccurately monitor the database. Clicking Set timed statistics now corrects theproblem only if the Spotlight user has Alter System privileges. Ensure theSpotlight user has sufficient privileges. See "Spotlight Users" (page 19) for moreinformation.

l Spotlight operates at reduced functionality once the initial trial period has expired.See "Authorize Spotlight" (page 18) for more information.

Add A New ConnectionNotes: (Before you add a new connection)

l Ensure you create a Spotlight user with sufficient privileges to diagnose the Oracledatabase. See "Spotlight Users" (page 19) for more information.

l Ensure the Unix or Windows server on which the Oracle database is installed isaccessible to Spotlight on Oracle. See the Spotlight on Unix or Spotlight on WindowsGetting Started Guide.

3

Spotlight on Oracle Getting Started GuideSpotlight on Oracle

23

1. Click File |Connect

2. Select Spotlight on Oracle on the Connections menu.

3. Double-click Add new connection.

4. Ensure Details is selected. Fill in the Properties window.

Oracle connection details

Field Description

ConnectionString

Select the name of the database that you wish to connect to.Note: When selected, the Connection name field resets to this value.You can then edit the Connection name field if required.

OracleUsername

Enter the Spotlight user name. See "Spotlight Users" (page 19) formore information.Tip: If you enter a Username that does not exist, Spotlight willprompt you when you complete this dialog to create a new Spotlightuser.

OraclePassword

Enter the Spotlight user password. See "Spotlight Users" (page 19) formore information.

OracleHome

Select an appropriate Oracle Home. Select a location where Oracleclient software is installed.Note:Choose a home that contains a version of the Oracle client thatis compatible with the Oracle server.

ASM connection details

ASM is built into the Oracle database kernel. It simplifies the creation and spacemanagement in Oracle databases.

Note: Available only in Oracle 10g and later.


24

Field Description

MonitorASM

Select to monitor the performance of Oracle's Automated StorageManagement (ASM) technology.

ASMConnectionString

Enter the connection string required to link to the ASM databaseinstance that is used for storage management for this Oracle database.Note:The TNSNAMES entry for the ASM instance must include theoption (UR = A) for the Spotlight connection to work properly.

ASMPassword

Enter the user password for Spotlight to login to the ASM databaseinstance.The user that Spotlight uses to login to the ASM database is SYS.This is the only user name that can provide access to ASM-specificinformation.Providing SYS access to an ASM instance does NOT create a securityissue for the Oracle database, but you should limit access to the SYSaccount to trusted individuals.Because the SYS password is stored in encrypted format, non-privileged users can still access Spotlight on Oracle. To do so:

l Use a pre-configured or shared Spotlight on Oracleconnection, OR

l Use a Spotlight connection without ASM access. Spotlightwill display Oracle-related data, but the ASM drilldown willcontain no data.

Server connection details

Field Description

Monitor OS Select to monitor the operating system on the database server.

OS Server Enter the name of the server.Tip: Use a period . to monitor the local database.


25

Field Description

OS Type Select the type of operating system.

l Windows

l Unix(REXEC)

l Unix(SSH)

l Unix(SSH using Public/Private Keys)

If you select one of the Unix systems, make sure the SSH or REXECdaemon is running on the database server and is configured toreceive remote connections. Spotlight supports both SSH1 and SSH2protocols. Public-key encryption is supported under SSH2 only.

SSH Port Enter the port number for SSH (secure shell) connection to thedatabase server. The default value is 22.Note: Applicable when OSType is Unix(SSH) or Unix(SSH usingPublic/Private Keys).

OSUsername

Type the user name to login to the database server.Note: When OSType is Windows:

l Not required if your login details have been assigned remoteaccess to the registry of the database server.

l Ensure the user has access to the registry of the server.

SSH KeyType

Select the key type from the options:RSA and DSA.Note: Applicable when OSType is Unix(SSH using Public/PrivateKeys).

SSH PrivateKeyFilename

Locate the filename that contains the private key for the Spotlightconnection.Hint: Use the "..." button.Note: Applicable when OSType is Unix(SSH using Public/PrivateKeys).

SSHPassphrase

Type the passphrase used to decrypt the private key.Note: Applicable when OSType is Unix(SSH using Public/PrivateKeys).

OSPassword

Type the user password to login to the database server (if required).Note: Not applicable when OSType is Unix(SSH usingPublic/Private Keys).

Connect toOSSpotlight

Select to have the database server connection open automaticallywhen you open the connection to the Oracle database.


26

Field Description

whenstarting thisconnection

ESX connection details

Note: Requires Server connection details | Monitor OS selected.

Field Description

MonitorESX

Select to monitor the virtual machine that hosts the database.

ESXWebService

Type or select the URL of the ESX Web Service that manages thevirtual machine.

ESXUsername

Type the username required to connect to the ESX Web Service.Note: The username specified must have read only permissions for boththe ESX virtual machine and the ESX host that the virtual machineresides on, or may reside on.

ESXPassword

Type the password required to connect the specified user to the ESXWeb Service.

VirtualMachineName

Click ... to select the virtual machine.

5. Click Test to verify Spotlight can connect successfully with the details provided.

Note:For connections to virtual machines, testing the connection validates that Spotlightcan connect to the Web Service using the specified username and password combination,and that the virtual machine exists. It does not check permissions, that the virtual machineis turned on, or that it is the same machine as the one specified in the Server ConnectionDetails section.

6. Click Save password details (for this connection) to save all the enteredpassword details.

Configuring Oracle User for Spotlight DialogNote:This dialog is applicable to Oracle 11g Release 2 and above.


27

This dialog will open from Spotlight Browser | All connections or the Connection Detailsdialog when selecting to monitor an Oracle or Oracle RAC connection, and the setup of theSpotlight User through the Oracle User Wizard is incomplete.

Field Description

Install Objects inmy schema

Click to install Spotlight objects in your schema. This automates therunning of the Oracle User Wizard (page 19) up to PredictiveDiagnostic Settings if you have DBAprivileges. If you do not haveDBAprivileges you will be prompted for a DBAlogin.

Configure anotheruser

Click to configure another user to connect to this Oracle instance. Tocomplete these steps you require SYSDBAprivileges.

Show connectiondetails

Click to view or edit the connection details. Open the ConnectionDetails dialog. See "Connect to Oracle Instance" (page 22) for moreinformation.

View informationabout objectsinstalled andprivileges requiredby spotlight

Click to open the Spotlight on Oracle Database Objects andPrivileges document.

Toad DBA Suite for OracleNote: This section is relevant for systems with the following software installed: Toad DBASuitefor Oracle with Spotlight on Oracle.

To connect to an Oracle database from within Toad

1. Click the Spotlight button where it appears in Toad.

2. Spotlight on Oracle opens at the Oracle database connection associated with the Toad forOracle connection.

The first time you open the Oracle database connection from within Toad

You are required to fill in the Details page with the following additions. See "Connect to OracleInstance" (page 22) for more information.

l The Oracle database and username fields are pre filled in as per the Toad connection.Security requires that you re-enter your Oracle password.

l The default connection name is Toad followed by the name of the Oracle database. Youmay choose to replace this with any connection name of your choice. Note that alimitation of the integration of Toad with Spotlight on Oracle is that this is a newSpotlight connection, ignoring all other pre-existing Spotlight connections.


28

l You may choose a different Oracle username to connect to Spotlight. For example, in anenvironment with multiple database administrators there may be a communal username forSpotlight connections. Simply replace the filled in Oracle username with the appropriateOracle username and password details.

l You can enter additional parameters to monitor ASM, the operating system and forvirtualization.

Using Spotlight on Oracle

l Spotlight opens at the Home Page or Drilldown relevant to the location in Toad fromwhere Spotlight was opened.

l You are prompted to create a new Spotlight user if the Oracle username has not beencreated in Spotlight. See "Spotlight Users" (page 19) for more information.

l This Spotlight connection can be found in the Spotlight Connection Manager. It can bemanaged the same as any other Spotlight connection. For example, you can close theconnection, edit the connection details and re-open the connection in the SpotlightConnection Manager.

Caution:When you click on the Spotlight button in Toad to connect to the Oracle database,a new Spotlight connection to the Oracle database is created, regardless of any pre-existingSpotlight connections to the Oracle database. To avoid the annoyance of setting up Spotlightconnections in Spotlight that will be set up identically through Toad later, you may chooseto create all connections through Toad if Toad integration is desired.

Install And Configure DTrace ScriptsDTrace is a dynamic analysis tool for Sun platforms. Spotlight on Oracle uses DTrace to capturedetails of the interaction between Oracle and the operating system (Solaris 10 onwards).

To use DTrace within Spotlight, you need to install and configure a number of DTrace scripts onthe server. Once installed, use the DTrace drilldown to view the DTrace analysis. See "Spotlighton Oracle Drilldowns" (page 41) for more information.

To install and configure DTrace scripts

1. Locate in the Spotlight installation folder: Plug-ins\SoO\SoO_DTrace_scripts.tar

2. Unpack SoO_DTrace_scripts.tar to a directory on the database server.

3. Ensure the unpacked files have execute permission.

4. For the account that will be used to run operating system commands from Spotlight:

l Edit .profile to define a QUEST_SOO_HOME variable that points to the directory onthe database server where the unpacked files are.

l Provide DTrace authority. Add this line to /etc/user_attr:

::::defaultpriv=basic,dtrace_proc,dtrace_kernel


29

Spotlight on Oracle Home PageThe Spotlight home page shows the flow of information and commands between various sub-components and the size and status of internal resources such as processes, disk files and memorystructures.

Related statistics are grouped together on panels that are connected by a series of graphical flowsand icons. Spotlight updates these flows in real time so you can see how quickly data is movingthrough the system. The icons change color as Spotlight alarms are raised, upgraded, downgradedand canceled.

The panels on the Spotlight on Oracle home page are described in more detail next. For a fulldiscussion of the panels, see the Spotlight online help.

Tip: Hover the mouse pointer over a panel component for more information.


30

Sessions Panel

The Sessions panel indicates the status of the Oracle communicationsbetween client and server. Information displayed here includes:

l The time (in milliseconds) that it takes for the query select user fromdual to be submitted, executed and returned.

l The total number of users connected to the database.

l The number of users who are currently active.

l The sum of DB time over all sessions, divided by elapsed time.

Note: The Average Active Sessions gauge is available for (Oracle 10g andlater).

The flows between the Sessions panel and the Server Processes panel represent the rate of datatransfer in both directions across the SQL*NET interface.

Host Panel

The Host panel shows metrics for the operating system:

l The total amount of CPU being used.

l The number of tasks waiting for CPU resources.

l The percentage of CPU that is unavailable to the virtual machine (ifapplicable).

l The available physical RAM.

l The amount of memory that has been reclaimed from the virtualmachine by ESX (if applicable).


31

Server Processes Panel

The Server Processes panel indicates the status of Oracle server processesthat perform database activities on behalf of end users, and mediate databaseconnections:

l Statistics for the PGA memory allocated by server processes.

l Dedicated server processes that perform work on behalf of a singleclient process.

l Shared or multi-threaded servers (MTS) that perform work on behalfof more than one client process.

l MTS dispatchers that coordinate the allocation of shared servers toclient tasks.

l Server processes that support parallel execution of queries.

l The current length of the Parallel Execution Queue.Note: The Parallel Exec Queue gauge applies only to (Oracle 11gRelease 2).

l Job queue server processes that run PL/SQL commands submitted tothe Oracle job queue via the DBMS_JOB package.

l The state of the EM agent connection between Oracle EnterpriseManager (if installed) and the database.

l The number of server processes that are currently active.

The flow to the Server Processes panel from the Disk Storage panel represents the rate of directphysical reads from disk.

The flows between the Server Processes panel and the SGA panel represent the rates of:

l Blocks reads from the SGA by all server processes.

l Block changes in the SGA by all server processes.

l Redo buffer entries made by all server processes.

l SQL parse requests per second by all server processes.

l SQL execution requests per second by all server processes.


32

SGAPanel

The SGA panel shows details of specific memory areas within the systemglobal area (SGA). The SGA is an area of shared or common process memorythat is used to cache frequently used data, SQL statements, procedures andother structures:

l The size of the buffer cache and its components. These hold data thatis accessed frequently.

l The current size of the SGA.

l The size of the Java Pool, which caches class definitions, Javamethods and Java objects.

l The size of the Large Pool allocation heap, which is used in MTS forsession memory. It can be used by parallel execution and backupprocesses.

l The size of the redo buffer. This contains redo entries that areeventually written to the redo log.

l The size of the Shared Pool, which caches SQL statements, PL/SQLprograms, object definitions and session memory for MTS sessions.

l The size and activity of the result cache (Oracle 11g and later).

l Whether the database is in manual or auto memory managementmode (Oracle 10g and later).

The flow between the SGA panel and the Disk Storage panel represents the rate of blockreads from disk.

The flows between the SGA panel and the Background Processes panel represent the rates of:

l Block writes by the Database writer (DBWR) process.

l I/O writes by the Recovery writer (DBWR) process.

l Redo log block writes by the Redo Log Writer (LGWR) process.


33

Background Processes Panel

The Background Processes panel displays the following key Oraclebackground processes:

l Database Writer (DBWRn).

l The Recovery Writer (RVWRn).

l The Log Writer (LGWRn).

l The Archiver (ARCHn).

l Data Guard Overhead and Apply Lag

l Predictive Diagnostics.

Note: The n value in each process icon indicates the number of processesthere are of that type.The flows between the Background Processes panel and the Disk Storagepanel represent:

l The rate at which modified blocks are written from the SGA to diskby the DBWR processes.

l The rate of I/O write operations to the flash recovery area by theRVWR processes.

l The rate at which redo log entries are written to the redo log files bythe LGWR processes.

l The rate at which the archiver copies from redo log files to archivedlogs.

The Background Processes panel also contains a status indicator forPredictive Diagnostics (if installed). See "Using Predictive Diagnostics"(page 50) for more information.


34

Disk Storage Panel

The Disk Storage panel represents Oracle database and log files on disk,including:

l Oracle database files.

l Size of the flash cache.

l Oracle flashback recovery files.

l Oracle redo logs.

l The archive log destinations.

Right-click the Disk Storage icon to open Space Manager. See "Tools|Space Manager" (page 46) for more information.

Spotlight on Oracle AlarmsSpotlight alerts you to problems with your system by issuing an alarm. You can configureSpotlight in the level of severity that constitutes an alarm, to disable an alarm, and the actionsSpotlight takes on raising the alarm.

When an alarm is raised Spotlight performs one or more of the following actions:

l Changes the color or intensity of relevant components.

l Gives audible warnings.

l Sends email notifications.

Actions you can take on an alarm being raised include:

l View details of the statistics that caused the alarm in a Spotlight drilldown page. See"Spotlight on Oracle Drilldowns" (page 41) for more information.

l View the Spotlight online help.

l View details of the alarm in the Spotlight Alarm Log drilldown.

l Snooze the alarm.


35

l Save the alarm details.

l Filter the list of alarms.

Following are the alarms specific to Spotlight on Oracle. For possible solutions to the problemsindicated by these alarms or for information on how to diagnose problems further, see Spotlighton Oracle alarms in the online help.

Alarm Description

Agent Status alarm Spotlight raises the Agent Status alarm when the status of the EMagent is down.OEM is a Web-based database administration and monitoring toolthat manages Oracle components and administers Oracle databases.When installed and active, OEM communicates with the databases itadministers via a set of EM agent processes under the automatedDBSNMP user account.

Archive CriticalFailure alarm

An Archive Critical Failure alarm indicates that the number ofworking archive destinations for the database instance is about to fallbelow the minimum number needed to archive Oracle redo logssuccessfully. When you see this alarm, you should make space in thearchive log destination by purging or by backing up to an offlinemedium any archived files that are not immediately needed.

ArchiveDestination Failedalarm

Spotlight raises an Archive Destination Failed alarm when at leastone of the archive destinations specified for redo log files has becomefull. See Archive Destination Failure alarm below.

ArchiveDestination Failurealarm

An Archive Destination Failure alarm when at least one of thearchive destinations specified for redo log files is about to becomefull. When you see this alarm, you should make space in the archivelog destination by purging it, or by backing it up to an offlinemedium.

Array Fetch Sizealarm

When more than one row is retrieved from an Oracle database in asingle SELECT (an array fetch), the amount of network trafficassociated with SELECT operations is radically reduced. The ArrayFetch Size alarm is raised when the array fetch size for the databasemay be too low.

ASM Balancealarm

This alarm indicates that there is unbalanced access to one or more ofthe disks in the ASM (Automatic Storage Management) disk groupthat has been allocated to the Oracle instance.

ASM Service Timealarm

This alarm indicates that the average time taken to perform I/O on theASM instance has exceeded a specified threshold value. (Service timerepresents the response time of a disk plus any wait time experiencedduring disk operations.)


36

Alarm Description

ASM Status alarm When a Spotlight connection is lost, Spotlight automatically tries tore-establish the connection. The ASM Status alarm indicates thatSpotlight is re-establishing its connection to the ASM diskgroup forthe Oracle instance.

Average Time toSync a Redo LogEntry alarm

The Average Time to Sync a Redo Log Entry alarm (formerly theAverage Redo Write Time alarm) is activated when the time taken towrite redo log entries exceeds a threshold. Unlike most other Oraclewrite I/Os, Oracle sessions must wait for redo log writes to completebefore they can continue processing.

Buffer Busy Waitalarm

The Buffer Busy Wait alarm occurs when a session cannot access ablock because it is in use by another session. The two most commoncauses are insufficient free lists for a table or insufficient rollbacksegments.

Cache BufferChains Latch Waitalarm

The Cache Buffer Chains Latch Wait alarm occurs becausecontention for the cache buffer chains latch exceeds a threshold.

Cache Buffer LRUChains Latch Waitalarm

The Cache Buffer LRU Chains Latch Wait alarm occurs becausecontention for the cache buffer chains latch exceeds a threshold.

Collection Statusalarm

Spotlight raises the Collection Status alarm when collection of datafor the OEM (Oracle Enterprise Manager) stops. Perhaps the server isrunning low on disk space.The Collection Status alarm often occurs with the Files PendingUpload Alarm.

Continued FetchRate alarm

Oracle sometimes needs to migrate a row in a database from itsoriginal block to a new block, or needs to chain an updated rowacross multiple blocks. These migrated and chained rows causeadditional I/O, and the Continued Fetch alarm occurs when thenumber of these "continued row" fetches exceeds a threshold.

CPU Busy alarm The CPU Busy alarm occurs when the total CPU utilization of thesystem exceeds a threshold.

CPU VirtualizationOverhead alarm

The CPU Virtualization Overhead alarm is activated whenVirtualization Overhead is greater than a threshold and CPU Usage is95% or higher.

Datafile ReadTime alarm

The Datafile Read Time alarm occurs if the average time for a randomdatafile read exceeds a given threshold.

Data Guard The Data Guard Configuration Alarm occurs on a Primary database in


37

Alarm Description

ConfigurationAlarm

an Oracle Data Guard environment. Please supply details of Standbyconnections to Spotlight so Spotlight can monitor the Oracle DataGuard environment. In monitoring the Oracle Data Guardenvironment Spotlight can alert you if key elements of the OracleData Guard environment are not working, or if Standby instances arelagging behind.

Dispatchers Busyalarm

The Dispatchers Busy alarm occurs when all or most dispatcherprocesses are busy. In an MTS environment, multiple sessions share asmaller number of shared server processes. The servers are usuallyallocated by a smaller number of dispatcher processes.

Estimated SharedPool Size alarm

This alarm occurs when the Shared Pool is set too small. A possiblecause is the use of literals rather than bind variables in SQLstatements. You can also increase the size of the Shared Pool.

Excessive RBSActivities alarm

The Excessive RBS Activities alarm occurs when there are highrollback extent/shrink/wrap activities.

Files PendingUpload alarm

Spotlight raises the Files Pending Upload alarm when a significantnumber of XML files are pending upload indicating a problem withthe upload process. The strategy for dealing with this alarm isdependent on the OEM control. The OEMcontrol is defined in theSpotlight configuration for OEM.This alarm may occur in conjunction with the Collection StatusAlarm.

Flash Cache InsertSkip: DBWROverloaded alarm

The Flash Cache Insert Skip: DBWR Overloaded alarm is raisedwhen the ratio of insert skips from the DBWR being overloadedexceeds a threshold.

Flash Cache ReadLatency

The Flash Cache Read Latency alarm is raised when the time taken toread a single block from flash cache exceeds a threshold. This may becaused by a low performance flash device.

Flash Cache WriteComplete Waits

The Flash Cache Write Complete Waits alarm is raised when the ratioof 'write complete waits: flash cache' to DB time exceeds a threshold.

Flashback BufferWait alarm

The Flashback Buffer Wait alarm indicates that the percentage ofavailable time spent waiting for the Recovery Writer (RVWR) towrite data to the flashback logs has exceeded a specified threshold.

Flashback SpaceUsage alarm

This alarm is raised when the amount of used space in the FlashbackRecovery Area exceeds a threshold.

Free Buffer Waitsalarm

The Free Buffer Waits alarm occurs when free buffer waits (as aproportion of total waits) exceeds a threshold.


38

Alarm Description

In Memory UndoLatch Wait alarm

The in memory undo latch is associated with Oracle's relatively newin memory undo (IMU) structures. These structures hold in memoryinformation formerly maintained in rollback (undo) segments.

Instance Failedalarm

Spotlight raises an Instance Failed alarm when a specified instancehas failed because of an Archiver error. See Archive Critical Failurealarm above.

InsufficientFlashback Logsalarm

This alarm fires when the flashback logs in the flashback area do notcover the amount of time specified in the parameter DB_FLASHBACK_RETENTION_TARGET.

Job Processes Busyalarm

The Job Processes Busy alarm occurs when all or most job queueprocesses are busy.

Large Pool Fullalarm

This alarm fires when the Large Pool allocation heap is full. TheLarge Pool is used in MTS for session memory, and can be used byparallel execution and backup processes.

Latch Free Waitsalarm

If a session needs to acquire a latch that is held by another session, alatch free wait may occur. The presence of latch free waits mayindicate a bottleneck within the SGA. (The specific action dependson the latch.)

Library CacheLatches alarms

These alarm fire when the time spent waiting on library cache latchesor library cache pin latches exceeds a threshold. Library cache latches(or mutexes in 10g R2 and above) protect the cached SQL statementsand object definitions held in the library cache within the SharedPool.

Library CacheMiss Rate alarm

The Library Cache Miss Rate alarm occurs when the library cache hitratio falls below a given threshold.

Lock Wait alarm The Lock Wait alarm occurs when the proportion of time sessionsspend waiting for locks exceeds a threshold.

Log Buffer SpaceWait alarm

The Log Buffer Space Wait alarm occurs if the amount of time spentwaiting for space in the redo log buffer exceeds a threshold.

Log Switch Timealarm

The Log Switch Time alarm occurs when the amount of time spentwaiting for log switch events exceeds a threshold.

Low Free PhysicalRAM alarm

The Low Free Physical RAM alarm occurs when the serversavailable RAM is low.

Maximum QueuedTime alarm

The Maximum Queued Time alarm becomes active when a sessionhas been in the parallel execution queue for longer than the specifiedthreshold.


39

Alarm Description

Note: Applies only to Oracle 11g Release 2.

Multi-threadedServer alarm

The Multi-threaded Server alarm occurs when all or most multi-threaded server (MTS) or dispatcher processes are busy.

Non-reclaimableFlashback Spacealarm

This alarm is raised when the amount of non-reclaimable used spacein the Recovery Area (flashback area) exceeds a threshold. Non-reclaimable files include archived redo logs, which must be backedup before being deleted.

Oracle Memory onDisk alarm

The Oracle Memory on Disk alarm is activated when the amount ofguest physical memory mapped to machine memory is less than thecurrent combined size of Oracle's SGA and PGA.

Oracle Parametersfor PredictiveDiagnostics alarm

The Predictive Diagnostics feature cannot predict the behavior ofOracle databases without the access to Oracle data that is granted byseveral Oracle parameters.

Oracle Status alarm When a Spotlight connection is lost, Spotlight automatically tries tore-establish the connection. The Oracle Status alarm indicates thatSpotlight is re-establishing its connection to the Oracle instance.

Parallel QueryServer alarms

The Parallel Query Server alarm occurs when all or most parallelservers are busy. Parallel server processes perform work to resolveSQL statements that are being executed in parallel. If all parallelserver processes are busy, parallel SQL may be downgraded, or evenexecuted serially, thus increasing its execution time. Spotlight canraise these Parallel Query Server alarms: Parallel Query Servers Busy,Parallel Queries Downgraded, and Parallel Queries Serialized.

Parse Ratio alarm The Parse Ratio alarm is raised when the ratio of parse calls toexecute calls exceeds a threshold.Note This alarm is initially disabled; see Total Parse Wait Time Ratioalarm.

PredictiveDiagnosticsalarms

Predictive Diagnostics alarm (SQL Collector, Bottleneck Collector orPerformance Modeling Collector alarm) is raised when theappropriate collection of predictive data has not been scheduled.

PredictiveDiagnosticsCustomSQL Error alarm

This alarm indicates that the customized SQL query use to representdatabase concurrency within Predictive Diagnostics has returned anerror.

PredictiveDiagnosticsOracleParameters alarm

The Predictive Diagnostics feature cannot predict the behavior ofOracle databases without the access to Oracle data that is granted byseveral Oracle parameters.


40

Alarm Description

Redo Allocationand Redo CopyLatch Wait alarms

The Redo Allocation and Redo Copy Latch Wait alarms are raisedwhen there is contention for the redo allocation latch and redo copylatch.

Result CacheFind/Create Ratioalarm

When Oracle executes SQL queries, the result cache stores the resultsets of specified queries in shared memory for later use. TheFind/Create Ratio alarm occurs when the rate of find operations onthe result cache drops below the rate of create operations, andindicates that the contents of the result cache are under-used.

Result CacheLatch Sleep Ratioalarm

Spotlight raises the Result Cache Latch Sleep Ratio alarm when therate of latch sleeps for the Result Cache latch indicates that there iscontention on the Result Cache latch, where multiple processes arecompeting to use the latch and are causing a significant degradationin response time.

Row Cache objectsLatch Wait

The row cache objects latch protects the contents of the row cache;the section of the shared pool to cache dictionary information.

Sequential ReadTime alarm

This alarm fires when the average time that Oracle takes to read asingle block of data typically an index-based lookup issignificantly higher than the average time recorded duringcalibration.

Sessions Queuedalarm

The Sessions Queued alarm becomes active when the number ofsessions in the parallel execution queue exceeds a threshold.Note: Applies only to Oracle 11g Release 2.

Shared Pool LatchWait alarm

Shared pool latches control access to the shared pool memory map.

Shared Pool LockPercentage alarm

This alarm is raised when one user session holds a lock on object(s)in the Shared Pool for a prolonged period, and causes other sessionsto wait on the same resource. This "blocking lock" happens rarely.When it does occur, use a SQL query to find the affected sessions.

Shared ServerConverted intoDedicated Serveralarm

Spotlight raises an alarm when many shared servers are converted,leaving too few shared servers to meet the demand. Converted sharedservers will have a status of WAIT(RECEIVE) in v$shared_server.

Status Error alarm This alarm is raised when there is a problem with the way the OEMconfiguration has been defined in Spotlight. Correct the valuesentered in the dialog View | Options | OEM Monitoring.

Temporary IOWait Ratio alarm

Spotlight raises the Temporary IO Wait Ratio alarm when the timespent in temporary IO waits is too high as a percentage of the time


41

Alarm Description

spent performing user calls on the Oracle database.

Total Parse WaitTime Ratio alarm

Spotlight raises this alarm when the amount of time spent parsingSQL statements as a percentage of the active database time exceeds alevel that may cause latch contention on the shared pool and librarycache latches.Note: The Total Parse Wait Time Ratio alarm is available for (Oracle10.2 and later). When monitoring earlier Oracle systems, considerreactivating the (disabled) Parse Ratio alarm.

Unarchived Logsalarm

The Unarchived Logs alarm occurs when the number of unarchivedredo logs exceeds a threshold. If all online logs need archiving,database update activity can be suspended while the redo log writerwaits for the next log to be archived.

Unix Status alarm When a Spotlight connection is lost, Spotlight automatically attemptsto re-establish the connection. Where the Oracle database is hostedon a Unix server, this alarm indicates that Spotlight is re-establishingits connection to the Unix server.

WMI Status alarm When a Spotlight connection is lost, Spotlight automatically attemptsto re-establish the connection. Where the Oracle database is hostedon a Windows server, this alarm indicates that Spotlight is re-establishing its WMI connection to the Windows server.

Write CompleteWait alarm

Spotlight raises the a Write Complete Wait alarm when writecomplete waits (as a proportion of total waits) exceed a threshold.Write complete waits occur when a session tries to modify a blockthat is currently being written to disk by the database writer process.

Note: Spotlight on Oracle Data Guard alarms occur in the Alarm Log of the Primary Databasein the Oracle Data Guard environment. See Spotlight on Oracle Data Guard Alarms in the onlinehelp for more information.

Spotlight on Oracle DrilldownsWhen you have isolated a problem, you can display a drilldown page, whose charts and tablesprovide a detailed breakdown of the underlying statistics.

Following are Spotlight on Oracle drilldowns.


42

Drilldown Clicktoopen

Description

Top Sessions The Sessions drilldown displays information about theusers connected to the Oracle database. The pages andsub-pages in the drilldown display information on:

l All users connected to the Oracle database.

l Session details for those users, including waits,SQL statements executed, locks, resource usage,and Oracle statistics.

l Parallel queries executed for all users connected tothe Oracle database.

l Oracle jobs currently scheduled for the database.

SQL&ApplicationWorkload

The SQL & Application Workload drilldown allows youto select, sort and display SQL statements executedaccording to specified criteria. The drilldown includes:

l The Top SQL page, which displays the SQLstatements that consume the most system resourcesfor the current connection.

l The Array Fetch page, which displays informationon array processing for the Oracle database underinvestigation.

l The Parse Activity page, which displaysinformation on SQL statements associated withhigh parse rates (Oracle 10.2 and later).

l The Sort Activity page, which displaysinformation on SQL statements associated withhigh sort activity (Oracle 10.2 and later).

l The Analyze Trace page (formerly a separatedrilldown), which displays the contents of tracefiles that store information on how Oracleprocesses the execution of SQL statements, and onthe resources it uses to do so (Oracle 10.2 andlater).

l The Result Cache page (Oracle 11g and later)shows information about the result cache, whichstores the result sets of specified queries so theycan be re-used.

Activity The Activity drilldown displays the activity on thedatabase. The pages and sub-pages in the drilldowndisplay information on waits, waiting events, blocking


43


Description

locks, lock activity, latches, query servers, transactions,rollbacks, and buffer busy waits.

I/O The I/O drilldown displays the input and output activityfor the database. The pages and sub-pages in thedrilldown display information on I/O by category, session,SQL, tablespaces, datafiles, segments and row chaining.

Configuration &Memory

The Configuration and Memory drilldown shows thememory utilization for the Shared Pool and the SGA, aswell as shared pool and buffer cache hit ratios, Oraclealert log and Oracle parameters.

Operating System The Operating System drilldown provides informationabout the performance, processes, and storage on thedatabase machine under investigation.

Disk Storage The Disk Storage drilldown displays information aboutonline redo logs, tablespace utilization, archivedestinations, and Flashback recovery.Right-click the tables in the Disk Storage drilldown toopen Space Manager. See "Tools |Space Manager" (page46) for more information.

Tuning You can use the Tuning drilldown to view informationabout Oracle memory resources in the Oracle SGA andPGA (System and Program Global Areas), to tune memoryresources where feasible, and to view and tune latchactivity for systems with multiple CPUs.

ASM If your system uses Automatic Storage Management(ASM) to manage disk storage, you can use the ASMdrilldown within Spotlight on Oracle to diagnose ASMperformance. ASM is a storage management solution fromOracle Corporation that automatically reorganizes databased on statistics collected from usage and user accesspatterns over time.

Analyze Trace To understand how well SQL is being processed by thesystem under investigation, and how to improve thatprocessing, you must trace the execution of SQLstatements for:

l How Oracle is processing the statements.


44


Description

l The resources that are required to execute thosestatements.

Oracle stores that information in trace files. The pages inSpotlight's Analyze Trace page allow you to view thecontents of those trace files and save the contents to theclient's computer.

DTrace DTrace is a dynamic analysis tool for Sun platforms.Spotlight on Oracle uses DTrace to capture details of theinteraction between Oracle and the operating system(Solaris 10 onwards). See "Install And Configure DTraceScripts" (page 28) for more information.

PredictiveDiagnostics (ifenabled)

Predictive Diagnostics (if available) allows Spotlight tocollect and analyze performance metrics for an Oracledatabase in order to predict its future behavior. In doingso, it identifies:

l SQL statements ("degrading SQL") whoseperformance may not scale adequately in the futureas data volumes and SQL execution rates increase.

l Waiting events ("bottlenecks") that may in thefuture affect database throughput and responsetime.

l Database resources (CPU, memory, and disk I/O)whose limitations may in the future affect databaseperformance.

See "Using Predictive Diagnostics" (page 50) for moreinformation.

Virtualization Use the Virtualization drilldown to view detailedinformation about the CPU and memory use of a virtualmachine.

Data Guard Monitor an Oracle Data Guard environment withSpotlight on Oracle Data Guard.Note: The Data Guard icon is enabled for Primarydatabases configured in the Oracle Data Guardenvironment.

Reports The Report wizard generates printable reports of fourtypes of monitoring data: alarms by percentage of time,waits, Top SQL and IO. The reports are designed and


45


Description

delivered through SAP Crystal Reports.

Alarm Log The Alarm Log drilldown displays information on thealarms associated with the Oracle database.You can filter the Alarm Log to display only the alarmsthat meet specified criteria.

Spotlight Tools

Tools |Explain PlanTools |Explain Plan allows you to determine the execution plan that Oracle applies to aparticular SQL statement. Spotlight on Oracle allows you to view graphical representations ofthe execution plan for SQL statements being executed by a user, or for those identified by theTop SQL Page.

To open the Explain Plan

1. Open a connection to an Oracle Database.

See "Connect to Oracle Instance" (page 22) for more information.

2. Click Tools |Explain Plan.

Tools |SQLOptimizerSQL Optimizer provides context-sensitive tuning advice for SQL statements based on the Oracleexecution plan and the database structure.

Before you open SQL Optimizer

l Ensure SQL Optimizer is installed on your machine.

l Ensure you have DBA privileges for the database you are diagnosing.

To open SQLOptimizer



2. Click Tools |SQLOptimizer.


46

Tools |Space ManagerSpace Manager provides a comprehensive solution for space management and reorganization inOracle databases. Space Manager offers preventive maintenance, problem detection andresolution, and capacity planning across any number of databases.

Before you open Space Manager

l Ensure Space Manager (version 4.5 or later) is installed on your machine.

l Ensure you have DBA privileges for the database you are diagnosing.

To open the Space Manager



2. Click Tools |Space Manager.

Tune Your DatabaseSpotlight supports an approach to Oracle performance tuning that could be described as "tuningby bottleneck". Spotlight alerts you if any component of the Oracle architecture is forming abottleneck. Additionally, Spotlight provides tools that allow you to identify and rectify anyinefficiency in your databases configuration.

In general, you tune your Oracle instance by reiteratively identifying bottlenecks, contention andcritical resources, and by using Spotlight's online advice to remove the bottleneck or contentionto improve the performance of the resource.

The following is a summary of steps that you can undertake to tune Oracle using Spotlight.

l Use the Spotlight home page to alert you to any obvious bottlenecks. If Spotlight detectsthat some component of Oracle constitutes a performance bottleneck, the correspondingcomponent generates a visual or auditory alarm. The exact appearance of the alarmdepends on how you have configured alarm severities. When an alarm is current you canpress F1 to display help for the alarm. Clicking the object (in the main window) displaysthe relevant drilldown.

See "Spotlight on Oracle Home Page" (page 29) for more information.

l Use the Activity drilldown to display overall efficiency and resource usage. In particular,the Event waits chart shows the amount of time database sessions are spending waitingfor various resources, as well as the amount of CPU being utilized. Tuning efforts aremost likely to succeed if they are concentrated on the resources being used most heavily.The Wait activity topic contains a discussion of the meaning and implications of variouswait events.

See "Spotlight on Oracle Drilldowns" (page 41) for more information.


47

l Other pages help you examine detailed activity. For instance, the Top Sessions drilldownshows the users who contribute most to database activity. You can display various detailsfor these sessions, including the SQL statements, locks and resource usage.

For further information on tuning your database, see the online help.

Troubleshooting

Connection Problems

Cannot Connect to Database

Possible cause Solution

Oracle is not running. Verify that the Oracle instance is running andcan be accessed from a host or clientworkstation.

The connect string and alias of thedatabase are incorrect or have changed.

Check your SQL* Net configuration andverify the connect string and alias you areusing are correct.

The user ID you are using to log intoSpotlight does not exist in the database.

Run the User Wizard to establish the user IDin the database.

The Spotlight user has the wrongpermissions in the database.

Contact your DBA to add relevantpermissions to the user.

Spotlight takes too long to open theOracle connection so it appears to beunable to connect.

Turn on Fast Initialize.More information on Fast Initialize isavailable in the online help.

1. Click View | Options.

2. Select Data Collection |OracleInitialization | Turn Fast Initializeon.

3. Restart Spotlight.

Cannot Connect to ASM

When installed and available, Automatic Storage Management (ASM) stores Oracle data in aseparate ASM database instance. Spotlight on Oracle displays performance metrics for ASM onthe home page and in the ASM drilldown.

If the database under investigation uses ASM but Spotlight cannot connect to it to retrieve therequired information, you may need to edit the TNSNAMES entry for the ASM instance.

1. Locate the tnsnames.ora file in Oracles network/admin sub-folder of the Oracleinstallation folder on your Spotlight client.

4

Spotlight on Oracle Getting Started Guide 49

2. Edit tnsnames.ora so that the TNSNAMES entry for the target ASM instance contains theoption (UR = A).

Dead Sessions in Top Sessions DrilldownThe solution is to reset the Top Sessions drilldown.

To reset the Top Sessions drilldown

From the Top Sessions drilldown

1. Click Filter.

2. Select Reset

3. Select the appropriate frequency.

Data That Does Not Appear in The History BrowserCertain features in Spotlight on Oracle cannot be captured for display in the Spotlight Historybrowser, even when the relevant Spotlight options are chosen. The features that cannot becollected in historical snapshots are:

l The SQL & Application Workload Drilldown.

l The Activity Drilldown | Waits Page | Grid.

l The Activity Drilldown | ( Waiting Events Page (Oracle 10g and later) | Waiting EventsPage (Oracle 9i))

l The I/O Drilldown | Summary Page | Bottom chart.

l The Configuration & Memory Drilldown | Buffer Cache Page | lower grid.

l The Configuration & Memory Drilldown | Oracle Parameters Page.

l The Operating System Drilldown | OS Storage Page.

Spotlight on Oracle Getting Started GuidePredict Database Performance

50

Predict Database Performance

Using Predictive DiagnosticsUse Predictive Diagnostics to predict the future performance of a database (and identify how toimprove that performance):

l Identify SQL statements whose performance may not scale adequately as data volumesand SQL execution rates increase.

l Identify bottlenecks that may in the future affect database throughput and response time.

l Predict when database resources such as CPU, memory and disk I/O are expected to reachtheir full capacity, and recommending strategies to enable the database to handle highloads at those times.

Notes:

l To ensure that Predictive Diagnostics has access to all the Oracle information it needs,check that the Oracle Parameters for Predictive Diagnostics are set to the required values.

l It takes about 10 days for Spotlight to collect enough data to make valid predictions onthe future performance of the database.

How to use Predictive Diagnostics

1. Select the Spotlight on Oracle connection in the Spotlight Browser.

2. Click Predictive Diagnostics.

The page contains several sections Configuration, Find Impending Problems,Performance Modeling, and What To Do Next and a set of images that indicate a basicworkflow through the predictive process.

3. Click a hyperlink in the Predictive Diagnostics home page to open the correspondingdiagnostic page.

First time users

Note: Although you can use any feature in Predictive Diagnostics at any time, we recommendthe following procedure for first-time users.


51

1. Follow the suggestions in What To Do Next.

The quality of predictions depends on the quality of the data on which they are based.What To Do Next tracks the current status of the data and indicates when that data isincomplete or outdated.

2. Wait until enough real data is available.

This usually occurs about 10 days after you begin monitoring the database. Before thattime, become familiar with how Predictive Diagnostics works by using the suppliedsample data.

3. Improve how the database is used now by using Identify Degrading SQL and IdentifyDatabase Bottlenecks to optimize data processing on the host system.

Note: The changes you make as a result of using these features MAY cause the data youcollected earlier to become outdated. If so, you may need to set Trend Changes andcollect new data for another 10 days.

4. Establish the resources available to the database.

Use Configuration to set realistic parameters for the CPU, memory and disk I/O resourcesavailable to the database. (The default settings we provide for these parameters may notbe appropriate for your database.)

In particular, use Peak Processing Periods to identify WHEN database resources are ingreatest demand. This will improve the precision of your predictions.

5. Use Performance Modeling to predict when and why database resources will run out,what will cause performance to suffer, and what you need to do to counter those

Predictive Diagnostics And SpotlightFor detailed information on Predictive Diagnostics, see the Spotlight online help.

When... Do this...

Predictive Diagnosticsidentifies a SQL statement thatmay cause problems

Open the Top SQL drilldown to view further detailsabout the SQL statement.

Open Explain Plan to view further information aboutthe statement's execution plan.

Open SQL Optimizer to view context-sensitive tuningadvice for the SQL statement based on the Oracleexecution plan and the structure of the database.


52

When... Do this...

Predictive Diagnosticsidentifies a bottleneck that maycause problems

Open the Activity |Waiting Events drilldown to viewfurther details about the current behavior of potentialbottlenecks.

Predictive Diagnosticsidentifies a database resourcewhose limitations may causeproblems

Open the I/O, Configuration & Memory, and Tuningdrilldowns to view further details about the currentbehavior of CPU, memory, and disk I/O resources.

Requirements for Predictive DiagnosticsThe disk storage requirements for Predictive Diagnostics schemas comprise:

Situation Requirement

Sample data Approximately 70 MB of disk space.

SQLstatements

A fixed overhead of approximately 10 MB per schema.An additional requirement of approximately 1 KB per snapshot for eachSQL statement.The number of snapshots collected depends on the application workload andsize of the shared pool, the collection interval for snapshots, and the lengthof time each snapshot is retained.ExampleIf there are 500 eligible SQL statements per snapshot, and snapshots arecollected twice a day, and each snapshot is kept for one year, the diskstorage requirement for SQL statements is:10MB + (500 x .001MB x 2 x 365) = 375 MB

Impendingbottlenecks

Disk storage requirements depend on how frequently data is collected, andthe length of time that snapshots are retained.ExamplesIf data is collected every 10 minutes and retained for six months:approximately 200 MB.If data is collected every 30 minutes and retained for six months:approximately 65 MB.If data is collected every 60 minutes and retained for six months:approximately 35 MB.

Performancemodeling

Disk storage requirements depend on how frequently data is collected, andthe length of time that performance data is retained.Example


53

Situation Requirement

If data is collected every 10 minutes and retained for one year:approximately 130 MB.

Note: Parameters set in View | Options | Predictive Diagnostics and the SQL Statement TrendAnalysis wizard affect the storage requirements for Predictive Diagnostics data. See the onlinehelp more for more information.

Troubleshooting

Predictive Diagnostics UnavailableYou can use Predictive Diagnostics to collect and analyze performance metrics for SQLstatements as they are executed on the database instance, system bottlenecks as they affectdatabase throughput and response time, and database resources whose limitations affect databaseperformance.

If Predictive Diagnostics is unavailable for your Spotlight connections, re-run the Oracle userwizard to install Predictive Diagnostics.

1. Close and re-open Spotlight on Oracle.

2. Click File | Oracle User Wizard...

3. Create a spotlight user. Make sure that you install Predictive Diagnostics as part of theprocedure. See "Spotlight Users" (page 19) for more information.

Note: When you start using Predictive Diagnostics, Spotlight will take about 10 days to collectenough data to make valid predictions on the future performance of the database. During thattime, you can use the supplied sample data to view Predictive Diagnostics at work.

Spotlight on Oracle Getting Started GuideSpotlight on Oracle RAC

54

Spotlight on Oracle RAC

What is Spotlight on Oracle RAC?Note: Spotlight monitoring of Oracle RAC systems requires additional licensing.

Use Spotlight on Oracle RAC to diagnose Oracle databases on multiple host machines under theReal Application Clusters (RAC) architecture.

When monitoring the behavior of an Oracle RAC cluster, a database administrator needsto know:

1. How is each of the instances in my cluster performing?

2. How is the database as a whole performing?

3. Is my database experiencing cluster-related contention?

To further investigate the behavior of individual database instances within the cluster, thedatabase administrator can use Spotlight on Oracle RAC to launch Spotlight on Oracle on anyparticular instance within a cluster. See "Instance Panels" (page 62) for more information.

What Is Oracle RAC?Real Application Clusters (RAC) is a clustering technology introduced by Oracle in release 9i. Itsupersedes Oracles previous clustering technology Oracle Parallel Server (OPS).

Oracle RAC is a shared disk clustering solution that is significantly different from thearchitectures offered by Microsoft SQL Server and IBM DB2. Each node in an Oracle RACcluster has equal access to all of the database data through a shared disk subsystem. Data isneither partitioned to specific nodes, nor replicated across nodes. A high-speed networkinterconnect allows each node to keep its in-memory view of data consistent.

An Oracle RAC configuration has the following architectural characteristics:

1. Many instances of Oracle run across several nodes.

2. Many instances share a single physical copy of a shared Oracle database.

3. All instances have common data and control files.

4. Every instance has individual redo logs and undo segments.

5. Every instance can simultaneously execute transactions against the same database.

The following diagram illustrates the basic components of an Oracle RAC cluster.


55

A user should find an Oracle RAC database operationally identical to a database hosted on asingle server. From top to bottom, the important features represented are:

l Cluster interconnect - A high-speed, high-bandwidth communication facility thatconnects the nodes in an Oracle RAC cluster.

l Multiple nodes/instances - Oracle instances run on the nodes (host machines) in theOracle RAC cluster. Each instance comprises an Oracle System Global Area (SGA)plus the corresponding Oracle background processes that retain and process Oracledatabase requests.

l Shared disk subsystem - Database files in Oracle RAC systems are stored on multipledisks that are shared by all the nodes in the cluster, and all nodes must be able read andwrite to those disks.

Connect to Spotlight on Oracle RACUse this procedure to connect to an Oracle instance running on an Oracle RAC cluster.

1. From the Spotlight Browser select All Connections |Spotlight on Oracle RAC

2. Select the RAC to connect to. If the RAC is not on the list then Add a new connection


56

(page 56).

3. The Verify Cluster Connection Details dialog is displayed if Spotlight encounters anerror during the connection process. See "Verify Cluster Connection Details Dialog" inthe online help for more information.

Note:You can request Spotlight displays this dialog every time you connect to theOracle RAC. On the connection Properties | Details dialog select Display ConnectionDetails Verification Dialog. Alternatively, on the Verify Cluster Connection Detailsdialog select Display this dialog for the current connection.

Add a new connection

1. Ensure you create a Spotlight user with sufficient privileges to diagnose the Oracle RAC.See "Spotlight Users" in the online help for more information.

2. Click File |Connect

3. Select Spotlight on Oracle RAC on the Connections menu.

4. Double-click Add new connection.

5. Ensure Details is selected. Fill in the Properties window.

Oracle Connection Details

Field Description

ConnectionString

Select the name (the SQL*Net alias) of the Oracle RAC instance thatyou wish to connect to.Notes:


57

Field Description

l The database name is defined within SQL*Net configurationutilities.

l While this can be the name of any instance in the Oracle RACcluster, we recommend using the name of the load-balancedinstance the instance whose TNSNAMES entry includes theoption (LOAD_BALANCE = yes)

l When selected, the Connection name field resets to this value.You can then edit the Connection name field if required.

OracleUsernam

Spotlight on Oracle Getting Started

Documents

Transcript of Spotlight on Oracle Getting Started