ASEMON

JEAN-PAUL MARTIN

Oct 2009

Installation and usage of Asemon_logger and Asemon_report

2 – October 2009

AGENDA

– Presentation– Installation and setup– Asemon_report Usage– Analyse ASE metrics– Advanced configuration

3 – October 2009

PRESENTATIONWhy use Asemon

4 – October 2009

ORIGIN

• Needing tool for exploiting new MDA tables in ASE 12.5.3• Developed and maintained by me (no official project)• Used for all my performance analysis consulting engagements• First version was interactive (real time monitoring)

– Not good for post-incident analysis– Still exists but not maintained by me– See « www.asemon.se » for a new version

• Current version based on– Asemon_logger : capture and archive metrics– Asemon_report : web application for results analysis

History of Asemon

5 – October 2009

OBJECTIVES

• Free (Gnu GPL license) • Sources available• As simple as possible

– Asemon_logger written in Java Easily extensible (add configuration files in XML) Auto admin (automatic schema admin, auto purge)

– Asemon_report Written in HTML and PHP Tested with Apache Any browser (prefer FIREFOX)

• Extensible (ASE 12, ASE 15, RS12, RS15, IQ12.7, IQ15, RAO)

Principles retained for building Asemon

6 – October 2009

REQUIREMENTS

• For Asemon_logger :– Install on a remote machine (not the production server)– Can be Linux, Windows, Solaris, AIX, … (Any plateform

supporting Java JRE 1.5)– For archive database : need an ASE (12.5 or 15) database

(preferably not on the production server)• For Asemon_report :

– Can be installed on Windows– PHP pages can be installed on an existing Web server

Need php 5.2 Need Sybase Open Client (12.5 or 15)

– Any browser

What do I need to run and use asemon

7 – October 2009

VERSIONS

• Current version (as of 25 Oct 2009)– Asemon_logger : V2.4.2– Asemon_report : V2.4.3

• Download– http://sourceforge.net/projects/asemon/

(go to "View all files" to get all downloadable modules)

8 – October 2009

FUNCTIONALITIES

• Practically all ASE MDA data– General counters (CPU, I/O, deadlocks, …)– Objects activity– Statements (all heavy SQL statements)

With context of execution Plan

– Locks : blocking situations, and deadlocks situations– …

• Other ASE metrics (not MDA based)– Tables fragmentation– Sysmon, spinlocks (all sysmonitors counters captured)

Metrics captured by asemon_logger for ASE

9 – October 2009

FUNCTIONALITIES

• RS :– Queues usage– Queues activity– Rep Agent activity– DSI activity– Capture all new RS 15 counters

• IQ :– Iq_status– Asa statistics– Connections– versionning

Metrics captured by asemon_logger for RS, IQ

10 – October 2009

FUNCTIONALITIES

– Read an XML config file– Start a Java thread for each group of metrics– Delay for data collection depends of metrics (can be

changed) Every 1 minute for most of metrics Every 5 s for blocking locks or statements Every 5 mn for objects activity Every 24 h for tables fragmentation

– Archive metrics in an ASE database– A purge thread automatically remove old data

(configurable)

What does Asemon_logger

11 – October 2009

FUNCTIONALITIESWhat is Asemon_report

12 – October 2009

INSTALLATIONSetup and configuration

13 – October 2009

ASEMON_LOGGER - INSTALL

1. Get asemon_logger_V2.4.2.tgz (Oct 2009 current version)

2. Untar it in any directory (ex. asemon)3. Check JRE install

– need JRE 1.5 or higher– Verify with "java –version"By default asemon_logger search JRE in

"$ASEMON_LOGGER_HOME/../jre1.5.0_11" (can be edited)

4. Create an ASE database on Archive server. Size depends on : Monitored server activity (captured statements, processes, …) Number of kept days (default is 3 month) Number of monitored servers

– Start with a 1 Gb db, and monitor usage

First steps

14 – October 2009

ASEMON_LOGGER - CONFIG

5. Copy "conf/config.xml" to "conf/YourMonitoredServer.xml"6. Edit "conf/YourMonitoredServer.xml" :

Basic configuration

<ArchiveSrv> <name> enter_your_archive_server_here </name><user> enter_archive_user_name_here </user><database> enter_archive_database_name_here </database><charset> </charset>

</ArchiveSrv><MonitoredSrv>

<SRV> <name> enter_monitored_server_name_here </name><user> enter_monitored_user_name_here </user><charset> </charset><srvDescriptor> enter_server_discriptor_name_here_(ex._:_ASEV125) </srvDescriptor><purgearchive daysToKeep="90" deleteSleep="100" /> <RSSDServer> optional_name_of_RSSD_server_if_RS15_monitored </RSSDServer><RSSDUser> optional_username_of_RSSD_server_if_RS15_monitored </RSSDUser><RSSDDatabase> optional_name_of_RSSD_database_if_RS15_monitored

</RSSDDatabase> </SRV>

</MonitoredSrv>

15 – October 2009

ASEMON_LOGGER - CONFIG

– Depends on your monitored server– Can be : ASEV125, ASEV15, IQ127, IQ15, RS126, RS15,

RAO15– Is a reference to the list of XML descriptors next in the

config file (each <md> describes a group of metrics)

What is server descriptor (<srvDescriptor> )

<SD> <name> ASEV15 </name><type checkMonitoringConfig="yes"> ASE </type><version> 1500 </version><metricsDescriptorsFiles >

<md> CachePool.xml </md><md> CnxActiv.xml </md><md> DataCache.xml </md><md> DeadLock.xml </md><md> DevIO_V15.xml </md><md> Engines_V15.xml </md>….

16 – October 2009

ASEMON_LOGGER - Before start

– Give "mon_role" to "sa"If "sa" is used. Can use another login with "sa_role"

– Can use a loging without "sa_role" but more configuration steps (see later)

– Configure MDA tables (not necessary with V15)– Configure monitoring parameters– Minimum configuration :

– Dynamic configuration after start, asemon_logger automatically see ASE configuration

changes, and can capture more or less metrics

7. Configure monitored ASE

exec sp_configure "max SQL text monitored", 2048exec sp_configure "enable monitoring ", 1 exec sp_configure "object lockwait timing ", 1 exec sp_configure "wait event timing ", 1

17 – October 2009

ASEMON_LOGGER - Before start

– Maximum configuration :

This has impact on performanceUse on DEV servers or sometimes on PROD servers

Full monitoring

exec sp_configure "max SQL text monitored", 2048exec sp_configure "enable monitoring ", 1 exec sp_configure "object lockwait timing ", 1 exec sp_configure "wait event timing ", 1exec sp_configure "SQL batch capture ", 1 exec sp_configure "per object statistics active ", 1 exec sp_configure "process wait events ", 1 exec sp_configure "sql text pipe active ", 1 exec sp_configure "sql text pipe max messages ", 1000 exec sp_configure "statement pipe active ", 1 exec sp_configure "statement pipe max messages ", 1000 exec sp_configure "statement statistics active ", 1 exec sp_configure "deadlock pipe active",1exec sp_configure "deadlock pipe max messages",100exec sp_configure "errorlog pipe active",1exec sp_configure "errorlog pipe max messages",100exec sp_configure "enable stmt cache monitoring ", 1 -- V15

18 – October 2009

ASEMON_LOGGER - START

– Start command (Unix)asemon_logger.sh –c conf/YourMonitoredSrv.xml

– Start command (Windows)asemon_logger –c conf\YourMonitoredSrv.xml

– First time : it asks for passwords archive server passwd Monitored server passwd

– Save encrypt passwd in "passwords" file (in asemon_logger home directory)

– First time : creates archive tables in archive dbName : YourMonitoredSRV_XXXXXXXX = CachePool, Cnx, CnxActiv, DataCache, …

Start monitoring session

19 – October 2009

ASEMON_LOGGER - START

– Can be edited (to reflect change in monitored or archive server)– Ex. :

Passwords file

# Store your passwords here. Asemon_logger encrypts not yet encrypted passwords when it starts## Format of a not yet encrypted row :# SERVER;user;clear_password## Format of an encrypted row :# --SERVER;user;encrypted_password##JPMV15;sa;new_password--JPMV125;sa;6ea3e8c4ed91a5ff7aee05ee6f187515--JPMRS126;sa;184143b6c5cfe5cb--JPMIQ;DBA;72be430d17dd9b28--JPMIQ15;DBA;945f14e36fa58556--JPMRS15;sa;184143b6c5cfe5cb--JPMRS15_RSSD;JPMRS15_RSSD_prim;591dc2d4f7408fbb662388fedaa9d17bfbf9d3aed5491662

No double dash . This line will be re-

encoded at next start

20 – October 2009

ASEMON_LOGGER - START

– Redirect output to a fileBy default go to stdout

Output

C:\jpm\java\Asemon_logger>asemon_logger -c ../asemon_logger_work/config_JPMV125.xml

2009/10/02 13:55:40.263 Config file used : ../asemon_logger_work/config_JPMV125.xml

2009/10/02 13:55:40.263 Start Asemon_logger Version V2.4.1

2009/10/02 13:55:40.856 Try to connect to srv : JPMV125

2009/10/02 13:55:40.872 Srv found in interfaces or SQL.INI file. Host=loopback Port=7100

2009/10/02 13:55:40.872 Using password from passwords file for 'JPMV125.sa'

2009/10/02 13:55:41.091 Connected to archive server : JPMV125 Database : test

2009/10/02 13:55:41.091 Try to connect to srv : JPMV125

2009/10/02 13:55:41.091 Srv found in interfaces or SQL.INI file. Host=loopback Port=7100

2009/10/02 13:55:41.106 Using password from passwords file for 'JPMV125.sa'

2009/10/02 13:55:41.153 connectMonitoredASE - connected to : JPMV125 Version : 1254

2009/10/02 13:55:41.169 Warning : Server option 'SQL batch capture' is not set, not all statistics will be captured

2009/10/02 13:55:41.184 Warning : Server option 'per object statistics active' is not set, not all statistics will be captured

2009/10/02 13:55:41.200 Warning : Server option 'statement statistics active' is not set, not all statistics will be captured

2009/10/02 13:55:41.247 Start thread : JPMV125_CachePool

2009/10/02 13:55:41.247 Start thread : JPMV125_CnxActiv

2009/10/02 13:55:41.247 Start thread : JPMV125_DataCache

2009/10/02 13:55:41.263 Start thread : JPMV125_DevIO

2009/10/02 13:55:41.263 Start thread : JPMV125_DeadLock

2009/10/02 13:55:41.263 Start thread : JPMV125_IOQueue

2009/10/02 13:55:41.263 Start thread : JPMV125_Engines

2009/10/02 13:55:41.263 Start thread : JPMV125_NetworkIO

2009/10/02 13:55:41.263 Start thread : JPMV125_MonState

2009/10/02 13:55:41.263 Start thread : JPMV125_OpenDbs

2009/10/02 13:55:41.263 Start thread : JPMV125_ProcCache

2009/10/02 13:55:41.263 Start thread : JPMV125_MonSQL

…

21 – October 2009

ASEMON_LOGGER

– No administration– Leave it running all the time– In case of asemon_logger upgrade, lastest version (2.4)

automatically creates or alter archive tables– Configure and start one asemon_logger per monitored

server (even if asemon_logger can monitor several server if configured)

Remarks

22 – October 2009

ASEMON_REPORT - INSTALL

1. Get "AsemonReportSRV_V2.4.3.zip" (Oct 2009 current version)This is a bundle with Asemon_report, Apache, PHP and Sybase OCS

2. Unzip in "c:\"(or anywhere else but more changes in config file)

3. Edit " C:\AsemonReportSRV\Apache Group\Apache2\conf\httpd.conf"– Change " ServerName martin-w2k.sybase.com:80"

to " ServerName yourPCname:80"– If not installed in "c:\"

Change all references to "C:/AsemonReportSRV" and "C:\AsemonReportSRV" to reflect your installation

4. Edit "C:\AsemonReportSRV\Sybase\ini\sql.ini" (add your archive srv)

On windows

23 – October 2009

ASEMON_REPORT - START

1. Open a cmd windows2. Execute "C:\AsemonReportSRV\setup.bat"

(not necessary if OCS already installed elsewhere on the PC)

3. Start Apache : "C:\AsemonReportSRV\StartApache.bat"Note : Apache can be configured as a service (and then starts automatically when PC reboots)To setup service, execute :

C:\AsemonReportSRV\Apache Group\Apache2\bin\apache –k install

On windows

24 – October 2009

ASEMON_REPORT on UNIX

1. Get "asemon_report_V2.4.3.tgz" (Oct 2009 current version)

2. Untar it in web server environment

Notes : – Apache must be configured with PHP 5.2– Sybase OCS 12.5 or 15 must be installed

With an already installed Apache server

25 – October 2009

ASEMON_REPORT USAGEHow to get results

26 – October 2009

ASEMON_REPORT USAGE

• In a browser call :http://localhost/www/Asemon_report/

Access the main page

27 – October 2009

ASEMON_REPORT USAGEConnect to Archive server

1 - Enter Server, user and password of Archive Server

2 – Click connect

3 – Choose archive

database

4 – Choose monitored server type

5 – Choose monitored

server

7 – Refine analyzed period

6 – Quick period setup

8 – Choose type of metrics 9 – Ask results

28 – October 2009

ASEMON_REPORT USAGE

• Start with short analyzed period : ex. one day(hit the "today" button for exemple)

• Avoid "large" analysisdata for a full month may be very long

• After reporting data, can refine the search period (edit "from" / "to" fields )

• Choose another type of metric for the same period

Remarks

29 – October 2009

ANALYSE ASE METRICSAnalyse results

30 – October 2009

ASE METRICS

• SummaryGeneral counters, CPU graphs, I/O graphs, locks summary, caches

summary ….• Statements

List of captured statements. Click a statement to get detail• Objects Stats

Statistics on tables and indexes (logical IO, physical IO, DML oper, …)Click an object to get detail usage during the analyzed period

• Objects cachedSize of objects in cachesCan get of graph of top 5 objects in each cache during the analyzed

period

Current functionalities

31 – October 2009

ASE METRICS - CONTINUED

• ProcessesStatistics of captured process (CPU, IO, transactions, network, …)

• LocksList of blocking stituations (time blocked, who blocks, who is blocked,

application /proc blocking / blocked, …)• Deadlocks

List of deadlocks• LockWaits

Summary of time wait per table and type of lock• Procedures

Summary of resource usage (cpu, io, …) per procedure captured at statement level

Current functionalities

32 – October 2009

ASE METRICS - CONTINUED

• FragmentationSize and fragmentation of tables and indexes

• SysmonEquivalent of "sp_sysmon" on the analyzed period

• SpinlocksUsage of all spinlocks (top use, contention)

• MissStatsV15 : reports missing statistics on objects

• TrendsDisplay graphs for KPI (about 25 key performance indicators)(One point per day for each KPI)Can see ASE usage evolution on time

Current functionalities

33 – October 2009

STATEMENTS ANALYSISStatement list

This list is sorted by LReads DESC

Plan is available for this statement

Click one statement to get

detail

Can get results in Excel as well as

SQL executed on Archive DB to get

this result

34 – October 2009

STATEMENTS ANALYSISStatement detail

35 – October 2009

PROCESS ANALYSISProcess detail

Identification and Summary activity

during the analyzed period

Graph of CPUGraph of I/O

Reads and Writes

Graph of network I/O

Graph of TEMPDB usage

Captured statements

executed by this process

36 – October 2009

Fragmentation ANALYSISFor Tables and Indexes

YELLOW : not so bad RED : bad

Display level : 2 means display only rows with a bad ratio

37 – October 2009

SYSMONEquivalent of sp_sysmon Choose the section of

interest

38 – October 2009

SPINLOCKSFor advance monitoring Can have decomposed

spinlocks

39 – October 2009

TRENDSGraphs of KPI's (one point per day)

Choose the KPI to graph

40 – October 2009

ADVANCED CONFIGURATIONHow to change default behaviour

41 – October 2009

UPDATE JCONNECT META DATA

• Check Jconnect metadata version on Archive ServerSp_version

Script Version Status

------ ------- ------

ODBC MDA Scripts 15.0.0.335/Mon 10-27-2008 15:55:38.67 Complete

OLEDB MDA Scripts 15.0.0.335/Mon 10-27-2008 15:55:38.67 Complete

installjdbc jConnect (TM) for JDBC(TM)/6.05(Build 26564)/P/EBF16676/JDK1 Complete

• Need EBF 16676 minimum• If not, execute "sql_server_XXX.sql" from :

"$Asemon_logger_home\jConnect-6_0\sp\"Where XXX corresponds to your Archive Server version

For Archive Server only

42 – October 2009

NON SA_ROLE LOGIN

• Often asked by customers (sa_role is often very restricted, or audited)

Configuration (only for ASE monitored server) :1. Create an asemon login with mon_role2. Execute the perl script 'install_asemon_procs_non_sa.pl'

Creates stored procs with sa_role (using Rob Vershoor’s technic)Alias asemon user to every database dbo (needed to get object

names in any database)

3. Use "config_for_non_sa.xml" rather than "config.xml" as template for your config file

Configuration

43 – October 2009

NON SA_ROLE LOGINTemplate config file (extract, no need to modify)

<SrvDescriptors> <SD> <name> ASEV125 </name> <type checkMonitoringConfig="yes"> ASE </type> <version> 1250 </version> <metricsDescriptorsFiles > <md> CachePool.xml </md> <md> CnxActiv.xml </md> <md> DataCache.xml </md> <md> DeadLock.xml </md> <md> DevIO.xml </md> <md> Engines.xml </md> <md> IOQueue.xml </md> <md> MonState.xml </md> <md> NetworkIO.xml </md> <md> OpenDbs.xml </md> <md> OpObjAct.xml </md> <md> ProcCache.xml </md> <md> SysWaits.xml </md> <md sp_showplan_proc="sp_asemon_showplan" > MonSQL.xml </md> <md> MonConf.xml </md> <md> Cnx.xml </md> <md> BlockedP.xml </md> <md sp_repagent_proc="sp_asemon_repagentstats" > RaActiv.xml </md> <md> LogsHold.xml </md> <md> ErrLog.xml </md> <md> CnxWaits.xml </md> <md> WEvInf.xml </md> <md> WClassInf.xml </md> <md sp_objstats_proc="sp_asemon_objstats" > LockWaits.xml </md> <md> SysMonFld_nonSA.xml </md> <md> SysDev.xml </md> <md> SysCaches.xml </md> <md> SysConf.xml </md> <md> SysMon_nonSA.xml </md> <md HOUR_EXEC="10" minpages="1000"> Fragment.xml </md> <md> Trends.xml </md> </metricsDescriptorsFiles> </SD>

44 – October 2009

TRACE FLAGS

• Debug : -D• Special traceflags

-T1 : skipRetreiveSQLText-T2 : disableBulkLoad-T3 : enableJNDI-T4 : disable ENCRYPT PASSWORD

Used at Asemon_logger startup