tipsandtricksforsybaseiq-121108084411-phpapp02

8/11/2019 tipsandtricksforsybaseiq-121108084411-phpapp02

1/27

Tips and Tricks for Optimizing

Performance with SAP SybaseIQ

Mark Mumy

IQ Technology Specialist

SAP

[email protected]


2/27

2November 8, 2012

Disclaimer

In this session we will be giving tips and guidelines for IQ to

provide novices and experienced DBAs with information theymay not know

This is by no means exhaustive. We strongly suggest taking

advantage of the education course offerings available to

enhance your in-depth knowledge of IQ


3/27

3November 8, 2012

Agenda

Sizing

Configuration

Indexes

Monitoring


4/27

4November 8, 2012

Sizing


5/27

5November 8, 2012

Memory Summary

Operating System .5 to 1 GB RAM

Filesystem Cache 20% of RAM

All Other Applications

IQ Catalog Memory -c/-cl/-ch parameters

IQ Thread Memory stack size * thread count

Load Memory (pre 15.2) per concurrent load

Bitmap Memory per concurrent load

IQ Main Cache 40% of remaining RAM

IQ Temporary Cache 60% of remaining RAM

Backup Memory per backup instance

Monitor the IQ Dynamic

Memory in sp_iqstatus Lists the current, and

maximumRAM that IQ has

allocated for all memory

structures

Swap recommendation isstill 1x virtual memory

Some customers run with 4-

8 GB

I prefer to run with 25-100GB depending on total RAM

May seem excessive, but it

can help avoid the dreaded

all available virtual memory

exhausted error!


6/27

6November 8, 2012

Quick Sizing Reference

RAM: 4-8 GB per core (prefer 8, settle for 4)

RAM: Give IQ 75% of available RAM (assumes there are no other majorconsumers of RAM on the host)

Storage: Prefer RAID 10 for write intensive systems and temp store

IQ can drive 50-500 MB/sec on and off disk per core in the multiplex sosize disks accordingly, but ground the sizing in reality and service levels

MAIN Store disk: 2-5 drives per core on the host or in the entiremultiplex

TEMP Store disk: 2-5 drives per core on the host The number of drives does NOT include any overhead needed by RAID

MAIN Store Fiber Controllers/HBAs: 1 per 5-10 cores

TEMP Store Fiber Controllers/HBAs: 1 per 5-10 cores


7/277November 8, 2012

Sizing CPUs for Load

For systems with 4 or fewer CPUs, expect to load roughly 10-20

GB of data per hour per CPU A 4 CPU system should be able to load about 40 GB of raw data per hour

For systems with 8 or more CPUs, expect a load rate of 20-50 GB

per hour per CPU

An 8 CPU system should be able to load between 160 and 400 GB of rawdata per hour

Load times with this approach will vary greatly based on CPU

count / speed and the number and types of indexes on the table

being loaded

For each BLOB or CLOB being loaded into IQ a single CPU will be

necessary for maximum performance

IQ 16 will change these ratios as we move to a 100% parallel load

engine


8/278November 8, 2012

Sizing CPUs for Queries

On average, 1-2 CPU per active querymore if queries are

complex and can be run in parallel

As IQ matures, we constantly push the engine to run more

queries fully parallel

This changes the sizing from how many cores per query to what is the

service level we need to achieve and how many cores will that take

Most queries are now run in parallel and can consume all

CPU resources on the host

IQ will blend single and multi-user query performance and

rebalance resources as the workload changes As more queries appear on the run queue, the available resources will

change and IQ will adjust accordingly


9/279November 8, 2012

Sizing Memory

Total system RAM should be based on 4-8 GB RAM per core

Threads do not matter

Allocate no more than 75% of total RAM to the main and temp caches

Shared memory is not used by Sybase IQ except in small amounts for

client/server communication where both are on the same host

Typically, we want more temp cache for loading as the HGindexes are built in temp cache

For queries the mix of main to temp will vary depending on

a few factors:

If there are temp tables, we want more temp cache

If the queries contain a lot of ordering and grouping, this will use work

tables based in temp cache

Typically, we weigh main and temp cache at 50/50 or 40/60 split


10/2710November 8, 2012

Configuration


11/2711November 8, 2012

Creation Options

Uppercase and lowercase?

Can force IQ to treat all text data as CASE IGNORE or CASE RESPECT

CASE RESPECT has slightly better performance but puts a burden on

applications to compare data with the appropriate case

Page size

Default is 128k which is acceptable for most sites

Increase the page size as the database and table sizes (rowcounts)

increase

Increasing the page size should be accompanied by an increase in

RAM

Collation and sort orders

Can only be set at database creation

If it needs to be changed the database must be rebuilt


12/2712November 8, 2012

Runtime Options (config file)

-iqnumbercpus

Should be set to match the physical core count, not the total

processing threads that are on the host

Caveat is IBM p-series where this can be tested and adjusted to a

value between physical cores and total SMT threads

-gm Total number of users that can connect

Keep this setting low as memory is carved based on this value. This

can impact query performance.

-gn Should be set to 1.5x thegm setting (default isgm + 5)


13/2713November 8, 2012

Database Options (set options)

FORCE_NO_SCROLL_CURSORS

Should always be set to ON

Very few applications require this to be OFF

Can improve query performance

Append_Load

Can be used to improve load performance

Will not reuse Row IDs or the space occupied by those Row IDs

Great for systems where large, contiguous chunks of data are deleted


14/2714November 8, 2012

Indexes


15/2715November 8, 2012

Key Characteristics

Most columns will have at least one index

Index selection decisions based on columncardinality (number of unique values)

Multiple indexes used to resolve a query

Indexes are self maintaining

No optimizer statistics to update

Indexes are compressed

Index advisor demystifies index selection

TYPE USAGEFast Projection Compressed raw data for result sets

(Default)

Low Fast Low cardinality data (up to 1000

unique values)

High Non-Group Aggregation on the fly and range

searches

High Group Key fields and groupings for cross-

tabular

Date, Time, DT Date ranges, date part operations

Multi-Column Concatenated indexes

Word, Text Sophisticated key word or phrase

string searches with boolean,ranking, proximity, fuzzy features

Compare Column comparisons

SAP Sybase IQ Index Types


16/2716November 8, 2012

Should We Index?

Is the column searchable?

Is the column a join column?

Is the column a date/time/datetime

datatype?

Will the column be part of anaggregation?

Will the column be part of a group

by?


17/2717November 8, 2012

When To Use Indexes

TYPE When to use it

Fast Projection This is the default storage for a column. You can only influence the type of FastProject via the IQ UNIQUE() syntax or Minimize_Storage option

High Group Low cardinality data (over 2000 unique values) where the columns are used in aWHERE clause for searching or joins. Also include columned used in a GROUP BY.

Low Fast Low cardinality data (up to 2000 unique values) where the columns are used in a

WHERE clause for searching or joins

High Non-Group Not used that much as the functionality has been implemented in other indexes(date, hg, lf)

Date, Time, DTTM All date fields should have these indexes

Word, Text Use only if you will be using the word or text search capabilities in IQ

Compare Use when comparing 2 columns in the same table.

Multi-Column Usually used for primary keys and for columns in a GROUP BY. Make sure thatevery column has an LF or HG as well!


18/2718November 8, 2012

Monitoring


19/2719November 8, 2012

Monitoring

Hardware and storage utilities

iostat, vmstat, top, glance, gpm, sar, etc.

Use tools to monitor the hardware and storage for any potential

bottlenecks outside of IQ

IQ utilities

sp_iqsysmon

Sybase Control Center

3rdparty tools like Bradmark Surveillance DBA and White Sands

ProActive DBA


20/27

20November 8, 2012

Monitoring via sp_iqsysmon

sp_iqsysmonis a great tool to consolidate the individual

monitoring!!

SAP Sybase IQ Monitor offers a series of views of the

counters to showing differing aspects of the server and

buffer cache workload

A consolidated interface for the iq utilitiescommand that is

much easier to use and provides a way to run multiple

reports with a single command


21/27

21November 8, 2012

Monitoring via Sybase Control Center


Sybase Central is being deprecated in version 16

Is a separate download via the product download center

Leverages a lightweight client (browser) with a server component so

that users need not install software

Graphical interface that presents the data in a way thathighlights issues visually

Most counters that are in sp_iqsysmon are available in



22/27

22November 8, 2012

Monitoring Guidance

Use sp_iqsysmonor Sybase Control Center when issues arise

and active monitoring is needed

Frequency and key items

Run at 10-15 second intervals for the duration of the issue

(sp_iqsysmon), or actively via Sybase Control Center

Thread use/starvation

Cache usage

Cache hit rates

Dirty page grabs

Page rereads Physical and logical i/os


23/27

23November 8, 2012

Query Monitoring

Leverage HTML query plans!

Query plan settings (off by default) to provide optimal query information toDBAs and engineering

set temporary option query_plan='off';

set temporary option query_plan_as_html='on';

set temporary option query_plan_as_html_directory=qplans';

set temporary option query_plan_after_run='on';

set temporary option query_timing='on';

set temporary option query_detail='on';

set temporary option DML_options10='on';

set temporary option Query_Name = Query Name

Should not be set globally as the Sybase IQ MSG file or query plan directorywill grow rapidly

No need to set both Query_Plan and Query_Plan_As_HTML

Query_Plan is on by default, but should be turned off immediately to keep the IQmessage file output to a minimum

This is much more difficult to read than the HTML plans and is note widelyused


24/27

24November 8, 2012

Index Monitoring

Index advice and monitoring is off by default

Turn on the index advisor globally and keep 1000 rows of advice

set option public.index_advisor='on';

set option public.index_advisor_max_rows=1000;

Will output data to HTML and text based query plans, if on

Will output advice to an in-memory structure that can be accessedvia sp_iqindexadvice()

Captures the advised index type, where it was used, the last time the

advice was generated, and the number of times this piece of advice

was made

The advisor will recommend any index that will cost less

This could be 1 ms or 1 i/o less

Be smart about the advice and whether or not you want to implement

it


25/27

25November 8, 2012

Troubleshooting Advice

Determine where the issue is

External to IQ Monitor the OS, hardware, and storage for any bottlenecks or issues

Look for high CPU use, high CPU system time, low CPU user time, high wait time

Look for I/O service times that are more than 10 ms

Internal to IQ Enable the index advisor and look for missing indexes (this is usually the culprit)

Enable HTML query plans and the index advisor if the issue is with a single query

If the overall health is in question, leverage sp_iqsysmonat 30-60 intervals for a

few days

Sysmon output can then be parsed looking for the key items previously discussed


26/27

26November 8, 2012


27/27

Useful links

SAP Sybase IQ Users Group

http://iqug.dssolutions.com/

Send an email to [email protected] follow the info in

the reply email to finalize subscription

SAP Sybase Education Curriculum

https://training.sap.com/us/en/courses-and-curricula/sybase

Sybooks IQ 15.4 Documentation

http://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.infoc

enter.help.iq.15.4/title.htm

SAP Sybase IQ Sizing Guide

http://www.sybase.com/detail?id=1093493
http://iqug.dssolutions.com/mailto:[email protected]://training.sap.com/us/en/courses-and-curricula/sybasehttp://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.infocenter.help.iq.15.4/title.htmhttp://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.infocenter.help.iq.15.4/title.htmhttp://www.sybase.com/detail?id=1093493http://www.sybase.com/detail?id=1093493http://www.sybase.com/detail?id=1093493http://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.infocenter.help.iq.15.4/title.htmhttp://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.infocenter.help.iq.15.4/title.htmhttp://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.infocenter.help.iq.15.4/title.htmhttps://training.sap.com/us/en/courses-and-curricula/sybasehttps://training.sap.com/us/en/courses-and-curricula/sybasehttps://training.sap.com/us/en/courses-and-curricula/sybasehttps://training.sap.com/us/en/courses-and-curricula/sybasehttps://training.sap.com/us/en/courses-and-curricula/sybasemailto:[email protected]:[email protected]:[email protected]://iqug.dssolutions.com/http://iqug.dssolutions.com/

tipsandtricksforsybaseiq-121108084411-phpapp02

Documents

Transcript of tipsandtricksforsybaseiq-121108084411-phpapp02