Ingres Configuration Ingres Configuration and Performanceand Performance

OrOr

How to get Ingres to perform as well How to get Ingres to perform as well as the English as the English <insert sport here><insert sport here> Team Team

by Rob Acramanby Rob AcramanDept Natural ResourcesDept Natural Resources

3030thth May 2007 May 2007

SQL performanceSQL performance Database DesignDatabase Design

– Over-NormalisationOver-Normalisation– Table StructureTable Structure

(B-Tree vs ISAM vs (B-Tree vs ISAM vs Hash)Hash)

– Secondary IndexesSecondary Indexes

Application BehaviourApplication Behaviour– Query StructureQuery Structure

eg. avoid Sub-SELECTseg. avoid Sub-SELECTs

– Check QEPsCheck QEPs– Locking ContentionLocking Contention

Ingres ConfigurationIngres Configuration

$II_SYSTEM/ingres/files/config.dat$II_SYSTEM/ingres/files/config.dat Maintain by text editor (eg. vi)Maintain by text editor (eg. vi) Maintain by ‘cbf’Maintain by ‘cbf’ Changes take effect at next Changes take effect at next

startupstartup

Ingres Health CheckIngres Health Check LockstatLockstat

– Locking StatisticsLocking Statistics LogstatLogstat

– Logging StatisticsLogging Statistics DM420DM420

– DMF Cache StatisticsDMF Cache Statistics ImaIma

– Query interfaceQuery interface– SELECT what you wantSELECT what you want

All produce statistics SINCE THE LAST STARTUP.

/home/somedba> /home/somedba> iishowres -diishowres -d

Current lock/log Allocations Current lock/log Allocations (not (not

DMF)DMF) !Retrieving configuration data for the logging system.!Retrieving configuration data for the logging system.!!!Display the configuration data entered.!Display the configuration data entered.!!!The log file size: 2096128KB!The log file size: 2096128KB!The number of log file partitions: 2!The number of log file partitions: 2!The number of log buffers in the memory : 20!The number of log buffers in the memory : 20!The maximum number of transactions in the logging system : 1700!The maximum number of transactions in the logging system : 1700!The maximum number of databases in the logging system: 160!The maximum number of databases in the logging system: 160!The max consistency point interval for invoking archiver : 7!The max consistency point interval for invoking archiver : 7!The block size of the log file : 4 Kbytes!The block size of the log file : 4 Kbytes!The log-full-limit in percentage : 90!The log-full-limit in percentage : 90!The percentage of log file for each consistency point : 5.00!The percentage of log file for each consistency point : 5.00!The force-abort-limit in percentage : 72!The force-abort-limit in percentage : 72!The size of the lock hash table : 18199!The size of the lock hash table : 18199!The size of the resource hash table : 18199!The size of the resource hash table : 18199!The maximum number of locks in the locking system : 136548!The maximum number of locks in the locking system : 136548!The maximum number of resources in the locking system: 91032!The maximum number of resources in the locking system: 91032!The maximum number of lock lists in the locking system : 1700!The maximum number of lock lists in the locking system : 1700!The maximum number of locks allowed per transaction : 500!The maximum number of locks allowed per transaction : 500Memory allocation for 1700 lock lists = 564400 bytesMemory allocation for 1700 lock lists = 564400 bytesMemory allocation for 136548 locks = 11652096 bytesMemory allocation for 136548 locks = 11652096 bytesMemory allocation for 91032 resource blocks = 46244256 bytesMemory allocation for 91032 resource blocks = 46244256 bytesMemory allocation for hash tables (18199 lock hash table,18199 resource hash table) = Memory allocation for hash tables (18199 lock hash table,18199 resource hash table) =

6406048 bytes.6406048 bytes.Memory allocation for 20 (4096-sized) log buffers = 10196 bytesMemory allocation for 20 (4096-sized) log buffers = 10196 bytesMemory allocation for 1700 transactions = 1013200 bytesMemory allocation for 1700 transactions = 1013200 bytesMemory allocation for 160 databases = 85760 bytesMemory allocation for 160 databases = 85760 bytesMemory allocation for internal data structures = 51564 bytesMemory allocation for internal data structures = 51564 bytesTotal allocation for logging/locking resources = 66027520 bytesTotal allocation for logging/locking resources = 66027520 bytes/home/somedba>/home/somedba>

lockstatlockstat/home/somedba> lockstat -statistics/home/somedba> lockstat -statistics==============Tue May 1 16:41:56 2007==============Tue May 1 16:41:56 2007

Total Locks 50000Total Locks 50000

Lock hash table 67819Lock hash table 67819 Resource hash table 67819Resource hash table 67819 Total lock lists 5000Total lock lists 5000

=============Tue May 1 16:41:56 2007=============Tue May 1 16:41:56 2007

Create lock list 3260256Create lock list 3260256 Request lock 333614341Request lock 333614341 Convert lock 4621641Convert lock 4621641 Escalate 2Escalate 2 Convert wait 0Convert wait 0 Deadlock Wakeups 13Deadlock Wakeups 13 Deadlock Search 13Deadlock Search 13 Cancel 5Cancel 5 Allocate CB 336767602Allocate CB 336767602 LBK Highwater 4LBK Highwater 4 SBK Highwater 3SBK Highwater 3 RBK Highwater 10RBK Highwater 10 Max Local dlk srch 0Max Local dlk srch 0 Max rsrc chain len 16Max rsrc chain len 16 Max locks per txn 2633Max locks per txn 2633 Callbacks Invoked 0Callbacks Invoked 0

Locking System Summary=============

Total Resources 50000 Locks per transaction 5000 Locks in use 8271 Resources in use 7611 Lock lists in use 154

Locking System Statistics===========

Release lock list 3259965 Re-request lock 18057160 Release lock 318460770 Lock wait 22348 Convert Deadlock 0 Max dlk queue len 1 Deadlock 0 Convert Search 0 Deallocate CB 668207774 LLB Highwater 200 LKB Highwater 5760 RSB Highwater 10936 Dlk locks examined 13 Max lock chain len 16 Callback Wakeups 0 Callbacks Ignored 0

CP Thread

WriteBack Threads

Logging and CacheLogging and Cache

TransactionLog File

Database 1Files

Journal Files(database 1)

Journal Files(database 2)

DMF Cache

Log Buffers

Ingres Server

dmfacp(Archiver Process)

LogWriter Threads

logstatlogstat======================Mon May 28 17:08:31======================Mon May 28 17:08:31

Database add 225111Database add 225111 Transaction begins 6146608Transaction begins 6146608 Log read i/o's 499109Log read i/o's 499109 Log writes 2925788Log writes 2925788 Log waits 620508Log waits 620508 Log free waits 39Log free waits 39 Log BCP waits 18Log BCP waits 18 Log group commit 6486Log group commit 6486 Check commit timer 3709Check commit timer 3709 Timer write, time 0Timer write, time 0 Inconsistent db 0Inconsistent db 0 ii_log_file read 499109ii_log_file read 499109 write complete 605350write complete 605350 All logwriters busy 11442All logwriters busy 11442

----Current log file header---------------------------------------------------------Current log file header-----------------------------------------------------

2007 Logging System Summary===========

Database removes 224813 Transaction ends 6146178 Log write i/o's 605483 Log forces 12388 Log split waits 1212 Log stall waits 18 Logfull stall waits 0 Log group count 6502 Timer write 3709 Timer write, idle 3709 Kbytes written 357648 ii_dual_log read 0 dual write complete 0 Max write queue len 20 Max write queue cnt 48

Block size: 4096Block size: 4096 Block count: 1310720Block count: 1310720 Partitions: 1Partitions: 1 Buffer count: 20Buffer count: 20

• Increase number of Log Buffers (maybe double number with half the size)

• If All Logwriters Busy > 10% of all Log Waits, increase number of Log Writers

(if half the size, don’t forget to also double Block count = number of blocks in Transaction Log File)

Buffer UtilizationBuffer Utilization

----Buffer utilization profile------------------------------------------------------Buffer utilization profile-------------------------------------------------- <10% ****************************************<10% **************************************** 10-19% ****10-19% **** 20-29% *****20-29% ***** 30-39% *30-39% * 40-49% **40-49% ** 50-59% *50-59% * 60-69% *60-69% * 70-79% *70-79% * 80-89% *80-89% * >90% *************>90% *************

----Buffer utilization profile------------------------------------------------------Buffer utilization profile-------------------------------------------------- <10% ****************<10% **************** 10-19% *10-19% * 20-29% ***20-29% *** 30-39% *30-39% * 40-49% *40-49% * 50-59% *50-59% * 60-69% *60-69% * 70-79% *70-79% * 80-89% *80-89% * >90% **********************************************>90% **********************************************

• Increase number of Log Buffers (maybe double number with half the size)

Good:Good:

BAD:BAD:

logstatlogstat======================Mon May 28 17:08:31======================Mon May 28 17:08:31

Database add 225111Database add 225111 Transaction begins 6146608Transaction begins 6146608 Log read i/o's 499109Log read i/o's 499109 Log writes 2925788Log writes 2925788 Log waits 620508Log waits 620508 Log free waits 39Log free waits 39 Log BCP waits 18Log BCP waits 18 Log group commit 6486Log group commit 6486 Check commit timer 3709Check commit timer 3709 Timer write, time 0Timer write, time 0 Inconsistent db 0Inconsistent db 0 ii_log_file read 499109ii_log_file read 499109 write complete 605350write complete 605350 All logwriters busy 11442All logwriters busy 11442

----Current log file header---------------------------------------------------------Current log file header-----------------------------------------------------

2007 Logging System Summary===========

Database removes 224813 Transaction ends 6146178 Log write i/o's 605483 Log forces 12388 Log split waits 1212 Log stall waits 18 Logfull stall waits 0 Log group count 6502 Timer write 3709 Timer write, idle 3709 Kbytes written 357648 ii_dual_log read 0 dual write complete 0 Max write queue len 20 Max write queue cnt 48

Block size: 4096Block size: 4096 Block count: 1310720Block count: 1310720 Partitions: 1Partitions: 1 Buffer count: 20Buffer count: 20

Fill Factor% =

“Kbytes written”,000 * 100“Log write i/o’s” * “Block size”

=35764800000605483 * 4096

= 14%

Aim for 60%Aim for 60%

CPs and WriteBackCPs and WriteBack Both :Both :

– Write from Transaction Log FileWrite from Transaction Log File– To Database FilesTo Database Files– Freeing pages in DMFFreeing pages in DMF

CPs triggered by percentage of Transaction Log File usageCPs triggered by percentage of Transaction Log File usage– Default 1% - Don’t exceed 5%Default 1% - Don’t exceed 5%– ii.myserver.rcp.log.cp_interval: 5ii.myserver.rcp.log.cp_interval: 5– Logstat (26201 = 5% of 524032)Logstat (26201 = 5% of 524032)----Current log file header---------------------------------------------------Current log file header-----------------------------------------------

Block size: 4096 Block size: 4096 Block count: 524032Block count: 524032 Partitions: 1 Buffer count: 20 Partitions: 1 Buffer count: 20

CP interval: 26201CP interval: 26201 Logfull interval: 471628 Abort interval: 377303 Logfull interval: 471628 Abort interval: 377303 CPs also trigger ArchiverCPs also trigger Archiver

– Archiver process (dmfacp) moves pages from Transaction Log File to JournalsArchiver process (dmfacp) moves pages from Transaction Log File to Journals– Freeing Transaction Log File pages in the processFreeing Transaction Log File pages in the process– This happens every <archiver_interval> CPsThis happens every <archiver_interval> CPs– ii.myserver.rcp.log.archiver_interval: 7ii.myserver.rcp.log.archiver_interval: 7– Typically, you want it 1 or 2 (to avoid large updates)Typically, you want it 1 or 2 (to avoid large updates)

Writeback ThreadsWriteback Threads WriteBack Threads triggered by number of modified pages in DMFWriteBack Threads triggered by number of modified pages in DMF

– ii.myserver.dbms.private.*.dmf_cache_size: 928ii.myserver.dbms.private.*.dmf_cache_size: 928– ii.myserver.dbms.private.*.dmf_wb_end: 464ii.myserver.dbms.private.*.dmf_wb_end: 464– ii.myserver.dbms.private.*.dmf_wb_start: 557ii.myserver.dbms.private.*.dmf_wb_start: 557– ii.myserver.dbms.private.*.p16k.dmf_cache_size: 928ii.myserver.dbms.private.*.p16k.dmf_cache_size: 928– ii.myserver.dbms.private.*.p16k.dmf_wb_end: 464ii.myserver.dbms.private.*.p16k.dmf_wb_end: 464– ii.myserver.dbms.private.*.p16k.dmf_wb_start: 557ii.myserver.dbms.private.*.p16k.dmf_wb_start: 557– ii.myserver.dbms.private.*.p4k.dmf_cache_size: 928ii.myserver.dbms.private.*.p4k.dmf_cache_size: 928– ii.myserver.dbms.private.*.p4k.dmf_wb_end: 464ii.myserver.dbms.private.*.p4k.dmf_wb_end: 464– ii.myserver.dbms.private.*.p4k.dmf_wb_start: 557ii.myserver.dbms.private.*.p4k.dmf_wb_start: 557

………………

CBF Defaults :CBF Defaults :– dmf_wb_start = dmf_wb_start = dmf_modify_limit - 15% * dmf_cache_sizedmf_modify_limit - 15% * dmf_cache_size– dmf_wb_end = 50% * dmf_cache_sizedmf_wb_end = 50% * dmf_cache_size

DM420DM420Displays DMF Cache StatisticsDisplays DMF Cache Statistics

$ sql iidbdb$ sql iidbdb* set trace output ‘dm420.log’ \p\g* set trace output ‘dm420.log’ \p\g* set trace point dm420\p\g* set trace point dm420\p\g\q\q

$ ls –l $II_SYSTEM/ingres/files/dm420.log$ ls –l $II_SYSTEM/ingres/files/dm420.log-rw-r--r-- 1 ingres dba 5335 Apr 27 16:06 -rw-r--r-- 1 ingres dba 5335 Apr 27 16:06

dm420.logdm420.log

DM420 DM420 APPENDSAPPENDS to the file, not Overwrites ! to the file, not Overwrites ! (so make sure you delete first).(so make sure you delete first).

DM420DM420!Begin Trace Log at Fri Apr 27 16:06:40 2007!Begin Trace Log at Fri Apr 27 16:06:40 2007!Fri Apr 27 16:06:50 2007 [29097,01A6B4A0] iidbdb >>>>> set trace point !Fri Apr 27 16:06:50 2007 [29097,01A6B4A0] iidbdb >>>>> set trace point

'dm420''dm420'

! Buffer Status: MT! Buffer Status: MT! Buffer Manager Id: 12 Connected servers 1! Buffer Manager Id: 12 Connected servers 1! CP count: 725 CP index : 0 CP check : 0! CP count: 725 CP index : 0 CP check : 0! Database cache size: 40 Table cache size: 40! Database cache size: 40 Table cache size: 40! Statistics-----------------------------------------------! Statistics-----------------------------------------------! Lock reclaim: 0 ! Lock reclaim: 0 CP flushes: 724CP flushes: 724! Buffer Cache Configuration ( 2K) ------------------------! Buffer Cache Configuration ( 2K) ------------------------! Buffer count: 1064 Bucket count: 1063 Group count: 42 Size: 8! Buffer count: 1064 Bucket count: 1063 Group count: 42 Size: 8! Free count: 327 Limit: 23 Modify count: 401 Limit: 546! Free count: 327 Limit: 23 Modify count: 401 Limit: 546! Free group count: 41 Modify group count: 1! Free group count: 41 Modify group count: 1! Fixed count: 0 Group fixed count: 0! Fixed count: 0 Group fixed count: 0! WB start limit: 437 WB end limit: 364! WB start limit: 437 WB end limit: 364! ! WB flushes: 5519WB flushes: 5519, WB threads created: 0, WB threads created: 0! WB threads active: 0, WB thread hwm: 1! WB threads active: 0, WB thread hwm: 1! WB pages flushed: 73, WB groups flushed: 0! WB pages flushed: 73, WB groups flushed: 0

WB Flushes > CP FlushedWB Flushes > CP Flushed

System can’t keep pages in memory long enough.System can’t keep pages in memory long enough.

Increase dmf_wb_start (or reduce CP_Interval)Increase dmf_wb_start (or reduce CP_Interval)

DM420 by Page SizeDM420 by Page Size! Summary Statistics ( ! Summary Statistics ( 2K2K) --------------------------------------) --------------------------------------! GWAIT GSYNC FREEWAIT GW PAGES GW I/O! GWAIT GSYNC FREEWAIT GW PAGES GW I/O! 5 0 ! 5 0 00 570318 57072 570318 57072! FIX CALLS HITS CHECK REFRESH READ TOSS! FIX CALLS HITS CHECK REFRESH READ TOSS! ! 414658001414658001 332355910332355910 82787709 63 82787709 63 5030104550301045 74932 74932! UNFIX CALLS DIRTY FORCE WRITE IOWAIT SYNC! UNFIX CALLS DIRTY FORCE WRITE IOWAIT SYNC! 217670172 1907841 25639 919761 1961 0! 217670172 1907841 25639 919761 1961 0! MWAIT FCWAIT RECLAIM REPLACE GREADS GWRITES! MWAIT FCWAIT RECLAIM REPLACE GREADS GWRITES! 9 2 47251 50824104 ! 9 2 47251 50824104 247568079247568079 0 0

FREEWAIT FREEWAIT MUSTMUST be zero – be zero –

otherwise SERIOUSLY increase dmf_cache_size for the pagesizeotherwise SERIOUSLY increase dmf_cache_size for the pagesize

HitRate = “HITS” / “FIX CALLS” Should be >= 95%HitRate = “HITS” / “FIX CALLS” Should be >= 95%

Increase dmf_cache_size and dmf_group_countIncrease dmf_cache_size and dmf_group_count

Check Check EACHEACH Page Size Page Size

““READ” should be less than “GREADs” (Group Reads = Table Scans)READ” should be less than “GREADs” (Group Reads = Table Scans)

Perform Optimize, Missing Indexes, Check SQLPerform Optimize, Missing Indexes, Check SQL

Modifying Modifying ConfigurationConfiguration

MEASURE weeklyMEASURE weekly– Keep history of snapshotsKeep history of snapshots– Take snapshot before weekly Take snapshot before weekly

bouncebounce– If no bounce, then subtract valuesIf no bounce, then subtract values

DON’T make changes blindlyDON’T make changes blindly

CHANGE and bounceCHANGE and bounce MEASURE the following week(s)MEASURE the following week(s)

??