B2: What’s New in 10.1 RDBMS? So many features, so few releases Richard Banville Progress Fellow.

B2: What’s New in 10.1 RDBMS?

So many features, so few releases

Richard BanvilleProgress Fellow

© 2008 Progress Software Corporation2

Agenda

High Availability• Problem Avoidability

• Visibility

• Scalability

Maintainability

The database “ilities”


Database Consistency Checking

Index orderId in order for recid 2010 could not be deleted.

Wrong key in idx 10 for record 2010.

Invalid size of an index entry.

Seen these messages before?



Invalid RM block for area 10

rmdoins: pbk->free went negative dbkey 4096

bkwrite: bktbl dbk 4096 not equal to bkbuf dbk -1234

bkaddr called with negative blkaddr: -1234

Or how about these…



Shared memory overwrite protection• -MemCheck

Physical block consistency checking• -DbCheck

• -AreaCheck “area name”

• -Index Check “index name”

• -TableCheck ‘table name”

Stop problems before they happen



Database startup parameter Managed via promon R&D Admin Functions

• 8. Block level consistency check

Enabling the consistency checks

Current consistency check status:

1. -MemCheck: enabled2. -DbCheck: enabled3. -AreaCheck in area: "customer" enabled4. -IndexCheck: disabled5. -TableCheck: disabled

Enter the option to enable/disable a consistency check:



Memory checking: unnoticeable impact Block level checking

Performance impact

Current consistency check status:

1. -MemCheck: enabled2. -DbCheck: enabled3. -AreaCheck in area: "customer" enabled4. -IndexCheck: disabled5. -TableCheck: disabled

Enter the option to enable/disable a consistency check:

~5%

< 1%



dbtool block consistency checking

5. Read or Validate Database Block(s)

Validation levels• 0: Block header info only• 1: Record header & record size• 2: Record overlap checking

3. Record Validation (logical validation)

Looking for existing inconsistencies online


Online Tools

Online dbanalys - Includes chanalys info Tools now online

proutil <db> -C chanalys [ -scan ]

• -scan: fewer locks, less consistent

proutil <db> -C idxcheck

• Idxcheck validation levels– Physical consistency– Keys to records– Records to keys– Validate key order– Locks associated tables

More analysis available online

contention


The roll forward process

myDb

myDb.bak



myDb

ai ai ai ai

ftp



myDbHot

Standbyai ai ai ai

ftp

ai ai ai ai

X

SYSTEM ERROR: Attempt to read block 18446744073709550382 which does not exist in area 8, database x.

** Save file named core for analysis by Progress Software Corporation.

Roll forward


Ai Verify

rfutil <db> -C aiverify <type>

• Partial: ai block and note header validation– Increases reliability of archived ai files

• Full: partial + note data validation– Identifies point in time recovery

• Running– At ai switch or on ai archival– Just before roll forward of extent

Preferably on hot standby

Ai validation before application


Roll forward verification

myDbHot

Standbyai ai ai ai

ftp

ai ai ai ai

X

Hot Stand by:• Validate/fix production db• Re-base hot standby

Recovery Scenario:• Roll forward to transaction

rfutil myDb -C aiverify full

rlNoteVerify: Note dbkey is negative -1234. (14099)Trid: 358 code = RL_CXINS version = 2 (12528)


More tools for high availability

“Online” backup of replication target (foundational work)• Normal operating state required

• SHR schema lock on source

• ai file stores changes until complete

EMC’s SRDF* certification• Real time copies of logical data volumes

• Data replication

• Remote backup

Replication enhancements

*Symmetrix Remote Data Facility (SRDF)

• Disaster recovery

• Fail over/fail back


The problem

Out of free shared memory. Use -Mxs to increase.

Lock table overflow, increase -L on server.

Have you ever seen these error messages…

or heard these…

Why can’t you improve the buffer pool hit ratio on the database?

The recovery subsystem is a bottle neck. Look at that BI buffer wait %.


Increase startup parameters online

proutil <db> -C increaseto <params> <params>: -L, -B, -bibufs, -aibufs, -Mxs Increase, not decrease Resource restrictions apply New shared memory segments

• Security restrictions– Servers: automatically attach quickly– Self serve: attach w/db action over time

• Segment size

Increase startup parameters without database restart


Increase startup parameters online

Increasing available locks online

proutil myDb -C increase -L 10000

Waiting for broker connection to newly added shared memory segments. Usr Name Type Pid 7 richb ABL 5957

The connections above have not attached to recently added shm segments.Do you wish to recheck? (y/n)

(n): Increase params aborted because of shared memory allocation issue.

(y): Increase params increasing lock table size (-L) from 1025 to 10016.


Agenda


• Visibility

• Scalability

Maintainability



Promon

Promon R&D Status• 17. Servers By Broker

• A more organized view of existing data

Better organized server grouping

Sv Pend. Cur. Max. Port No Pid Type Protocol Logins Users Users Users Num

0 15275 Login TCP 5 0 0 15 2053 2 15501 Auto TCP 1 0 1 15 1025 3 15509 Auto TCP 1 0 1 15 1026 4 15511 Auto TCP 1 0 1 15 1027 5 15514 Auto TCP 1 0 1 15 1028 1 15381 Login TCP 5 0 0 5 2051 7 15609 Auto TCP 1 0 1 5 1030 8 15617 Auto TCP 1 0 1 5 1031 9 15629 Auto TCP 1 0 1 5 1032 10 15638 Auto TCP 1 0 1 5 1033


Promon

Promon R&D Other Displays• 7. Total Locks per User

User type display – _Connect-ClientType

Improved user information

User Name Type PID TTY Total Record SHR/EXCL... 5 richb SELF/ABL 15494 /dev/pts/16 1 1 1 11 richb SELF/ABL 16101 /dev/pts/13 3 3 0 24 richb REMC/SQLC 15530 2 1 0 48 richb REMC/WTA 20182 mysystem 2 2 0100 richb REMC/ABL 20183 mysystem 2 2 0101 richb SQFC 20100 mysystem 5 4 0150 richb REMC/APSV 20101 mysystem 1 1 0175 richb SELF/APSV 20102 mysystem 1 1 0


Statement Caching

List recent client statements• Promon R&D Status

18. Client Database-Request statement Cache– By user/server/all users current and future.– Last line or entire stack– ABL info obtained from DEBUG-LIST output

.i’s are in-lined

What code is executing against my database


Statement Caching

Where is that ABL code executing

User number : 24 User name : richb User type : REMC/ABL Login date/time : 03/06/08 15:30

Statement caching type : ABL Program StackStatement caching last updated : 03/06/08 15:35

Statement cache information : 39 : proc7 /usr1/richb/x.p 26 : proc6 /usr1/richb/x.p 22 : proc5 /usr1/richb/x.p 18 : proc4 /usr1/richb/x.p 14 : proc3 /usr1/richb/x.p 10 : proc2 /usr1/richb/x.p 6 : proc1 /usr1/richb/x.p 3 : /usr1/richb/x.p


Statement Caching

What’s that SQL code executing

User number : 23 User name : richb User type : REMC/SQLCLogin date/time : 03/06/08 15:42

Statement caching type : SQL StatementStatement caching last updated : 03/06/08 15:42

Statement cache information : select count(*) from pub.customer


Statement Caching

proutil <db> -C updatevst• Need to load new schema fields

_Connect vst• _Connect-CachingType

• _Connect-CacheLastUpdate

• _Connect-CacheInfoType

• _Connect-CacheLineNumber[32]

• _Connect-CacheInfo[32]

VST support - _Connect


Agenda


• Visibility

• Scalability

Maintainability



Scalability

Large file support for bulk load (> 2Gb)• Independent of DB large file status

IPv6 support• More ip addresses

– only 30% ip addrs left (7 yrs)

• Routing improvements• Required by government contracts• Configuration

– -ipver IPv4 (default) or IPv6– Property file: ipver=– Explorer option

I want more


Scalability

IPv6 support• Mixed mode (dual stack)

– IPv6 can convert IPv4 address

– Not available on windows

• Avoiding confusion– -minport/-maxport

– use IPv6 configured –H hostnames

I want more


Internal Performance Improvements

“clean” shutdown indicator• Avoids long redo in roll forward

• Last 2 clusters always redone

Improved read operation concurrency• Latch enhancements and usage optimization

Multi-user bi activity optimization• Avoid rollback “block jump” notes

• Fewer notes written

The need for speed


Agenda


• Visibility

• Scalability

Maintainability



Roll forward protection

myDb

ai ai ai ai ai ai ai ai

myDb.bak



myDb


X** The database was last changed Mon Apr 1 15:37:38 2008.** The after-image file expected Mon Apr 1 15:33:45 2008.** Those dates don't match, so you have the wrong copy of one of them.roll forward open /usr1/x.a4 error: -1.

Roll forward



myDb


XIn the .lg file:

rfutil -C roll forward session end. Single-user session begin for richb on /dev/pts/101. Begin Physical Redo Phase at 256 .



rfutil <db> -C roll forward oplock Prevents “stray” database connections

• Prostrct add allowed Automatic disablement

• At roll forward completion Explicit disablement

rfutil <db> -C roll opunlock• Recovers db

• Stops the roll forward process

Non interruptible roll forward



myDb


- Access to the database during roll forward process is not allowed because it will modify the database. - Write access to the database will not be allowed until the roll forward operations have completed.

rfutil <db> -C roll forward oplock –a myDb.a1

Connection attempts:


Index Rebuild

Index rebuild packing factor• Max % of space used

• Avoids costly index block splits

proutil <db> -C idxbuild –pfactor <60 – 100>

Examine utilization % in idxanalys

More control


Index Fix Interface

Idxfix uses idxbuild interface• Select indexes to fix by

– Table, schema, area or activation state

Specific choices

Select one of the following:

All (a/A) - Fix all the indexes

Some (s/S) - Fix only some of the indexes

By Area (r/R) - Fix indexes in selected areas

By Schema (c/C) - Fix indexes by schema owners

By Table (t/T) - Fix indexes in selected tables

By Activation (v/V) - Fix selected active or inactive indexes


SQL Stored Procedures

64 bit JVM availability• Java™ 1.5 certification

• Can use same drivers

• Additional schema– _SysProcBin, _SysProcText

• 32 bit databases have schema already

• Databases created in 64 bit environmentproutil <db> -C enablestoredproc

64 bit stored procedure support.


Binary Dump Specified - Improved

proutil <db> -C dumpspecified <field-info>

<op1> <low-value>

AND <op2> <high-value> <dir>

Option values: GT, GE, LT, LE, EQ Dump specific ranges Improved parallelism

proutil db –C dumpspecified cust.custnum GE 5 and LE 100 /dumpdir

Binary dump specified with “between” range dumping


In Summary

We’ve made it even easier… to achieve high availability to see what’s going on to maintain


Relevant Exchange Sessions

OPS-1: How Healthy is Your Database Today?

OPS-18: Data Management Roadmap

OPS-19: What’s IPV6 and Why Should I Care?

OPS-28: A New Spin on Some Old Latches


Questions?


What’s here that is also in 10.1B03?

Database Consistency Checking• -memCheck, 0DbCheck, -AreaCheck, -IndexCheck, -

TableCheck Dbtool: Read or Validate Database Blocks Rfutil:

• aiverify• roll forward oplock

Promon:• Servers by Broker• Total Locks per User

Record lock removal on NO-LOCK reads


Thank You

B2: What’s New in 10.1 RDBMS? So many features, so few releases Richard Banville Progress Fellow.

Documents

Transcript of B2: What’s New in 10.1 RDBMS? So many features, so few releases Richard Banville Progress Fellow.