BravePoint Progress System Tables Dan Foreman BravePoint, Inc. Email: [email protected].

BravePoint

Progress System Progress System TablesTables

Dan ForemanDan Foreman

BravePoint, Inc.BravePoint, Inc.

Email: [email protected]: [email protected]

BravePoint

Introduction - Dan ForemanIntroduction - Dan Foreman• Progress User since 1984 (V2.1)Progress User since 1984 (V2.1)• Presenter at lots of Progress Presenter at lots of Progress

conferences dating back to 1990conferences dating back to 1990

BravePoint

Introduction - Dan ForemanIntroduction - Dan ForemanAuthor of:Author of:• Progress Performance Tuning GuideProgress Performance Tuning Guide• Progress Database Admin GuideProgress Database Admin Guide• Progress Virtual System TablesProgress Virtual System Tables• Pro D&LPro D&L - Dump/Load with short down time no - Dump/Load with short down time no

matter how large the Database ismatter how large the Database is• ProMonitorProMonitor – DB monitoring tool (uses VSTs) – DB monitoring tool (uses VSTs)• ProCheckProCheck – Monitor AppServers & WebSpeed – Monitor AppServers & WebSpeed

BravePoint

What are Virtual System TablesWhat are Virtual System Tables

Virtual System tables (VSTs) provide Virtual System tables (VSTs) provide access to Database Shared Memory data access to Database Shared Memory data that was previously only accessible that was previously only accessible through through promonpromonIt’s possible to customize the user It’s possible to customize the user interfaceinterfaceAccessible Locally or Remotely via the Accessible Locally or Remotely via the 4GL or SQL4GL or SQL

BravePoint


VSTs are Schema TablesVSTs are Schema TablesThey do not interfere with Application tables and IndexesVSTs are not part of the empty Database in V8Enable with proutil <db> -C enablevstAre part of the V9/V10/V11 empty DatabaseAre visible in the Dictionary (must view ‘Hidden’ tables)


Some new tables and fields are not in Some new tables and fields are not in the “native” empty DB unless:the “native” empty DB unless:

proutil –C updatevst is run

Example: New VST fields added in V10.2B SP6

The feature must be enabled:Auditing: proutil enableauditing

TDE

Multi-Tenancy

BravePoint

BravePoint


VSTs in the Standard empty DBVSTs in the Standard empty DBV8.2V8.2 30 tables30 tablesV8.3V8.3 35 tables35 tablesV9.0-V9.1CV9.0-V9.1C 39 tables39 tablesV9.1DV9.1D 42 tables42 tablesV10V10 45 tables45 tablesV10.1BV10.1B 47 tables47 tables

What are Virtual System TablesWhat are Virtual System Tablespromonpromon can lag behind VSTs; can lag behind VSTs; Examples:Examples:

_TableStat & _IndexStat (V8.3)_AreaStatus (V9 and later)

Conversely VSTs can lag behind also:Conversely VSTs can lag behind also:_Startup

BravePoint

BravePoint

Resource Impact of VSTsResource Impact of VSTsThere is no performance penalty for accessing There is no performance penalty for accessing VSTs with 2 exceptions that will be covered VSTs with 2 exceptions that will be covered laterlaterVSTs take up no additional disk space except VSTs take up no additional disk space except for the VST schemafor the VST schemaVSTs take up a small amount of memory to VSTs take up a small amount of memory to cache the additional schemacache the additional schema

Except –tablerangesize & -indexrangesize can increase DB Shared Memory by a surprising amount

BravePoint

VST ImplementationVST Implementation

Whenever a record is requested, the Whenever a record is requested, the table is checked to see if it’s a VST table is checked to see if it’s a VST table.table.

VST tables have table numbers between -16385 to -32768.The check is simple arithmetic which takes place regardless of whether VSTs have been enabled or not

BravePoint

VST ImplementationVST Implementation

If a requested record is a VST record, If a requested record is a VST record, the fields are populated from shared the fields are populated from shared memory structures instead of from a memory structures instead of from a database recorddatabase record

BravePoint

VST BasicsVST BasicsEach VST has one, unique index on the Each VST has one, unique index on the ‘Id’ field which is a ‘virtual’ RECID ‘Id’ field which is a ‘virtual’ RECID Each VST record is numbered starting Each VST record is numbered starting at 1 and continues sequentially (some at 1 and continues sequentially (some exceptions)exceptions)It is not possible to add new indexes or It is not possible to add new indexes or triggers to VSTstriggers to VSTs

BravePoint

VST BasicsVST BasicsSome VST Record Counts are controlled by startup Some VST Record Counts are controlled by startup parametersparameters_connect_connect -n & -Mn-n & -Mn_LockReq_LockReq -n & -Mn-n & -Mn_UserIO_UserIO -n & -Mn-n & -Mn_UserLock_UserLock -n & -Mn-n & -Mn_UserStatus_UserStatus -n & -Mn-n & -Mn_Trans_Trans -n-n_Servers_Servers -Mn-Mn_Lock_Lock -L-L

BravePoint

VST BasicsVST BasicsThe tables on the last slide are pre-populated The tables on the last slide are pre-populated at Database Broker startup timeat Database Broker startup timeThe empty slots have “?” valuesThe empty slots have “?” valuesTo avoid displaying the empty slots:To avoid displaying the empty slots:FOR EACH _connect FOR EACH _connect

WHERE _connect-usr <> ?WHERE _connect-usr <> ?The <> operator is not a performance The <> operator is not a performance problem since VSTs are not really ‘indexed’problem since VSTs are not really ‘indexed’

BravePoint

Which DatabaseWhich Database

If a Client is connected to multiple If a Client is connected to multiple Databases, the VST Statistics will be Databases, the VST Statistics will be based upon the Current Working based upon the Current Working DatabaseDatabase

BravePoint

Updatable VSTsUpdatable VSTs

_Startup-spin_Startup-spin -spin-spin

_Startup-APWQTime_Startup-APWQTime -pwqdelay-pwqdelay

_Startup-APWSTime_Startup-APWSTime -pwsdelay-pwsdelay

_Startup-APWBuffs_Startup-APWBuffs -pwscan-pwscan

_Startup-APWMaxWrites_Startup-APWMaxWrites -pwwmax-pwwmax

_Startup-_Startup-

Updateable in V10.2B SP6Updateable in V10.2B SP6

_Startup._Startup-LRU-Skips_Startup._Startup-LRU-Skips

_Startup._Startup-LRU2-Skips_Startup._Startup-LRU2-Skips

_Startup._Startup-NmsgWait_Startup._Startup-NmsgWait

_Startup._Startup-Pefetch-Delay_Startup._Startup-Pefetch-Delay

_Startup._Startup-Prefetch-Factor_Startup._Startup-Prefetch-Factor

_Startup._Startup-Prefetch-Num-Recs_Startup._Startup-Prefetch-Num-Recs

_Startup._Startup-Prefetch-Priority_Startup._Startup-Prefetch-Priority

BravePoint

BravePoint


_MyConnect-NumSeqBuffers_MyConnect-NumSeqBuffers

Added in V9Added in V9

Same as –Bp parameterSame as –Bp parameter

Number of Sequential Read BuffersNumber of Sequential Read Buffers

Makes it possible to run a large report Makes it possible to run a large report with a less severe impact on the -B with a less severe impact on the -B Buffer CacheBuffer Cache

Example on the next slideExample on the next slide

BravePoint


DO FOR _MyConnect TRANSACTION :DO FOR _MyConnect TRANSACTION :

FIND FIRST _MyConnect.FIND FIRST _MyConnect.

_myconnect-numseqbuffers = 32._myconnect-numseqbuffers = 32.

END.END.

RUN killer_report.p.RUN killer_report.p.

DO FOR _MyConnect TRANSACTION : DO FOR _MyConnect TRANSACTION :

FIND FIRST _MyConnect.FIND FIRST _MyConnect.

_myconnect-numseqbuffers = 0._myconnect-numseqbuffers = 0.

END.END.

BravePoint


_StatBase._TableBase_StatBase._TableBase_StatBase._IndexBase_StatBase._IndexBaseV9 and laterV9 and laterUsed to set the window size of Tables and Used to set the window size of Tables and Indexes monitored using the _TableStat Indexes monitored using the _TableStat and _IndexStat VSTs or to reset (zero out) and _IndexStat VSTs or to reset (zero out) the statistics by moving the “window” up the statistics by moving the “window” up and then downand then down

BravePoint

Table & Index I/O VSTsTable & Index I/O VSTs

Note that _TableStat and _IndexStat Note that _TableStat and _IndexStat are available in V8.3 and very helpful are available in V8.3 and very helpful for Storage Area Planningfor Storage Area PlanningUse _IndexStat to identify unused Use _IndexStat to identify unused indexes; COMPILE/XREF is only good indexes; COMPILE/XREF is only good for finding indexes that are defined but for finding indexes that are defined but not used in the code; COMPILE/XREF not used in the code; COMPILE/XREF is useless for Dynamic Queriesis useless for Dynamic Queries

BravePoint

VST ProblemsVST Problems

Potential Performance KillersPotential Performance Killers_Block ( V8, useless in V9/V10/V11 )

_Lock ( FOR EACH _lock WHILE … )

_UserLock ( DO loops of 512 iterations )

Too Much Data for One ScreenToo Much Data for One Screen_Block-block x(1024)

_UserLock-* Several arrays of 512

Missing DataMissing Data_DBStatus

BravePoint

VST ProblemsVST ProblemsStorageStorage

VSTs are defined as INTEGERs which limits the maximum value to about 2 billion – prior to V10.1B

Poorly documented (see my Poorly documented (see my System Table System Table GuideGuide for better documentation) for better documentation)Stale DataStale Data

_Lock (total # of records) compared to _DbStatus-NumLocks

System Tables - MetaSchemaSystem Tables - MetaSchema

_File_File

_Field_Field

_Index_Index

_Index-Field_Index-Field

_File-Trig_File-Trig

_Field-Trig_Field-Trig

_Db_Db

_Sequence_Sequence

BravePoint

System Tables - OtherSystem Tables - Other

SQLSQLAuthentication (GRANTs & REVOKEs)

Views

Constraints

Query Optimizer statistics

AuditingAuditing

EncryptionEncryption

Multi-TenancyMulti-Tenancy

BravePoint

BravePoint

Usefulness Usefulness

VSTs are useful for two types of VSTs are useful for two types of Progress usersProgress users

Application Developers

Database Administrators

Who else is there?

BravePoint

For DevelopersFor Developers

Can view the lock status for an individual Can view the lock status for an individual recordrecord_Lock VST shows all active locks _Lock VST shows all active locks andand who who has the lockhas the lockWe can easily find the Table name of the We can easily find the Table name of the record being locked by doing a join to the record being locked by doing a join to the _File table_File table_Trans VST shows all active transactions_Trans VST shows all active transactions

BravePoint

For DevelopersFor DevelopersDeadly Embrace (Deadlock) DetectionDeadly Embrace (Deadlock) Detection

_Lock_Connect

““Delinquent” Active Transactions that Delinquent” Active Transactions that grow the BI File to an abnormally large grow the BI File to an abnormally large sizesize

_Connect_Trans

BravePoint


Improve code efficiency by measuringImprove code efficiency by measuring::User DB Accesses, Reads, etc.

Reads, Writes, Creates, Deletes by Table_TableStat

Unused indexes; Logical Scatter Factor_IndexStat


Name & attributes of a FieldName & attributes of a Field

find _file “customer” no-lock.find _file “customer” no-lock.

for each _field of _file no-lock :for each _field of _file no-lock :

displaydisplay

_field-name_field-name

_data-type_data-type

_format_format

_extent._extent.

end.end.

BravePoint

BravePoint

DBAs - Tables of interestDBAs - Tables of interest

Monitoring the BI High Water MarkMonitoring the BI High Water Mark_DbStatus._dbstatus-bisize

Available in V8.3 and later

V8: BI Size in Bytes

V9 & later: BI Size divided by BI Block Size

BravePoint

DBAs - Tables of interestDBAs - Tables of interestMonitoring Database SizeMonitoring Database Size

promon doesn’t have the correct data

Each Storage Area has a separate High Water Each Storage Area has a separate High Water MarkMark

_AreaStatus_AreaStatus-AreaName_AreaStatus-HiWater_AreaStatus-TotBlocks_AreaStatus-LastExtent_AreaStatus-Extents

BravePoint

DBAs - Tables of InterestDBAs - Tables of InterestRecord FragmentationRecord Fragmentation

_Record._Record-RecRead

_Record._Record-FragRead

dbanalys/tabanalysdbanalys/tabanalys only shows only shows how many Fragments exist, not how many Fragments exist, not how often the Fragments are how often the Fragments are accessedaccessed

DBAs - Tables of InterestDBAs - Tables of InterestWhat Client is running what ProgramWhat Client is running what Program

_Connect-Cache*

Also Line# for 4GL Code

Or SQL Statement

Since V10.1C

What Tables is a Client HittingWhat Tables is a Client Hitting_UserTableStat

Since V10.1B

BravePoint

BravePoint

For More InformationFor More Information

Progress System Tables GuideProgress System Tables Guide200+ pages

Covers V8 through V11

Covers VSTs and System Tables

Has lots of undocumented information

BravePoint

ProMonitorProMonitorWritten using (mostly) VSTsWritten using (mostly) VSTsSource Code provided for most ProgramsSource Code provided for most ProgramsThreshold Alerts warn of problemsThreshold Alerts warn of problemsMonitor an unlimited number of DBsMonitor an unlimited number of DBsMonitors more than just VST StatisticsMonitors more than just VST Statistics

Session Temp File I/OClient Memory UtilizationDB Log File MonitoringDB Fragmentation/Scatter MonitoringAppServers, WebSpeed, AdminServer, etc

BravePoint

ConclusionConclusionQuestions?Questions?Dan ForemanDan Foreman

[email protected]

BravePoint Progress System Tables Dan Foreman BravePoint, Inc. Email: [email protected].

Documents

Transcript of BravePoint Progress System Tables Dan Foreman BravePoint, Inc. Email: [email protected].