1 RAC Internals Julian Dyke Independent Consultant Web Version © 2007 Julian Dyke

Click here to load reader

download 1 RAC Internals Julian Dyke Independent Consultant Web Version   © 2007 Julian Dyke

of 19

  • date post

    13-Jan-2016
  • Category

    Documents

  • view

    220
  • download

    0

Embed Size (px)

Transcript of 1 RAC Internals Julian Dyke Independent Consultant Web Version © 2007 Julian Dyke

Forgotten Features© 2007 Julian Dyke
System Change Number
In RAC clusters SCN must be maintained across all nodes in cluster
SCN propagation scheme differs according to version
In Oracle 9.2 and below defaults to Lamport algorithm
Lamport or SCN Scheme 2 in alert.log
SCN piggy-backed on GCS/GES messages
Recorded in redo log
In Oracle 10.1 and above uses a new algorithm
SCN Scheme 3 in alert.log
Broadcast on commit
Apparently no delay
Default value is 700 centiseconds (7 seconds)
Specifies maximum time taken for a COMMIT on one node to be reflected on other nodes in the cluster
For some applications, value must be set to 0 (Broadcast on commit) including:
E-Business suite
juliandyke.com
© 2007 Julian Dyke
LMS Background Processes
LMS background processes:
Implement cache fusion
Serve both consistent and current versions of blocks in cache of local instance to other instances
Maintain local part of Global Resource Directory
Minimum of 1 LMS process per instance
Maximum is version dependent
Prior to Oracle 10.1, could be configured using _lm_lms parameter
In Oracle 10.1 and above, initial number of LMS processes specified by gcs_server_processes parameter
juliandyke.com
Each LMS background process manages a set of blocks
Determined by hash function based on number of LMS background processes
Consequently
a block will always be handled by the same LMS process
Number of blocks served recorded in
Session / System statistics
slot 0
col1: ENG
col2: 340
col3: 1
slot 1
col1: AUS
col2: 99
col3: 10
CURRENT_REQUESTS
NUMBER
DATA_REQUESTS
NUMBER
UNDO_REQUESTS
NUMBER
TX_REQUESTS
NUMBER
CURRENT_REQUESTS
NUMBER
PRIVATE_REQUESTS
NUMBER
ZERO_RESULTS
NUMBER
DISK_READ_RESULTS
NUMBER
FAIL_RESULTS
NUMBER
FAIRNESS_DOWN_CONVERTS
NUMBER
FAIRNESS_CLEARS
NUMBER
FREE_GC_ELEMENTS
NUMBER
FLUSHES
NUMBER
FLUSHES_QUEUED
NUMBER
FLUSH_QUEUE_FULL
NUMBER
FLUSH_MAX_TIME
NUMBER
ERRORS
NUMBER
juliandyke.com
© 2007 Julian Dyke
Light Works Rule
In theory, once a block has been written to disk, the LMS process will not attempt to read it again when responding to a consistent read request
Light Works Rule
Prevents LMS processes from going to disk when responding to CR requests for data, undo or undo segment blocks
Can prevent LMS process from completing its response to a CR request
juliandyke.com
22:10
22:10
GC Read Uncommitted Block
Uncommitted changes MUST be flushed to the redo log before the LMS process can ship a consistent block to another instance
Reading process must wait until redo log changes have been written to redo log by LMS process
Bad for standard RAC databases
Reads must wait for redo log writes
Worse for extended / stretch RAC clusters
Increased latency of cross site disk communications
juliandyke.com
GC Read Uncommitted Block
For each block on which a consistent read is performed, a redo log flush must first be performed
Number of redo log flushes is recorded in the FLUSHES column of V$CR_BLOCK_SERVER
Redo log flush time
is recorded in the gc cr block flush time statistic for the LMS process
will increase time taken to serve consistent block
will increase time taken to perform consistent read
If LMS processes become very busy, consistent reads will experience high wait times e.g. for a full table scan
gc cr multi block request
juliandyke.com
22:10
22:10
SEE SLIDE NOTES FOR ADDITIONAL INFORMATION
In this example, Instance 1 is attempting to read a block which has been updated by Instance 2. .The data block has been flushed to disk by Instance 2 from its buffer cache before Instance 1 initiates the read operation.
As the data block is no longer in the buffer cache of Instance 2, Instance 1 is responsible for building the read-consistent version.
Instance1 therefore reads the data block from disk. However, in this example, the undo is still the buffer cache of Instance 2. Therefore, in order to build the read-consistent image of the block, Instance 1 must request the relevant undo block(s) from Instance 2. The LMS process on Instance 2 is responsible for serving the undo blocks to Instance 1. When Instance 1 receives the undo block(s) it will apply them locally to build the consistent-read version of the data block.
In this example, there is only one undo block in the buffer cache of Instance 2. Therefore it is only necessary to serve one undo block to Instance 1. However, if, for example, the data block had contained a sequence number which had been updated many times without committing, there could be many undo blocks in the buffer cache of Instance 2 which contained undo information for the data block. It would be necessary to serve all those undo blocks to Instance 1 which contained changes since the start of the transaction on Instance 2.
juliandyke.com
Consistent Reads in RAC
If possible, blocks will always be read from the cache of another instance
Undo blocks will be flushed to disk more frequently when:
All columns are updated
Indexed columns are updated
Transactions are regularly rolled back
Rows locked using SELECT FOR UPDATE
Data blocks will be flushed to disk more frequently when:
Most transactions are read-only
Consistent read response times in RAC can be reduced by:
Avoid reading uncommitted blocks on remote nodes
Partitioning
Specifying SCN
Must retain ACID properties
May be possible to use application logic to synchronize writes and reads
Increasing number of LMS processes on remote node
Should be added dynamically by kernel
Also by obvious hardware changes such as
reducing latency of interconnect
For more information and to provide feedback
please contact me