Post on 03-Jan-2016
EM414 – SQL Remote Tips & Techniques
Robert WaywellSenior Product Support EngineeriAnywhere SolutionsRwaywell@sybase.com
Objectives
• Provide technical examples showing how to implement real-world business requirements using SQL Remote
• Develop an understanding of these examples which will enable their extension and customization
Topics
• Multiple Subscriptions
• Message System Control ‘Chart’
• One-Way Publications
• Self-Refreshing Primary Key Pool
• Passthrough in a Multi-Tier Environment
• Request Queue
• DBRemote and DBXtract Exclusivity
• Last Chance Threshold
Sample Databases
Head Office
Regional Office
Manager1SalesRep1 SalesRep2
Multiple Subscriptions
Business RequirementA given user or type of user requires a super-set of the
data sent to other users or classes of users
Example: Each sales rep is assigned customers and the manager needs to see all the customers for each of their sales reps
Regional Office
Multiple Subscriptions Business Requirement
Customer
1 Jones Rep12 Rice Rep23 Smith Rep3
SalesRep1
Customer
1 Jones Rep1
2 Rice Rep2
SalesRep2Customer
Manager 1Customer
1 Jones Rep12 Rice Rep2
Multiple Subscriptions Design Considerations
Publications are handled by the engineThe more publications you have the more work the engine has
to do to evaluate each publication and the more information that has to be written to the log file
When using SQL Remote for ASE this work is handled by the scan phase (-i) of SSRemote.exe
Subscriptions are handled by DBRemote/SSRemoteDuring the send phase, DBRemote/SSRemote maps the current
Subscriptions to the Publication information in the log file and generates appropriate messages for each remote user
2 Rice Rep1
Publications are handled by the engine
Multiple Subscriptions Design Considerations
Customer
UPDATE Customer SET sales_rep = Rep1 WHERE cust_id = 2
Log File
Customer_Pub2OLD(…)NEW(…)
Customer_Pub1
OLD(…)
NEW(…)
UPDATE Customer. . .
Rep2
Multiple Subscriptions Design Considerations
Subscriptions are handled by DBRemote/SSRemoteLog FileCustomer_Pub1
OLD(Rep1)NEW(Rep2)
UPDATE Customer. . .
SaleRep1
Delete . . .
SalesRep2
Insert . . .
Manager 1
Update . . .
DBRemote/SSRemote
sys.syssubscriptionsCustomer_Pub1 SalesRep1 Rep1Customer_Pub1 SalesRep2 Rep2Customer_Pub1 Manager1 Rep1Customer_Pub1 Manager1 Rep2
Multiple Subscriptions Design Alternatives
Create Two Publications One for the Managers
One for the Sales Reps
Create One PublicationPartition records at the Sales Rep level
Create multiple subscriptions for a Manager
Multiple Subscriptions Implementation
Customer
customer_idcompany_nameaddress
Linksalesrep_idcustomer_iddate_assigned
SalesRepsalesrep_idregion_idsalesrep_namesalesrep_phonedepartment
Multiple Subscriptions Implementation
CREATE PUBLICATION Customer_Pub( TABLE Customer SUBSCRIBE BY
( SELECT salesrep_id FROM Link WHERE Customer.customer_id = Link.customer_id
))
CREATE SUBSCRIPTION TO Customer_Pub(‘1’) FOR SalesRep1;CREATE SUBSCRIPTION TO Customer_Pub(‘2’) FOR SalesRep2;
CREATE SUBSCRIPTION TO Customer_Pub(‘1’) FOR Manager1;CREATE SUBSCRIPTION TO Customer_Pub(‘2’) FOR Manager1;
Topics
• Multiple Subscriptions
• Message System Control ‘Chart’
• Self-Refreshing Primary Key Pool
• Passthrough in a Multi-Tier Environment
• Request Queue
• DBRemote and DBXtract Exclusivity
• Last Chance Threshold
Message System Control ‘Chart’
Business RequirementSQL Remote has inherent latency in communications, this
is one of the characteristics of message based replication.
Remote users are typically expected to replicate on a semi-regular schedule.
Need to differentiate between expected latency and unexpected latency.
Message System Control ‘Chart’ – Business Requirements
User_name … Log_sent Confirm_sent …
Rem1 332214 331903
Rem2 332214 331353
…
Message System Control ‘Chart’
Design ConsiderationsLatency can be measured as the difference between
the log_sent and confirm_sent values for a given remote (or consolidated) user.
Amount of expected latency is specific to a given system.
Some factors contributing to latency include:• Volume of system activity• Replication schedule• Business travel• Vacations
Message System Control ‘Chart’ – Design Considerations
Concept of a Statistical Control Chart has existed since 1920.• Plot sample values relative to a mean value
• Values that fall within +/- 3 standard deviations of the mean are considered to be “chance variation” and the process is “in control”
• Values that fall outside the limits are considered to be “assignable variation” and the process is “out of control”
Message System Control ‘Chart’ – Design Considerations
0
5
10
15
20
25
1 2 3 4 5 6 7 8 9 10
UCL
Mean
LCL
Sample Values
Message System Control ‘Chart’ – Design Alternatives
Flag any remote user whose time_received is not current as an exception.• This would flag anyone who is out of the office of a
day as an exception• Could set a range for this, but the value would still
be arbitrary
Generate a control chart• Over time it will reflect the normal variations in the
system• Will filter out the ‘chance variation’ and be more
likely to correctly flag the ‘assignable variation’
Message System Control ‘Chart’ – Implementation
Formulas:UCLX = X + A2R
LCLX = X - A2R
X = ( ∑X )/ N
R = ( ∑ R ) /N
Message System Control ‘Chart’ – Implementation
Use a database event to capture the daily average latency, range of latency, and record ‘out of control’ remote users in an exception report
Requires 3 working tables:Control_chart Latency_calculationLatency_Exception_Report
Message System Control ‘Chart’ – Implementation
CREATE TABLE control_chart
( sample_number INT DEFAULT AUTOINCREMENT PRIMARY KEY,
sample_mean NUMERIC(20,0) NOT NULL,
sample_range NUMERIC(20,0) NOT NULL,
sample_date DATE DEFAULT now()
);
Message System Control ‘Chart’ – Implementation
CREATE GLOBAL TEMPORARY TABLE
latency_calculation
( user_id INT PRIMARY KEY,
latency NUMERIC(20,0)
);
Message System Control ‘Chart’ – Implementation
CREATE TABLE latency_exception_report
( user_name CHAR(128),
exception_date DATE DEFAULT now(),
latency NUMERIC(20,0),
upper_control_limit NUMERIC(20,0),
lower_control_limit NUMERIC(20,0),
PRIMARY KEY (user_name, exception_date)
);
Message System Control ‘Chart’ – Implementation
CREATE EVENT populate_control_chartSCHEDULE daily_sampleSTART TIME '01:00AM' EVERY 24 HOURSAT ALLHANDLERBEGINDECLARE grand_mean NUMERIC(20,0);DECLARE average_range NUMERIC(20,0);DECLARE a_2 NUMERIC(20,3);DECLARE ucl NUMERIC(20,0);DECLARE lcl NUMERIC(20,0);
Message System Control ‘Chart’ – Implementation
MESSAGE 'Firing event "populate_control_chart"';
SET a_2 = 0.729; //this assumes a sample size of 4 remote users, you may want to use a larger sample size
INSERT INTO latency_calculation SELECT TOP 4 user_id, (log_sent-confirm_sent) AS latency FROM sys.sysremoteuser;
INSERT INTO control_chart(sample_mean, sample_range) SELECT avg(latency) AS mean, ( max( latency )-min( latency ) ) AS range FROM latency_calculation;
TRUNCATE TABLE latency_calculation;
Message System Control ‘Chart’ – Implementation
SELECT avg(sample_mean) INTO grand_mean FROM control_chart;
SELECT avg(sample_range) INTO average_range FROM control_chart;
SET ucl = grand_mean + (a_2*average_range);
SET lcl = grand_mean - (a_2*average_range);
Message System Control ‘Chart’ – Implementation
INSERT INTO latency_exception_report( user_name, latency, upper_control_limit, lower_control_limit)
SELECT user_name, ( log_sent - confirm_sent ) AS latency, ucl, lcl FROM sys.sysremoteusers
WHERE latency > ucl
OR latency < lcl;
END;
Topics
• Multiple Subscriptions
• Message System Control ‘Chart’
• Self-Refreshing Primary Key Pool
• Passthrough in a Multi-Tier Environment
• Request Queue
• DBRemote and DBXtract Exclusivity
• Last Chance Threshold
Self-Refreshing Primary Key Pool Business Requirement
In a distributed system, the generation and assignment of unique primary key values must be built in to the application.
Self-Refreshing Primary Key Pool Design Considerations
Since the assignment of a key value must be done consistently regardless of the front-end application the proper place to implement this functionality is in the database.
A Primary Key Pool allows you to generate a single column primary key as opposed to a using a composite key.
The Global Autoincrement default (introduced in 7.0) may still require refreshing over time.
Manual maintenance of a key pool would be an unacceptable risk.
Self-Refreshing Primary Key Pool Design Considerations
Trigger Actions Don’t Replicate!
Except:• Running DBRemote with the -t switch
• Running SSRemote
• Result of a RESOLVE UPDATE trigger
• Modification to the current row in a BEFORE trigger
• An UPDATE … PUBLICATION statement
• An update to a SUBSCRIBE BY column that changes “ownership” of the row
Self-Refreshing Primary Key Pool Design Alternatives
Use database EVENTs to automate the maintenance of the Primary Key PoolEvents were introduced in ASA 7.0.Events can be triggered by a type of action or they can be
scheduled• “Connect”• EVERY …
Can not be used on ASEUse a trigger to maintain the Primary Key Pool
Take advantage of the exceptions under which trigger actions will replicate
Self-Refreshing Primary Key Pool Implementation
Regional Office
SalesRep1
Expense
Expense KeyPoolExpense 1 SalesRep1Expense 2 SalesRep2Expense 3 SalesRep1
KeyPool
Expense 1Expense 3
Mileage $40.001
1 Mileage $40.00
Expense 4 NoOneSalesRep1
Expense 4
Self-Refreshing Primary Key Pool Implementation
Regional Office - Table Definition
CREATE TABLE KeyPool
(
table_name VARCHAR(128) NOT NULL,
value INTEGER NOT NULL,
remote_location VARCHAR(128) NOT NULL,
PRIMARY KEY (table_name, value)
);
Self-Refreshing Primary Key Pool Implementation
SalesRep1 - Table Definition
CREATE TABLE KeyPool
(
table_name VARCHAR(128) NOT NULL,
value INTEGER NOT NULL,
PRIMARY KEY (table_name, value)
);
Self-Refreshing Primary Key Pool Implementation
Regional Office - Publication & Subscriptions
Create Publication Admin_Pub( TABLE KeyPool( table_name, value ) SUBSCRIBE BY ( remote_location ));
CREATE SUBSCRIPTION TO Admin_Pub('SalesRep1') FOR SalesRep1;CREATE SUBSCRIPTION TO Admin_Pub('SalesRep2') FOR SalesRep2;CREATE SUBSCRIPTION TO Admin_Pub('Manager1') FOR Manager1;
No Publication or Subscriptions for the Key Pool on the Remotes
Self-Refreshing Primary Key Pool Implementation
Regional Office - Trigger Definition
CREATE TRIGGER BI_Expense BEFORE INSERT ORDER 1 ON Expense
REFERENCING NEW AS newrow
FOR EACH ROW
BEGIN
DECLARE row_id_var INT;
DECLARE new_row_id INT;
SELECT (MAX(value)+1) INTO new_row_id FROM KeyPool WHERE table_name='Expense';
Self-Refreshing Primary Key Pool Implementation
Regional Office - Trigger Definition (cont.)
IF (CURRENT PUBLISHER = 'RegionalOffice' AND CURRENT REMOTE USER IS NOT NULL ) THEN
INSERT INTO KeyPool VALUES('Expense',new_row_id,'NoOne'); UPDATE KeyPool SET remote_location = CURRENT REMOTE USER WHERE table_name = 'Expense' AND value = new_row_id;
DELETE FROM KeyPool WHERE table_name = 'Expense' AND value = newrow.expense_id;
Self-Refreshing Primary Key Pool Implementation
Regional Office - Trigger Definition (cont.)ELSE/* Else, this is a new insert so do the following*/
SET row_id_var = -1;SELECT MIN(value) INTO row_id_var FROM KeyPool WHERE table_name = 'Expense' AND remote_location = CURRENT PUBLISHER;
IF ( row_id_var = -1 OR row_id_var IS NULL) THENMESSAGE 'Out of key values' TYPE ACTION TO CLIENT;RETURN
END IF;
Self-Refreshing Primary Key Pool Implementation
Regional Office - Trigger Definition (cont.)
SET newrow.expense_id = row_id_var;
DELETE FROM KeyPool
WHERE table_name = 'Expense'
AND value = newrow.expense_id;
INSERT INTO KeyPool VALUES('Expense', new_row_id, CURRENT PUBLISHER );
END IF;
END;
Self-Refreshing Primary Key Pool Implementation
SalesRep1 - Trigger DefinitionCREATE TRIGGER BI_Expense BEFORE INSERT ORDER 1 ON Expense
REFERENCING NEW AS newrow
FOR EACH ROW
BEGIN
DECLARE row_id_var INT;
IF ( CURRENT REMOTE USER IS NULL ) THEN
SET row_id_var = -1;
SELECT MIN(value) INTO row_id_var FROM KeyPool
WHERE table_name = 'Expense';
Self-Refreshing Primary Key Pool Implementation
SalesRep1 - Trigger Definition (cont.) IF ( row_id_var = -1 OR row_id_var IS NULL) THEN
MESSAGE 'Out of key values’ TYPE ACTION TO CLIENT; RETURN
END IF;
SET newrow.expense_id = row_id_var; DELETE FROM KeyPool WHERE table_name = 'Expense' AND value = newrow.expense_id;
END IF;END;
Topics
• Multiple Subscriptions
• Message System Control ‘Chart’
• Self-Refreshing Primary Key Pool
• Passthrough in a Multi-Tier Environment
• Request Queue
• DBRemote and DBXtract Exclusivity
• Last Chance Threshold
PASSTHROUGH in a Multi-Tier Environment
Business Requirement
Two reasons for a Multi-Tier distributed environmentFunctional design differences between tiers
• Sales Reps may share customer details via the Regional Office
• Regional Offices may share budget data with the Head Office
Load Balancing across multiple mid-tier consolidated servers
PASSTHROUGH in a Multi-Tier Environment Design Considerations
PASSTHROUGH syntax is additive
• Additional PASSTHROUGH FOR … statements add recipients, it does not nest
PASSTHROUGH works on only 1 level of a hierarchy
PASSTHROUGH lets you perform DDL which is difficult to accomplish with a Request Queue
PASSTHROUGH in a Multi-Tier Environment Design Considerations
Head Office
Regional Office
SalesRep1
PASSTHROUGH FOR RegionalOffice;
ALTER TABLE Expense ADD expense_date DATETIME DEFAULT CURRENT TIMESTAMP;
PASSTHROUGH STOP;
PASSTHROUGH FOR SalesRep1;
PASSTHROUGH in a Multi-Tier Environment Design Considerations
Regional Office
SalesRep1
PASSTHROUGH FOR SalesRep1;
ALTER TABLE Expense ADD expense_date DATETIME DEFAULT CURRENT TIMESTAMP;
PASSTHROUGH STOP;
PASSTHROUGH FOR SalesRep2;
SalesRep2
PASSTHROUGH in a Multi-Tier Environment Design Alternatives
Distribute changes outside of SQL Remote• requires an alternative communication link to distribute
script files
Disconnected from the data flow in SQL Remote
• difficult to ensure both timing and compliance
Use a Request QueueThis is our next topic.
Nest PASSTHROUGH statements within stored procedures
PASSTHROUGH in a Multi-Tier Environment Implementation
Head Office - Script
ALTER TABLE Expense ADD expense_date DATETIME DEFAULT CURRENT TIMESTAMP;
CREATE PROCEDURE Regional_Passthrough_Proc()BEGIN MESSAGE 'This is a dummy procedure that is created so that when the PASSTHROUGH
session is parsed, the procedure name can be found'; MESSAGE 'This procedure will be dropped once the PASSTHROUGH session
is completed';END;
PASSTHROUGH ONLY FOR RegionalOffice;
PASSTHROUGH in a Multi-Tier Environment Implementation
Head Office - Script (cont.)
CREATE PROCEDURE Regional_Passthrough_Proc ( )
BEGIN
PASSTHROUGH FOR SUBSCRIPTION TO Admin_Pub;
ALTER TABLE Expense
ADD expense_date DATETIME DEFAULT CURRENT TIMESTAMP;
PASSTHROUGH STOP;
END;
CALL Regional_Passthrough_Proc();
DROP PROCEDURE Regional_Passthrough_Proc;
PASSTHROUGH in a Multi-Tier Environment Implementation
Head Office - Script (cont.)
PASSTHROUGH STOP;
DROP PROCEDURE Regional_Passthrough_Proc;
PASSTHROUGH in a Multi-Tier Environment Implementation
Regional Office - What gets executed?
CREATE PROCEDURE Regional_Passthrough_Proc ( )
BEGIN
PASSTHROUGH FOR SUBSCRIPTION TO Admin_Pub;
ALTER TABLE Expense
ADD expense_date DATETIME DEFAULT CURRENT TIMESTAMP;
PASSTHROUGH STOP;
END;
CALL Regional_Passthrough_Proc();
DROP PROCEDURE Regional_Passthrough_Proc;
PASSTHROUGH in a Multi-Tier Environment Implementation
SalesRep1/SalesRep2/Manager1 - What gets executed?
ALTER TABLE Expense
ADD expense_date DATETIME DEFAULT CURRENT TIMESTAMP;
Topics
• Multiple Subscriptions
• Message System Control ‘Chart’
• Self-Refreshing Primary Key Pool
• Passthrough in a Multi-Tier Environment
• Request Queue
• DBRemote and DBXtract Exclusivity
• Last Chance Threshold
Request Queue
Business RequirementFrom an application standpoint, some operations may
have to be performed at a central location.• Approval of a loan application
• Assignment of an e-mail id
For Administrative reasons you may need to send out commands to be executed at remote sites.
• Retrieve information from the sys.sysremoteuser table
For performance reasons you may want to perform a batch operation
• Archive records older than a certain date
Request Queue Design Considerations
The only operations that replicate automatically are INSERTs, UPDATEs and DELETEs.
Any operation that is the result of a PASSTHROUGH session will not replicate.
• You can’t force a return code to replicate back
PASSTHROUGH only affects 1 tier down• Additional PASSTHROUGH FOR … statements
add recipients to the list, they do not nest
Request Queue Design Alternatives
Monitor queue with a database EVENT
Monitor queue with an external process
Use a trigger to handle requests as they are entered
You could use different tools for different operation types
Request Queue Implementation
Request_Queue
request_idremote_locationop_codestmtstatustimestamp
Request_Queue_Return_Codes
request_idremote_locationreturn_codesubscribe_by
Composite Primary Key
Table Definitions
Request Queue Implementation
Head Office
Regional Office
1 ‘ALL’ 3 ‘DELETE FROM Expense’Request_Queue
1 ‘RegionalOffice’ 0 HeadOffice
Request_Queue_Return_Codes
1 ‘ALL’ 3 ‘DELETE FROM Expense’Request_Queue
Expense
1 Mileage $40.00
1 ‘RegionalOffice’ 0 NoOne
Request_Queue_Return_Codes
HeadOffice
Request Queue Implementation
Head Office - Publication & SubscriptionsCREATE PUBLICATION Admin_Pub
(
TABLE Request_Queue SUBSCRIBE BY remote_location
);
CREATE SUBSCRIPTION TO Admin_Pub('ALL') FOR RegionalOffice;
CREATE SUBSCRIPTION TO Admin_Pub('RegionalOffice') FOR RegionalOffice;
CREATE SUBSCRIPTION TO Admin_Pub('SalesRep1') FOR RegionalOffice;
CREATE SUBSCRIPTION TO Admin_Pub('SalesRep2') FOR RegionalOffice;
CREATE SUBSCRIPTION TO Admin_Pub('Manager1') FOR RegionalOffice;
Request Queue Implementation
Regional Office - PublicationsCREATE PUBLICATION Admin_Pub
( TABLE KeyPool( table_name, value ) SUBSCRIBE BY ( remote_location ),
TABLE Request_Queue SUBSCRIBE BY ( remote_location )
);
CREATE PUBLICATION Response_Pub
( TABLE Request_Queue_Return_Codes SUBSCRIBE BY subscribe_by
);
Request Queue Implementation
Regional Office - SubscriptionsCREATE SUBSCRIPTION TO Admin_Pub('SalesRep1') FOR SalesRep1;
CREATE SUBSCRIPTION TO Admin_Pub('SalesRep2') FOR SalesRep2;
CREATE SUBSCRIPTION TO Admin_Pub('Manager1') FOR Manager1;
CREATE SUBSCRIPTION TO Admin_Pub('ALL') FOR SalesRep1;
CREATE SUBSCRIPTION TO Admin_Pub('ALL') FOR SalesRep2;
CREATE SUBSCRIPTION TO Admin_Pub('ALL') FOR Manager1;
CREATE SUBSCRIPTION TO Response_Pub('HeadOffice') FOR HeadOffice;
Request Queue Implementation
Regional Office - TriggerCREATE TRIGGER AI_Request_Queue
AFTER INSERT ON request_queue
REFERENCING NEW AS newrow
FOR EACH ROW
BEGIN
DECLARE command_stmt LONG VARCHAR;
IF ( CURRENT PUBLISHER = newrow.remote_location OR
'ALL' = newrow.remote_location ) THEN
SET command_stmt = newrow.stmt;
Request Queue Implementation
Regional Office - Trigger (cont.)CASE newrow.op_codeWHEN 1 THEN // Do nothing, this is an information message for the // application to display MESSAGE 'Display message: '+command_stmt TYPE WARNING TO CLIENT;
SET command_stmt = '';
WHEN 2 THEN// This is a command that should be executed on the // databaseMESSAGE 'Executed: '+command_stmt TYPE WARNING TO CLIENT;EXECUTE IMMEDIATE command_stmt;
Request Queue Implementation
Regional Office - Trigger (cont.)WHEN 3 THEN
// This command should be executed on the database
// and the return code should be replicated back up
MESSAGE 'Executed: '+command_stmt TYPE WARNING TO CLIENT;
EXECUTE IMMEDIATE command_stmt;
INSERT INTO Request_Queue_Return_Codes( request_id, return_code,
subscribe_by )
VALUES( newrow.request_id, SQLCODE, 'NoOne' );
UPDATE Request_Queue_Return_Codes
SET subscribe_by = 'HeadOffice'
WHERE request_id = newrow.request_id;
Request Queue Implementation
Regional Office - Trigger (cont.) ELSE
// Do nothing
MESSAGE 'Ignoring: '+command_stmt TYPE INFO TO CLIENT;
SET command_stmt = '';
END CASE;
ELSE
MESSAGE 'Message not for me: '+newrow.stmt;
END IF;
END;
Topics
• Multiple Subscriptions
• Message System Control ‘Chart’
• Self-Refreshing Primary Key Pool
• Passthrough in a Multi-Tier Environment
• Request Queue
• DBRemote and DBXtract Exclusivity
• Last Chance Threshold
DBRemote and DBXtract Exclusivity
Business RequirementDBRemote and DBXtract must be run exclusive of each
other in order to ensure system integrity.
DBRemote and DBXtract Exclusivity
Design ConsiderationsDBRemote reads the sys.sysremoteuser table when it
starts up. When run in batch mode, the sys.sysremoteuser table is
only updated after that.DBXtract modifies the sys.sysremoteuser table by issuing
a REMOTE RESET command at the end of the extraction process.
If DBRemote and DBXtract are allowed to run at the same time, then the sys.sysremoteuser entries for the user being extracted can be left out of synch.
DBRemote and DBXtract Exclusivity – Design Considerations
In a future version of ASA, DBRemote & DBXtract will enforce this behavior automatically.
In the meantime:Need to be able to identify when DBRemote or DBXtract
is connecting.Multiple DBXtract sessions can run concurrently.
e.g. Extract multiple remote users at the same time.Multiple DBRemote sessions can run concurrently.
e.g Separate processes for sending and receiving.
DBRemote and DBXtract Exclusivity – Design Alternatives
ManualHave the system administrator ensure that DBRemote is shut down
before running DBXtract.
• Anything relying on human intervention is error prone
• If DBRemote is run be a scheduler, then it may start before an extract (or set of extracts) completes
Use “Connect” and “Disconnect” Database EventsDetect and record when DBRemote or DBXtract connects to the
database
Still relies on specific user ids being used to run each of DBRemote and DBXtract
DBRemote and DBXtract Exclusivity – Implementation
Create specific user ID’s to be used by DBRemote and DBXtract.GRANT CONNECT TO dbremote_user IDENTIFIED BY sql;GRANT REMOTE DBA TO dbremote_user;
GRANT CONNECT TO dbxtract_user IDENTIFIED BY sql;GRANT REMOTE DBA TO dbxtract_user;
Ensure that these user ID’s are used in your scripts for running DBRemote and DBXtract
DBRemote and DBXtract Exclusivity – Implementation
CREATE TABLE dbremote_dbxtract_exclusive
( connection_type CHAR(128) PRIMARY KEY,
instance_count INT
);
connection_type instance_count
dbxtract_user 0
dbremote_user 0
DBRemote and DBXtract Exclusivity – Implementation
“Connect” EventCREATE EVENT connect_force_dbremote_dbxtract_exclusive
TYPE "Connect"
AT ALL
HANDLER
BEGIN
DECLARE dbxtract_count INT;
DECLARE dbremote_count INT;
DECLARE connection_id_var INT;
DBRemote and DBXtract Exclusivity – Implementation
“Connect” Event (cont)SELECT EVENT_PARAMETER('ConnectionID') INTO connection_id_var;
IF ( EVENT_PARAMETER('User') = 'dbxtract_user' ) THEN MESSAGE 'Connect event was fired by the dbxtract_user connecting';
SELECT instance_count INTO dbremote_count FROM dbremote_dbxtract_exclusive
WHERE connection_type = 'dbremote_user'; IF ( dbremote_count != 0 ) THEN
MESSAGE 'Dropping connection for dbxtract_user'; EXECUTE IMMEDIATE 'DROP CONNECTION '||
connection_id_var; END IF;
DBRemote and DBXtract Exclusivity – Implementation
“Connect” Event (cont)UPDATE dbremote_dbxtract_exclusive
SET instance_count = instance_count + 1 WHERE connection_type = 'dbxtract_user';
END IF;
IF ( EVENT_PARAMETER('User') = 'dbremote_user' ) THEN Message 'Connect event was fired by the dbremote_user connecting';
SELECT instance_count INTO dbxtract_count FROM dbremote_dbxtract_exclusive
WHERE connection_type = 'dbxtract_user';
DBRemote and DBXtract Exclusivity – Implementation
“Connect” Event (cont)IF ( dbxtract_count != 0 ) THEN
MESSAGE 'Dropping connection for dbremote_user'; EXECUTE IMMEDIATE 'DROP CONNECTION '||
connection_id_var; END IF;
UPDATE dbremote_dbxtract_exclusive SET instance_count = instance_count + 1 WHERE connection_type = 'dbremote_user';
END IF; END;
DBRemote and DBXtract Exclusivity – Implementation
“Disconnect” EventCREATE EVENT
disconnect_force_dbremote_dbxtract_exclusive
TYPE "Disconnect"
AT ALL
HANDLER
BEGIN
DECLARE dbxtract_count INT;
DECLARE dbremote_count INT;
DBRemote and DBXtract Exclusivity – Implementation
“Disconnect” Event (cont.)IF ( EVENT_PARAMETER('User') = 'dbxtract_user' ) THEN
Message 'Disconnect event was fired by the dbxtract_user connecting';
UPDATE dbremote_dbxtract_exclusiveSET instance_count = instance_count - 1WHERE connection_type =
'dbxtract_user';END IF;
DBRemote and DBXtract Exclusivity – Implementation
“Disconnect” Event (cont.)
IF ( EVENT_PARAMETER('User') = 'dbremote_user' ) THEN
Message 'Disconnect event was fired by the dbremote_user connecting';
UPDATE dbremote_dbxtract_exclusiveSET instance_count = instance_count - 1WHERE connection_type = 'dbremote_user';
END IF;END;
Topics
• Multiple Subscriptions
• Message System Control ‘Chart’
• Self-Refreshing Primary Key Pool
• Passthrough in a Multi-Tier Environment
• Request Queue
• DBRemote and DBXtract Exclusivity
• Last Chance Threshold
Last Chance Threshold
Business RequirementBusiness rules may be dependent on certain events.
• Once an order has been submitted there may be a fee to change or cancel the order
• airline ticket purchase
These events may relate to when the record is replicated rather than when it was inserted/updated/deleted
• Need to know when DBRemote/SSRemote was actually run and modify records in the database accordingly
Last Chance Threshold Design Considerations
SQL Remote is log based replication• once an operation is in the log file it will be replicated
• DBRemote/SSRemote do not rely on the database file when sending messages
SQL Remote is message based replication• From the remote, you can’t tell when the consolidated
actually receives the record
• You can only flag a record with a status to indicate whether or not it has already been sent
Last Chance Threshold Design Alternatives
Implement a custom wrapper to SQL Remote • use the SQL Remote API provided in dbrmt.h
• call a stored procedure before running dbremote to flag rows as ‘pending’
• commit the operation once dbremote returns successfully
Use the following new features:• CurrentRedoPos database property implemented in 7.0.0
• SQL Remote hooks implemented in 7.0.1
Last Chance Threshold Design Alternatives
CurrentRedoPos Database PropertyThe current offset in the transaction log file, where the next database
operation is to be logged.
SQL Remote HooksStored procedures which can be defined for DBRemote/SSRemote to call at
particular points in the process• sp_hook_dbremote_begin, sp_hook_ssrmt_begin• sp_hook_dbremote_receive_end, sp_hook_ssrmt_recv_end• sp_hook_dbremote_send_end, sp_hook_ssrmt_send_end• …
Hook actions Replicate!
Last Chance Threshold Implementation
order_idcustomer_idproduct_idquantitystatus_idorder_datecreated_log_offsetcreated_location
OrderOrder_Status
status_id description
0 'New Order' 1 'Submitted' 2 'Received' 3 'Processing' 4 'Shipping' 5 'Back Ordered' 6 'Filled' 7 'Approved' 8 'Denied' 9 'Pending'
Tables
Last Chance Threshold Implementation
SalesRep1
DBRemote/SSRemote
123 … 0 … 251802 SalesRep1
Order
RegionalOffice
Insert . . .
CALL sp_hook_dbremote_send_end
252130
1
… log_sent confirm_sent …
sys.sysremoteuser
… 251656 251656 ...
Last Chance Threshold Implementation
Regional Office
DBRemote/SSRemote
Order
123 … 0 … 251802 SalesRep1
RegionalOffice
Insert . . .
CALL sp_hook_dbremote_receive_end
2
SalesRep1
Update . . .
Last Chance Threshold Implementation
SalesRep1
DBRemote/SSRemote
SalesRep1
Update . . .
Order
123 … 1 … 251802 SalesRep12
Last Chance Threshold Implementation
CREATE TABLE "DBA"."Order”(
"order_id" integer NOT NULL,"customer_id" integer NOT NULL,"product_id" integer NOT NULL,"quantity" integer NOT NULL,"status_id" tinyint NOT NULL DEFAULT 0,"order_date" date NOT NULL DEFAULT current date,"created_log_offset" NUMERIC(20,0) DEFAULT
db_property('CurrentRedoPos'),"created_location" VARCHAR(128) DEFAULT CURRENT PUBLISHER,PRIMARY KEY ("order_id")
);
Last Chance Threshold Implementation
SalesRep1 - sp_hook_dbremote_send_end
CREATE PROCEDURE sp_hook_dbremote_send_end()
BEGIN
UPDATE "Order", sys.sysremoteusers
SET "Order".status_id = 1
WHERE "Order".created_log_offset < sys.sysremoteusers.log_sent
AND "Order".status_id = 0
AND "Order".created_location = CURRENT PUBLISHER;
END;
Last Chance Threshold Implementation
Regional Office -sp_hook_dbremote_receive_end
CREATE PROCEDURE sp_hook_dbremote_receive_end()
BEGIN
UPDATE "Order"
SET "Order".status_id = 2
WHERE "Order".status_id = 1
OR "Order".status_id = 0
AND "Order".created_location != CURRENT PUBLISHER;
END;
Topics
• Multiple Subscriptions
• Message System Control ‘Chart’
• Self-Refreshing Primary Key Pool
• Passthrough in a Multi-Tier Environment
• Request Queue
• DBRemote and DBXtract Exclusivity
• Last Chance Threshold
Recommended Sessions
Morning SessionsAM33 - Adaptive Server Anywhere (ASA) Internals
Performance and TuningAM38 – Data Replication with ASAAM36 – MobiLink Synchronization
Afternoon SessionsEM415 – Custom Extraction TechniquesEM417 - Setting Up a SQL Remote System for PerformanceEM418 - SQL Remote for Adaptive Server Anywhere Internals EM408 - Back-up and Recovery of SQL Anywhere: Tips &
Techniques
Newsgroups
forums.sybase.com
\sybase.public.sqlanywhere.general\sybase.public.sqlanywhere.linux\sybase.public.sqlanywhere.mobilink\sybase.public.sqlanywhere.product_futures_discussion\sybase.public.sqlanywhere.replication\sybase.public.sqlanywhere.ultralite
iAnywhere Solutions Highlights
• Ask the Experts - about Mobile & Wireless Solutions-Mezzanine Level Room 15B Mon./Tues. 11:30 am - 3:30 pm; Wed. 11:30 - 1:30;Thurs. 9 am - 12 noon-Exhibit Hall - Demo Center (truck) exhibit hall hours
• SIG (Special Interest Group)- Tuesday 5:30pm Mobile & Wireless SDCC, Upper level, Room 11
• Keynote - Enabling m-Business SolutionsWednesday 1:30 pm - 3:00 pm
• iAnywhere Solutions Developer Community-Excellent resource for commonly asked questions, newsgroups, bugfixes, newsletters, event listings - visit www.ianywhere.com/developer