1 EM413 Using Adaptive Server Anywhere’s Remote Data Access Feature Mark Culp Software Development...
Transcript of 1 EM413 Using Adaptive Server Anywhere’s Remote Data Access Feature Mark Culp Software Development...
1
EM413Using Adaptive Server Anywhere’s Remote Data Access Feature
Mark CulpSoftware Development ManageriAnywhere [email protected]
2
Goals of this presentation
• Enable you to access remote data from your ASA database
• Show you how to migrate your data to ASA
• Help you improve your cross database join performance
• What should you not do when using ASA's remote data access feature
3
Contents
• What is “Remote Data Access” ?• Basics – How to get started• What if it doesn't work? – Some helpful
hints• Data migration tools• Cross database joins• Limitations – What to avoid• Questions
4
What is “Remote Data Access” ?
5
What is “Remote Data Access” ?
• Allows an ASA server to access data from a remote source as if it were local
• Remote data may be :• iAnywhere ASA, Sybase ASE• Oracle, IBM DB2, MS SQL Server• Other ODBC data sources
• E.g. MS Access, MS Excel
6
Why use “Remote Data Access” ?
• Cross database joins• Maintain real time access to remote
data• Integration to other projects and other
data sources• Migration of legacy data• Quickly set up a remote ASA database
for Mobilink
7
ASA Server
ASAStore
QueryExecution
Engine
ClientComm.
What is … - Architecture
RemoteStore
Other
ODBCASE
jConnect ASA
Remote
Server
8
What is … - Platforms
Available on• Win32 platforms:
95,98,ME,NT,2000• UNIX platforms (since ASA 7.0.2):
Linux and SPARC Solaris• Netware (since ASA 7.0.2)
9
What is … – Also known as …
Remote Data Access capability is also known as:
• OMNI - OmniConnect (from ASE 10.5)• CIS - Component Integration Services• Proxy Tables
10
Basics – How to get started
11
Basics
Three steps to accessing remote data1. Define your remote server2. Map your external login3. Create your remote proxy tables
Then use the proxy tables in the same way you would use any other table in your query
12
ASA needs to know:• What you will call the remote server• How to connect to the remote server• The capabilities of the remote server
Basics – Step 1:Define your remote server
SQL statement:create server <server_name>
class <connection_class>
using <remote_location>
13
Step 1 – Create server … class
Connection class:• JDBC (jConnect) classes:
• asajdbc, asejdbc
• ODBC classes:• asaodbc, aseodbc – ASA and ASE• db2odbc – IBM DB2• mssodbc – MS SQL Server• oraodbc – Oracle server (version 8.0 or later)• odbc – any other ODBC data source
14
Step 1 – Create server … location (JDBC)
Remote location (JDBC):<machine_name>:<port> [ /<db_name> ]
Location appended to JDBC URL specification:'jdbc.sybase.TDS.'
/<db_name> only required if more than one database loaded on remote server
15
Step 1 – Create server … example (JDBC)
Example 1 (using jConnect):create server DemoServerViaJconnect
class 'asajdbc'using 'localhost:2638/asademo'
jConnectasajdbcDemoServer
ViaJconnect
ASA server:
asademolocalhost
:2638/asademo
Remote server:
16
Step 1 – Create server … location (ODBC)
Remote location (ODBC):<DSN_name> [ ; <other_options> ]
DSN (data source name) normally specifies parameters on how to find remote server
• Driver name • Driver specific parameters:
• E.g. ServerNameConnection is in context of the ASA server
17
Step 1 – Create server … location (ODBC)
Example 2 (using an ODBC DSN):create server DemoServer
class 'asaodbc'using 'demoDSN'
DemoServer
ASA server:
asaodbc ODBCDriver
Manager
ODBCDriver
demoDSN
Note: You can use dbdsn to create ASA DSNs:dbdsn –w demoDSN –c “eng=asademo”
asademoEng=
asademo
Remote server:
18
Step 1 – Create server … location (ODBC)
Example 3 (connect to an Excel Spreadsheet):create server ExcelData
class 'odbc'using 'ExcelDataDSN'
Note 'Read Only' Check Box
19
Step 1 – Create server … location (ODBC)
Example 4 (no ODBC DSN):create server DemoServerNoDSN
class 'asaodbc'using 'driver=Adaptive Server Anywhere
8.0; ServerName=asademo'
DemoServerNoDSN
ASA server:
asaodbc ODBCDriver
Manager
ODBCDriver
Driver=…asademo
ServerName=
asademo
Remote server:
20
Step 1 – Create server … location (UNIX)
Remote location (UNIX ODBC):<DSN_name> [ ; driver=<driver_path> ]
There is no standard ODBC driver manager on UNIX, so ASA needs to know where to find the driver (manager) that you want to use.
You must specify a driver-path either:• In your location string, or• In your file DSN specification
21
Step 1 – Create server … location (UNIX)
Example 5 (UNIX - no driver manager): create server DemoServerDirect
class 'asaodbc'using 'demoDSN'
Contents of ~/.odbc.ini:[demoDSN]driver=/opt/sybase/SYBSsa8/lib/dbodbc8_r.soservername=asademo
22
Step 1 – Create server … location (UNIX)
Example 5 continued (UNIX - no driver manager):
• Since no "driver=" found in location string, ASA looks for location of driver in the DSN
asademoDemoServer
Directasaodbc demoDSN
ASA server:
ODBCDriver
dbodbc8_r.so
Remote server:
23
Step 1 – Create server … location (UNIX)
Example 6 (UNIX - with driver manager): create server DemoServerUsingManager
class 'asaodbc'using 'driver=<manager_path>;
DSN=demoDSN'
Contents of ~/.odbc.ini:[demoDSN]driver=/opt/sybase/SYBSsa8/lib/dbodbc8_r.soservername=asademo
24
Step 1 – Create server … location (UNIX)
Example 6 continued (with driver manager):
• ASA loads ODBC driver manager• ODBC driver manager loads ODBC driver
dbodbc8_r.so
asademoDemoServer
UsingManagerasaodbc .odbc.ini
driver=…
ASA server:
ODBCDriver
Manager
ServerName=
asademo
ODBCDriver
Remote server:
25
Basics – Step 2:Map your external login
ASA may need to know:• User id and password to use when
connecting to the remote server
SQL statement:create externlogin <login_name>
to <remote_server>remote login <remote_user>[ identified by <remote_password> ]
26
Step 2 – When to create externlogin
• By default, ASA uses the names and passwords of its clients whenever it connects to a remote server on behalf of those clients.
• CREATE EXTERNLOGIN statement assigns an alternate login name and password to be used when communicating with a remote server
27
Step 2 – Map your external login
Example 7:create externlogin dba
to DemoServerremote login dba identified by sql
28
Basics – Step 3:Create proxy tables
ASA needs to know how to make the mapping from a local proxy table to its remote data:
• If the remote table already exists, use the CREATE EXISTING statement.
• If the remote table is to be created, use the CREATE … AT statement.
29
Step 3 – Create existing table
To map an existing remote table:
SQL Statement:create existing table [<owner>.]<table_name>
[ ( <column_definitions> ) ]at <location_clause>;
• This statement defines the proxy table for an existing table on the remote server.
30
Step 3 – Location clause
location_clause specifies where to locate the remote table, and is of the form:
'<server>;<database>;<owner>;<table>'where:• server - name of server as known by current
database from create server command• database - depends on type of remote server• owner - owner of remote table• table – name of the remote table
31
Step 3 – Create existing example
Example 8:create existing table dba.p_employee
at 'DemoServer;;dba;employee';
dba.p_employee
ASA local database: DemoServer:
dba.employee
proxy table: remote table:
32
Step 3 – Create new table
To create a new remote table:
SQL Statement:create table [<owner>.]<table_name>
( <column_definitions> )at <location_clause>;
This statement creates a new table on the remote server, and also defines the proxy table for that table.
33
Step 3 – Create new example
Example 9:create table p_employee_sheet
( emp_id numeric(9,0), emp_fname varchar(30) )at 'ExcelData;;;employee_sheet'
Copy data into the remote table using: insert into p_employee_sheet
select emp_id, emp_fname from p_employee
34
Some Helpful Hints
35
Hints – Dropping proxy tables
To drop a proxy table, use:
SQL Statement:drop table [<owner>.]<table_name>;
• If the proxy table was created using CREATE EXISTING, only the proxy table is dropped
• If the proxy table was created using CREATE … AT statement, then the proxy table and the remote table are both dropped.
36
Hints – Dropping proxy tables example
Example 10:drop table p_employee• Since this proxy table was created using
CREATE EXISTING, this statement drops the proxy table only; the remote table is not affected.
drop table p_employee_sheet• Since this proxy table was created using
CREATE TABLE … AT, this statement drops the proxy table, AND will also drop the remote table at 'ExcelData;;;employee_sheet'
37
Remote Data Access passthru mode
To send a statement directly to the remote server, use:
SQL statement:forward to <server_name>;<remote_statement(s)>forward to;
• The remote_statements text is sent to the remote server as given
• Text is terminated by the "forward" and "to" tokens
38
Hints – Useful system tablessys.sysservers – list of remote servers
select * from sys.sysservers
39
Hints – Useful system tablessys.systable – list of proxy tables
select table_name, remote_location, existing_objfrom systablewhere remote_location is not NULL;
40
Hints – Useful built-in procedures
sp_remote_tables server_name [, table_name ][, table_owner ] [, table_qualifier ][, with_table_type ]
• Lists tables available at remote server
sp_remote_columns server_name, table_name[, owner ] [, database]
• Lists columns in remote table
41
Remote Data Access trace CIS option
You diagnose remote data access problems by using the cis_option connection property
SQL statement:set temporary option cis_option = 7;
• ASA will display messages in console log.• Set cis_option value to 0 (zero) to turn off.
42
Trace CIS option example
Example 11:set temporary option cis_option=7;select * from p_employee;
43
Data Migration Tools
44
Data migration tools - Availability
You can import remote data using the sa_migrate set of stored procedures
• Available in ASA 7.0.2, 7.0.3 and 8.0• Source data may be from one of:
• iAnywhere ASA• Sybase ASE• Oracle, IBM DB2, MS SQL Server, MS
Access
45
Data migration tools - Basics
Basic methodology:• create base table(s) and proxy table(s)• copy data from remote table(s)• create foreign keys
proxytable
ASA local database: Remote data source:
remotetable
basetable
46
Migration – Single step process
To migrate entire remote database in one step, use stored procedure:
dbo.sa_migrate(IN local_table_owner VARCHAR(128),IN server_name VARCHAR(128),IN table_name VARCHAR(128) DEFAULT NULL,IN owner_name VARCHAR(128) DEFAULT NULL,IN database_name VARCHAR(128) DEFAULT NULL,IN migrate_data BIT DEFAULT 1,IN drop_proxy_tables BIT DEFAULT 1 )
47
Migration – Single step example
Example 12: call dbo.sa_migrate(
'dba', 'DemoServer', 'Customer', 'dba' )• This migrates the customer table definition and
data from DemoServer to my database by:• creating customer table in my database• creating customer_et proxy table• copying the customer data• dropping the customer_et proxy table
48
Migration – Single step example results
49
Migration – Multi-step process
To have more control over what is migrated, use:
1. sa_migrate_create_remote_table_list( … )2. sa_migrate_create_tables( … )3. sa_migrate_data( … )4. sa_migrate_create_remote_fks_list( … )5. sa_migrate_create_fks( … )6. sa_migrate_drop_proxy_tables( … )
50
Migration – Step 1: create table list
1. Create a list of tables to migrate usingdbo.sa_migrate_create_remote_table_list(
IN server_name varchar(128),IN table_name varchar(128) default NULL,IN owner_name varchar(128) default NULL,IN database_name varchar(128) default NULL )
• this populates the dbo.migrate_remote_table_list table
• delete any rows that you do not want
51
Migration – Step 2: create proxy tables
2. Create proxy and base tables using:dbo.sa_migrate_create_tables(
IN local_table_owner varchar(128) )
• uses rows in dbo.migrate_remote_table_list
• creates a proxy table and a base table for each remote table listed
• also creates all primary key indexes
52
Migration - Step 3: migrate data
3. Migrate data into base tables using:dbo.sa_migrate_data (
IN local_table_owner varchar(128) )
• uses rows in dbo.migrate_remote_table_list
• populates base tables with rows from proxy tables
53
Migration – Step 4: create fks list
4. Create list of foreign keys to migrate:dbo.sa_migrate_create_remote_fks_list(
IN server_name varchar(128) )
• uses rows in dbo.migrate_remote_table_list
• creates dbo.migrate_remote_fks_list table• delete any rows that you do not want
54
Migration – Step 5: create foreign keys
5. Create the foreign keys using:dbo.sa_migrate_create_fks(
IN local_table_owner varchar(128) )
• uses rows in dbo.migrate_remote_fks_list• creates a foreign key for each listed
55
Migration - Step 6: drop proxy tables
6. Drop all of the proxy tables:dbo.sa_migrate_drop_proxy_tables(
IN local_table_owner varchar(128) )
• uses rows in dbo.migrate_remote_table_list
• drops the proxy table (created in step 2) for each remote table listed
56
Migration – Multi-step example
Example 13:• call dbo.sa_migrate_create_remote_table_list
( 'DemoServer', NULL, 'dba' )• select * from dbo.migrate_remote_table_list• delete from dbo.migrate_remote_table_list
where table_name = 'customer' or table_name like 'sales_%' or table_name like 'fin
%'• call dbo.sa_migrate_create_tables( 'dba' )• call dbo.sa_migrate_data( 'dba' )• call dbo.sa_migrate_create_remote_fks_list( 'DemoServer' )• select * from dbo.migrate_remote_fks_list• call dbo.sa_migrate_create_fks( 'dba' )• call dbo.sa_migrate_drop_proxy_tables( 'dba' )
57
Migration – In case of a failure
Migration may fail if a foreign key referenced table does not exist
• You may choose to update the dbo.migrate_remote_fks_list table (delete the offending entry) and continue
• Minimally you must run dbo.sa_migrate_drop_proxy_tables to delete the temporary proxy tables that were created
58
Cross Database Joins
59
Cross database joins – What is it?
A "Cross Database Join" is a join between two or more tables where the tables reside on two or more different servers
Table A(ASA)
Join
Table B(remote)
Example A:
Table A(remote a)
Join
Table B(remote b)
Example B:
60
Cross database joins – Implementation
ASA handles cross database joins by:• Parse the query and identify remote tables• Push predicates down to remote server when
possible• Reads remote rows using forward-only cursorNote:• 6.0.x and 7.0.x uses only naïve nested loop
join operator
61
Cross database joins – Remote Join
• When the join is on tables from the same server, the join is pushed to the remote
Table A(remote a)
Join
Table B(remote a)
Example C:
62
Cross database joins – Example 14
Example 14:• R is remote table, S is ASA local table
Select R.* from R join Swhere R.k = S.k
For each row r in Rif exists( select 1 from S where S.k = r.k )
match row
63
Cross database joins – Example 15
64
Cross database joins – Example 16
65
Cross Database Joins – Why is it slow?
If ASA uses a plan that has the remote table on the right side of the join operator,then the remote data will be hauled across the communication link multiple times – once for each left hand side row.
For each row s in Sfor each row r in ( select * from R )
if s.k = r.k match row
66
Cross Joins – How to make it go faster?
• Only include remote columns that are absolutely required by the query
• Run the query using dbisql, and check the plan that is being used
• Simplify query to have fewer remote tables• Consider using local temporary tables to
hold intermediate results• Migrate data to ASA and perform join on
local table
67
Limitations:
What should you avoid ?
68
Limitations
Features not supported on remote data:• ALTER TABLE statement against remote tables• Triggers defined on proxy tables will not fire• SQL Remote• Java data types• Foreign keys that refer to remote tables are
ignored• Positioned UPDATE and DELETE• UPDATE and DELETE requiring an intermediate
temporary table• Backwards scrolling on cursors
69
What to avoid … No cycles
1. Don't create a proxy table that refers back to the same database.
Reason:Deadlock can occur if remote access waits for a lock caused by the same connection.
Corollary:Don't create cycles in your databases' remote access definitions.
70
What to avoid … Separate Servers
2. Don't create a proxy table that refers back to the same server.
Reason:Deadlock will occur when more than one concurrent remote access is attempted.
Work around: increase –gx parameterSolution:
Use separate servers to host the local and remote databases.
71
What to avoid … Join Large Tables
3. Don't join two large remote tables from different remote servers.
Reason:Slow response
Solution:Migrate one of the tables into an ASA table first.
72
What to avoid … N-way joins
4. Don't create complicated joins containing many remote tables from many different servers.
Reason:Slow response
Solution:Migrate table(s) into ASA first.
73
Summary
ASA's Remote Data Access feature is a powerful tool to access your remote data from your ASA database:
• Easy to migrate your legacy data to ASA, or• Use your remote data in place as if it were
part of the ASA database
More detailed information found in:ASA SQL User's Guide
15. Accessing Remote Data