Why Use xDB Replication Server from...

Why Use xDB Replication Server from EnterpriseDB?

An EnterpriseDB White Paper For DBAs, Application Developers, & Enterprise Architects April 2013


EnterpriseDB The Enterprise PostgreSQL Company 2

© Copyright 2013 EnterpriseDB Corporation. All rights reserved. EnterpriseDB and Postgres Plus are trademarks of EnterpriseDB Corporation. Other names may be trademarks of their respective owners. http://www.enterprisedb.com

Introduction

Within every successful company, there is a constant flow of data moving between various application systems. Data typically enters an organization through online transaction processing (OLTP) applications and is fed into a relational database (RDBMS). Those databases may be used in turn to service other OLTP systems or feed into business intelligence and reporting applications that are used to assist key decision makers in the company. Managing the flow of data throughout an organization can be a challenging task, even for seasoned database professionals. While there are various data integration and extract-transform-load (ETL) solutions available, the most used mechanism for moving data between various databases is replication. Replication, as its name implies, involves the copying of data between one or more databases so that more than one replica of a whole or partial database is available for use (usually on another physical machine). The use cases of replication can be many, however the core reasons that replication is used are the following:

• Simple backup: replication is used to provide a simple backup in case of a failure on the primary database.

• Sharded or read scale-out architecture: an application architect may use replication to duplicate lookup/read-intensive data across many servers that are fronted by a load balancer, with the entire system serving something such as a very demanding application that has lots of concurrent users.

• Business intelligence/reporting: a very common use of replication is to offload transactional data onto another machine that is used for query-intensive BI and reporting work.

• Simple high availability: especially in a cloud environment, replication may be used to support the elastic and high-availability nature of a cloud database.

• Improved price/performance: allows organizations to utilize low cost commodity hardware to relieve performance limitations instead of buying expensive hardware and / or network infrastructure.

Although replication has been a feature for a long time in most every database, it still can be a difficult thing to implement and manage well as it has grown into a versatile tool for solving more and more problems. IT professionals today are looking for ways that reduce the load of dealing with their replication needs, as well as solutions that cross database vendor boundaries (i.e. are heterogeneous in nature). Needless to say, solutions that don’t break an IT area’s budget are also highly desirable. EnterpriseDB offers just such a replication solution in its xDB Replication Server. Being the primary worldwide provider of PostgreSQL products and services, EnterpriseDB is dedicated to making enterprise database management available to everyone. Through its Postgres Plus products, EnterpriseDB builds on top of the world’s most advanced open source database and supplies additional




enterprise database server functionality such as powerful Oracle compatibility. That, plus a suite of management tools, gives database users increased performance, high-availability, productivity, and ease of use. This paper provides an overview of EnterpriseDB’s xDB Replication Server, describes how the solution functions, and then looks at the question of why and when the xDB Replication Server should be considered over the built-in replication functionality offered in PostgreSQL 9.0 and above.

What is xDB Replication Server?

xDB Replication Server is an asynchronous, master-to-slave replication system (supporting both single and multi-master configurations) enabling replication of tables from Postgres to Postgres databases, as well as from Oracle and Microsoft SQL Server databases to a Postgres database. Through a graphical user interface, a DBA or developer can easily create a variety of replication topologies. xDB Replication Server is created and marketed by EnterpriseDB.

How does xDB Replication Server Work? From a global perspective, xDB is architected to manage replication in a publish-subscribe manner. The data to be made available for copying is made public as a publication. A publication is defined as a named set of tables and views within a source database. The database that contains the publication is called the publication database of that publication.




To get a copy of that data, a DBA or developer must subscribe to the publication. A subscription is a named association with a publication. Along with its association with a publication, a subscription is associated with a destination database to which the publication is to be copied. This database is called the subscription database. The subscription tables are the tables in the subscription database where each subscription table is created from a corresponding table or view in the publication.

Why xDB over PostgreSQL Replication?

Since PostgreSQL 9.0, replication is now built into the database server. Previous editions of PostgreSQL had to rely on third party or community solutions to carry out replication needs, but that all changed with PostgreSQL 9.0. However, the streaming replication found in 9.0 can only replicate a full instance of PostgreSQL to another PostgreSQL database; there is no way to replicate only a particular database or set of tables. Other limitations such as no strong monitoring capabilities, limited topologies, no record filtering, and replication across different database versions also exist. EnterpriseDB’s xDB Replication Server can help those whose needs cannot be met with PostgreSQL 9.x replication. The sections that follow discuss the many benefits of using the xDB Replication Server over PostgreSQL 9.x replication and other community or third party replication solutions.

Multi-Master Replication Another interesting and increasingly popular aspect of replication is a configuration where users can make updates to any member in a cluster of database instances and have that member's update replicated to all the other members. In other words each member of the cluster is an updatable master instance and the data on each member stays synchronized with all the other members. This is significantly different than the one master one direction replication in wide use today and allows xDB Replication Server to address an ever-increasing variety of issues faced by DBAs in growing businesses.




xDB Replication Server's multi-master configuration allows DBAs new options for increasing read and write performance, providing scalability to serve more users, and keeping data available for around the clock access. Specifically, multi-master replication can offer organizations with geo-dispersed locations a number of advantages:

• Daily batch jobs that only temporarily synchronize databases can be replaced with near-real time updates for all database instances keeping information fresh and up to date around the clock for everyone,

• Write loads funneled to a single master and suffering from network latencies or hardware bound performance can be replaced with local masters each handling their own write load quicker than the overloaded single master,

• A single master presenting a single point of failure for write transactions can be replaced with multiple masters which can be used as failover backups for any downed master keeping write and read transactions available around the clock to all users

The flexibility afforded to DBAs in solving a wide variety of database issues makes a product like xDB Replication Server an indispensable tool for any Postgres shop.

Graphical Replication Management Replication between databases can sometimes be a difficult thing to manually setup, configure, monitor, and keep going. Such a thing can be tough enough when the task is to replicate data between the same database (e.g. Oracle to Oracle), but the job becomes ever harder when replication is required between different databases (e.g. from Oracle to PostgreSQL). The xDB Replication Server makes the job of setting up and maintaining various replication topologies easy through its graphical console. Whether it’s configuring PostgreSQL to PostgreSQL replication, Oracle to PostgreSQL replication, or




Microsoft SQL Server to PostgreSQL replication, xDB lets a DBA point-and-click their way through the process from start to finish.

Oracle to PostgreSQL Replication One of the major features and benefits of using xDB Replication Server is being able to migrate data straight from Oracle databases to a PostgreSQL database. All of the setup and connectivity is handled in a graphical style via the xDB Replication Server console, so there is no difficult manual configuration or scripting necessary. This configuration helps organizations reduce their costs for additional Oracle licenses yet utilize the Oracle data in a more liberal fashion. To set up Oracle replication to PostgreSQL, the DBA or developer connects to the Oracle database via xDB by entering in host and login information:




Once a connection is established with an Oracle host, the DBA or developer then creates a publication from the Oracle database (i.e. one or more tables to replicate) and then connects to a PostgreSQL server target that will serve as the subscriber. Once the Oracle source and PostgreSQL targets have been defined, the DBA or developer then performs a ‘snapshot’ that creates an initial copy of the Oracle table(s) on the PostgreSQL server (both schema and data). Once that has been done, they can choose to perform either full manual refreshes in the future, or schedule either recurring snapshots or synchronizations to occur, the latter which just replicates the changes made on the Oracle side to PostgreSQL rather than make an entirely new copy.

Microsoft SQL Server to PostgreSQL Replication In the same way that the xDB Replication Server replicates data from an Oracle database to PostgreSQL, it can do the same thing for Microsoft SQL Server databases. A DBA or developer adds a publication database by supplying xDB with the hostname, port number, security credentials, and database on the source Microsoft SQL Server database, and it is added as an xDB publication source. Once the SQL Server database is added as a publication, the DBA or developer then selects the option to create a new publication. The xDB Replication Server queries the SQL Server database data dictionary and lists the available tables in the database. One or more tables are then chosen by the DBA and a new publication is created. The next step is for the DBA or developer to create a subscription target for the new SQL Server publication. This is accomplished by supplying xDB with the host, database, and login information for the target PostgreSQL server. Once this has been done, a snapshot task is performed to initialize the PostgreSQL server from the SQL Server source, which involves making a copy of the entire SQL Server table and porting it to PostgreSQL. From that point, a schedule can be set up to perform either an entire refresh of the table or to only synchronize the PostgreSQL table with changes that occur to the published table on the SQL Server system.




Database and Object Filtering While PostgreSQL 9.x and above has built-in streaming replication, it currently does not offer any filtering or options to just replicate either selected databases within a PostgreSQL cluster or objects (selected tables and records) within a database. In other words, streaming replication replicates an entire instance of PostgreSQL data to another PostgreSQL machine or instance. A nice feature of xDB is that it offers filtering at both the database and object level, for both PostgreSQL source and targets as well as for Oracle and SQL Server source systems. This means a DBA or developer can select only the objects they need replicated and not have to bear the network or storage overhead of replicating objects and data that aren’t required.

Data Filters The xDB Replication Server provides a SQL WHERE clause filtering capability so only the specific data returned by the custom WHERE predicate will be replicated from the published source to the subscribed target. This feature becomes especially useful when a DBA or developer only wants to replicate a small portion of a very large table. Note that the filter is applied during the initialization of the subscribed table so only the rows desired are replicated from the very outset.




Flexible Replication Modes xDB offers a variety of replication modes that enable the DBA or developer to replicate in exactly the way they need. The two primary modes are snapshot and continuous. The snapshot mode makes a completely new copy of the source table(s) on the target (or subscriber) system. The continuous mode only replicates the data changes in near real time from the publisher database to the subscriber.

Custom Replication Scheduler Most (if not all) built-in replication mechanisms in databases replicate data from a source to a target system in a non-scheduled fashion, with the speed of replication being affected by a variety of factors (e.g. data load on the system, whether the replication is single or multi-threaded, etc.) In other words, the DBA or developer cannot control when replication between systems occurs.

The xDB Replication Server provides a custom scheduling interface that allows a user to schedule replication when they want it to happen. Schedules can be set up to occur as granular as every second or less often such as once a week or month. The scheduler wizard also provides an interface for a custom expression to be passed to the replication scheduler to define a flexible schedule.




Reporting Facility Another very useful feature of xDB is the reporting function that provides a useful history of replication activities. A DBA or developer can easily view the replication events for each published and subscribed object and even view the data that was replicated.

Command Line Interface A command line interface to all of xDB Replication Servers features also adds additional flexibility for DBAs. If desired, the functions of xDB can be carried out via an operating system’s command shell/line interface. Using the following command provides a full listing of all the options available: java -jar edb-repcli.jar –help. This enables the creation of handy scripts for manual management tasks or automated tasks.




Conclusions

Every successful business has its hands full managing the flow of data that runs through its IT systems. Database replication is oftentimes the primary mechanism that’s used to manage the flow of that data between various databases and database platforms for a variety of solutions. EnterpriseDB’s xDB Replication Server is a smart choice for database professionals who need to replicate data between two or more PostgreSQL systems, or who want to replicate data from either Oracle or Microsoft SQL Server to PostgreSQL. The flexibility and ease of use that xDB brings to what is normally an error prone and time-consuming process will be welcomed by anyone wishing to make life easier where database replication is concerned. XDB Replication Server also offers many replication options and features not found in streaming replication which is built into the PostgreSQL database server such as: multi-master replication, partial database replication, cross-version replication, row filters, a graphical replication management console, a history viewer, a command line interface for scripting, and a flexible replication scheduler. These features make available to DBAs an interesting toolset to solve a wider variety of data management problems.

The xDB Replication Server is easy to download, install and evaluate. More information and downloads of xDB can be found at: http://www.enterprisedb.com/download-xdb-replication-server-mmr For more information on all Postgres Plus products and services available from EnterpriseDB visit the EnterpriseDB website at http://www.enterprisedb.com .




About EnterpriseDB

EnterpriseDB, based in Bedford, MA, is the leading worldwide provider of enterprise-class PostgreSQL and Oracle database compatibility products and services. Used by thousands of business and government customers globally, EnterpriseDB delivers the industry’s best price-performance in a relational database. Employing some of the most innovative members of the PostgreSQL Community, EnterpriseDB builds upon 25-plus years of development to provide companies a proven, secure and feature-rich alternative to costly traditional databases. EnterpriseDB’s Postgres Plus family of products includes the open source community PostgreSQL, Postgres Plus Advanced Server with Oracle database compatibility, a Database-as-a-Service with its Postgres Plus Cloud Database and enterprise management and monitoring tools. EnterpriseDB understands that adopting an open source database is not a trivial task. You have lots of questions needing answers, schedules and budgets to keep, and processes to follow. We have helped thousands of organizations like yours through the steps to investigate, evaluate, prove, develop, and deploy their open source solutions. If you would like to discuss products, services, training, consulting, or enterprise support options, please do not hesitate to contact EnterpriseDB directly at: http://www.enterprisedb.com/general-inquiry-form . EnterpriseDB has offices in North America and around the world. The company was founded in 2004 and is headquartered in Bedford, MA. For more information, please visit http://www.enterprisedb.com. Sales Inquiries:

US +1 781-357-3390 or 1-877-377-4352 ; EMEA +44 (0) 123 522 7276 Japan +81-3-5488-6007

India +91-20-30589500/01

Why Use xDB Replication Server from...

Documents

Transcript of Why Use xDB Replication Server from...